Tighten rules and add Vercel-style validate/extract pipeline

README.md

@@ -58,13 +58,15 @@ Design the interface for this service layer
 ## Skill Structure
 
 Each skill contains:
-- `SKILL.md` - Entrypoint loaded into agent context
-- `README.md` - Human-facing overview
-- `metadata.json` - Version, abstract, references
-- `AGENTS.md` - Compiled document with all rules expanded
-- `rules/` - Individual rule files (one per rule)
-
-See `AGENTS.md` at the repo root for guidance on authoring new rules or sections.
+- `SKILL.md` - Entrypoint loaded into agent context (quick reference)
+- `README.md` - Human-facing overview and authoring workflow
+- `metadata.json` - Version, abstract, references, Python version floor
+- `AGENTS.md` - (generated) Compiled document with all rules expanded
+- `test-cases.json` - (generated) LLM evaluation data extracted from rule examples
+- `rules/` - Individual rule files (one per rule), plus `_sections.md` and `_template.md`
+- `src/` - Build, validate, and extract-tests scripts (`build.py`, `validate.py`, `extract_tests.py`)
+
+`AGENTS.md` and `test-cases.json` are generated outputs — do not edit them by hand. See the per-skill README and `AGENTS.md` at the repo root for guidance on authoring new rules or sections.
 
 ## License
 

skills/python-best-practices/README.md

@@ -2,96 +2,141 @@
 
 A structured skill for writing and reviewing Python code. Rules are derived from real PR review patterns, organized by impact, and formatted for AI-agent consumption.
 
+**Python version baseline:** 3.11+ (some rules note higher-version features explicitly — e.g., `warnings.deprecated()` is 3.13+).
+
 ## Structure
 
-- `rules/` — Individual rule files (one per rule)
-  - `_sections.md` — Section metadata (titles, impacts, descriptions, prefixes)
-  - `_template.md` — Template for creating new rules
-  - `{prefix}-{name}.md` — Individual rule files
-- `SKILL.md` — Entrypoint loaded into agent context
-- `AGENTS.md` — Compiled document with all rules expanded
-- `metadata.json` — Version and abstract
+```
+python-best-practices/
+├── SKILL.md                # Entrypoint loaded into agent context (quick reference)
+├── README.md               # This file — human-facing overview and contribution notes
+├── metadata.json           # Version, abstract, references, Python version floor
+├── AGENTS.md               # (generated) Compiled document with every rule expanded
+├── test-cases.json         # (generated) LLM evaluation data extracted from rule examples
+├── rules/                  # Individual rule files (one rule per file)
+│   ├── _sections.md        # Section metadata (titles, impacts, descriptions, prefixes)
+│   ├── _template.md        # Template for new rules (with required `references` line)
+│   └── {prefix}-{name}.md  # Rule files; `prefix` matches a section in `_sections.md`
+└── src/                    # Build, validate, and extract-tests scripts
+    ├── build.py            # Compile rules into AGENTS.md
+    ├── validate.py         # Lint rule files (frontmatter, examples, references)
+    └── extract_tests.py    # Generate test-cases.json from rule examples
+```
 
 ## Sections
 
 ### 1. Data Modeling (CRITICAL) — `data-`
 
-Derive over store, discriminated unions, explicit variants, mutation contracts. The architectural foundation — mistakes here compound hardest.
+Derive over store, discriminated unions, explicit variants, mutation contracts, mutable defaults, sentinels, timezone-aware datetimes. The architectural foundation — mistakes here compound hardest.
 
 ### 2. Type Safety (CRITICAL) — `types-`
 
 No `Any` drift, precise annotations, proper narrowing. The type checker is load-bearing; keep it that way.
 
 ### 3. API Design (HIGH) — `api-`
 
-Keyword-only params, private underscores, immutable transforms. Interface decisions that compound over years.
+Keyword-only params, private underscores, immutable transforms, no boolean flag soup. Interface decisions that compound over years.
 
 ### 4. Error Handling (HIGH) — `error-`
 
-Specific exceptions, fail-fast validation, consolidated try/except. Sloppy exceptions hide bugs; good ones localize them.
+Specific exceptions, fail-fast validation, consolidated try/except, context managers for resources, exhaustiveness via `assert_never`. Sloppy exceptions hide bugs; good ones localize them.
 
 ### 5. Code Simplification (MEDIUM-HIGH) — `simplify-`
 
 Comprehensions, `any()`/`all()`, early returns, dead-code removal. Python idioms that reduce LOC and mental load.
 
 ### 6. Performance (MEDIUM) — `perf-`
 
-Module-level compilation, set/dict lookups, cached properties. Python-specific optimizations that matter on hot paths.
+Module-level compilation, set/dict lookups, cached properties. Python-specific optimizations applied where the hot path is measured.
 
 ### 7. Naming (MEDIUM) — `naming-`
 
 Specific names, consistent terminology, no type suffixes. Names are the most-read interface in any codebase.
 
 ### 8. Imports & Structure (LOW-MEDIUM) — `imports-`
 
-Top-of-file imports, optional dependency handling. Module hygiene.
+Top-of-file imports (with documented exceptions), optional dependency handling, no import-time side effects. Module hygiene.
 
-## Creating a New Rule
+## Authoring Workflow
 
 1. Copy `rules/_template.md` to `rules/{prefix}-{name}.md`
 2. Choose the appropriate prefix from `_sections.md`
-3. Fill in the frontmatter and content
-4. Ensure you have clear incorrect/correct examples with explanations
+3. Fill in the frontmatter (including a primary-source `references` line for any rule that depends on language version or library behavior)
+4. Write a short explanation, an Incorrect/Correct pair, and a closing note about edge cases
+5. Run the build / validate / extract-tests scripts (below)
+
+## Scripts
+
+The `src/` directory contains the maintenance pipeline:
+
+```bash
+# Compile rules into AGENTS.md
+python src/build.py
+
+# Lint rule files (frontmatter, references, example structure, broken links)
+python src/validate.py
+
+# Extract Incorrect/Correct example pairs into test-cases.json (for LLM evals)
+python src/extract_tests.py
+```
+
+A typical authoring loop is `validate.py` → fix → `build.py` → `extract_tests.py` before committing. `AGENTS.md` and `test-cases.json` are generated outputs — do not edit them by hand.
 
 ## Rule File Format
 
-Each rule file should follow this structure:
+Each rule file follows this structure:
 
 ```markdown
 ---
 title: Rule Title Here
 impact: MEDIUM
 impactDescription: brief phrase describing the payoff
-tags: tag1, tag2
+tags: tag1, tag2, applicability:pydantic   # `applicability:` for ecosystem-specific rules
+references: https://docs.python.org/3/library/...
 ---
 
 ## Rule Title Here
 
-Brief explanation of the rule and why it matters. One or two sentences.
+Brief explanation of the rule and why it matters. Name the impulse the agent is tempted to take.
 
-**Incorrect (why this is wrong):**
+**Incorrect (what's wrong with this):**
 
-\`\`\`python
+```python
 # Bad example
-\`\`\`
+```
 
-**Correct (why this is right):**
+**Correct (what's right about this):**
 
-\`\`\`python
+```python
 # Good example
-\`\`\`
+```
 
-Optional closing paragraph with nuance or references.
+Optional closing paragraph with nuance, edge cases, or version notes.
 ```
 
+### When `references` is required
+
+`references` is **required** when the rule depends on:
+
+- A specific Python version (3.10 union types in `isinstance`, 3.11 `assert_never`, 3.13 `warnings.deprecated`)
+- Standard-library behavior (`assert` under `-O`, `cached_property` thread safety)
+- Third-party library behavior (Pydantic, mypy, ruff)
+- A PEP
+
+Pure judgment-call rules (naming preferences, taste) may omit `references` but adding one is encouraged.
+
+### Tagging applicability
+
+Rules that only apply within a specific ecosystem (e.g., Pydantic) carry an `applicability:{name}` tag and call it out in the body. This lets future filtering/eval pipelines skip rules that don't apply to a given codebase.
+
 ## Impact Levels
 
 - `CRITICAL` — Highest priority; prevents classes of bugs or unmaintainable code
 - `HIGH` — Significant maintainability or correctness improvements
 - `MEDIUM-HIGH` — Noticeable improvements worth enforcing
 - `MEDIUM` — Good practices for cleaner, clearer code
 - `LOW-MEDIUM` — Marginal improvements
-- `LOW` — Incremental; apply opportunistically
+- `LOW` — Incremental; apply opportunistically (e.g., micro-optimizations on profiled hot paths only)
 
 ## Acknowledgments
 

skills/python-best-practices/SKILL.md

@@ -4,14 +4,15 @@ description: Python software engineering guidelines from real PR review patterns
 license: MIT
 metadata:
   author: python-best-practices
-  version: "1.0.0"
+  version: "1.1.0"
+  pythonVersion: ">=3.11"
 ---
 
 # Python Best Practices
 
-Comprehensive guidelines for Python codebases that must resist drift over time. Contains 50+ rules across 8 categories, prioritized by impact to guide automated refactoring and code generation.
+Comprehensive guidelines for Python codebases that must resist drift over time. 60+ rules across 8 categories, prioritized by impact to guide automated refactoring and code generation.
 
-The rules codify the failure modes agents fall into: reaching for `Any`, stacking optional fields into grab-bag models, catching bare `Exception`, and bypassing type checkers with `# type: ignore`. Each rule names the impulse, shows the failure, and points at the better path.
+The rules codify the failure modes agents fall into: reaching for `Any`, stacking optional fields into grab-bag models, catching bare `except:`, using mutable defaults, and bypassing type checkers with `# type: ignore`. Each rule names the impulse, shows the failure, and points at the better path.
 
 ## When to Apply
 
@@ -24,6 +25,17 @@ Reference these guidelines when:
 - Adding exception handling, validation, or error paths
 - Optimizing hot paths for performance
 
+## Python Version Baseline
+
+Rules assume **Python 3.11+** as the floor (for `assert_never`, `Self`, exception groups, `tomllib`). Rules that depend on a higher version call it out inline:
+
+- `warnings.deprecated()` — 3.13+
+- `zoneinfo` — 3.9+
+- Union types in `isinstance()` — 3.10+
+- `assert_never` — 3.11+ (use `typing_extensions` to backport)
+
+Rules tagged `applicability:pydantic` are Pydantic-specific.
+
 ## Rule Categories by Priority
 
 | Priority | Category               | Impact       | Prefix      |
@@ -46,7 +58,10 @@ Reference these guidelines when:
 - `data-explicit-variants` — Concrete classes per mode beat one class with `is_thread`/`is_edit`/`is_forward` flags
 - `data-phased-composition` — Group co-present optional fields into one nested optional, not eight siblings
 - `data-mutation-contract` — Mutate OR return; never both (callers can't tell which to use)
-- `data-encapsulate-mutable-state` — Trap mutable state in the smallest possible scope
+- `data-encapsulate-mutable-state` — Trap mutable state in the **narrowest clear scope** — closure, focused class, or instance attribute as the case demands
+- `data-mutable-defaults` — Never `def f(items=[])`; use `None` + body construction or `default_factory`
+- `data-sentinel-when-none-is-valid` — Use a private sentinel when `None` is itself a meaningful domain value
+- `data-aware-datetimes` — Timezone-aware `datetime.now(timezone.utc)` at every boundary; `datetime.utcnow()` is deprecated
 - `data-delete-dead-variants` — Remove union/enum branches that are never constructed
 - `data-newtype-for-ids` — Brand primitive IDs (`NewType('UserId', str)`) so they aren't interchangeable
 
@@ -67,23 +82,27 @@ Reference these guidelines when:
 
 - `api-required-before-optional` — Required fields before optional in dataclasses (Python enforces this)
 - `api-keyword-only-params` — `*` or `KW_ONLY` marker for optional/config params to prevent breakage
+- `api-no-boolean-flag-params` — `Literal`/`Enum` over positional `True, False` soup; split functions when bodies barely overlap
 - `api-underscore-for-private` — `_prefix` for internals; exclude from `__all__`
 - `api-immutable-transforms` — Return new collections; don't mutate inputs (unless named `update_*` / `*_inplace`)
-- `api-deprecated-aliases` — Old names stay as aliases when renaming public API
+- `api-deprecated-aliases` — `warnings.deprecated()` (3.13+) for renamed funcs/classes; compatibility kwargs + `warnings.warn` for renamed parameters
 - `api-no-private-access` — Don't reach into `_prefixed` names from outside the module
 - `api-model-cohesion` — Keep models flat; avoid duplicate/single-key-wrapped/redundant fields
-- `api-instance-vs-module-fn` — Instance methods for stateful behavior; module-level fns for pure utilities
+- `api-instance-vs-module-fn` — Pick the simplest namespace that matches ownership and polymorphism
 
 ### 4. Error Handling (HIGH)
 
-- `error-specific-exceptions` — Catch specific types; never bare `except Exception`
+- `error-no-bare-except` — `except:` catches `KeyboardInterrupt`/`SystemExit`/`CancelledError`; never use it
+- `error-specific-exceptions` — Catch specific types; `except Exception:` only at outer-loop log-and-reraise sites
+- `error-context-managers` — `with` / `async with` for files, locks, sessions, temp dirs; not manual `close()`
 - `error-consolidate-try-except` — Merge blocks that catch the same exception with similar handling
-- `error-assert-invariants` — `assert` for invariants that can't fail; not `RuntimeError('internal error')`
+- `error-assert-debug-only` — `assert` is stripped under `-O`; only use it for debug-only invariants
+- `error-assert-never-exhaustiveness` — `typing.assert_never` for exhaustiveness checks (3.11+)
 - `error-validate-at-boundaries` — Validate input before expensive work; fail fast at system edges
 - `error-inherit-base-exceptions` — New exceptions inherit from existing bases for backward compatibility
 - `error-repr-in-messages` — `f"tool {name!r}"` for identifiers in error text; consistent quoting
 - `error-raise-from-for-chains` — `raise NewErr(...) from original` to preserve causality
-- `error-trust-validated-state` — No defensive re-checks after earlier validation; trust the invariant
+- `error-trust-validated-state` — Trust validated, immutable, locally-constructed state in the same trust domain; keep checks for mutable/external/rehydrated objects
 - `error-preserve-cancellation` — `CancelledError` is `BaseException` on 3.8+; don't false-flag `except Exception:` for "swallowing cancellation"
 
 ### 5. Code Simplification (MEDIUM-HIGH)
@@ -93,20 +112,20 @@ Reference these guidelines when:
 - `simplify-fallback-or` — `x or default` over verbose `if`/`else` (when falsy values aren't semantic)
 - `simplify-inline-single-use-vars` — Drop `_filtered`, `_copy` intermediates that are used once
 - `simplify-flatten-nested-if` — Combine into `if cond1 and cond2:` when there's no intervening code
-- `simplify-cached-property` — `@cached_property` for expensive derived attributes
+- `simplify-cached-property` — `@cached_property` for derived attrs on **immutable** instances with `__dict__`; not thread-safe
 - `simplify-extract-after-duplication` — Extract helpers once a pattern repeats; don't copy-paste a third time
 - `simplify-remove-dead-code` — Delete commented-out code and unused definitions; git preserves history
 - `simplify-early-return` — Return early; don't nest the happy path three levels deep
 
 ### 6. Performance (MEDIUM)
 
 - `perf-compile-regex-module-level` — Compile static regex at module scope; not inside hot functions
-- `perf-type-adapter-constant` — Define `TypeAdapter` instances at module scope
+- `perf-type-adapter-constant` — Define Pydantic `TypeAdapter` instances at module scope *(applicability: pydantic)*
 - `perf-set-for-membership` — `set` for repeated `in` checks; O(1) beats `list.__contains__`
 - `perf-dict-index-over-nested-loops` — Build a `dict` for lookups; not nested `for` + `if`
-- `perf-generator-over-list` — Generators for streaming iteration; materialize only when needed
+- `perf-generator-over-list` — Stream with generators when memory or first-result latency matters; lists are fine when you re-iterate or need `len()`
 - `perf-lru-cache-pure-fns` — `functools.lru_cache` / `functools.cache` for pure functions
-- `perf-isinstance-tuple-syntax` — `isinstance(x, (A, B))` over `isinstance(x, A | B)` (tuple is faster)
+- `perf-isinstance-tuple-syntax` — Tuple form is marginally faster; **only rewrite on profiled hot paths**, not as a stylistic crusade
 - `perf-combine-iterations` — Fuse `filter` + `map` into one pass when possible
 
 ### 7. Naming (MEDIUM)
@@ -120,7 +139,8 @@ Reference these guidelines when:
 
 ### 8. Imports & Structure (LOW-MEDIUM)
 
-- `imports-top-of-file` — All imports at the top; not inline in function bodies
+- `imports-top-of-file` — Imports at the top by default; documented exceptions for circular imports, optional heavy deps, and side-effect deferral
+- `imports-no-side-effects` — Modules must be cheap to import — no network calls, model loads, env reads, or registrations at import time
 - `imports-optional-dependencies` — `try`/`except ImportError` with helpful install hints
 - `imports-remove-unused` — Delete unused imports; keep module namespace tight
 - `imports-no-duplicates` — One import per name
@@ -140,8 +160,19 @@ Each rule file contains:
 - Brief explanation of why it matters
 - Incorrect code example with explanation
 - Correct code example with explanation
-- Additional context, nuance, or references
+- Primary-source `references` (PEPs, stdlib docs, library docs) when version- or library-dependent
+- Closing notes on edge cases or applicability
+
+## Authoring & Maintenance
+
+The skill ships with a build/validate/extract-tests pipeline (see `README.md`):
+
+- `python src/build.py` — compile rules into `AGENTS.md`
+- `python src/validate.py` — lint frontmatter, references, and example structure
+- `python src/extract_tests.py` — generate `test-cases.json` for LLM evals
+
+Rule files live under `rules/`; `AGENTS.md` and `test-cases.json` are generated outputs.
 
 ## Full Compiled Document
 
-For the complete guide with every rule expanded: `AGENTS.md`
+For the complete guide with every rule expanded: `AGENTS.md`. Note that `AGENTS.md` is large by design (every rule body) — agents can grep individual rule files in `rules/` instead when only one or two rules are relevant.

skills/python-best-practices/metadata.json

@@ -1,14 +1,24 @@
 {
-  "version": "1.0.0",
+  "version": "1.1.0",
   "organization": "Python Best Practices",
   "date": "April 2026",
-  "abstract": "Comprehensive Python software engineering guidelines designed for AI agents. Contains 50+ rules across 8 categories, prioritized by impact from critical (data modeling, type safety) to low (import hygiene). Each rule names the failure mode agents tend toward, shows incorrect and correct code, and explains the payoff. Rules are derived from real PR review patterns and production experience.",
+  "pythonVersion": ">=3.11",
+  "abstract": "Comprehensive Python software engineering guidelines designed for AI agents. 60+ rules across 8 categories, prioritized by impact from critical (data modeling, type safety) to low (import hygiene). Each rule names the failure mode agents tend toward, shows incorrect and correct code, cites primary-source references where the rule depends on language or library behavior, and explains the payoff. Rules assume Python 3.11+ as a baseline; rules that depend on a higher version (e.g., 3.13 for warnings.deprecated) are tagged accordingly.",
   "references": [
     "https://docs.python.org/3/library/typing.html",
+    "https://docs.python.org/3/library/dataclasses.html",
+    "https://docs.python.org/3/library/exceptions.html",
+    "https://docs.python.org/3/reference/simple_stmts.html#the-assert-statement",
     "https://docs.pydantic.dev/",
     "https://mypy.readthedocs.io/",
+    "https://docs.astral.sh/ruff/",
+    "https://peps.python.org/pep-0008/",
     "https://peps.python.org/pep-0544/",
+    "https://peps.python.org/pep-0604/",
+    "https://peps.python.org/pep-0615/",
+    "https://peps.python.org/pep-0661/",
     "https://peps.python.org/pep-0695/",
+    "https://peps.python.org/pep-0702/",
     "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/pydantic/pydantic-ai"
   ]
 }