Skip to content

docs: preregister engineering pilots and evaluation rubric - #413

Merged
TheAmericanMaker merged 1 commit into
mainfrom
docs/engineering-pilot-evaluation
Sep 17, 2026
Merged

TheAmericanMaker merged 1 commit into
mainfrom
docs/engineering-pilot-evaluation

Conversation

@TheAmericanMaker

Copy link
Copy Markdown
Member

Summary

  • Preregister a real test-isolation bug pilot (Tests: the git-config guard misses config injected through GIT_CONFIG_COUNT #412) and a bounded capability-filter feature pilot for E11, with exact target revision, preserved behavior, allowed scope, and observable acceptance checks.
  • Add an evaluation rubric covering correctness, evidence identity, independent review, actual human acceptance, interruption/recovery, negative controls, second-change continuity, effort, and overhead.
  • Clarify that dependency eligibility does not override maintainer scheduling holds. E09 remains on hold; E01 ownership and all E11 prerequisites are unchanged.

Scope and evidence

Documentation only. Neither pilot has been implemented, no new feature assignment was created, and this PR does not authorize execution or provider spend.

Selection checks on pinned target ee091ac680eef4824ad378b4aa958829a21d89cd:

  • Build and full suite: 948/948 passed.
  • Library test control: 72/72 passed.
  • Injected Git configuration reproduction: 71 passed, 1 named failure, process exit 1 as expected.
  • Synthetic capability metadata is stored, but current core/public schema lacks the proposed capability filter. This is feature-absence evidence, not a regression claim.

The known-answer bug is explicitly not a diagnostic benchmark. Historical target and future engine revisions are distinct. The application test-data library is not a required synthesis-reference library.

Verification and publication safety

  • Independent read-only review: APPROVE for the staged documentation and retained selection evidence.
  • Incorporated the nonblocking suggestion to explicitly cover MCP empty-string and non-string filter inputs in the acceptance checklist; these behaviors were already specified in the reviewed contract.
  • Build and full test suite passed; staged diff, relative links, and sensitive-content checks passed.
  • Only three Markdown files. No raw machine logs, private paths/configuration, credentials, or session transcripts included.

Refs #409
Refs #398

No version bump, release, deployment, E09 implementation, or E01 contract change.

Plan E11 without implementing pilot changes or starting held E09 work. Refs #409 and #398.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant