Repository navigation
fix(ledger): exclude scope boundaries from analyzer work - #802
Conversation
Signed-off-by: Efe <efe@rexcode.co.uk>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
@efegokdemir thank you for tracing #800 to the one summarizer every analyzer shares, and for a regression that fails on the old accounting.
Value and readiness
Real report-integrity defect. Static analyzers emit out_of_scope SCOPE_BOUNDARY rows for unreferenced binary members. analyzer_status_for_events counted those rows as planned work, and finalize_ledger maps the unknown out_of_scope outcome to failed. Reports therefore said completed with failed: 8, which SkillEvaluator rejects (102 of 105 skills in #800). Excluding scope boundaries at the summarizer (inspection_ledger.py:529-532) fixes this at the source, and the exclusions still appear in scope_exclusions. Ready for final maintainer review.
Material findings
None blocking.
- Optional: a CLI-level regression using #800's shape (a DOCX member with embedded fonts) would pin the end-to-end JSON. The new
finalize_ledgertest already covers the accounting.
PIC tradeoffs
Output shift, not a defect. planned_work for static analyzers drops by the number of unreferenced binary members. An analyzer whose only rows are scope boundaries now reports not_applicable (no_applicable_files) instead of completed. Consumers that compare counts across versions will see the change.
Verification and gaps
- The only analyzer-tagged SCOPE_BOUNDARY producer is
static_runner.py:2530-2540, for unreferenced BINARY content. Referenced binaries stay WORK_ITEM PARTIAL, so real gaps still count as planned work.build_context's discovery scope rows carry no analyzer and are not summarized. - I traced
finalize_ledger. Path coverage already ignored OUT_OF_SCOPE outcomes, and out-of-scope inventory is excluded from relevant components unless referenced, so removing these targets fromprimary_targetsdoes not change the fully/partially/entirely-inspected counts. If every analyzer's rows were scope boundaries, the fallback branch only touches components whose inventory is already out of scope, so no gap can become complete. static_yaraandsemantic_security_discoverybuildplanned_workby hand but never emit scope boundaries.guard_analyzer_nodeis unaffected.- Tests: the new finalize test fails on the old code (planned 2, one counted failed), and the unit test pins
not_applicable. - Main (
b07f181d) is 49 commits ahead; main's new RULE_SET rows sit nearby but do not overlap, andgit merge-treeis clean. - CI: all 6 checks are green on
28ecaf3. I did not run PR code, per policy.
What
scope_exclusions.analyzer_statusesreportscompletedwhile counting out-of-scope binary members asfailed, so SkillEvaluator rejects the report #800.Why
Binary members that are intentionally out of scope were included in each analyzer's planned work. Finalization then counted the unrecognized
out_of_scopeoutcome as failed work even though the analyzer status wascompleted, producing an inconsistent report that SkillEvaluator rejects.How
analyzer_status_for_eventsnow excludesscope_boundaryrecords from the analyzer work plan. If an analyzer has only scope boundaries, its status isnot_applicable; the exclusions are still reported separately.Testing
make test-unit: 10,539 passed, 14 skipped, 133 deselected, 4 xfailed.make lint,make format-check, andpython -m build --no-isolationpassed.call-overloaderror atinspection_ledger.py:1081; the same error is present on the base commit. The changed module passes when that existing diagnostic is suppressed and imports are silenced.AI assistance
ChatGPT assisted with the implementation and regression tests. The commit is DCO signed off.