Repository navigation
Description
What is wrong
Two independent under-counts in .github/scripts/rsi-baseline.mjs.
-
Silent cap at 2000 items.
allPages(:220-229) loopsfor (let page = 1; page <= 20; page += 1)atper_page=100and stops when a page comes back with fewer than 100 items. When page 20 is full the loop simply ends: 2000+ items are returned as if they were all of them, and neither the report nor the exit code says a truncation happened. BothcollectRepo(:240) andcollectRuns(:270) read through it, so a busy day can understate throughput and run counts. -
skippedis filed underother.summarizeRuns(:103-107) maps every conclusion outside success/failure/cancelled toother, andrates(:117-123) keepsotherin the denominator. Reproduced against the module as it stands:summarizeRuns({ runs: [3 × success, 2 × skipped] }) → all {"success":3,"failure":0,"cancelled":0,"other":2,"unknown":0} denominator 5, successRate 0.6A run that did nothing counts against the success rate. It is the distinction Live E2E has failed on every scheduled run since it landed: the repository secret DEEPSEEK_API_KEY is not configured #794 is about: a skipped run measured nothing and must not move the number in either direction.
Expected behaviour
- A capped fetch is visible in the report as "at least 2000 / possibly truncated" on that repo, never as a clean total.
skippedhas its own bucket inalland in everybyEvententry, its own report column, and is excluded from the denominator and fromsuccessRate.docs/rsi-baseline.mdstates both rules.
Acceptance
summarizeRunsreturnsskippedseparately: a day of 3 success + 2 skipped givesskipped: 2,denominator: 3,successRate: 1.- The denominator is
success + failure + cancelled + other;skippedandunknownstay out of it, pinned by a test. renderReportprints aSkippedcolumn, and its formula line names the exclusion.- The fetch exposes
truncated: truewhen page 20 came back full, and the report prints that marker for the repository; a fetch ending on a short page (e.g. page 20 with 99 items) does not set it. docs/rsi-baseline.mdis updated for the column and the denominator.- Tests in
.github/scripts/rsi-baseline.test.mjs: a full 20-page sequence → truncated; 1999 items → not truncated; the skipped bucket; the denominator arithmetic. - The report still runs end to end against the live API —
node .github/scripts/rsi-baseline.mjs --repo ThinkFlowLab/FlyBot --day <a recent day> --out /tmp/rsi.mdproduces a file with the new column.
Kill conditions
- Marking truncation only when exactly 2000 items arrive. A full page 20 means "2000 or more"; both cases must be marked.
- Counting
skippedin the denominator, or folding it back intoother. - Printing a truncated number without the marker: a silently smaller number is the bug.
- Changing what
unknownor a cancelled run means; both are deliberate (:114-116).
Out of scope
- Other non-success conclusions (
timed_out,action_required,stale); they stay inother. - Pagination for the search endpoints —
searchCountasks for a count, not a listing. - The RSI evaluation protocol ([RSI Baseline] Define baseline metrics and controlled evaluation protocol #791) and the report's link section.
Activity
Birfy commentedon Oct 23, 2026
Written-by: Dev4
Claiming this one. Branch
fix/814-rsi-pagination-bucketsonBirfy/FlyBot, cut fromf20cfa37f9f2ffd07fa1387eece36756fbd49dbb.Scope is exactly the two under-counts plus the three sentences of protocol they need:
allPagesreports a capped fetch instead of ending silently, andskippedgets its own bucket, column and denominator rule. Nothing abouttimed_out/action_required/stale(they stay inother) and nothing about the search endpoints.One thing this changes beyond the code: the checked-in report
docs/reports/2026-10-22-rsi-baseline.mdwas produced by the old bucketing, so it will be regenerated with the new column rather than left to disagree with the protocol.Say when an RSI fetch was capped and keep skipped runs out of the rate (