fix(i18n): register every backend error in the wordlist, and stop the comment-stripper from mangling UTF-8 - #249
Merged
Conversation
… comment-stripper from mangling UTF-8
`ui/js/api.js` hands the backend's `error` string to `I18n.mapErr()`. `mapErr`
translates only the Chinese literals hand-listed in `ERR_MAP` (`ui/js/i18n.js`)
and returns everything else **verbatim** — so every backend message nobody
remembered to register reaches an `lang=en` user in Chinese, and `cargo test`
stays green (the three existing i18n gates cannot see `mapErr("中文")`).
Measured with jsdom on the real `ui/index.html` + the four real scripts (only
`fetch` stubbed): 20 of the backend's 45 user-visible Chinese messages came back
in Chinese under `lang=en`. One of them — `部门下还有 N 名成员，请先调整成员部门`
(`org.rs` DELETE → 409) — is reachable from a real control, so an English user
actually read Chinese on screen.
This turns the hand-written roster into a gate and closes the 20 gaps.
While building the gate, its extractor reported **45** unmatched messages while
an independent scanner reported **0**. The divergence was the real story:
`strip_js_comments` copied bytes with `out.push(b[i] as char)`, so every UTF-8
byte became its own code point. The Chinese side of `ERR_MAP` was mangled, the
corpus (read raw) was not, so nothing could ever match. Every existing consumer
asks only `is_ascii()` — and mojibake is non-ASCII too — so the corruption passed
every old gate for three rounds. Acting on that false red would have meant adding
45 entries that can never match. Fixed at the root (byte-accurate copy +
`String::from_utf8`), with a fidelity control that the old implementation fails.
Merged
12 tasks
argszero
added a commit
that referenced
this pull request
Sep 14, 2026
…response data fields (#252) * fix(i18n): stop the backend from inventing Chinese display labels in response data fields The client localizes the `error` field (`api.js` hands it to `I18n.mapErr()`, wordlist in `ui/js/i18n.js` — see #249). Response **data** fields have no such path: the UI renders them raw. So a backend that invents a Chinese label and puts it in a data field prints Chinese on the `en` interface, and `cargo test` stays green. Two reachable cases: 1. `GET /api/admin/usage` — the department bucket was `COALESCE(d.name, '（未分配）')` and `app.js` prints it through `barRow()` into `#usage-dept`. Any user with `dept_id IS NULL` and usage this month puts Chinese on an English screen — while the member table on the *same page* has used `T("common.unassigned")` all along. 2. `GET /api/plans` — when config leaves `name` unset the backend derived a display name from `type` (`API（按量）` / `Token Plan` / `Coding Plan`). Neither `config.toml` nor `config.example.toml` sets `name` for any `[[plans]]`, so it always fired, into `#sf-plan` and the share toast. Fix — the backend returns data or a language-neutral marker, the client owns the wording: - the no-department bucket is `''` (same shape as the neighbouring `users[].dept_name`), and the client renders `d.name || T("common.unassigned")` — an existing key, zero new ones; - `/api/plans` returns `p.name` verbatim (empty when unset), and the client gets `planLabel(pl)`: config's own name when present, else a new key per type (`share.planName.paygo|token|coding`), reused by both render sites. Three CI contracts, one per direction: - `i18n_pack::backend_data_fields_are_language_neutral` — scans every `src/**/*.rs` and extracts Chinese literals delivered through a `json!` data field (key ≠ `error`), including through a local `let` binding (the plan-name fallback was written that way; without that layer the gate is green on the pre-change tree). The result must be *exactly* the adjudicated exemption list — a new invented label turns it red, and a stale exemption turns it red too. Its scope is stated in the doc comment: values manufactured by SQL (`COALESCE(..., '中文')`) are covered by the runtime contract below instead. - `i18n_pack::backend_neutral_data_labels_are_localized_in_the_client` — both render sites must have the localized fallback, otherwise a neutral backend just yields a blank label. - `routes::tests::usage_department_bucket_without_a_department_is_language_neutral` — the runtime contract: with a `dept_id IS NULL` user who has usage this month, the bucket name carries no CJK, and a real department name still passes through verbatim. The one exemption (`routes::mod.rs` registration's `unwrap_or("用户")`) is a *user-data* default — the same kind of value as an account name — and is unreachable (`split('@').next()` is always `Some`); it is not an invented display label. * test(i18n): give the data-label exemption roster a deliberate budget * fix(i18n): stop the backend from inventing Chinese display labels in response data fields The client localizes the `error` field (`api.js` hands it to `I18n.mapErr()`, wordlist in `ui/js/i18n.js` — see #249). Response **data** fields have no such path: the UI renders them raw. So a backend that invents a Chinese label and puts it in a data field prints Chinese on the `en` interface, and `cargo test` stays green. Two reachable cases: 1. `GET /api/admin/usage` — the department bucket was `COALESCE(d.name, '（未分配）')` and `app.js` prints it through `barRow()` into `#usage-dept`. Any user with `dept_id IS NULL` and usage this month puts Chinese on an English screen — while the member table on the *same page* has used `T("common.unassigned")` all along. 2. `GET /api/plans` — when config leaves `name` unset the backend derived a display name from `type` (`API（按量）` / `Token Plan` / `Coding Plan`). Neither `config.toml` nor `config.example.toml` sets `name` for any of the `[[plans]]`, so it always fired, into `#sf-plan` and the share toast. Fix — the backend returns data or a language-neutral marker, the client owns the wording: - the no-department bucket is `''` (same shape as the neighbouring `users[].dept_name`), and the client renders `d.name || T("common.unassigned")` — an existing key, zero new ones; - `/api/plans` returns `p.name` verbatim (empty when unset), and the client gets `planLabel(pl)`: config's own name when present, else a new key per type (`share.planName.paygo|token|coding`), reused by both render sites; - the plan dropdown is now rebuilt when its **data source** changes (`dataset.plansSrc`) instead of once (`dataset.init`). The view renders before `loadSharing()` resolves, so the fallback table `D.PLANS` always built the dropdown first and the one-shot guard then kept it forever — the backend fix was invisible on that control. The cascading fills read the source at call time, so the once-registered listeners cannot pin the old table either. Three CI contracts, one per direction: - `i18n_pack::backend_data_fields_are_language_neutral` — scans every `src/**/*.rs` and extracts Chinese literals delivered through a `json!` data field (key ≠ `error`), including through a local `let` binding (the plan-name fallback was written that way; without that layer the gate is green on the pre-change tree). The result must be *exactly* the adjudicated exemption list — a new invented label turns it red, a stale exemption turns it red too, and the list has a budget of one, because it is a sunset list rather than a registry (the tempting "keep inventing, register the label" fix has to edit a second place to pass). Scope is stated in its doc comment: values manufactured by SQL (`COALESCE(..., '中文')`) are covered by the runtime contract instead. - `i18n_pack::backend_neutral_data_labels_are_localized_in_the_client` — both render sites must have the localized fallback (otherwise a neutral backend just yields a blank label), and the dropdown must be rebuilt from source evidence rather than a one-shot flag. - `routes::tests::usage_department_bucket_without_a_department_is_language_neutral` — the runtime contract: with a `dept_id IS NULL` user who has usage this month, the bucket name carries no CJK, and a real department name still passes through verbatim. The one exemption (`routes::mod.rs` registration's `unwrap_or("用户")`) is a *user-data* default — the same kind of value as an account name — and is unreachable (`split('@').next()` is always `Some`); it is not an invented display label. A/B — the JS probe (`tmp/c2133_probe.js`; jsdom, real `index.html` + 4 real scripts, only `fetch` stubbed, `atp_lang=en` forced, every check compared against its **expectation** so the pre-change leg's demonstration checks are not confused with acceptance failures): | leg | mismatched checks | |-----|-------------------| | working tree | `{}` — 17/17 | | pre-change tree (client + payloads from `origin/main`) | `{B2b,D1,D2,E2,F1,F2}` | | render sites read `pl.name` raw (backend neutral, no client label) | `{A2b,D2}` | | dropdown keeps the one-shot guard (no rebuild) | `{F1,F2}` | | dropdown reads a source snapshot taken at first render | `{B2b,D2}` | | no-department bucket loses its localized fallback | `{D1,E2}` | Each leg is generated by string substitution from the working tree, so the red sets are reproducible; `F1`/`F2` (the provider list must follow the live plan source) are what make the rebuild half observable at all — without them that leg reads green, which is how the one-shot guard survived the first pass. Rust side, `cargo test` per leg (mutated in place, restored against `HEAD` with md5 + `git diff --stat` + `diff` checks) — see the PR body for the recorded sets. Same axis, deliberately not bundled (recorded in `ui/README.md`): `ui/js/data.js` carries its own `PLANS[].name` display labels (`API（按量）`, `Kimi Code 会员`…), which the client shows when `/api/plans` fails. Different producer (a client data table, not a response field) and it needs a brand-vs-generic ruling first — `provLabel()` right next to it is the pattern to copy (`I18n.lang === "zh"` gates the label table; `en` gets the id).
argszero
added a commit
that referenced
this pull request
Sep 15, 2026
Ships the 18 PRs merged since v0.7.24 (#242-#259). Schema 14 -> 15 (two covering indexes, applied at startup). No config change, so no deployment-side config.toml edit is needed. Two themes: Perf on the NFS dev database - #259: stop mapping the db (PRAGMA mmap_size 64MB -> 0) and stop a real write per request (dao::touch_api_key gains a 60s guard). Measured on the live dev db: mmap=64MB 1.7-3.1s per COUNT / 250 MiB read vs mmap=0 ~10.5ms / 80 KiB; mmap=0 alone still leaves ~1.2s behind any write, so the pair is required. - #242: codify the two emergency indexes in a v15 migration and gate the conditional joins at the plan level. - #243: read the sharing page's earn total from one batched aggregate. Frontend: display must equal what it filters on, and one fact, one source - #250 one writer for the transaction cache; #251 clear every session slot at the identity boundary and give the wallet view a loader; #253 one shared writer for the wallet/dashboard month-changes; #254 boot loads only the destination view; #255 a model row's identity is the model, not its index; #256 the marketplace source follows the session, not whether data arrived; #257 the sidebar advertises only digits that work; #258 the admin total-balance card sums the gift amount its caption names. i18n - #249 every backend error reaches the wordlist, and the comment stripper stops mangling UTF-8; #252 the backend stops inventing Chinese display labels in response data fields. Forms and robustness - #244 a non-auth boot failure no longer looks like being logged out; #245 a credential 401 is no longer read as a session expiry; #246 wire timestamps reach the renderer unsliced; #247 inline cards submit from every field; #248 a market row's availability label comes from that row. - Cargo.toml / Cargo.lock: 0.7.24 -> 0.7.25. - CHANGELOG.md: v0.7.25 entry. - ui/index.html: cache-bust left as-is; the UI PRs in this release already advanced it past the value deployed with v0.7.24 (app.js 20260915-13, i18n.js 20260915-3). cargo test 288 passed; cargo fmt --check clean; clippy unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ui/js/api.jshands the backend'serrorstring toI18n.mapErr().mapErrtranslates only the Chinese literals hand-listed inERR_MAP(ui/js/i18n.js:1684) and returns everything else verbatim — so every backend message nobody remembered to register reaches anlang=enuser in Chinese, whilecargo teststays green. The three existing i18n gates structurally cannot see this: the key-existence gate sees both packs have the keys, the usage gate only recognisesT("literal"), and the placeholder gate never opensapi.js. The roster is hand-written; the backend is not.Measured in jsdom with the real
ui/index.html+ the four real scripts (onlyfetchstubbed), underlang=en:删除部门on a department that still has members →org.rsDELETE → 409部门下还有 3 名成员，请先调整成员部门. There is no client-side guard for "department not empty", so an English user actually read that Chinese sentence on screen.Positive control on the same code path: the same button answering 404
部门不存在(a message that is registered) renders the Englisherr.notFoundtext — so the path renders mapped text and a red axis leg is not "the toast never fired".Related Issue
Changes
ui/js/i18n.js— 20 newERR_MAPentries derived verbatim from the backend sources, plus the 20 matching keys in both packs (zh/en). Messages carrying runtime interpolation register only the stable prefix before the placeholder (部门下还有,部门「,流式协议转换):mapErris a substring match, so the rendered message still hits, whereas the full template can never match.src/i18n_pack.rs— new gateevery_backend_error_message_reaches_the_wordlist: the corpus is the backend's own"error"literals (both writing shapes,#[cfg(test)]truncated), and the invariant is "every Chinese message must be matched byERR_MAP", usingmapErr's own longest-match rule. Two positive controls (entry count + CJK message count) separate "the extractor silently returned empty" from "the corpus really changed".src/i18n_pack.rs— the roster is derived, not hand-written:backend_error_sources_cover_every_file_that_emits_an_error_literalscanssrc/*.rs+src/routes/*.rsand requires the set of files that emit an error literal to be exactlyBACKEND_ERROR_SOURCES(a missing emitter = the gate silently skips it; a stale entry = the roster is rotting).src/i18n_pack.rs— root-cause fix:strip_js_commentscopied bytes without.push(b[i] as char), so every UTF-8 byte became its own code point and any non-ASCII literal was mangled. The Chinese side ofERR_MAPwas corrupted while the corpus (read raw) was not, so nothing could ever match. Every existing consumer asks onlyis_ascii()— and mojibake is non-ASCII too — so it passed all three old gates; the corruption had survived three rounds unnoticed. Now byte-accurate (Vec<u8>+String::from_utf8), with a fidelity control that the old implementation fails (the previous controls were allis_ascii()-based, which is exactly why the distortion had no witness).ui/index.htmlcache-busti18n.js;ui/README.md— the i18n convention now states the rule (backend writes a Chinese error ⇒ it must be inERR_MAP), the stale key count (787 → 806) and the stale "15 组" count are corrected.m_minleg below).Tests
cargo test— 265 passed / 0 failed (was 262; +3 new gate tests)cargo fmt --check— cleancargo clippy— only the pre-existing warning atsrc/protocol.rs:662A/B evidence (
tmp/c2129_*; all UI mutations injected fromtmp/throughC2129_I18N— the work tree is never rewritten):v0_orig(pinned818ed88i18n.js){A4, B1, B2}m_min(register only the reachable message){A4}m_e2e(full fix minus the reachable entry){A4, B1, B2}live{}In-place gate legs (byte-identical restore verified by md5):
src/routes/byte as charChecklist
fix/）