Skip to content

feat(scripts): delete chats that carry Russian, after archiving them (CLEAN-169) - #143

Merged
maksymhryzodub-prog merged 1 commit into
mainfrom
feat/CLEAN-169-purge-russian-chats
Oct 19, 2026
Merged

maksymhryzodub-prog merged 1 commit into
mainfrom
feat/CLEAN-169-purge-russian-chats

Conversation

@maksymhryzodub-prog

@maksymhryzodub-prog maksymhryzodub-prog commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

CLEAN-169

The customer-facing analytics dashboard shows Russian text. This PR removes the chats themselves. The rollup cleanup and dropping Rancher from the dashboard are the other half, and they can only live in the customer fork — the analytics slice does not exist here.

Why not insights.language

That looked like the obvious key and it is the wrong one.

  • ChatInsightService only summarizes chats with at least three user messages (gate.minUserMessages: 3), so a two-message "привет" has no insights at all. Most of the offending chats are two messages long.
  • The parser keeps whatever string the model returned and defaults to en (chatInsight.service.ts:273). It is unvalidated LLM output.

A rule that decides what gets deleted has to be a rule. This one is: Cyrillic in the transcript, /[\u0400-\u04FF]/.

Cyrillic in the bot reply counts too — the chat list preview renders the last message whatever its role, so a Russian answer to an English question is just as visible to the customer. The dry run prints per-chat counts split by side, so the borderline case (one Russian word in an English conversation) is visible before anything is deleted.

Cyrillic is wider than Russian; Ukrainian and Bulgarian would match. Accepted for this installation, and detect.ts is the one place to narrow it.

Safety

  • The dry run is the default, the inverse of the other scripts. --delete is required to act, and --delete without --archive is refused: the archive is the only thing that makes this reversible.
  • The archive is one JSONL line per chat (index row + full transcript) written inside the pod, copied off with kubectl cp. No copy stays on prod.
  • Per chat: archive, then the ChatSession row, then the S3 object. This is the reverse of the CLEAN-168 copy and deliberately so — a row pointing at a missing transcript opens as an empty chat, while an object with no row is invisible. A failed object delete is reported by key for a manual sweep.
  • ChatFeedback follows the row by onDelete: Cascade. Lead is untouched: it is installation-wide and the same person may also have English chats.
  • A session whose object is gone cannot be judged, so it is reported as unreadable and left alone rather than guessed about.

Runbook

kubectl -n <ns> exec <pod> -- bun dist/scripts/purgeRussianChats.js
# review the report, then:
kubectl -n <ns> exec <pod> -- bun dist/scripts/purgeRussianChats.js \
  --delete --archive /tmp/purged-ru.jsonl
kubectl -n <ns> cp <pod>:/tmp/purged-ru.jsonl ./purged-ru.jsonl

Twin consoles

Nothing to check: this touches api/src/scripts/ only, no slice in admin/ or app/.

Gates

  • bunx jest src/scripts — 18 pass, 2 suites (written first, watched fail, then implemented)
  • bunx jest — 2154 pass, 1 fail: slices/bridle/domain/documentText.extractor.spec.ts:363 (PDF text extraction). Pre-existing on origin/main — verified by stashing this branch and re-running that spec, which fails identically. Not touched by this change.
  • bun run build — clean (needs prisma-import --force + prisma generate first after a branch switch, or the stale client reports 133 phantom errors)
  • bunx eslint src/scripts — 0 real findings; the 508 errors are all Delete ␍ CRLF noise, the known repo-wide line-ending issue
  • Compiled CLI refusals smoke-tested: --archive <path> is required with --delete and unknown flag --nope, both before any DB connection

🤖 Generated with Claude Code

https://claude.ai/code/session_011B9tuWEQvYg5YQY9SQZN2u

…(CLEAN-169)

The customer-facing dashboard must show no Russian. This removes the chats
themselves; the rollup cleanup and the Rancher exclusion live in the fork,
where the analytics slice is.

Selection is Cyrillic in the transcript, not ChatSession.insights.language.
That field is written by the summarizer, which only runs on chats with three
or more user messages, so a two-message "привет" never gets one — and the
parser keeps whatever string the model returned, defaulting to 'en'. A rule
that decides what gets deleted has to be a rule, not a judgment.

Cyrillic in the bot reply counts too: the chat list preview renders the last
message whatever its role, so a Russian answer to an English question is
just as visible to the customer.

The dry run is the default here, the inverse of the other scripts, and
--delete without --archive is refused: the archive is the only thing that
makes this reversible. It is written inside the pod and copied off with
kubectl cp, so no copy is left behind on prod.

Per chat the row goes first and the S3 object second — the reverse of the
CLEAN-168 copy. A row pointing at a missing transcript opens as an empty
chat; an object with no row is invisible, and failures are reported by key
for a manual sweep.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011B9tuWEQvYg5YQY9SQZN2u
@maksymhryzodub-prog
maksymhryzodub-prog merged commit cde1ca1 into main Oct 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant