Skip to content

Group tests into unit, system, SIT, performance, accessibility and UAT - #41

Draft
amar-python wants to merge 4 commits into
mainfrom
claude/elegant-bohr-zn28zn
Draft

amar-python wants to merge 4 commits into
mainfrom
claude/elegant-bohr-zn28zn

Conversation

@amar-python

@amar-python amar-python commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

Reorganises the test suite into six category folders and adds new test cases to each. It also fixes the defects the new tests found: accessibility issues in the frontend, plus two bugs tracked as #42 (preview parse_failed) and #43 (CORS port). Finally, it fixes the CI database setting that has kept the integration (postgres service) job red on main.

Fixes #42. Fixes #43. Both are already closed, with the test evidence in their comments.

Group Folder Moved in New
Unit tests/unit/, csv-table-hub-main/src/__tests__/unit/ 9 pytest files + 6 vitest files test_csv_parse.py, test_config_auth.py, test_db_pool.py, preview-contract.test.ts
System tests/system/ api_coverage, e2e_pipeline, arbitrary_shapes test_system_api.py: full upload→list→rows→delete→audit lifecycle, typing, duplicates, overwrite, limits, T&E mode
System integration tests/system_integration/ api_integration, te_loader_integration, parity, environments/ test_sit_frontend_contract.py: live API responses checked against the zod enums parsed from csv.functions.ts, plus API↔PostgreSQL bootstrap and schema checks
Performance tests/performance/ — 100k-row parse, 20k-row upload, 8 concurrent uploads, p95 read latency, constant query count. PERF_BUDGET_SCALE scales the budgets
Accessibility tests/accessibility/ — Playwright + axe-core (WCAG 2.1 A/AA) against the Vite dev server with a mocked API, plus keyboard, focus-trap, heading and 320px-reflow checks
UAT tests/uat/ issue #4 / #5 acceptance tests test_uat_user_stories.py: 7 user stories as Given/When/Then, using build/csv/samples
  • tests/conftest.py gives every test the marker for its folder (system, sit, performance, accessibility, uat, plus the existing unit). There are new shared fixtures: live_api, registry_cleanup, te_org_cleanup, destructive_enabled.
  • docs/TEST_CASES.md is generated by scripts/test_catalogue.py. It lists every test case with ID, file and objective, grouped by category. CI runs --check to keep it current.
  • There are new Makefile targets test-unit, test-system, test-sit, test-performance, test-accessibility and test-uat. tests/README.md explains each group.
  • The psql-driven SQL suite (tests/suites, tests/framework, run_all_tests.sql) stays where it is, because the Tier S eval and the deploy scripts reference those paths. The catalogue lists it under system integration.

Accessibility fixes (found by the new tests)

  • Accessible names added to the unlabeled Select triggers (Rows per page, column type, T&E table, status filter), to the icon-only download menu on the audit page, and to the upload progress bar.
  • Contrast fixes: inline <code> chips now use text-foreground; the "ok" count and the error pill now use darker emerald-700 / red-700 text.
  • Sonner toast text (light theme, richColors) is darkened in styles.css. All four types were below AA: success 4.29, info 4.35, warning 3.07, error 4.36:1. They are now 5.23 / 6.46 / 6.72 / 5.75:1.
  • The test harness now waits for CSS animations to finish before each axe scan. One CI run failed because the dialog was scanned mid-fade-in. Reproduced by freezing the fade at 50ms: 6 color-contrast nodes, versus none once it finishes.

API ↔ frontend fixes

CI

  • Pre-existing failure fixed: the PostgreSQL test step now uses PGDATABASE=te_mgmt_dev. te_dev is deployed there, so every T&E-mode test was failing on main with relation te_dev.organisations does not exist.
  • That job now runs --markers "not accessibility". A new accessibility job runs vitest and the Playwright + axe suite.
  • bun.lock is synced. It was missing jsdom and vitest, so bun install --frozen-lockfile failed. package-lock.json is out of sync too; this PR does not touch it.

Test plan

🤖 Generated with Claude Code

https://claude.ai/code/session_012CS4GeHuWThWvU74AkBL6E

…lity and UAT

Reorganise the pytest suite into one folder per test category and add test
cases for each:

- tests/unit (+ frontend src/__tests__/unit): moved existing DB-free tests;
  new coverage for csv_parse, config/auth, the connection pool, app error
  handlers and loader guard paths
- tests/system: moved API/e2e tests; new full-lifecycle API tests (typing,
  duplicates, overwrite, limits, audit log, T&E mode)
- tests/system_integration: moved integration/parity/env-contract tests;
  new API<->frontend contract tests checked against the zod enums in
  csv.functions.ts, plus API<->PostgreSQL bootstrap/schema tests
- tests/performance: parser throughput, 20k-row upload, concurrent uploads,
  p95 read latency, constant query count (PERF_BUDGET_SCALE scales budgets)
- tests/accessibility: WCAG 2.1 AA via Playwright + axe-core against the Vite
  dev server with a mocked API, plus keyboard, focus-trap and reflow checks
- tests/uat: moved issue #4/#5 acceptance tests; new Given/When/Then user
  stories using the shipped sample CSVs

conftest.py tags every test with its folder's marker; new Makefile targets
test-unit/system/sit/performance/accessibility/uat. docs/TEST_CASES.md is
generated by scripts/test_catalogue.py and CI checks it is current.

Fix the accessibility defects the new tests found: accessible names on the
Select triggers, the audit-page download menu button and the upload progress
bar; darker text for inline code chips and the ok/error status pills.

CI: run the PostgreSQL test step against te_mgmt_dev (where te_dev is
deployed; T&E-mode tests failed on main against the postgres DB), add an
accessibility job (vitest + Playwright/axe), and sync bun.lock, which was
missing jsdom and vitest so --frozen-lockfile failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CS4GeHuWThWvU74AkBL6E
…t in CORS

- csvPreviewResultSchema now accepts reason "parse_failed" (plus its
  message), which /api/csv/preview returns when the parser throws (BUG-025
  handler). Previously the zod parse failed and the upload flow silently
  dropped the T&E table suggestion.
- Default CORS_ORIGINS now includes http://localhost:8080, the port the
  Vite dev server actually listens on; 5173/3000 are kept. Docs,
  env.dev.example and start-frontend.ps1 updated to match.
- Tests: SIT-17/18 check every preview invalid_structure reason the API can
  return (including a simulated parser crash) is in the UI's preview enum;
  unit tests check the default origins and a preflight from :8080.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CS4GeHuWThWvU74AkBL6E
…8080)

- New vitest suite src/__tests__/unit/preview-contract.test.ts drives the
  real previewCsv() with a mocked fetch: parse_failed (+ message) is
  accepted, the three original reasons still are, and an unknown reason is
  still rejected.
- New unit test: an unlisted origin is still refused by CORS, so adding
  :8080 did not loosen the policy.
- Existing SIT-17/18 and CORS default tests now reference the issues.

Verified against the pre-fix source (15268ae~1): the #42/#43 tests fail
(4 failures); with the fix all pass.

Refs #42, #43

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CS4GeHuWThWvU74AkBL6E
CI failed test_table_preview_dialog_has_no_violations with a color-contrast
violation on the dialog's <th> cells. Cause: the dialog fades in (200ms) and
the scan could start mid-animation, when semi-transparent text measures low
contrast. Reproduced deterministically by freezing the fade at 50ms (6
color-contrast nodes) vs. after it finishes (none). _axe() now waits until
document.getAnimations() has nothing running.

Waiting for animations also lets the import toast render before the scan,
which exposed a real defect: sonner's richColors light-theme text is below
WCAG AA on its pale backgrounds (success 4.29:1, info 4.35:1, warning
3.07:1, error 4.36:1). Darker text, same hues: 5.23 / 6.46 / 6.72 / 5.75:1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CS4GeHuWThWvU74AkBL6E
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants