Repository navigation
Claude/bold fermat 5swa7s - #9
Open
JaykibyMinhDung wants to merge 8 commits into
Open
JaykibyMinhDung wants to merge 8 commits into
JaykibyMinhDung wants to merge 8 commits into
Conversation
…port - test_review_yaml_router: YAML loading, TestsRouter routing and error fallbacks - test_blackbox_cli: 8 black-box test cases for the ccheck CLI (TC-01..TC-08) - test_e2e_qa_flow: question -> answer -> ccheck regression flow against a mock RAG API, including an empty retriever - test_concurrency: 50 concurrent requests, deadlock and cross-talk checks - mock_rag_server: in-process HTTP server imitating the RAG API - docs/testing/BAO_CAO_KIEM_THU.md: report (Vietnamese) Defects found are recorded as strict xfail tests describing the desired behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1q6SVuJiH6yWpSsXH7tMh
Vietnamese docs under docs/testing describing what was reviewed and tested, which actor each test case serves, how each test is implemented, and the defects found with reproduction steps and proposed fixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1q6SVuJiH6yWpSsXH7tMh
Move docs/testing into the MkDocs docs_dir, add it to the nav, enable Mermaid diagrams, and explain in ABOUT_DOCS that mkdocs must be run via poetry run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1q6SVuJiH6yWpSsXH7tMh
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1q6SVuJiH6yWpSsXH7tMh
…s/docs Covers TC-01..TC-20 and TC-31..TC-50 of the group SQA plan: - test_sqa_connectors_endpoints: HTTP timeout/4xx/5xx/429, OpenAI payload, endpoint defaults, YAML errors, request/response schemas - test_sqa_generators: token limit, duplicates, JSON parsing, wrapper parameter mapping, 1000-chunk stress test, prompt injection / Jinja SSTI - test_sqa_metrics: deterministic metrics, ground-truth agreement, factory routing, LLM judge prompts and verdict parsing (offline stub judge) - test_sqa_interfaces_docs: console width and legacy code pages, read-only output, JSON report schema, flake8, pytest config, docs vs code, notebook - sqa_http_stub: configurable local HTTP stub recording requests Defects found are recorded as strict xfail tests describing the expected behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1q6SVuJiH6yWpSsXH7tMh
- docs/docs/testing/07: install and run on Windows (PowerShell), run the automated suite, and step-by-step manual checks for every test case with expected and recorded results - examples/manual_testing: stdlib mock server (echo, RAG QA, slow, HTTP status codes, wrong types, empty retriever), 21 sample scenarios, a --folder demo and helper scripts (OpenAI payload, Jinja SSTI, Vietnamese preprocessing, 50 concurrent requests) - Console subprocess tests now force UTF-8 so they pass on Windows code pages; the encoding defect itself stays covered by TC-42 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1q6SVuJiH6yWpSsXH7tMh
loaders/yaml.py opens scenario files with the OS default encoding, so a Vietnamese scenario crashes on Windows (cp1252/cp1258). The strict xfail test reproduces it with a POSIX C locale and XPASSes once the loader uses UTF-8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1q6SVuJiH6yWpSsXH7tMh
…nd console Each test names the SRS-CC-N11 requirement it verifies. Confirmed defects (LLM called without chunks, multi-key assertions, anchored regex, invalid JSON schema, exit code 0 when a later step fails with --exit-on-failure, truncated console tables) are strict xfails describing the expected behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y1q6SVuJiH6yWpSsXH7tMh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.