Skip to content

Release 0.1.0: project settings, real-model fixes, PyPI publishing - #24

Merged
pro-utkarshM merged 5 commits into
masterfrom
claude/amazing-johnson-ehm81c
Oct 15, 2026
Merged

pro-utkarshM merged 5 commits into
masterfrom
claude/amazing-johnson-ehm81c

Conversation

@pro-utkarshM

Copy link
Copy Markdown
Member

Summary

Everything for the 0.1.0 release (Python-only alpha). After this merges, master gets the v0.1.0 tag. Publishing a GitHub release from that tag uploads the package to PyPI.

Changes

  • code2test init settings are used. init saves confidence_threshold, output_dir and exclude_patterns to .code2test/config.json, and code2test test uses them as defaults. Command-line options win. Config files from older versions still load, and invalid values stop the run with a clear message.
  • --exclude adds to the built-in exclusions and can be repeated. It used to replace them, so generated tests were analyzed as project code on the next run. The output directory is now always excluded.
  • --framework removed: only pytest tests are generated.
  • Run reports are written under the project root, like the tests.
  • One version source, code2test.__version__. pyproject.toml and the reports read it.
  • Last CodeWiki wording removed. The attribution and the ~/.codewiki settings location remain.

Bugs found by the real-model checks (each has a test)

  • Truncated intents. The intent summary stopped at the first period anywhere: "(0.2 means 20%)" became "(0", and "e.g." ended the summary. Wrapped docstring lines also kept a newline inside the intent.
  • Interactive mode lost edits. It never stored intents, so an edited intent was lost and the next session inferred it again.
  • Wrong interactive summary. It reported "Tests Generated 0, Verified 0, Avg Confidence 0%" after saving 44 verified tests.

Release material

  • CHANGELOG.md with the 0.1.0 entry.
  • .github/workflows/publish.yml: when a GitHub release is published, it builds, runs twine check, and uploads to PyPI with trusted publishing, so no token is stored. It refuses a release whose tag doesn't match the package version.
  • The README and quickstart install with pip install code2test, with the GitHub tag as the alternative. The CI examples install the package.

Verification

Unit tests: pytest passes (122 tests plus the new ones); ruff check and ruff format --check are clean.

Real-model checks (MiniMax MiniMax-M3.1-Flash-Preview, on a 3-function sample project shop/pricing.py):

Command Result
code2test test . interactive: accept apply_discount, edit total_with_tax's intent, skip format_price 2 files / 43 tests saved, 1 skipped; summary correct; the edited intent is stored (intent show returns it at 95% confidence) and the model generated tests for the edited detail; verify: 74/74 pass
Inject a bug (percent / 1000), then code2test verify tests/ exit 1, 17 failing test cases
code2test verify --diagnose tests/ 9 of 9 diagnoses CODE_BUG (97–98%), suggested fix "change the divisor back to 100"
Intended change (discount capped at 90%, docstring updated), then code2test verify --fix tests/ file repaired; tests now assert 91% and 100% are rejected with the new message; verify: 75/75 pass
code2test config validate real model answers, exit 0; with an unknown model name, exit 2 with the provider's error

Package:

  • Wheel (172 KB) and sdist (155 KB), built from a clean git archive of the final commit.
  • twine check: PASSED for both.
  • Installed into a fresh Python 3.12.3 environment, importing from site-packages: code2test --version gives 0.1.0.
  • code2test test --auto --exit-code . run from that install on the sample:
    • all 3 of 3 components got passing tests;
    • 94% of tests passed before any repair;
    • exit 0;
    • verify: 123/123 pass.

Release notes (paste into the GitHub release for v0.1.0)

Code2Test 0.1.0 — first release, an alpha for Python projects.

Code2Test generates pytest suites for existing Python code with an LLM. It infers what each function and class should do, generates tests against that intent, runs and repairs them, and keeps a test that exposes a suspected bug as an expected failure instead of deleting it. The suite it writes always passes.

  • code2test test (interactive or --auto), verify --diagnose/--fix, benchmark, report coverage, intent, init, config.
  • Works with any model pydantic-ai supports, including OpenAI-compatible endpoints.
  • Measured on inflection and TinyDB with two MiniMax models, two runs each:
    • components with passing tests: 96–100%;
    • tests passing before repair: 84–99%;
    • injected bugs caught: 59–84%;
    • diagnosis correct: 97–100%.
  • It found a real defect in inflection 0.5.1: singularize("campuses") returns "campuse".

Install: pip install code2test. Python 3.12 or later. Each run sends source code to your model provider. See CHANGELOG.md for details and known limitations.

After merge

  1. I tag v0.1.0 on the merge commit.
  2. You add a "pending trusted publisher" once on pypi.org:
    • project code2test
    • owner Cirkitly, repository code2test
    • workflow publish.yml, environment pypi
  3. You publish a GitHub release from the v0.1.0 tag with the notes above. That triggers the PyPI upload.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VGBhCd3VqGRk2KLZg1Tqu2


Generated by Claude Code

…init`

`init` now saves only what `test` uses, in .code2test/config.json:
confidence_threshold, output_dir and exclude_patterns (settable with
--confidence, --output-dir and repeatable --exclude). `test` takes them
as defaults; options on the command line win. Files from older versions
(with language/framework keys) still load; invalid values stop the run
with a clear message.

Also:
- `--exclude` adds to the built-in exclusions instead of replacing
  them (replacing them made the next run analyze the generated tests)
  and can be repeated; the output directory is always excluded.
- `--framework` is removed: only pytest tests are generated.
- Run reports go under the project root, like the tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGBhCd3VqGRk2KLZg1Tqu2
The version is defined once, as code2test.__version__: pyproject reads
it (dynamic version) and the JSON reports use it instead of a literal.
Module docstrings and comments that still described CodeWiki's
documentation generator now describe the dependency analysis, the
exception base class is Code2TestError, and `python -m code2test`'s
docstring names the right module. The CodeWiki attribution and the
~/.codewiki settings location (kept so saved settings keep working)
remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGBhCd3VqGRk2KLZg1Tqu2
The static intent took everything before the first "." in a docstring,
so "Return the sum ... plus tax at rate (0.2 means 20%)" became
"Return the sum ... plus tax at rate (0", and "e.g." ended a summary
early. The summary now ends at ".", "!" or "?" followed by a space or
the end of the paragraph (skipping e.g., i.e., etc., vs., cf.), only the
first paragraph is used, and wrapped lines are joined instead of
keeping a newline inside the intent. Found while preparing the real
model checks for 0.1.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGBhCd3VqGRk2KLZg1Tqu2
Found by running `code2test test` interactively against a real model:

- Intents were never stored in interactive mode, so an edited intent
  was lost (`intent show` found nothing) and the next session inferred
  it again. Every presented intent is now stored, an edit is saved as
  user-edited, and a later session starts from the edited intent.
- The end-of-session table read keys the session never set, so it
  showed "Tests Generated 0, Verified 0, Avg Confidence 0%" after
  saving 44 verified tests. It now lists components, test files saved,
  tests saved and skipped components.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGBhCd3VqGRk2KLZg1Tqu2
- CHANGELOG.md with the 0.1.0 entry: features, measured results, known
  limitations.
- .github/workflows/publish.yml builds, checks (twine) and uploads to
  PyPI with trusted publishing when a GitHub release is published; it
  refuses a release whose tag does not match code2test.__version__.
- README and quickstart install with `pip install code2test`, with the
  GitHub tag as the alternative; the CI examples install the package.
- docs/task.md lists the release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGBhCd3VqGRk2KLZg1Tqu2

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@pro-utkarshM
pro-utkarshM merged commit 1b99980 into master Oct 15, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants