Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
297 changes: 0 additions & 297 deletions DEVELOPMENT.md

This file was deleted.

23 changes: 23 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
MIT License

Copyright (c) 2026 Code2Test Contributors
Copyright (c) 2025 FSoft-AI4Code (portions derived from CodeWiki,
https://github.com/FSoft-AI4Code/CodeWiki)

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
26 changes: 18 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Code2Test picks the model in this order:
4. Gemini, when the API key saved with `code2test config set --api-key` is a Google key.
5. `openai:gpt-4o-mini` otherwise.

Each provider reads its usual key variable, for example `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`.
Each provider reads its usual key variable, for example `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`. A key saved with `code2test config set --api-key` is stored in the system keychain and exported as `OPENAI_API_KEY` (and as the Google key variables) when Code2Test runs, replacing any value already in the environment.

```bash
export ANTHROPIC_API_KEY=...
Expand Down Expand Up @@ -100,8 +100,9 @@ The suite Code2Test writes therefore always passes: every test either passes or
| `code2test benchmark [PATH]` | Measure mutation score and diagnosis accuracy of the generated tests |
| `code2test intent show\|edit\|export [NAME]` | Review or correct inferred intents |
| `code2test report` | Summarize stored intents by confidence |
| `code2test init` | Create a `.code2test` configuration for a repository |
| `code2test config set\|show\|validate` | Store API credentials and model settings |
| `code2test init` | Write a starter `.code2test/config.json` (the other commands take their settings from options, not from this file) |
| `code2test config set\|show\|validate` | Store the API key and model (`--api-key`, `--main-model`); `validate` sends the model one test request |
| `code2test version` | Show the installed version |

### `code2test test`

Expand Down Expand Up @@ -161,6 +162,7 @@ The benchmark also scores the diagnosis agent on failures whose cause is known.
| `.code2test/metrics.json` | Metrics of the last `test --auto` run |
| `.code2test/benchmark.json` | Results of the last `benchmark` run, including uncaught mutants |
| `.code2test/dependency_graphs/` | The analyzed dependency graph |
| `.code2test/config.json` | Starter settings written by `code2test init` (not read by other commands) |
| `tests/reports/report.html`, `report.json` | Run reports, when `--report` is used |

## Results
Expand All @@ -187,29 +189,37 @@ On these well-documented libraries, nearly every inferred intent came from the d

## Limitations

- **Python only.** Other languages are analyzed but get no tests; the tool says so when it skips them.
- **Python only.** Files in other languages are part of the dependency graph but get no tests; the tool says so when it skips them.
- **Behavior is inferred, not specified.** Expected values come from the inferred intent and the source. Review generated tests before relying on them, as you would review a colleague's.
- **Each run calls the model.** A run sends source code to the configured provider and costs API usage: one or more calls per component, plus repairs.
- **Results come from two libraries and two models of one provider, one run each.** Larger codebases, code with external services, other models and run-to-run variance are not yet measured.

## Development

```bash
git clone https://github.com/Cirkitly/code2test.git
cd code2test
pip install -e ".[dev]"
pytest
```

The test suite drives the whole pipeline offline with scripted models, so no API key is needed. CI runs it on every push.
The tests in `tests/` drive the whole pipeline offline with scripted models, so no API key is needed. CI ([`.github/workflows/ci.yml`](.github/workflows/ci.yml)) runs them on every push and pull request.

When changing the pipeline:

- Add or update a test in `tests/` for every behavior change. `tests/test_pipeline.py` shows how to script model responses.
- To check a change against a real model, run `code2test test --auto`, `code2test verify` and `code2test benchmark` on a small library with its own tests removed, as [`research/scripts/run_experiments.sh`](research/scripts/run_experiments.sh) does.
- Keep [docs/architecture.md](docs/architecture.md) in step with the code it describes.

Further documentation:

- [docs/architecture.md](docs/architecture.md): components and data flow
- [docs/architecture.md](docs/architecture.md): modules and data flow
- [docs/quickstart.md](docs/quickstart.md): a step-by-step first run
- [docs/task.md](docs/task.md): implementation status
- [docs/proposal.md](docs/proposal.md): the original project proposal and its success criteria
- [docs/real_model_run.md](docs/real_model_run.md): the first real-model runs and the failures they exposed
- [research/](research/): the paper, experiment scripts and raw results
- [docs/task.md](docs/task.md): implementation status

## License

MIT
MIT; see [LICENSE](LICENSE). The dependency analyzer is derived from [CodeWiki](https://github.com/FSoft-AI4Code/CodeWiki) (MIT, copyright 2025 FSoft-AI4Code), whose notice is kept in the LICENSE file.
Loading
Loading