Skip to content

The version test asserts the fact instead of a copy of it - #290

Merged
blooop merged 1 commit into
mainfrom
fix/version-test-cannot-rot
Aug 20, 2026
Merged

blooop merged 1 commit into
mainfrom
fix/version-test-cannot-rot

Conversation

@blooop

@blooop blooop commented Aug 20, 2026 •

Copy link
Copy Markdown
Owner

main is red on cargo test --workspace

Found while running the full suite for another change:

test tests::versioned_with_the_workspace ... FAILED
assert_eq!(env!("CARGO_PKG_VERSION"), "0.3.0");

devlaunch-test-support asserted the literal "0.3.0", and #289 (Release 0.3.1) did not update it. My miss.

Two things wrong, and the literal is the interesting one

1. The literal cannot express the claim. The crate declares version.workspace = true — that is what makes CARGO_PKG_VERSION the workspace's version. So comparing that variable against a hardcoded string does not test "versioned with the workspace"; it tests whether someone retyped a string. The test now reads Cargo.toml and asserts version.workspace = true, which has no second copy to fall out of step with.

Note this is the opposite call from the one in #282, where a literal budget was kept deliberately. There the literal was a ratchet — a real value with no other source of truth, whose failure asks a human to confirm an intended change. Here there is a source of truth one line away in Cargo.toml, and the literal only duplicated it.

2. It failed nothing for an entire release. The rust job names crates one at a time — on purpose, so each gets its own timeout — and devlaunch-test-support was not among them:

-p devlaunch-runner   -p devlaunch-core   -p aid   -p dl

So the only command that reaches this test is cargo test --workspace, which only a developer runs — and CLAUDE.md documents exactly that command. Added as its own CI step. What lives in that crate is the fake devpod every integration suite uses as its whole devpod, so a fidelity gap there is the one way a fake can do real harm.

The gate job already covers rust, so it needs no change.

Checks

  • cargo test --locked --workspace — green, all crates
  • cargo clippy --locked --all-targets -- -D warnings — clean
  • cargo fmt --check — clean

🤖 Generated with Claude Code

Summary by Sourcery

Ensure the test-support crate verifies workspace version inheritance and is covered by CI.

Bug Fixes:

  • Replace the brittle hardcoded version assertion with a check that the test-support crate inherits the workspace version.
  • Add CI coverage for the devlaunch-test-support crate so its tests run in the Rust workflow.

CI:

  • Run devlaunch-test-support tests as a dedicated step in the Rust CI job.

Tests:

  • Validate workspace version inheritance directly from the crate manifest.

`cargo test --workspace` failed on a clean main: devlaunch-test-support
asserted `CARGO_PKG_VERSION == "0.3.0"` and the 0.3.1 release did not
update the literal.

The literal was the bug, not the missed edit. The crate declares
`version.workspace = true`, which is what makes CARGO_PKG_VERSION the
workspace's version in the first place -- so comparing that variable to a
hardcoded string tests only whether somebody remembered to retype the
string, and the claim in the test's name ("versioned with the workspace")
is a Cargo.toml fact that a literal cannot express. It now reads
Cargo.toml and asserts `version.workspace = true`, which cannot go stale
because there is no second copy of the version to keep in step.

And it failed nothing for a release, which is the other half. The rust
job names the crates it tests one at a time -- deliberately, so each
carries its own timeout -- and this crate was not among them, so the one
command that reaches the test is one only a developer runs. Added as its
own step. What is in there is the fake devpod every integration suite
uses as its whole devpod, so a gap in it is the one way a fake can do
real harm.

cargo test --workspace is green, clippy -D warnings and fmt clean.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @blooop, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@sourcery-ai

sourcery-ai Bot commented Aug 20, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR fixes a brittle versioning test in devlaunch-test-support by asserting the workspace-version configuration directly from Cargo.toml, and adds a dedicated CI step so that crate’s tests run in the rust job like the other Rust crates.

File-Level Changes

Change Details Files
Make the version test assert the workspace-version configuration from Cargo.toml instead of a hardcoded version string.
  • Replace env!("CARGO_PKG_VERSION") string comparison with parsing the crate’s Cargo.toml via include_str!
  • Scan manifest lines and assert that a normalized line equals "version.workspace=true"
  • Provide a clear failure message explaining the invariant that devlaunch-test-support must use the workspace version
rust/devlaunch-test-support/src/lib.rs
Add devlaunch-test-support to the Rust CI workflow so its tests are executed with their own timeout.
  • Insert a new CI step in the rust job to run cargo test --locked -p devlaunch-test-support
  • Use a 300-second timeout and the same test-threads and nocapture flags as other crate-specific steps
  • Document in comments that each crate gets its own timeout and that this crate’s fake devpod is safety-critical for integration suites
.github/workflows/ci.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 42.98%. Comparing base (ee8e1cf) to head (f4ea484).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #290   +/-   ##
=======================================
  Coverage   42.98%   42.98%           
=======================================
  Files           3        3           
  Lines         221      221           
=======================================
  Hits           95       95           
  Misses        126      126           

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@blooop
blooop merged commit e1151a5 into main Aug 20, 2026
11 checks passed
@blooop
blooop deleted the fix/version-test-cannot-rot branch August 20, 2026 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant