Skip to content

No CI workflow runs the test suite, so two red goldens survived a release cycle #55

Open

Description

@NewGraphEnvironment

Problem

.github/workflows/ holds only pkgdown.yaml and update-citation-cff.yaml. Nothing runs the test suite.

That is why #48's two red goldens survived a release cycle: every CI run is green because nothing runs the tests. v0.9.0 and v0.10.0 were both tagged over FAIL 2 on main — verified for v0.9.0 by checking out the tag and running the suite, not assumed.

$ git worktree add /tmp/drift-pre51 66b2c36 && cd /tmp/drift-pre51
$ Rscript -e 'devtools::test(filter="dft_stac_")'
test-dft_stac_cube.R:62   Expected `cube_key()`  to equal "638a2be11fdf"
test-dft_stac_fetch.R:88  Expected `cache_key()` to equal "79f67b7b9dae"
[ FAIL 2 | WARN 0 | SKIP 5 | PASS 177 ]

Why this is separate from #48

#48 is the bug. This is the reason nobody was told. Fixing #48 without this leaves the next frozen guardian free to go red the day it lands, with the same silence.

Fix

usethis::use_github_action("check-standard").

Premise update (2026-09-05)

#48 is fixed (v0.12.0), so the workflow would be green on arrival, not red — the two decisions below are resolved by that. Better: the goldens now assert a cross-platform property (content-only key, digest xxhash64 over canonical bytes), and check-standard runs on Linux, macOS and Windows runners. That makes this workflow the cheapest possible witness for #59 — three clean machines instead of a trip to the m4. If a runner disagrees with the pinned 8b02f87e393e7f9a / 3e704313eb1f1c00, that is a real failure of the property, not a golden to re-pin. #58 was closed as a duplicate of this.

Also worth knowing since #44: the suite now has a test asserting lwgeom is never loaded, and devtools::check() reports one pre-existing WARNING (non-ASCII characters in R/dft_stac_fetch.R), which error_on = "warning" — r-lib's default — will turn red. Fix the file or set error_on: '"error"' deliberately; do not discover it from the first red run.

Two things to decide before it goes green (original — both now moot)

  1. It will be red on arrival, because of Cache key moved on an rlang upgrade, silently orphaning every cached cube and fetch #48. That is the correct behaviour and the point of adding it — but it means Cache key moved on an rlang upgrade, silently orphaning every cached cube and fetch #48 has to be resolved (or the goldens deliberately skipped with a stated reason) or the badge is red from day one and stops being read.

  2. The goldens may be environment-specific, so CI is a second environment that can disagree. If Cache key moved on an rlang upgrade, silently orphaning every cached cube and fetch #48's drift hypothesis is right, 2264b5dbef6e and 45685ccbda33 are this machine's values and a GitHub runner will produce different ones. That is not a reason to skip the check — it is arguably the strongest evidence available for Cache key moved on an rlang upgrade, silently orphaning every cached cube and fetch #48, since a runner is a clean, reproducible environment. Worth running once deliberately to see what it yields; that single data point would settle whether the drift is environmental.

Network tests already opt in via DRIFT_TEST_NETWORK, so the default run stays network-free and CI needs no secrets.

Context

Found while merging #52 (#51). Split out of #54, which was closed as a duplicate of #48.

Activity

  1. NewGraphEnvironment commented on Sep 5, 2026

    @NewGraphEnvironment
    OwnerAuthor

    Two things from the #48 work that raise the stakes here, recorded so they are not re-derived:

    1. The goldens now assert a cross-platform property. #48 replaced the rlang::hash() key with one that depends only on content, and the frozen values are documented — in NEWS.md, in dft_stac_fetch()'s roxygen, and in the goldens' own comments — as portable facts: identical on any machine, R version, rlang version and architecture. Nothing verifies that anywhere but one macOS arm64 machine. That claim will be believed and acted on (someone copies a cache between machines) with no check behind it.

    Cross-machine verification is filed separately as #59, and does not need CI.

    2. There is a blocker to check-standard worth deciding before wiring it. devtools::check() on main currently reports 1 WARNING — pre-existing non-ASCII em dashes in R/dft_stac_fetch.R roxygen, present long before #48 and house style in that file. r-lib's default error_on: "warning" would fail the build on it. So either fix the non-ASCII or set error_on: "error"; otherwise the first green-CI attempt fails for a reason unrelated to the tests.

    Also note the suite has network-gated tests (DRIFT_TEST_NETWORK, and DRIFT_TEST_CACHE_DAMAGE added in #41) that must stay skipped in CI.

    For the record, this issue's evidence held up: at 66b2c36 both goldens were red, which is the FAIL 2 that #48's body reported. Current main is FAIL 1 only because #51 re-pinned the fetch golden after the drift — the cube one stayed red until #48.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions