ci: make the PR gate fast and deterministic, move ct install off PRs

[mosoriob]

Lint and Test Charts failed 15 of its last 17 runs. It is not flaky in the random sense. Three deterministic faults make it fail.

Cause 1 — Python was not pinned

This is the real "flakiness". setup-python was set to python-version: "3.x" with check-latest: true, so it floats to whatever Python is newest. Today that is 3.14, and ct lint crashes:

AttributeError: module 'ast' has no attribute 'Num'
  .../ct/3.12.0/amd64/venv/lib/python3.14/site-packages/yamale/syntax/parser.py

chart-testing bundles yamale, yamale references ast.Num, and ast.Num was removed in Python 3.12. The workflow broke because upstream Python moved, not because anything in this repo changed. Now pinned to 3.11.

Cause 2 — ct.yaml is never read

chart-testing resolves its config from CT_CONFIG_DIR inside the tool cache, not from the repository root, and the workflow never passed --config. So helm-extra-args: --timeout 600s was dropped in silence, helm used its 300s default, and every run died at ~4m58s:

Error: INSTALLATION FAILED: client rate limiter Wait returned an error: context deadline exceeded

ct-arm-64.yaml is dead for the same reason.

Cause 3 — ct install cannot converge

A longer timeout does not help. In the same run:

• model-catalog failed readiness and liveness with HTTP 503, continuously (9 and 6 consecutive failures)
• hasura-auth-webhook refused connections on port 3000

The full stack does not come up on a single-node kind cluster on a 2-vCPU runner without real backing services.

Changes

The PR gate is now fast and deterministic. ct lint plus a helm template + kubeconform matrix over four value permutations. Every check finishes in under 20 seconds and every failure is a real chart fault.

kubeconform is the part helm lint cannot do. It validates the rendered manifests against the Kubernetes schemas, so it catches duplicate keys and bad apiVersions.

The fourth permutation is all-components-arm64. data_catalog is off by default, so it is the only one that renders datacatalog.yaml at all, and the only one that reaches its two arm_support branches. Those branches were covered by nothing before.

ct install moves to chart-install.yaml, nightly and on demand, where it reports without blocking a merge.

Also pins Python to 3.11, passes --config ct.yaml, bumps checkout to v4, setup-helm to v4.3.0, chart-testing-action to v2.7.0 and helm to v3.16.4 (which clears the Node 20 deprecation warnings), and adds concurrency and job timeouts.

The chart fix

The new kubeconform step immediately found a real defect:

PersistentVolumeClaim mint-hasura-db failed validation:
  line 9: key "helm.sh/resource-policy" already set in map

pvc-hasura.yaml hardcoded helm.sh/resource-policy: keep and values.yaml supplies the same key through persistence.annotations, so the rendered PVC carried the annotation twice. Helm resolves the duplicate last-wins, so one annotation reached the cluster and behaviour was correct, but the manifest was not valid YAML. helm lint and helm template both pass it in silence.

The four sibling PVC templates never hardcoded it. Hasura was the only outlier. All five persistence blocks in values.yaml carry the keep annotation, so removing the hardcoded line makes Hasura consistent and leaves the rendered output unchanged.

Verified against the real deployment

Checked against In-For-Disaster-Analytics/dynamo, which pins MINT 9.0.0-beta.11, using its actual production values rather than synthetic ones:

• The full rendered release before and after differs by exactly the one duplicate line
• Both PVCs it creates still carry helm.sh/resource-policy: keep
• dynamo overrides hasura_db.persistence with storageClass and subPath only. It never sets commonAnnotations and never nulls annotations, so the one edge case where the old hardcode mattered cannot arise there

The keep annotation survives overrides because Helm coalesces user values onto chart defaults rather than replacing the map.

Note

The nightly install job will likely still go red. The model-catalog 503 is a chart or environment issue, not a CI one. It is now informational rather than a merge gate.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Gb5UJ1nbafcMMFbEoZfB41