Skip to content

Do not use core.exportVariable in unit tests (2nd attempt) - #4212

Open
mbg wants to merge 5 commits into
mainfrom
mbg/tests/fix-export-variable-2
Open

mbg wants to merge 5 commits into
mainfrom
mbg/tests/fix-export-variable-2

Conversation

@mbg

@mbg mbg commented Oct 7, 2026

Copy link
Copy Markdown
Member

This PR revives #3930 as per the discussion at #4198 (comment). This PR aims to accomplish the same goals as #3930, but is updated based on changes that have happened in the codebase since. The original PR description follows.


Although we already clear environment variables that are set during unit tests, that does not affect the behaviour of core.exportVariable which additionally sets environment variables for subsequent steps in a workflow. If the unit tests are run in CI, then core.exportVariable sets environment variables for subsequent steps in the workflow job which can interfere with them.

This PR improves the situation by introducing a wrapper around core.exportVariable which does not call core.exportVariable when NODE_ENV is test (which is set automatically by ava).


The main change compared to the previous PR is that we now have ReadOnlyEnv and Env and it makes sense to integrate the logic of our exportVariable wrapper with those.

At first glance, a potential concern with using the export method of the Env class is that, if NODE_ENV is test, it only stores the environment variable in the Env instance. Other functions which may depend on the environment variable, but still use process.env to access it, do not see the value. However, all of the unit tests pass, and so we can rule out that this is an issue affecting the tests.

Outside of unit tests, where NODE_ENV is not expected to be set to test, the implementation calls core.exportVariable as well, which does set the environment variable for the process.

Risk assessment

For internal use only. Please select the risk level of this change:

  • Low risk: Changes are fully under feature flags, or have been fully tested and validated in pre-production environments and are highly observable, or are documentation or test only.

Which use cases does this change impact?

Environments:

  • Testing/None - This change does not impact any CodeQL workflows in production.

How did/will you validate this change?

  • Unit tests - I am depending on unit test coverage (i.e. tests in .test.ts files).
  • End-to-end tests - I am depending on PR checks (i.e. tests in pr-checks).

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

  • Rollback - Change can only be disabled by rolling back the release or releasing a new version with a fix.
  • Development/testing only - This change cannot cause any failures in production.

How will you know if something goes wrong after this change is released?

  • Telemetry - I rely on existing telemetry or have made changes to the telemetry.
    • Dashboards - I will watch relevant dashboards for issues after the release. Consider whether this requires this change to be released at a particular time rather than as part of a regular release.
    • Alerts - New or existing monitors will trip if something goes wrong with this change.

Are there any special considerations for merging or releasing this change?

  • No special considerations - This change can be merged at any time.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Consider adding a changelog entry for this change.
  • Confirm the readme and docs have been updated if necessary.

@github-actions github-actions Bot added the size/M Should be of average difficulty to review label Oct 7, 2026
@mbg
mbg marked this pull request as ready for review October 8, 2026 09:57
@mbg
mbg requested a review from a team as a code owner October 8, 2026 09:57
Copilot AI balanced review requested due to automatic review settings October 8, 2026 09:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

  • Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.

🟡 Changes recommended

NODE_ENV-based detection can suppress cross-step exports in real workflows that set NODE_ENV=test.

1 open finding
What changed in this PR

Introduces an environment abstraction that prevents unit tests from exporting variables to subsequent workflow steps.

Changes:

  • Adds Env.export and test-environment detection.
  • Migrates environment exports away from core.exportVariable.
  • Updates tests, linting, and CodeQL query configuration.
File Description
src/​environment.ts Adds environment export and test-mode abstractions.
src/​testing-utils.ts Configures isolated test environments.
src/​actions-util.ts Removes exporting from ActionsEnv.
src/​util.ts Uses the environment export wrapper.
src/​upload-lib.ts Migrates SARIF-related exports.
src/​status-report.ts Migrates status and UUID exports.
src/​setup-codeql.ts Migrates setup-state export.
src/​setup-codeql-action.ts Uses action-state environment exporting.
src/​init.ts Migrates deprecation-warning export.
src/​init-action.ts Uses action-state environment access and exporting.
src/​init-action-post.ts Migrates final-status exports.
src/​debug-artifacts.ts Migrates artifact-scan export.
src/​config-utils.ts Migrates TRAP-cache exports.
src/​codeql.ts Migrates warning-suppression export.
src/​autobuild.ts Migrates autobuild exports.
src/​autobuild-action.ts Uses action-state environment exporting.
src/​api-client.ts Migrates analysis-key export.
src/​analyze-action.ts Uses action-state environment exporting.
src/​overlay/​caching.test.ts Updates the test-mode stub location.
src/​init.test.ts Updates environment export stubs.
queries/​default-setup-environment-variables.ql Allows NODE_ENV access.
eslint.config.mjs Prohibits direct core.exportVariable use.
lib/​entry-points.js Generated bundle update; excluded from review.
Files excluded by content exclusion policy (1)
  • lib/entry-points.js

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Comment thread src/environment.ts
Comment on lines +359 to +360
if (!this.isTestingEnv()) {
core.exportVariable(name, val);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a potential concern if we have advanced setup users running CodeQL in the same job as unit tests.  Hopefully that NODE_ENV variable wouldn't be exported, but I suppose it's possible.  Do you think we should instead set a specific CODEQL_ACTION_ environment variable in the npm test script?

Comment thread src/environment.ts
Comment on lines +359 to +360
if (!this.isTestingEnv()) {
core.exportVariable(name, val);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a potential concern if we have advanced setup users running CodeQL in the same job as unit tests.  Hopefully that NODE_ENV variable wouldn't be exported, but I suppose it's possible.  Do you think we should instead set a specific CODEQL_ACTION_ environment variable in the npm test script?

Comment thread eslint.config.mjs
Comment on lines +144 to +145
// A basic check that we don't use `exportVariable` from `@actions/core`. This rule depends on
// the module being imported as `core`, but that is a good enough check for us.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A CodeQL query might be more robust, but admittedly would give feedback later on in the development loop. Perhaps it's worth having both so we catch anything missed by this rule?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two thoughts here:

  1. That could probably be for a separate PR if we want to do it?
  2. That said, ideally, our exportVariable wrapper would go away sooner than later and we'd just use Env::export everywhere. In that case, the linter rule could just look for exportVariable and we wouldn't have to worry about where it came from. I suppose that we could equally rename our exportVariable to exportEnvVar or similar now to avoid the issue now. Do you think a query would still have a benefit over the linter rule in that case?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Should be of average difficulty to review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants