Say a dry-run action was not enforced only when the policy refused it and it went ahead - #530
Merged
davidmckayv merged 2 commits intoSep 14, 2026
Merged
Conversation
zopeVaibhav
requested review from
MikeRyanDev,
davidmckayv,
guidovizoso,
mxmzb and
tylerslaton
as code owners
September 14, 2026 15:48
zopeVaibhav
force-pushed
the
fix/audit-dry-run-label
branch
3 times, most recently
from
September 14, 2026 16:40
4ab50b1 to
3364875
Compare
… and it went ahead
zopeVaibhav
force-pushed
the
fix/audit-dry-run-label
branch
from
September 14, 2026 16:45
3364875 to
4625f77
Compare
# Conflicts: # CHANGELOG.md
davidmckayv
approved these changes
Sep 14, 2026
davidmckayv
left a comment
Contributor
There was a problem hiding this comment.
Deep-reviewed clean (validation, no secret leak, fail-closed, agrees with existing layers). CI green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #529
What this changes
On a deployment whose policy is in
dry-run, the Audit page printed "dry-run: recorded, not enforced" under rows that were never refused and under a tool call that was actually stopped. The line is meant for one case: the policy refused an action and dry-run let it through. It was drawn ondecision.mode === "dry-run" && decision.carriedOut, and an allowed action is always carried out, so every allowed row on a dry-run deployment qualified.Two small changes, one on each side.
The page now also requires
decision.allowed === falsebefore drawing the line. That is the whole definition of a refusal dry-run did not enforce, and it reads rows already in the trail correctly, including content-inspection refusals written before this change, because those carryallowed: truefrom the policy step.The content-inspection refusal row in
server/src/plugins/store.tsnow recordsdecision.carriedOut: false. It was built by spreading the policy step'sdecidedobject, which already heldcarriedOut: verdict.forward, so a call inspection refused was recorded as carried out in every mode, enforce included. The changelog entry that introduced the field defines it as what tells "a call this deployment stopped from one dry-run recorded and let past", and this row now agrees with that.Deliberately left alone: the page still shows no reason line for a content-inspection refusal (it prints
payload.reasonformcp.call_rejected, and this row storesrefusalinstead), and a call that a dry-run deny rule matches and that inspection then stops still writes two rows, the first of which correctly says the policy's refusal was not enforced. Both are separate questions about what the page should say, not about this label.Where it runs
Boundary and audit
The gateway order is unchanged, and nothing about what is refused changes. The content-inspection refusal still writes
mcp.call_rejectedbefore it throws; only the value ofdecision.carriedOutin that row differs. Nothing inserver,workerorappreadscarriedOutexcept this page, and rows written before the change are not rewritten.Changelog
A line under
Unreleaseddescribing the label and the corrected field.Proof
app/tests/audit-dry-run-label.test.tsxrenders the real Audit route with four rows. Againstmain(a97f775) in a separate worktree, the two bug cases fail (an allowed dry-run action, and a content-inspection refusal under dry-run, both showing the line) and the two controls pass (a dry-run refusal that went ahead shows the line, an enforce refusal does not). On this branch all four pass.server/tests/plugin-store.integration.test.ts"credential material is refused and never copied into the audit trail" now also assertsdecision: { carriedOut: false }. Againstmainit fails with"carriedOut": true; on this branch the file is 62 pass, 0 fail.{ mode: "dry-run", deny: [], allow: ["true"] }on a migrated test database: an allowed call recordsallowed: true, carriedOut: true, and a call carrying anapiKeyrecordsmcp.call_rejectedwithcarriedOut: trueonmainandcarriedOut: falseon this branch.bun test app/: 771 pass, 0 fail on this branch across 89 files, and 767 pass, 0 fail onmainacross 88 files in the same session. The difference is the four new tests.bun test server/withTEST_DATABASE_URLpointing at a dedicated test database: 2449 pass, 0 fail on both, with identical failure lines (none).bun run typecheck,bun run lintandbun run format:checkare clean.