chore(fullsend): bump agents harness pin, retire redundant sandbox policy fork - #4843
Merged
durandom merged 1 commit intoSep 17, 2026
Conversation
subhashkhileri
force-pushed
the
chore/fullsend-bump-harness-pin
branch
from
September 17, 2026 06:05
6f6489f to
60d97a0
Compare
Our code/fix/review harnesses pinned fullsend-ai/agents@4bbe4f50e, set once in redhat-developer#4496 and never bumped — 794 commits behind main. Move to the agents@v0 tag (7ab05564b, 794 commits later) to pick up two fixes relevant to the /fs-review timeout on PR redhat-developer#4808 (run 35097265739): - fullsend-ai/agents#1116 (PR redhat-developer#1181): sonnet-tier review sub-agents now request the bare `sonnet` alias, pinned via ANTHROPIC_DEFAULT_SONNET_MODEL=claude-sonnet-4-6 in env/gcp-vertex.env, instead of the hardcoded `claude-sonnet-4-6@default` literal our pin still carries. That literal was falling through the CLI's own alias resolution to claude-sonnet-4 (Sonnet 4.5 not available on our Vertex project), landing on a base model with a 120k output-tokens/min quota vs. 300k for sonnet-4-6 — three parallel sonnet sub-agents blew through it and all failed with 429 RESOURCE_EXHAUSTED. - review harness timeout_minutes raised 20 -> 45, which independently would have absorbed the degraded run instead of hard-timing-out. Scope: base: URL + content sha256 only. Our rhdh/policies/*.yaml overrides are left as-is for now — upstream moved network policy from policy: to composable providers:/openshell: profiles: in the interim (ADR 0065), which likely makes the claude.exe allowlist in our policy fork redundant (profiles/fullsend-vertex-ai.yaml already allowlists it upstream), but retiring that fork is deferred to a follow-up so this change stays a minimal, low-risk version bump.
subhashkhileri
force-pushed
the
chore/fullsend-bump-harness-pin
branch
from
September 17, 2026 06:09
60d97a0 to
11188a1
Compare
|
This was referenced Sep 18, 2026
Merged
durandom
pushed a commit
to redhat-developer/rhdh-plugin-export-overlays
that referenced
this pull request
Sep 18, 2026
…icy fork (#3857) Same fix as redhat-developer/rhdh-plugins#4843. Our code/fix/review harnesses pinned fullsend-ai/agents@4bbe4f50e (2026-07-13, set once and never bumped) — 794 commits behind. Move to agents@v0.43.0 (7ab05564b), the exact companion release for the fullsend@v0.43.0 CLI our .github/workflows/fullsend.yaml already pins. This picks up fullsend-ai/agents#1116 (PR #1181): sonnet-tier review sub-agents now request the bare `sonnet` alias, pinned via ANTHROPIC_DEFAULT_SONNET_MODEL=claude-sonnet-4-6 in env/gcp-vertex.env, instead of a hardcoded `claude-sonnet-4-6@default` literal that was falling through the CLI's own alias resolution to the much-smaller-quota claude-sonnet-4 base model when Sonnet 4.5 isn't available on a fleet Vertex project. Also picks up review.yaml's timeout_minutes 20 -> 45. Verified working end-to-end on rhdh-parasol/rhdh-plugins#54 (run 35214304712) before landing #4843. Also retires the now-redundant rhdh/policies/review.yaml fork: upstream migrated sandbox network policy from a monolithic policy: file to composable providers:/openshell: profiles: (ADR 0065), and its new default profiles/fullsend-vertex-ai.yaml already allowlists the claude.exe binary our fork existed solely to add. Diffed line-for-line against profiles/fullsend-vertex-ai.yaml + fullsend-github-ro.yaml — full coverage confirmed. rhdh/policies/code.yaml (used by both code.yaml and fix.yaml here) is kept as-is and NOT retired, unlike rhdh-plugins: it carries repo-specific additions (repo.yarnpkg.com + corepack binary, for corepack's yarn download) that upstream's default profiles/fullsend-package-registries.yaml doesn't cover. It already has the claude.exe binary entry, so no further change needed there.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
Our
.fullsend/rhdh/harness/{review,code,fix}.yamlall pinfullsend-ai/agentsat commit4bbe4f50e— set once in #4496 and never bumped since. That's 794 commits behindmainand predates a fix relevant to the/fs-reviewtimeout on PR #4808 (run 35097265739).What broke:
/fs-reviewdispatches 3 parallel sonnet-tier sub-agents (intent-coherence, style-conventions, docs-currency). Their hardcodedmodel: claude-sonnet-4-6@defaultwas falling through the Claude Code CLI's own alias resolution to the plainclaude-sonnet-4base model (log:Sonnet 4.5 not available — using Sonnet 4), which has a 120k output-tokens/min Vertex AI quota — 2.5x smaller thanclaude-sonnet-4-6's 300k. All 3 sonnet sub-agents hit429 RESOURCE_EXHAUSTEDwithin the same ~30s window, leaving only the opus correctness sub-agent to finish alone, which then ran long and blew the harness's hardtimeout_minutes: 20.Already fixed upstream:
fullsend-ai/agents#1116→PR #1181(merged 2026-09-05) changed the sonnet sub-agents to request the baresonnetalias and pinned it explicitly viaANTHROPIC_DEFAULT_SONNET_MODEL=claude-sonnet-4-6inenv/gcp-vertex.env, so it no longer falls through to the CLI's own default (which fleet Vertex projects don't serve). Follow-up#1186tracked getting this onto thev0tag. Separately, upstream also raisedreview.yaml'stimeout_minutesfrom 20 to 45, which independently would have absorbed the degraded run.Version match confirmed
The commit we're pinning to (
7ab05564b) is tagged bothv0andv0.43.0infullsend-ai/agents— and our own.github/workflows/fullsend.yamlalready pins thefullsendCLI tool itself tofullsend-ai/fullsend@v0.43.0(cut 8 minutes after the agents tag). These are companion releases versioned in lockstep, so this isn't just "a recent commit with the fix" — it's the exact harness content release the CLI version we already run was built and tested against.Changes
Bump the pin.
base:URL + contentsha256in all three harness files,4bbe4f50e(2026-07-13) →7ab05564b/agents@v0.43.0(2026-09-09).code.yaml/fix.yamlare bumped alongsidereview.yamlfor consistency — all three were pinned in the same original commit and are equally stale, though onlyreview.yaml's sub-agents were exposed to the sonnet quota bug (code/fix run single-agent onopus, unaffected).Retire the redundant sandbox policy fork. In the same 2 months, upstream migrated sandbox network policy from one monolithic
policy:file to composableproviders:/openshell: profiles:(ADR 0065). Ourrhdh/policies/{review,code}.yamlforks existed for exactly one reason — allowlisting theclaude.exebinary for Vertex AI, since Claude Code 2.1+ ships it under that name and the old upstream policy only matched**/claude. I diffed our fork against upstream's new default profiles and confirmed full coverage, in some cases stricter/better than what we had:vertex_ai→profiles/fullsend-vertex-ai.yaml— already allowlistsclaude.exeupstream now.github_api(review, read-only) →profiles/fullsend-github-ro.yaml— identical (api.github.com REST + GraphQL read-only, github.com read-only,gh/nodebinaries).github_api(code/fix) →profiles/fullsend-github-code.yaml— tightens our read-write API access to read-only (matches the harness's own documented intent: sandbox GH_TOKEN is read-only, only the runner-side post-script with a separate PUSH_TOKEN can write).gitleaks_releases→profiles/fullsend-gitleaks.yaml— identical.package_registries→profiles/fullsend-package-registries.yaml— identical, plus the npmallow_encoded_slashfix (needed for yarn's scoped-package URL encoding) now also applies toregistry.yarnpkg.com, and addsuv/uvxbinaries we didn't have.Removed the
policy:override line from all three harness files (falls through to base'spolicies/base.yaml+ provider/profile wiring) and deleted.fullsend/rhdh/policies/{review,code}.yaml, which nothing else in the repo references.Test plan
/fs-review(or a natural PR review trigger) completes without 429s / within timeout/fs-codeand/fs-fixstill dispatch correctly — network access to Vertex, npm/pip/go registries, and GitHub read-only/git still workspr-risk-assessmentskill replacingissue-labelsin the review skill list,validation_loop.max_iterations2→1)Assisted-by: Claude Code