Skip to content

chore(fullsend): bump agents harness pin, retire redundant sandbox policy fork - #4843

Merged
durandom merged 1 commit into
redhat-developer:mainfrom
subhashkhileri:chore/fullsend-bump-harness-pin
Sep 17, 2026
Merged

durandom merged 1 commit into
redhat-developer:mainfrom
subhashkhileri:chore/fullsend-bump-harness-pin

Conversation

@subhashkhileri

@subhashkhileri subhashkhileri commented Sep 17, 2026

Copy link
Copy Markdown
Member

Summary

Our .fullsend/rhdh/harness/{review,code,fix}.yaml all pin fullsend-ai/agents at commit 4bbe4f50e — set once in #4496 and never bumped since. That's 794 commits behind main and predates a fix relevant to the /fs-review timeout on PR #4808 (run 35097265739).

What broke: /fs-review dispatches 3 parallel sonnet-tier sub-agents (intent-coherence, style-conventions, docs-currency). Their hardcoded model: claude-sonnet-4-6@default was falling through the Claude Code CLI's own alias resolution to the plain claude-sonnet-4 base model (log: Sonnet 4.5 not available — using Sonnet 4), which has a 120k output-tokens/min Vertex AI quota — 2.5x smaller than claude-sonnet-4-6's 300k. All 3 sonnet sub-agents hit 429 RESOURCE_EXHAUSTED within the same ~30s window, leaving only the opus correctness sub-agent to finish alone, which then ran long and blew the harness's hard timeout_minutes: 20.

Already fixed upstream: fullsend-ai/agents#1116PR #1181 (merged 2026-09-05) changed the sonnet sub-agents to request the bare sonnet alias and pinned it explicitly via ANTHROPIC_DEFAULT_SONNET_MODEL=claude-sonnet-4-6 in env/gcp-vertex.env, so it no longer falls through to the CLI's own default (which fleet Vertex projects don't serve). Follow-up #1186 tracked getting this onto the v0 tag. Separately, upstream also raised review.yaml's timeout_minutes from 20 to 45, which independently would have absorbed the degraded run.

Version match confirmed

The commit we're pinning to (7ab05564b) is tagged both v0 and v0.43.0 in fullsend-ai/agents — and our own .github/workflows/fullsend.yaml already pins the fullsend CLI tool itself to fullsend-ai/fullsend@v0.43.0 (cut 8 minutes after the agents tag). These are companion releases versioned in lockstep, so this isn't just "a recent commit with the fix" — it's the exact harness content release the CLI version we already run was built and tested against.

Changes

  1. Bump the pin. base: URL + content sha256 in all three harness files, 4bbe4f50e (2026-07-13) → 7ab05564b / agents@v0.43.0 (2026-09-09). code.yaml/fix.yaml are bumped alongside review.yaml for consistency — all three were pinned in the same original commit and are equally stale, though only review.yaml's sub-agents were exposed to the sonnet quota bug (code/fix run single-agent on opus, unaffected).

  2. Retire the redundant sandbox policy fork. In the same 2 months, upstream migrated sandbox network policy from one monolithic policy: file to composable providers:/openshell: profiles: (ADR 0065). Our rhdh/policies/{review,code}.yaml forks existed for exactly one reason — allowlisting the claude.exe binary for Vertex AI, since Claude Code 2.1+ ships it under that name and the old upstream policy only matched **/claude. I diffed our fork against upstream's new default profiles and confirmed full coverage, in some cases stricter/better than what we had:

    • vertex_aiprofiles/fullsend-vertex-ai.yaml — already allowlists claude.exe upstream now.
    • github_api (review, read-only) → profiles/fullsend-github-ro.yaml — identical (api.github.com REST + GraphQL read-only, github.com read-only, gh/node binaries).
    • github_api (code/fix) → profiles/fullsend-github-code.yamltightens our read-write API access to read-only (matches the harness's own documented intent: sandbox GH_TOKEN is read-only, only the runner-side post-script with a separate PUSH_TOKEN can write).
    • gitleaks_releasesprofiles/fullsend-gitleaks.yaml — identical.
    • package_registriesprofiles/fullsend-package-registries.yaml — identical, plus the npm allow_encoded_slash fix (needed for yarn's scoped-package URL encoding) now also applies to registry.yarnpkg.com, and adds uv/uvx binaries we didn't have.

    Removed the policy: override line from all three harness files (falls through to base's policies/base.yaml + provider/profile wiring) and deleted .fullsend/rhdh/policies/{review,code}.yaml, which nothing else in the repo references.

Test plan

  • Confirm /fs-review (or a natural PR review trigger) completes without 429s / within timeout
  • Confirm the review sandbox can still reach Vertex AI and read-only GitHub API (no policy regression from dropping the fork)
  • Confirm /fs-code and /fs-fix still dispatch correctly — network access to Vertex, npm/pip/go registries, and GitHub read-only/git still works
  • Watch the first few runs post-merge for any behavior shift from other bundled upstream changes (new pr-risk-assessment skill replacing issue-labels in the review skill list, validation_loop.max_iterations 2→1)

Assisted-by: Claude Code

@subhashkhileri
subhashkhileri requested review from a team as code owners September 17, 2026 06:03
@subhashkhileri
subhashkhileri force-pushed the chore/fullsend-bump-harness-pin branch from 6f6489f to 60d97a0 Compare September 17, 2026 06:05
Our code/fix/review harnesses pinned fullsend-ai/agents@4bbe4f50e, set
once in redhat-developer#4496 and never bumped — 794 commits behind main. Move to the
agents@v0 tag (7ab05564b, 794 commits later) to pick up two fixes
relevant to the /fs-review timeout on PR redhat-developer#4808 (run 35097265739):

- fullsend-ai/agents#1116 (PR redhat-developer#1181): sonnet-tier review sub-agents now
  request the bare `sonnet` alias, pinned via
  ANTHROPIC_DEFAULT_SONNET_MODEL=claude-sonnet-4-6 in env/gcp-vertex.env,
  instead of the hardcoded `claude-sonnet-4-6@default` literal our pin
  still carries. That literal was falling through the CLI's own alias
  resolution to claude-sonnet-4 (Sonnet 4.5 not available on our Vertex
  project), landing on a base model with a 120k output-tokens/min quota
  vs. 300k for sonnet-4-6 — three parallel sonnet sub-agents blew
  through it and all failed with 429 RESOURCE_EXHAUSTED.
- review harness timeout_minutes raised 20 -> 45, which independently
  would have absorbed the degraded run instead of hard-timing-out.

Scope: base: URL + content sha256 only. Our rhdh/policies/*.yaml
overrides are left as-is for now — upstream moved network policy from
policy: to composable providers:/openshell: profiles: in the interim
(ADR 0065), which likely makes the claude.exe allowlist in our policy
fork redundant (profiles/fullsend-vertex-ai.yaml already allowlists it
upstream), but retiring that fork is deferred to a follow-up so this
change stays a minimal, low-risk version bump.
@subhashkhileri
subhashkhileri force-pushed the chore/fullsend-bump-harness-pin branch from 60d97a0 to 11188a1 Compare September 17, 2026 06:09
@subhashkhileri subhashkhileri changed the title chore(fullsend): bump agents harness pin (2026-07-13 -> 2026-09-09) chore(fullsend): bump agents harness pin, retire redundant sandbox policy fork Sep 17, 2026
@sonarqubecloud

Copy link
Copy Markdown

@durandom
durandom merged commit 13066a1 into redhat-developer:main Sep 17, 2026
11 checks passed
durandom pushed a commit to redhat-developer/rhdh-plugin-export-overlays that referenced this pull request Sep 18, 2026
…icy fork (#3857)

Same fix as redhat-developer/rhdh-plugins#4843. Our code/fix/review
harnesses pinned fullsend-ai/agents@4bbe4f50e (2026-07-13, set once and
never bumped) — 794 commits behind. Move to agents@v0.43.0 (7ab05564b),
the exact companion release for the fullsend@v0.43.0 CLI our
.github/workflows/fullsend.yaml already pins.

This picks up fullsend-ai/agents#1116 (PR #1181): sonnet-tier review
sub-agents now request the bare `sonnet` alias, pinned via
ANTHROPIC_DEFAULT_SONNET_MODEL=claude-sonnet-4-6 in env/gcp-vertex.env,
instead of a hardcoded `claude-sonnet-4-6@default` literal that was
falling through the CLI's own alias resolution to the much-smaller-quota
claude-sonnet-4 base model when Sonnet 4.5 isn't available on a fleet
Vertex project. Also picks up review.yaml's timeout_minutes 20 -> 45.
Verified working end-to-end on rhdh-parasol/rhdh-plugins#54 (run
35214304712) before landing #4843.

Also retires the now-redundant rhdh/policies/review.yaml fork: upstream
migrated sandbox network policy from a monolithic policy: file to
composable providers:/openshell: profiles: (ADR 0065), and its new
default profiles/fullsend-vertex-ai.yaml already allowlists the
claude.exe binary our fork existed solely to add. Diffed line-for-line
against profiles/fullsend-vertex-ai.yaml + fullsend-github-ro.yaml —
full coverage confirmed.

rhdh/policies/code.yaml (used by both code.yaml and fix.yaml here) is
kept as-is and NOT retired, unlike rhdh-plugins: it carries
repo-specific additions (repo.yarnpkg.com + corepack binary, for
corepack's yarn download) that upstream's default
profiles/fullsend-package-registries.yaml doesn't cover. It already has
the claude.exe binary entry, so no further change needed there.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants