Repository navigation
Add Helm chart to deploy Scope app onto the Bicep-provisioned AKS cluster #1481
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Josh Duffney (duffney)
wants to merge
18
commits into
main
Choose a base branch
from
duffney-helm-chart-app-deployment
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
2218d65
feat(deploy): add Helm chart skeleton for Scope app
duffney a0ff409
feat(deploy): split Helm core app onto GHCR, defer ACP workers to ACR…
duffney 622761e
fix(helm): resolve real-deployment bugs found validating chart on AKS
duffney 0e28bbd
fix(helm): default portal.authEnabled to false
duffney 71db4a9
fix: restore missing scripts/get-copilot-token.ts
duffney 6bb5924
Add Helm Deployment templates for ACP coding-agent workers
duffney f94998b
Add bootstrap-workers.sh to automate Phase 3 ACP worker setup
duffney c9613a0
Fix criteria/prompt-feature import missing projectId; doc sample-scen…
duffney 9fe2df6
Retry blob writes on transient authorization errors; fix stale --reus…
duffney 59e9a9a
Split the Helm chart README into a lean quickstart plus detail docs
duffney 46caf15
Add MCP gateway sidecar, bootstrap-core script, and doc fixes
duffney fd9410b
Split bootstrap-core.sh into values-only generator, fix set -u array bug
duffney 91e2863
docs(helm): tighten chart README for conciseness
duffney c99275f
helm: remove kv-secrets CSI mount and workload identity from ACP workers
duffney 6b4c847
helm: give ACP workers their own worker-scoped identity
duffney a0243d1
scripts: remove secret bootstrap logic from bootstrap-workers.sh
duffney 4bda1fb
fix(helm): default worker bootstrap image tag to git SHA, not latest
duffney d85ed7a
fix(helm): pin migrations to the app's revision; widen publish-images…
duffney File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| name: Publish core images to GHCR | ||
|
|
||
| # Builds and publishes the 6 core Scope app images (NOT the two ACP | ||
| # coding-agent workers, which are built/pushed to the Bicep-provisioned ACR | ||
| # as a separate step — see deploy/helm/scope/README.md "Phase 3") to GHCR, | ||
| # so the Helm chart under deploy/helm/scope/ can install the app straight | ||
| # from ghcr.io/<owner>/scope-<service> with no private registry required. | ||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "apps/api/**" | ||
| - "apps/judge/**" | ||
| - "apps/token-manager/**" | ||
| - "apps/scheduler/**" | ||
| - "apps/portal/**" | ||
| - "apps/workers/report-generator/**" | ||
| - "packages/shared/**" | ||
| - "packages/telemetry/**" | ||
| - "packages/db-migrations/**" | ||
| - "tsconfig.base.json" | ||
| - "pnpm-workspace.yaml" | ||
| - "pnpm-lock.yaml" | ||
| - ".github/workflows/publish-images.yml" | ||
| workflow_dispatch: {} | ||
|
|
||
| permissions: | ||
| contents: read | ||
| packages: write | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: false | ||
|
|
||
| env: | ||
| REGISTRY: ghcr.io | ||
|
|
||
| jobs: | ||
| publish: | ||
| # Fork PRs never trigger this workflow (push to main / manual dispatch | ||
| # only), so there's no need for the ci.yml TRUSTED_CODE gate here. | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| image: | ||
| - name: scope-api | ||
| dockerfile: apps/api/Dockerfile | ||
| - name: scope-judge | ||
| dockerfile: apps/judge/Dockerfile | ||
| - name: scope-token-manager | ||
| dockerfile: apps/token-manager/Dockerfile | ||
| - name: scope-scheduler | ||
| dockerfile: apps/scheduler/Dockerfile | ||
| - name: scope-portal | ||
| dockerfile: apps/portal/Dockerfile | ||
| - name: scope-report-generator | ||
| dockerfile: apps/workers/report-generator/Dockerfile | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v4.0.0 | ||
|
|
||
| - name: Log in to GHCR | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Build and push (production image) | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: . | ||
| file: ${{ matrix.image.dockerfile }} | ||
| push: true | ||
| tags: | | ||
| ${{ env.REGISTRY }}/${{ github.repository_owner }}/${{ matrix.image.name }}:latest | ||
| ${{ env.REGISTRY }}/${{ github.repository_owner }}/${{ matrix.image.name }}:${{ github.sha }} | ||
| build-args: | | ||
| GIT_COMMIT=${{ github.sha }} | ||
| BUILD_TIME=${{ github.event.head_commit.timestamp || '' }} | ||
| GIT_BRANCH=${{ github.ref_name }} | ||
| cache-from: type=gha,scope=${{ matrix.image.name }} | ||
| cache-to: type=gha,mode=min,scope=${{ matrix.image.name }} | ||
|
|
||
| # The db-migrate Helm Job runs migrations via the api Dockerfile's | ||
| # `builder` stage (full TS source + tsx) — the slim production image | ||
| # above only ships packages/db-migrations/dist, which mongo-migrate-ts's | ||
| # CLI can't use (see deploy/helm/scope/templates/jobs/db-migrate.yaml). | ||
| - name: Build and push (api builder stage, for db-migrate job) | ||
| if: matrix.image.name == 'scope-api' | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: . | ||
| file: apps/api/Dockerfile | ||
| target: builder | ||
| push: true | ||
| tags: | | ||
| ${{ env.REGISTRY }}/${{ github.repository_owner }}/scope-api:builder | ||
| ${{ env.REGISTRY }}/${{ github.repository_owner }}/scope-api:builder-${{ github.sha }} | ||
| cache-from: type=gha,scope=scope-api-builder | ||
| cache-to: type=gha,mode=min,scope=scope-api-builder | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| .git/ | ||
| .gitignore | ||
| *.swp | ||
| *.bak | ||
| *.tmp | ||
| .DS_Store |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| # Configuration reference | ||
|
|
||
| Details on values.yaml conventions and image registries. For the install | ||
| steps, see the [README](./README.md). | ||
|
|
||
| ## Bring your own secrets | ||
|
|
||
| Port-forward the Portal if it isn't already running (see README "Verify"): | ||
|
|
||
| ```bash | ||
| kubectl -n scope port-forward svc/scope-portal 5100:80 | ||
| ``` | ||
|
|
||
| Then, at <http://localhost:5100>: | ||
|
|
||
| 1. Go to the **Secrets** tab. | ||
| 2. Click **Register key**. | ||
| 3. Add your GitHub token and Anthropic API key. | ||
|
|
||
| > See [Workers](./WORKERS.md#registering-credentials-another-way) for other | ||
| > ways to register these credentials (a raw Key Vault secret, or the API | ||
| > directly). | ||
|
|
||
| ### GitHub OAuth token for the Copilot worker | ||
|
|
||
| The `coder-acp-copilot` worker needs a scopeless GitHub OAuth token (not a | ||
| personal access token). Generate one with: | ||
|
|
||
| ```bash | ||
| pnpm get-copilot-token | ||
| ``` | ||
|
|
||
| Follow the device-code prompt to approve access, then register the printed | ||
| token as the GitHub secret key in the Portal's **Secrets** tab above. | ||
|
|
||
| ## Values conventions | ||
|
|
||
| See [`values.yaml`](./values.yaml) for the full set of configurable values. | ||
| Every service section supports: | ||
|
|
||
| - `enabled` | ||
| - `image.repository`, `image.tag`, `image.registry` | ||
| - `replicas` | ||
| - `resources` | ||
|
|
||
| `image.registry` overrides `global.imageRegistry` for a single image. The | ||
| workers use this (they pull from ACR, not GHCR), as does `jobs.dbMigrate` | ||
| (which needs a separate `:builder`-tagged image — see | ||
| `templates/jobs/db-migrate.yaml`). | ||
|
|
||
| ## Image registry strategy | ||
|
|
||
| By default, the six core services (api, portal, judge, token-manager, | ||
| scheduler, report-generator) pull from `global.imageRegistry` | ||
| (`ghcr.io/microsoft` by default — public images published by | ||
| [`.github/workflows/publish-images.yml`](../../../.github/workflows/publish-images.yml)). | ||
| No registry build step is required for these. | ||
|
|
||
| If you publish your own fork's images, override the registry: | ||
|
|
||
| ```bash | ||
| helm install scope deploy/helm/scope -n scope \ | ||
| -f my-values.yaml \ | ||
| --set global.imageRegistry=ghcr.io/<your-fork-owner> | ||
| ``` | ||
|
|
||
| The two ACP worker images are different — they're never published to GHCR. | ||
| `scripts/bootstrap-workers.sh` builds and pushes them to the ACR instance | ||
| `deploy/azure/` provisions. See [Workers](./WORKERS.md). | ||
|
|
||
| ## Not covered | ||
|
|
||
| Bring your own ingress controller, TLS/cert-manager, and DNS — this chart | ||
| doesn't configure any of them. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| apiVersion: v2 | ||
| name: scope | ||
| description: >- | ||
| Helm chart for deploying the Scope application (API, Portal, Judge, Token | ||
| Manager, Scheduler, and coding-agent workers) onto an AKS cluster — pairs | ||
| with the infrastructure provisioned by deploy/azure/. | ||
| type: application | ||
| version: 0.1.0 | ||
| appVersion: "0.1.0" | ||
| home: https://microsoft.github.io/scope/ | ||
| sources: | ||
| - https://github.com/microsoft/scope | ||
| keywords: | ||
| - scope | ||
| - ai-benchmarking | ||
| - coding-agents | ||
| maintainers: | ||
| - name: Scope maintainers | ||
| url: https://github.com/microsoft/scope |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| # Scope Helm chart | ||
|
|
||
| Deploys the Scope application onto the AKS cluster that | ||
| [`deploy/azure/`](../azure/README.md) provisions. | ||
|
|
||
| | Phase | What it does | Tooling | | ||
| |---|---|---| | ||
| | 1. Infrastructure | AKS, Cosmos DB, Key Vault, Redis, Storage, ACR, networking | `deploy/azure/` (Bicep) | | ||
| | 2. Core app | api, portal, judge, token-manager, scheduler, report-generator | This chart | | ||
| | 3. Coding-agent workers | Worker images, agent registration | `scripts/bootstrap-workers.sh` | | ||
|
|
||
| This README covers phases 2 and 3. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Phase 1 infrastructure already deployed (AKS with Key Vault Secrets | ||
| Provider CSI driver + workload identity, Key Vault, Storage, Cosmos DB, | ||
| Redis, ACR). | ||
| - `helm` and `kubectl`, configured against the target cluster. | ||
|
|
||
| ## Phase 2: Deploy the core app | ||
|
|
||
| **1. Get a values file.** Deployed via `deploy/azure/`'s Bicep: | ||
|
|
||
| ```bash | ||
| ./scripts/generate-values-from-bicep.sh --resource-group <resource-group-name> | ||
| ``` | ||
|
|
||
| Otherwise, hand-author `my-values.yaml` — see [Configuration](./CONFIGURATION.md). | ||
|
|
||
| **2. Install the chart:** | ||
|
|
||
| ```bash | ||
| helm upgrade --install scope deploy/helm/scope \ | ||
| --namespace scope --create-namespace \ | ||
| -f my-values.yaml \ | ||
| --wait --timeout 5m | ||
| ``` | ||
|
|
||
| **3. Add any bring-your-own secrets** (GitHub token, Anthropic API key) | ||
| Bicep doesn't provision — see | ||
| [Bring your own secrets](./CONFIGURATION.md#bring-your-own-secrets). | ||
|
|
||
| ## Phase 3: Bring workers online | ||
|
|
||
| ACP coding-agent workers are disabled by default: | ||
|
|
||
| ```bash | ||
| ./scripts/bootstrap-workers.sh --acr <acrName> | ||
| ``` | ||
|
|
||
| Builds + pushes both worker images, enables their Deployments, and | ||
| registers both agents with the API. Register the GitHub token/Anthropic | ||
| API key they need yourself via the Portal's **Secrets** tab (or | ||
| `POST /api/v1/keys`) — see [Workers](./WORKERS.md) for flags and details. | ||
|
|
||
| ## Verify | ||
|
|
||
| ```bash | ||
| kubectl get pods -n scope | ||
|
|
||
| # No ingress yet — reach services via port-forward. | ||
| kubectl -n scope port-forward svc/api 18080:80 & | ||
| kubectl -n scope port-forward svc/scope-portal 5100:80 & | ||
|
|
||
| scope agent list --url http://localhost:18080 # or open http://localhost:5100 | ||
| ``` | ||
|
|
||
| Then try an end-to-end run — see [Run a sample scenario](./SAMPLE-SCENARIO.md). | ||
|
|
||
| ## Learn more | ||
|
|
||
| - [Configuration reference](./CONFIGURATION.md) — values.yaml conventions, | ||
| image registry strategy, secrets options | ||
| - [Workers](./WORKERS.md) — bootstrapping the ACP coding-agent workers | ||
| - [Run a sample scenario](./SAMPLE-SCENARIO.md) — seed criteria and submit | ||
| a run | ||
| - [`values.yaml`](./values.yaml) — every configurable value | ||
| - [Bicep infrastructure README](../azure/README.md) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| # Run a sample scenario | ||
|
|
||
| Once the core app and at least one [worker](./WORKERS.md) are up, try an | ||
| end-to-end run. | ||
|
|
||
| ## Why you need to seed criteria first | ||
|
|
||
| Scenarios and personas (`config/scenarios/`, `config/personas/`) aren't | ||
| server-side data — `scope run submit --scenario <path> --persona <path>` | ||
| reads them straight off disk, so there's nothing to import for those. | ||
|
|
||
| Criteria are different: they're server-side and project-scoped. Several | ||
| scenarios (the `version: v2` ones) reference reusable criteria IDs like | ||
| `has_react`/`has_typescript` instead of inlining their own, and those IDs | ||
| must exist in the target project before the judge can evaluate a run | ||
| against them. | ||
|
|
||
| This seed step isn't a Helm hook. Unlike `db-migrate`, it's project-scoped | ||
| business data, not infrastructure, and no project exists until the | ||
| `db-migrate` Job's "Initial Project" migration has run. Baking a project | ||
| assumption into the chart would surprise anyone bringing their own | ||
| projects — so run it yourself, once. | ||
|
|
||
| ## Steps | ||
|
|
||
| ```bash | ||
| kubectl -n scope port-forward svc/api 18080:80 & | ||
|
|
||
| # Find the project to seed into: the "Initial Project" created by | ||
| # db-migrate on first install, or one of your own. | ||
| scope project list --url http://localhost:18080 | ||
|
|
||
| # Import every reusable criterion. This is an upsert, so it's safe to rerun. | ||
| scope criteria import config/criteria --project <project-id> --url http://localhost:18080 | ||
|
|
||
| # Submit the simplest sample scenario. | ||
| scope run submit \ | ||
| --scenario config/scenarios/hello-world-express-v2.yaml \ | ||
| --persona config/personas/vibe-coder.yaml \ | ||
| --worker coder-acp-copilot \ | ||
| --project <project-id> \ | ||
| --url http://localhost:18080 | ||
| ``` | ||
|
|
||
| `--worker` must be an agent ID registered in [Workers](./WORKERS.md) — | ||
| check with `scope agent list --url http://localhost:18080`. | ||
|
|
||
| `hello-world-express-v2.yaml` only needs `has_azure_doc`/`has_azure_azd` | ||
| (plus their `has_iac`/`has_azure`/`has_cloud` ancestors). Importing the | ||
| whole `config/criteria` directory is simpler than cherry-picking a | ||
| dependency closure, and it covers every sample scenario, including | ||
| `react-snake-game-v2.yaml`. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.