Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
2218d65
feat(deploy): add Helm chart skeleton for Scope app
duffney Oct 8, 2026
a0ff409
feat(deploy): split Helm core app onto GHCR, defer ACP workers to ACR…
duffney Oct 9, 2026
622761e
fix(helm): resolve real-deployment bugs found validating chart on AKS
duffney Oct 9, 2026
0e28bbd
fix(helm): default portal.authEnabled to false
duffney Oct 9, 2026
71db4a9
fix: restore missing scripts/get-copilot-token.ts
duffney Oct 9, 2026
6bb5924
Add Helm Deployment templates for ACP coding-agent workers
duffney Oct 9, 2026
f94998b
Add bootstrap-workers.sh to automate Phase 3 ACP worker setup
duffney Oct 9, 2026
c9613a0
Fix criteria/prompt-feature import missing projectId; doc sample-scen…
duffney Oct 9, 2026
9fe2df6
Retry blob writes on transient authorization errors; fix stale --reus…
duffney Oct 9, 2026
59e9a9a
Split the Helm chart README into a lean quickstart plus detail docs
duffney Oct 9, 2026
46caf15
Add MCP gateway sidecar, bootstrap-core script, and doc fixes
duffney Oct 12, 2026
fd9410b
Split bootstrap-core.sh into values-only generator, fix set -u array bug
duffney Oct 12, 2026
91e2863
docs(helm): tighten chart README for conciseness
duffney Oct 12, 2026
c99275f
helm: remove kv-secrets CSI mount and workload identity from ACP workers
duffney Oct 12, 2026
6b4c847
helm: give ACP workers their own worker-scoped identity
duffney Oct 12, 2026
a0243d1
scripts: remove secret bootstrap logic from bootstrap-workers.sh
duffney Oct 12, 2026
4bda1fb
fix(helm): default worker bootstrap image tag to git SHA, not latest
duffney Oct 12, 2026
d85ed7a
fix(helm): pin migrations to the app's revision; widen publish-images…
duffney Oct 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 105 additions & 0 deletions .github/workflows/publish-images.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
name: Publish core images to GHCR

# Builds and publishes the 6 core Scope app images (NOT the two ACP
# coding-agent workers, which are built/pushed to the Bicep-provisioned ACR
# as a separate step — see deploy/helm/scope/README.md "Phase 3") to GHCR,
# so the Helm chart under deploy/helm/scope/ can install the app straight
# from ghcr.io/<owner>/scope-<service> with no private registry required.
on:
push:
branches: [main]
paths:
- "apps/api/**"
- "apps/judge/**"
- "apps/token-manager/**"
- "apps/scheduler/**"
- "apps/portal/**"
- "apps/workers/report-generator/**"
- "packages/shared/**"
- "packages/telemetry/**"
- "packages/db-migrations/**"
Comment thread
duffney marked this conversation as resolved.
Resolved
- "tsconfig.base.json"
- "pnpm-workspace.yaml"
- "pnpm-lock.yaml"
- ".github/workflows/publish-images.yml"
workflow_dispatch: {}

permissions:
contents: read
packages: write

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false

env:
REGISTRY: ghcr.io

jobs:
publish:
# Fork PRs never trigger this workflow (push to main / manual dispatch
# only), so there's no need for the ci.yml TRUSTED_CODE gate here.
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
image:
- name: scope-api
dockerfile: apps/api/Dockerfile
- name: scope-judge
dockerfile: apps/judge/Dockerfile
- name: scope-token-manager
dockerfile: apps/token-manager/Dockerfile
- name: scope-scheduler
dockerfile: apps/scheduler/Dockerfile
- name: scope-portal
dockerfile: apps/portal/Dockerfile
- name: scope-report-generator
dockerfile: apps/workers/report-generator/Dockerfile
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4.0.0

- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push (production image)
uses: docker/build-push-action@v6
with:
context: .
file: ${{ matrix.image.dockerfile }}
push: true
tags: |
${{ env.REGISTRY }}/${{ github.repository_owner }}/${{ matrix.image.name }}:latest
${{ env.REGISTRY }}/${{ github.repository_owner }}/${{ matrix.image.name }}:${{ github.sha }}
build-args: |
GIT_COMMIT=${{ github.sha }}
BUILD_TIME=${{ github.event.head_commit.timestamp || '' }}
GIT_BRANCH=${{ github.ref_name }}
cache-from: type=gha,scope=${{ matrix.image.name }}
cache-to: type=gha,mode=min,scope=${{ matrix.image.name }}

# The db-migrate Helm Job runs migrations via the api Dockerfile's
# `builder` stage (full TS source + tsx) — the slim production image
# above only ships packages/db-migrations/dist, which mongo-migrate-ts's
# CLI can't use (see deploy/helm/scope/templates/jobs/db-migrate.yaml).
- name: Build and push (api builder stage, for db-migrate job)
if: matrix.image.name == 'scope-api'
uses: docker/build-push-action@v6
with:
context: .
file: apps/api/Dockerfile
target: builder
push: true
tags: |
${{ env.REGISTRY }}/${{ github.repository_owner }}/scope-api:builder
${{ env.REGISTRY }}/${{ github.repository_owner }}/scope-api:builder-${{ github.sha }}
cache-from: type=gha,scope=scope-api-builder
cache-to: type=gha,mode=min,scope=scope-api-builder
2 changes: 2 additions & 0 deletions apps/cli/src/__snapshots__/cli-commands.test.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,7 @@ exports[`CLI command registration > full command tree matches snapshot 1`] = `
"import": {
"options": [
"--dry-run",
"--project",
"--url",
],
},
Expand Down Expand Up @@ -533,6 +534,7 @@ exports[`CLI command registration > full command tree matches snapshot 1`] = `
"import": {
"options": [
"--dry-run",
"--project",
"--url",
],
},
Expand Down
3 changes: 3 additions & 0 deletions apps/cli/src/commands/criteria.ts
Original file line number Diff line number Diff line change
Expand Up @@ -427,8 +427,10 @@ criteria
.argument("<path>", "Path to a .yaml file or a directory of .yaml files")
.option("--dry-run", "Preview what would be imported without sending to API")
.option("-u, --url <url>", "API base URL", getDefaultApiUrl())
.option("--project <id>", "Project ID for scoped operations (overrides SCOPE_PROJECT and the saved selection)")
.action(async (inputPath: string, options) => {
try {
const projectId = requireProjectId(options.project);
const absPath = resolve(process.env.INIT_CWD || process.cwd(), inputPath);
if (!existsSync(absPath)) {
console.error(errorText(`Path not found: ${absPath}`));
Expand Down Expand Up @@ -512,6 +514,7 @@ criteria
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ criteria: allCriteria }),
projectId,
});

if (!response.ok) {
Expand Down
3 changes: 3 additions & 0 deletions apps/cli/src/commands/prompt-feature.ts
Original file line number Diff line number Diff line change
Expand Up @@ -242,8 +242,10 @@ promptFeature
.argument("<path>", "Path to a .yaml file or a directory of .yaml files")
.option("--dry-run", "Preview what would be imported without sending to API")
.option("-u, --url <url>", "API base URL", getDefaultApiUrl())
.option("--project <id>", "Project ID for scoped operations (overrides SCOPE_PROJECT and the saved selection)")
.action(async (inputPath: string, options) => {
try {
const projectId = requireProjectId(options.project);
const absPath = resolve(inputPath);
if (!existsSync(absPath)) {
console.error(errorText(`Path not found: ${absPath}`));
Expand Down Expand Up @@ -320,6 +322,7 @@ promptFeature
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ features: allFeatures }),
projectId,
});

if (!response.ok) {
Expand Down
6 changes: 6 additions & 0 deletions deploy/helm/scope/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.git/
.gitignore
*.swp
*.bak
*.tmp
.DS_Store
74 changes: 74 additions & 0 deletions deploy/helm/scope/CONFIGURATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Configuration reference

Details on values.yaml conventions and image registries. For the install
steps, see the [README](./README.md).

## Bring your own secrets

Port-forward the Portal if it isn't already running (see README "Verify"):

```bash
kubectl -n scope port-forward svc/scope-portal 5100:80
```

Then, at <http://localhost:5100>:

1. Go to the **Secrets** tab.
2. Click **Register key**.
3. Add your GitHub token and Anthropic API key.

> See [Workers](./WORKERS.md#registering-credentials-another-way) for other
> ways to register these credentials (a raw Key Vault secret, or the API
> directly).

### GitHub OAuth token for the Copilot worker

The `coder-acp-copilot` worker needs a scopeless GitHub OAuth token (not a
personal access token). Generate one with:

```bash
pnpm get-copilot-token
```

Follow the device-code prompt to approve access, then register the printed
token as the GitHub secret key in the Portal's **Secrets** tab above.

## Values conventions

See [`values.yaml`](./values.yaml) for the full set of configurable values.
Every service section supports:

- `enabled`
- `image.repository`, `image.tag`, `image.registry`
- `replicas`
- `resources`

`image.registry` overrides `global.imageRegistry` for a single image. The
workers use this (they pull from ACR, not GHCR), as does `jobs.dbMigrate`
(which needs a separate `:builder`-tagged image — see
`templates/jobs/db-migrate.yaml`).

## Image registry strategy

By default, the six core services (api, portal, judge, token-manager,
scheduler, report-generator) pull from `global.imageRegistry`
(`ghcr.io/microsoft` by default — public images published by
[`.github/workflows/publish-images.yml`](../../../.github/workflows/publish-images.yml)).
No registry build step is required for these.

If you publish your own fork's images, override the registry:

```bash
helm install scope deploy/helm/scope -n scope \
-f my-values.yaml \
--set global.imageRegistry=ghcr.io/<your-fork-owner>
```

The two ACP worker images are different — they're never published to GHCR.
`scripts/bootstrap-workers.sh` builds and pushes them to the ACR instance
`deploy/azure/` provisions. See [Workers](./WORKERS.md).

## Not covered

Bring your own ingress controller, TLS/cert-manager, and DNS — this chart
doesn't configure any of them.
19 changes: 19 additions & 0 deletions deploy/helm/scope/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
apiVersion: v2
name: scope
description: >-
Helm chart for deploying the Scope application (API, Portal, Judge, Token
Manager, Scheduler, and coding-agent workers) onto an AKS cluster — pairs
with the infrastructure provisioned by deploy/azure/.
type: application
version: 0.1.0
appVersion: "0.1.0"
home: https://microsoft.github.io/scope/
sources:
- https://github.com/microsoft/scope
keywords:
- scope
- ai-benchmarking
- coding-agents
maintainers:
- name: Scope maintainers
url: https://github.com/microsoft/scope
79 changes: 79 additions & 0 deletions deploy/helm/scope/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Scope Helm chart

Deploys the Scope application onto the AKS cluster that
[`deploy/azure/`](../azure/README.md) provisions.

| Phase | What it does | Tooling |
|---|---|---|
| 1. Infrastructure | AKS, Cosmos DB, Key Vault, Redis, Storage, ACR, networking | `deploy/azure/` (Bicep) |
| 2. Core app | api, portal, judge, token-manager, scheduler, report-generator | This chart |
| 3. Coding-agent workers | Worker images, agent registration | `scripts/bootstrap-workers.sh` |

This README covers phases 2 and 3.

## Prerequisites

- Phase 1 infrastructure already deployed (AKS with Key Vault Secrets
Provider CSI driver + workload identity, Key Vault, Storage, Cosmos DB,
Redis, ACR).
- `helm` and `kubectl`, configured against the target cluster.

## Phase 2: Deploy the core app

**1. Get a values file.** Deployed via `deploy/azure/`'s Bicep:

```bash
./scripts/generate-values-from-bicep.sh --resource-group <resource-group-name>
```

Otherwise, hand-author `my-values.yaml` — see [Configuration](./CONFIGURATION.md).

**2. Install the chart:**

```bash
helm upgrade --install scope deploy/helm/scope \
--namespace scope --create-namespace \
-f my-values.yaml \
--wait --timeout 5m
```

**3. Add any bring-your-own secrets** (GitHub token, Anthropic API key)
Bicep doesn't provision — see
[Bring your own secrets](./CONFIGURATION.md#bring-your-own-secrets).

## Phase 3: Bring workers online

ACP coding-agent workers are disabled by default:

```bash
./scripts/bootstrap-workers.sh --acr <acrName>
```

Builds + pushes both worker images, enables their Deployments, and
registers both agents with the API. Register the GitHub token/Anthropic
API key they need yourself via the Portal's **Secrets** tab (or
`POST /api/v1/keys`) — see [Workers](./WORKERS.md) for flags and details.

## Verify

```bash
kubectl get pods -n scope

# No ingress yet — reach services via port-forward.
kubectl -n scope port-forward svc/api 18080:80 &
kubectl -n scope port-forward svc/scope-portal 5100:80 &

scope agent list --url http://localhost:18080 # or open http://localhost:5100
```

Then try an end-to-end run — see [Run a sample scenario](./SAMPLE-SCENARIO.md).

## Learn more

- [Configuration reference](./CONFIGURATION.md) — values.yaml conventions,
image registry strategy, secrets options
- [Workers](./WORKERS.md) — bootstrapping the ACP coding-agent workers
- [Run a sample scenario](./SAMPLE-SCENARIO.md) — seed criteria and submit
a run
- [`values.yaml`](./values.yaml) — every configurable value
- [Bicep infrastructure README](../azure/README.md)
52 changes: 52 additions & 0 deletions deploy/helm/scope/SAMPLE-SCENARIO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Run a sample scenario

Once the core app and at least one [worker](./WORKERS.md) are up, try an
end-to-end run.

## Why you need to seed criteria first

Scenarios and personas (`config/scenarios/`, `config/personas/`) aren't
server-side data — `scope run submit --scenario <path> --persona <path>`
reads them straight off disk, so there's nothing to import for those.

Criteria are different: they're server-side and project-scoped. Several
scenarios (the `version: v2` ones) reference reusable criteria IDs like
`has_react`/`has_typescript` instead of inlining their own, and those IDs
must exist in the target project before the judge can evaluate a run
against them.

This seed step isn't a Helm hook. Unlike `db-migrate`, it's project-scoped
business data, not infrastructure, and no project exists until the
`db-migrate` Job's "Initial Project" migration has run. Baking a project
assumption into the chart would surprise anyone bringing their own
projects — so run it yourself, once.

## Steps

```bash
kubectl -n scope port-forward svc/api 18080:80 &

# Find the project to seed into: the "Initial Project" created by
# db-migrate on first install, or one of your own.
scope project list --url http://localhost:18080

# Import every reusable criterion. This is an upsert, so it's safe to rerun.
scope criteria import config/criteria --project <project-id> --url http://localhost:18080

# Submit the simplest sample scenario.
scope run submit \
--scenario config/scenarios/hello-world-express-v2.yaml \
--persona config/personas/vibe-coder.yaml \
--worker coder-acp-copilot \
--project <project-id> \
--url http://localhost:18080
```

`--worker` must be an agent ID registered in [Workers](./WORKERS.md) —
check with `scope agent list --url http://localhost:18080`.

`hello-world-express-v2.yaml` only needs `has_azure_doc`/`has_azure_azd`
(plus their `has_iac`/`has_azure`/`has_cloud` ancestors). Importing the
whole `config/criteria` directory is simpler than cherry-picking a
dependency closure, and it covers every sample scenario, including
`react-snake-game-v2.yaml`.
Loading
Loading