Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 21 additions & 9 deletions 03-Azure/01-04-AI/06_AI_Governance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,29 +67,41 @@ Your lab automation deploys:
the later challenges call for.

Each challenge exercises one governance capability against these resources, using the
Citadel workshop's own Jupyter notebooks — unmodified, and already fully documented with
their own numbered sections, so the challenge files here exist to orient you and point
you at the right notebook, not to restate what's already in it.
Citadel workshop's own Jupyter notebooks — kept as close to upstream as possible, and
already fully documented with their own numbered sections, so the challenge files here
exist to orient you and point you at the right notebook, not to restate what's already
in it. (The only change made to them is in each notebook's first setup cell, so that
configuration can be read from the environment as well as from `azd`.)

## Requirements

To complete this MicroHack you'll need:

- Your lab's attendee credentials (`HackboxCredential` values) from the MicroHack
dashboard — resource group, subscription, and the Spoke Foundry/Key Vault/ACR names.
- `azd` installed, so [`setup-notebook-env.ps1`](labautomation/README.md#notebook-environment-setup)
can bridge those credentials into a local `azd` environment the (unchanged) workshop
notebooks read via `azd env get-value`.
- The Azure CLI (`az`), logged in (`az login`) against your lab's subscription/tenant.
- The workshop's Python environment set up (`uv sync` — see
[`challenges/workshop/readme.md`](challenges/workshop/readme.md) — or
`pip install -r requirements.txt` from the `challenges/workshop/` folder).
- Comfort running and reading Jupyter notebooks in VS Code.

`azd` is **optional**. The notebooks read their configuration from the environment
first, so you can simply paste your dashboard values into a `.env` file:

```bash
cd challenges/workshop
cp .env.template .env # then fill in the values from your dashboard
```

If you'd rather use the original `azd`-based workflow, install `azd` and run
[`setup-notebook-env.ps1`](labautomation/README.md#notebook-environment-setup)
instead — the notebooks fall back to `azd env get-value` for any key that isn't
already set in the environment. Both routes work; you only need one.

> [!TIP]
> **You're ready to start when** you've run `setup-notebook-env.ps1` once with your
> dashboard's credentials (see [Challenge 1, Part A](challenges/challenge-01.md)) and can
> open the first workshop notebook without any `azd env get-value` errors.
> **You're ready to start when** `challenges/workshop/.env` holds your dashboard
> values (see [Challenge 1, Part A](challenges/challenge-01.md)) and you can open the
> first workshop notebook and run its setup cell without errors.

## Challenges

Expand Down
44 changes: 36 additions & 8 deletions 03-Azure/01-04-AI/06_AI_Governance/challenges/challenge-01.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,46 @@ fragments, then prove it works through multiple API formats.

This challenge runs entirely against **Hub** resources — the APIM instance and its
backend pools sit in front of every model your governance hub serves. Before you start,
confirm your attendee credentials have been bridged into a local `azd` environment (see
Part A below); every workshop notebook reads its configuration via `azd env get-value`.
confirm your attendee credentials are available to the notebooks (see Part A below);
every workshop notebook reads its configuration from the environment, falling back to
`azd env get-value`.

## ✅ Tasks

### Part A — Confirm your notebook environment is ready (5 min)

1. Check with your facilitator (or your own setup) that
Every notebook in this MicroHack reads its settings from the environment, so do this
once and it applies to all nine challenges. Pick **either** route:

**Route 1 — `.env` file (recommended, no extra tooling):**

1. From the `challenges/workshop/` folder, copy the template:

```bash
cd challenges/workshop
cp .env.template .env
```

2. Open `.env` and paste in the matching `HackboxCredential` values from your MicroHack
dashboard. The template names the exact dashboard credential for every key.
For challenges 1-6 you only need `AZURE_RESOURCE_GROUP`, `AZURE_LOCATION`,
`AZURE_SUBSCRIPTION_ID` and `LLM_BACKEND_CONFIG`.
3. Paste `LLM_BACKEND_CONFIG` as a **single line**, exactly as the dashboard shows it —
it's a JSON document, so a stray line break will break it.

`.env` is gitignored, so your credentials stay out of source control.

**Route 2 — `azd` bridge (the original workshop workflow):**

1. Install `azd`, then run
[`setup-notebook-env.ps1`](../labautomation/README.md#notebook-environment-setup)
has already been run with your dashboard's `HackboxCredential` values.
2. If it hasn't, run it now — the workshop notebooks are **unchanged** and call
`azd env get-value` for every setting, so this step is required once before any
notebook in this MicroHack.
with your dashboard's `HackboxCredential` values. It writes a local `azd`
environment that the notebooks fall back to for any key not already set.

> [!TIP]
> To confirm you're ready, run the first setup cell of notebook 1. It prints your
> resource group, location, and the number of configured LLM backends. If it raises
> instead, the message tells you which key is missing and whether a `.env` was found.

### Part B — Run the LLM backend onboarding notebook (30 min)

Expand Down Expand Up @@ -61,7 +88,8 @@ Part A below); every workshop notebook reads its configuration via `azd env get-

| Symptom | Fix |
|---------|-----|
| `azd env get-value` errors / empty values | `setup-notebook-env.ps1` hasn't been run for your attendee credentials yet — see Part A. |
| Setup cell raises a missing-key error | That key isn't in your `.env` (and no `azd` environment supplied it) — see Part A. The error message tells you which key is missing and whether a `.env` file was found. |
| `LLM_BACKEND_CONFIG` fails to parse | It must be on a single line, pasted exactly as the dashboard shows it. A line break or a truncated copy will break the JSON. |
| Backend deployment succeeds but test calls 401/403 | APIM RBAC / managed-identity propagation can take a minute after a fresh deployment — wait and retry. |
| A model call fails with a region/quota error | Model availability is region-specific; double-check the model/SKU you configured is available in your lab's region. |

Expand Down
19 changes: 14 additions & 5 deletions 03-Azure/01-04-AI/06_AI_Governance/challenges/challenge-02.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,26 @@ right operation for each model (chat completions, embeddings, or the Responses A

This challenge deploys its own **Access Contract** (an APIM product + subscription) via
Bicep — a dynamically generated product policy with no model RBAC restriction and a
generous capacity allocation. Make sure your `setup-notebook-env.ps1` bridge is already
generous capacity allocation. Make sure your notebook configuration is already
in place (see [Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min))
before you start.

## ✅ Tasks

> [!IMPORTANT]
> **Challenge 1 must be completed first — not just its environment setup.**
> Notebook 1 deploys the APIM **policy fragments** (`set-llm-requested-model`,
> `validate-model-access`, `set-backend-pools`, `set-target-backend-pool`,
> `set-backend-authorization`, `set-llm-usage`) that this challenge's product
> policy includes with `<include-fragment>`. The lab's hub deployment
> intentionally ships only a minimal gateway, so if you jump straight here the
> Bicep deployment fails with a *"Policy fragment not found"* error.

### Part A — Confirm your notebook environment is ready (2 min)

Same prerequisite as every notebook in this MicroHack: confirm
[`setup-notebook-env.ps1`](../labautomation/README.md#notebook-environment-setup) has
been run for your attendee credentials.
Same prerequisite as every notebook in this MicroHack: confirm your attendee
credentials are in `challenges/workshop/.env` (or bridged via `azd`) — see
[Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min).

### Part B — Run the Universal LLM API validation notebook (25 min)

Expand Down Expand Up @@ -59,7 +68,7 @@ been run for your attendee credentials.

| Symptom | Fix |
|---------|-----|
| `azd env get-value` errors | `setup-notebook-env.ps1` hasn't been run — see Part A. |
| Setup cell raises a missing-key error | Your `challenges/workshop/.env` is missing that key (or you haven't run the `azd` bridge) — see [Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min). |
| A model in the catalogue fails every operation | Check whether that backend was onboarded in [Challenge 1](challenge-01.md) or is otherwise disabled in APIM. |
| Responses API calls fail for a `gpt` model | Some models don't implement the Responses API trio — the notebook's classification is a heuristic based on the model name, not a guarantee. |

Expand Down
18 changes: 13 additions & 5 deletions 03-Azure/01-04-AI/06_AI_Governance/challenges/challenge-03.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,20 @@ explore optional Key Vault and Microsoft Foundry connection integrations.

The Foundry connection name(s) this notebook prints are consumed by
[Challenge 4](challenge-04.md), so keep this notebook's output around before moving on.
As always, confirm
[`setup-notebook-env.ps1`](../labautomation/README.md#notebook-environment-setup) has
already been run.
As always, confirm your notebook configuration is in place — see
[Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min).

## ✅ Tasks

> [!IMPORTANT]
> **Challenge 1 must be completed first — not just its environment setup.**
> Notebook 1 deploys the APIM **policy fragments** (`set-llm-requested-model`,
> `validate-model-access`, `set-backend-pools`, `set-target-backend-pool`,
> `set-backend-authorization`, `set-llm-usage`) that this challenge's product
> policy includes with `<include-fragment>`. The lab's hub deployment
> intentionally ships only a minimal gateway, so if you jump straight here the
> Bicep deployment fails with a *"Policy fragment not found"* error.

### Part A — Confirm your notebook environment is ready (2 min)

Same prerequisite as every notebook in this MicroHack — see
Expand Down Expand Up @@ -64,9 +72,9 @@ Same prerequisite as every notebook in this MicroHack — see

| Symptom | Fix |
|---------|-----|
| `azd env get-value` errors | `setup-notebook-env.ps1` hasn't been run — see Part A. |
| Setup cell raises a missing-key error | Your `challenges/workshop/.env` is missing that key (or you haven't run the `azd` bridge) — see [Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min). |
| Key Vault integration fails to resolve a secret | Confirm the spoke Key Vault name resolved correctly and your identity has data-plane access (RBAC propagation can take a minute). |
| Foundry connection step fails | Confirm the spoke Foundry account/project names resolved from `azd env get-value` — re-check `setup-notebook-env.ps1` was run with the correct `SpokeAiFoundryAccountName` / `SpokeAiFoundryProjectName` values. |
| Foundry connection step fails | Confirm `SPOKE_AI_FOUNDRY_ACCOUNT_NAME` and `SPOKE_AI_FOUNDRY_PROJECT_NAME` match the `SpokeAiFoundryAccountName` / `SpokeAiFoundryProjectName` credentials on your dashboard. |
| Load test results look flat (no 429s) | Capacity limits are per-contract; make sure you're hitting the right subscription key for the contract you're testing. |

## 🚀 Go further
Expand Down
11 changes: 10 additions & 1 deletion 03-Azure/01-04-AI/06_AI_Governance/challenges/challenge-05.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,15 @@ don't need to provision anything extra beyond confirming your notebook environme

## ✅ Tasks

> [!IMPORTANT]
> **Challenge 1 must be completed first — not just its environment setup.**
> Notebook 1 deploys the APIM **policy fragments** (`set-llm-requested-model`,
> `validate-model-access`, `set-backend-pools`, `set-target-backend-pool`,
> `set-backend-authorization`, `set-llm-usage`) that this challenge's product
> policy includes with `<include-fragment>`. The lab's hub deployment
> intentionally ships only a minimal gateway, so if you jump straight here the
> Bicep deployment fails with a *"Policy fragment not found"* error.

### Part A — Confirm your notebook environment is ready (2 min)

Same prerequisite as every notebook in this MicroHack — see
Expand Down Expand Up @@ -56,7 +65,7 @@ Same prerequisite as every notebook in this MicroHack — see

| Symptom | Fix |
|---------|-----|
| `azd env get-value` errors | `setup-notebook-env.ps1` hasn't been run — see Part A. |
| Setup cell raises a missing-key error | Your `challenges/workshop/.env` is missing that key (or you haven't run the `azd` bridge) — see [Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min). |
| Cosmos DB query returns nothing / `Forbidden` | Section `5️⃣.1.1` grants your identity Cosmos DB data-plane access — re-run it and wait a minute for RBAC to propagate. |
| PII detection doesn't fire on a sample | Confirm the Azure AI Language Service instance is configured and reachable; some PII types need the exact regex/entity category the notebook defines. |
| Deanonymization returns masked text unchanged | The masking/deanonymization state must be saved via Event Hub first — confirm section `3️⃣.5` ran before attempting deanonymization. |
Expand Down
11 changes: 10 additions & 1 deletion 03-Azure/01-04-AI/06_AI_Governance/challenges/challenge-06.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,15 @@ start.

## ✅ Tasks

> [!IMPORTANT]
> **Challenge 1 must be completed first — not just its environment setup.**
> Notebook 1 deploys the APIM **policy fragments** (`set-llm-requested-model`,
> `validate-model-access`, `set-backend-pools`, `set-target-backend-pool`,
> `set-backend-authorization`, `set-llm-usage`) that this challenge's product
> policy includes with `<include-fragment>`. The lab's hub deployment
> intentionally ships only a minimal gateway, so if you jump straight here the
> Bicep deployment fails with a *"Policy fragment not found"* error.

### Part A — Confirm your notebook environment is ready (2 min)

Same prerequisite as every notebook in this MicroHack — see
Expand Down Expand Up @@ -69,7 +78,7 @@ Same prerequisite as every notebook in this MicroHack — see

| Symptom | Fix |
|---------|-----|
| `azd env get-value` errors | `setup-notebook-env.ps1` hasn't been run — see Part A. |
| Setup cell raises a missing-key error | Your `challenges/workshop/.env` is missing that key (or you haven't run the `azd` bridge) — see [Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min). |
| Test 3 (Responses API) or Test 4 (Gemini) show as skipped | These are conditional on your onboarded backends supporting that pattern — not every lab environment configures a Gemini backend. |
| Load test (Test 6) shows only `200`s, no `429`s | Capacity limits are per-contract; the load may not be high enough to trip the token bucket — increase the burst size in that cell. |

Expand Down
9 changes: 5 additions & 4 deletions 03-Azure/01-04-AI/06_AI_Governance/challenges/challenge-07.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,10 @@ questions.

**This notebook requires [Challenge 3](challenge-03.md)** to have created the
`Hub-HR-ChatAgent-DEV-LLM` BYO Gateway connection. Your lab's spoke Foundry resources and
Azure Container Registry are already provisioned — check
[`setup-notebook-env.ps1`](../labautomation/README.md#notebook-environment-setup) has
run so the notebook can resolve them.
Azure Container Registry are already provisioned — make sure `SPOKE_ACR_NAME` and
`SPOKE_ACR_LOGIN_SERVER` are set (see
[Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min))
so the notebook can resolve them.

## ✅ Tasks

Expand Down Expand Up @@ -72,7 +73,7 @@ Same prerequisite as every notebook in this MicroHack — see

| Symptom | Fix |
|---------|-----|
| `azd env get-value` errors | `setup-notebook-env.ps1` hasn't been run — see Part A. |
| Setup cell raises a missing-key error | Your `challenges/workshop/.env` is missing that key (or you haven't run the `azd` bridge) — see [Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min). |
| Connection `Hub-HR-ChatAgent-DEV-LLM` not found | Run [Challenge 3](challenge-03.md) first — it creates this Foundry connection. |
| Agent stays in a non-active state after step `5️⃣` | Container deployments can take a few minutes to come up; re-run the wait cell rather than assuming failure. |
| Step `6a` doesn't deny the disallowed request | Confirm `policies/hr-policy.yaml` was actually bundled into the container image built in step `3️⃣` — re-run `2️⃣`–`4️⃣` if the source files changed after the last build. |
Expand Down
11 changes: 9 additions & 2 deletions 03-Azure/01-04-AI/06_AI_Governance/challenges/challenge-08.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ disabled.

This notebook:

1. Resolves the agent-hosting Foundry account (the `-0` account) and project from `azd`
1. Resolves the agent-hosting Foundry account and project from the environment
no hardcoded names.
2. Temporarily enables public network access on that account so the notebook can reach
its data plane.
Expand All @@ -26,6 +26,13 @@ This notebook:
7. Calls the agent through APIM with just a subscription key — proving APIM reaches it
over its private endpoint even with public access disabled.

> [!IMPORTANT]
> Upstream, this notebook finds the agent-hosting account by looking for a backend id
> ending in `-0`. This lab names its Foundry accounts `aif-hub-*` / `aif-spoke-*`, so
> that lookup finds nothing. Set **`A2A_FOUNDRY_ACCOUNT_NAME`** in your
> `challenges/workshop/.env` to the **`A2aFoundryAccountName`** value from your
> dashboard (it's the spoke Foundry account) and the notebook resolves it directly.

## ✅ Tasks

### Part A — Confirm your notebook environment is ready (2 min)
Expand Down Expand Up @@ -71,7 +78,7 @@ Same prerequisite as every notebook in this MicroHack — see

| Symptom | Fix |
|---------|-----|
| `azd env get-value` errors | `setup-notebook-env.ps1` hasn't been run — see Part A. |
| Setup cell raises a missing-key error | Your `challenges/workshop/.env` is missing that key (or you haven't run the `azd` bridge) — see [Challenge 1, Part A](challenge-01.md#part-a--confirm-your-notebook-environment-is-ready-5-min). |
| `500` — backend "name is valid, but no data of the requested type was found" | A DNS/network error: APIM can't reach the agent's Foundry host. Confirm you're publishing through the APIM in the **same** azd environment as the agent. |
| `500` with a managed-identity error in the trace | APIM had no usable identity — section `10.3` auto-detects system- vs. user-assigned identity; if user-assigned, confirm the `client-id` was pinned. |
| `401`/`403` from the backend | The APIM managed identity lacks an Azure AI role (e.g. **Cognitive Services User**) on the agent's Foundry account — `azd up` grants this by default, but role propagation can take a few minutes. |
Expand Down
Loading