feat(den): centralize AI Gateway access and usage management - #5139
OmarMcAdam wants to merge 8 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
3 Skipped Deployments
|
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
Test evidence — GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension — Incomplete (visual review pending)SHA 33acec1 · engine v1 Behavioral journey: passed (1 passed, 0 failed, 0 skipped). Visual evidence: incomplete. 19 recorded assertions passed on the PR head, with no organization Gateway opt-in in the fixture. The 13 reference screenshots have no recorded automated visual validation and are not visual pass claims. Reproduce: [world] den(local) · [seed] api GET /v1/org · [seed] api GET /v1/inference-providers?scope=manageable · [seed] api GET /v1/gateway/usage-limit-policies · [seed] api POST /v1/inference-providers · [seed] api GET /v1/inference-providers/ipr_01m2twp15kek7r1fhy178m64s0/connect · web(signed in) · desktop(as member) ℹ️ ASSERTION — 1. Den policy editor: real app screenshot and DOM dimensions{"hash":"dd3b1cec0be06e52c294f0610807d71c40d08bfe23c10cce588829667d0cdbf5","width":1440,"height":1200,"viewport":{"width":1440,"height":1200,"scale":1,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 2. Den Limits policy rows: real app screenshot and DOM dimensions{"hash":"ba812a04602e7f242ec435a07bb6949ad243e7b79b9c06deeb4bc2ae2a954538","width":1440,"height":1200,"viewport":{"width":1440,"height":1200,"scale":1,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 3. Den central usage assignment form: real app screenshot and DOM dimensions{"hash":"586b363ea9d694e285e20e8efc7d1cb9a0eda4bc65130bb6c01d51c5fd69bc54","width":1440,"height":1200,"viewport":{"width":1440,"height":1200,"scale":1,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 4. Den Users and Teams assignment cards: real app screenshot and DOM dimensions{"hash":"3367c8aa257b48bc616a583ca6db0d4580885f2650ad0a6f92a3dc793b3111b5","width":1440,"height":1200,"viewport":{"width":1440,"height":1200,"scale":1,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":2}
ℹ️ ASSERTION — 5. Den OpenWork Models tab: real app screenshot and DOM dimensions{"hash":"bcd206eb35a62b2dc5da6338f5682912a717d29b9d0a746d826355234acebbe3","width":1440,"height":1200,"viewport":{"width":1440,"height":1200,"scale":1,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 6. AI Gateway tab and nested form ownershipNew/edit provider forms stay within AI Providers without saving changes. The old Gateway sidebar link is absent and its list/new/detail/edit URLs show 404 without redirecting. Central Users & Teams assigns only Usage Member; Limits inspector reports that member within allowance and the unassigned control unlimited. The legacy Models URL forwards to the OpenWork Models tab with repeated query values intact and no duplicate sidebar link or page heading. No upstream inference calls occurred.
ℹ️ ASSERTION — 7. Rendered Den assignment reaches only the intended member's real DesktopMonthly $1 hard/reset-enabled policy persisted through Den UI. Own-status identity injection did not change the control member; management requests returned 403; Desktop rendered zero used of $1.
ℹ️ ASSERTION — 8. Desktop exhausted usage: real app screenshot and DOM dimensions{"hash":"a9dc721415af79757803910a3428aacd70ca51102ede806cd2fe3dfcda0c1716","width":2360,"height":1640,"viewport":{"width":1180,"height":820,"scale":2,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 9. Gateway, not Desktop, blocks after known consumptionFirst request settled 1000000 micro-USD; second returned trusted policy HTTP 429 without a second upstream call or charge. Desktop rendered exhaustion; the unassigned control stayed unlimited.
ℹ️ ASSERTION — 10. Desktop blocked composer: real app screenshot and DOM dimensions{"hash":"f429ee4dd30fddb69b08eafd6e8ea1ebceffeacde35ddd177e40c8bb982f68e3","width":2360,"height":1640,"viewport":{"width":1180,"height":820,"scale":2,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 11. Native composer reaches the real Gateway and own status corroborates the custom notice{"engine":"v1","rejectedBefore":1,"rejectedAfter":2,"upstreamRequests":1,"nativePromptRecorded":true,"nativeAssistantErrorRecorded":true,"usedMicroUsd":1000000}
ℹ️ ASSERTION — 12. Desktop direct increase form: real app screenshot and DOM dimensions{"hash":"d22de898a6994a38781caafdd2363a6f9cb6e228d50981b0e82f4b15e227fb8b","width":2360,"height":1640,"viewport":{"width":1180,"height":820,"scale":2,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 13. Desktop pending increase: real app screenshot and DOM dimensions{"hash":"ebafa453aa5b31f3e1633a283247ef507509fcdcf588555cdf54c2657757347e","width":2360,"height":1640,"viewport":{"width":1180,"height":820,"scale":2,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 14. Den pending request row: real app screenshot and DOM dimensions{"hash":"b194d2c6947e44179da0c8cc8cf4102c90a498a776744370b5aa0b05decf9bef","width":1440,"height":1200,"viewport":{"width":1440,"height":1200,"scale":1,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":2}
ℹ️ ASSERTION — 15. Den approved history: real app screenshot and DOM dimensions{"hash":"04f83a206c0b469b0727987ceb1ec83e02fdb64e60f75c7e692e97b057b07b40","width":1440,"height":1200,"viewport":{"width":1440,"height":1200,"scale":1,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":2}
ℹ️ ASSERTION — 16. Desktop approved increase: real app screenshot and DOM dimensions{"hash":"08492fd0a333db27eaa35ea5a4cad1d55d8d76767bf37f0397cef856dfe7cd08","width":2360,"height":1640,"viewport":{"width":1180,"height":820,"scale":2,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 17. Desktop native recovery completed: real app screenshot and DOM dimensions{"hash":"3dc58fbc0a3c73567ae7d361913f65ad0401c1b888c6c7d0fca1793ac36970ca","width":2360,"height":1640,"viewport":{"width":1180,"height":820,"scale":2,"readyState":"complete","visibility":"visible","focused":true,"theme":"light"},"elements":1}
ℹ️ ASSERTION — 18. Den approval restores actual Desktop composer completion, not a direct HTTP bypass{"engine":"v1","nativeAssistantCompleted":true,"renderedAnswer":"Complete café","upstreamRequests":2,"streamed":true,"settledCostMicroUsd":1000000,"totalUsedMicroUsd":2000000,"additionalRejections":0}
ℹ️ ASSERTION — 19. Desktop request and Den approval restore the same native sessionThe member submitted a required reason through the blocked-notice dialog; Den displayed it and granted exactly 250000 micro-USD without forgiving consumption. After approval, a real Desktop composer send produced a completed native assistant and rendered answer with one streaming upstream call and a settled $1 cost. Total usage became $2, correctly exhausting the $1.25 allowance again; the control member remained unlimited.
⚪ UNVALIDATED — 20. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 1
⚪ UNVALIDATED — 21. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 3
⚪ UNVALIDATED — 22. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 5
⚪ UNVALIDATED — 23. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 7
⚪ UNVALIDATED — 24. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 9
⚪ UNVALIDATED — 25. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 13
⚪ UNVALIDATED — 26. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 16
⚪ UNVALIDATED — 27. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 19
⚪ UNVALIDATED — 28. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 21
⚪ UNVALIDATED — 29. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 23
⚪ UNVALIDATED — 30. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 25
⚪ UNVALIDATED — 31. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 27
⚪ UNVALIDATED — 32. GATEWAY-USAGE-01 admin policy blocks member Gateway calls until a reviewed 25% extension artifact 29
Test run created 2026-09-18T18:32:26.652Z · Source: |
1 participant
Summary
/dashboard/inferencelinks forward to the Models tab with repeated query parameters preserved.General availability
Retire the organization-level
gatewayDashboardrollout flag everywhere it controlled access: dashboard/migration gates, internal client capability types, admin toggle/write semantics, organization creation metadata, fixtures, and current operational documentation. Stored missing/false/malformed flag values cannot block Gateway. No opt-in or backfill is needed.For staggered-deployment compatibility, the API retains a deprecated
gatewayDashboard: trueresponse and accepts validated boolean/null admin inputs as no-ops. They are no longer feature controls. New clients do not consult the field; obsolete metadata is ignored/cleaned. API/SDK contracts were regenerated.Unchanged: admin/membership/reauthentication rules, provider/model/credential grants, usage enforcement, subscription/DPA rules, and deployment configuration.
GATEWAY_ENABLEDanddeploymentCapabilities.aiGatewayremain installation/configuration controls for deployments that may not run the Gateway service. This change does not enable inference without valid configuration and credentials.Database migration and rollout
One migration relative to
dev:0108_gateway_usage_organization_assignments.sql, plus snapshot/journal. Flag retirement adds no migration.Verification
Final head:
33acec1e3530664abffad1258cfb4ca2531b6945.pnpm evals:e2e gateway-usage-policy --local— 1 passed, 0 failed, 0 skipped in an isolated checkout. The fixture asserts a fresh organization has no Gateway opt-in override and management GETs return 200. The journey verifies central assignments, Models-tab redirect/query preservation, removed routes, nested provider forms, quota blocking, increase approval, Desktop recovery, and unauthorized-member denial using mocked inference. No real Stripe calls.Evidence scope
Focused verification, not the full repository matrix. Deployment-disabled behavior is covered by focused tests, not this browser journey. Current screenshots are synthetic-only 1440×1200 captures. The recorder contains 19 passed assertions and 13 reference screenshots without recorded automated visual validation; formal visual evidence completeness remains Incomplete.
Screenshots
Current-head Users & Teams, Limits, and OpenWork Models, with the top-level AI Gateway navigation.