Skip to content

supabase integration - #472

Merged
zfoong merged 2 commits into
V1.4.4from
ahmad/supabase-integration
Oct 5, 2026
Merged

zfoong merged 2 commits into
V1.4.4from
ahmad/supabase-integration

Conversation

@ahmad-ajmal

@ahmad-ajmal ahmad-ajmal commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #470.

What this adds

A full Supabase integration: one connected account covers every project the user can reach.

  • Management API (api.supabase.com/v1): projects, organizations, SQL, migrations, edge functions, branches, secrets, logs, security/performance advisors, service config.
  • Each project's own APIs (<ref>.supabase.co): table rows (PostgREST), storage files, auth users, edge-function calls. The project's secret key is fetched through the Management API on first use and kept in memory only. Users paste one token, never a per-project key, and the agent never sees the secret key.

71 operations in 10 action sets, plus a 24-op supabase shortlist set. It follows the full-port shape (craftos_integrations/providers/supabase/) and passes scripts/verify_integration.py supabase.

Auth: a personal access token (sbp_…) today. OAuth (PKCE, refresh) is implemented and turns on automatically once SUPABASE_SHARED_CLIENT_ID / _SECRET are set: auth_type reports both only then, so the UI never offers a button that can't work. Project keys pasted by mistake (sb_secret_, sb_publishable_, anon/service_role JWTs) are rejected before any request.

Safety:

  • A read-only mode setting blocks every write client-side. SQL reads go through Supabase's read-only database role.
  • Secret values, jwt_secret and SMTP passwords are never returned.
  • Row update/delete refuse empty filters.
  • Project delete needs an explicit ref.
  • Schema changes sent through raw SQL are recorded in migration history automatically.

Changes outside the provider

File Change
app/action/action_set.py The action-set catalog now describes each set from its actions (e.g. supabase_storage: 12 actions: buckets, files, signed_url) instead of Custom action set: <name>. Shortlist sets say they're a shortlist. Applies to all 179 sets; about +1.2k tokens in the cached prompt prefix.
craftos_integrations/core/system.py Providers can suggest a name for a new account (default_alias(credential)). Never overwrites a user's rename; a taken name is skipped.
IntegrationsSettings.tsx Supabase icon. The account list now shows the nickname, with the raw identity in a tooltip. Applies to every integration.
integration_management.py, app/config.py, app/agent_base.py connect_integration hints and the OAuth client-credential wiring.

Testing

Run end to end: CraftBot from this branch, a real Supabase account, agent model gpt-5.2, scratch project Test. Everything created during testing was removed afterwards.

Result
Offline: pytest tests/integrations tests/test_action_set_catalog.py 887 passed (66 Supabase, 5 catalog, 4 core alias)
Wider suite 1,383 passed. 1 failure and 2 collection errors also occur on the base branch (macOS shortcut test, missing app.usage.pricing).
Live API: scripts/smoke_supabase.py --connected --write 58 / 62. The 4 failures are Supabase/account-side (see below).
Agent scenarios in the CraftBot UI 14 / 14 behave correctly

Agent scenarios

# Request Result
1 List my projects and whether each is healthy ✅ project + 6 service statuses
2 Create a table with RLS, add 3 rows ✅ correct types, RLS on, rows in
3 Mark one done, show open tasks ✅
4 Delete all tasks (edge) ✅ asks first; "cancel" respected, 3 rows remain (verified)
5 Private bucket + upload + 10-min link ✅ link downloads (HTTP 200); public URL refused
6 Secret + deploy edge function + invoke ✅ function reads the secret; also verified with external curl
7 Auth user with role + security check ✅ advisor finds RLS-without-policies + leaked-password protection off
8–10 Missing table / invalid project / failing logs (edge) ✅ each reported accurately, nothing invented
11 Read-only mode on, then "add a task" (edge) ✅ refused before any request; no SQL workaround; reads still work
12 Clean up everything ✅ after fix (see bug 7)
13 Create a table (re-test) ✅ recorded as migration create_notes_table_enable_rls
14 Create a project ✅ Supabase 403 for this token, explained, no retry loop

Also tested live: connect edge cases (empty, sb_secret_, sb_publishable_, JWT, fake sbp_ → real 401), pause → restore (76s / 200s), Postgres + auth config writes.

Bugs found live and fixed in this PR

  1. Valid tokens rejected: /v1/profile returns 403 for personal access tokens. Validity is now checked with /v1/organizations; the account is identified by a token fingerprint.
  2. Logs: Supabase retired logs.all (410). Queries now use the unified logs table (ClickHouse SQL, filtered by source_name).
  3. The analytics endpoints return HTTP 200 with {"error": …} in the body; this is now treated as a failure.
  4. Rows written straight after CREATE TABLE hit PostgREST's stale schema cache (PGRST205). The client now asks PostgREST to reload and retries.
  5. Migration rollback is refused on production projects. The description and guidance now undo changes with a reversing migration.
  6. Log search promised regex but did substring matching; it now uses ClickHouse match().
  7. With only the shortlist set loaded, the agent fell back to raw SQL on storage / auth / vault and told the user to use the dashboard. Fixed in the guidance plus the catalog change above; the re-test passed.
  8. The connect message said "no organizations yet" on a working account.

Not covered

  • OAuth: needs a registered Supabase OAuth app (redirect http://localhost:8765) and its id/secret.
  • Logs success path: Supabase's logs backend failed for the test project, even for SELECT 1, while other analytics worked. The integration retries once, then explains.
  • Organization-level operations (list members, create project): the test token gets 403, most likely the account's organization role. Handled with an explanatory error.
  • Branches: need a paid plan.

Screenshots

Connecting (7)

Supabase card, connected

Supabase card, connected

Connect dialog with credential help

Connect dialog with credential help

Edge case: nothing pasted

Edge case: nothing pasted

Edge case: project secret key rejected

Edge case: project secret key rejected

Edge case: publishable key rejected

Edge case: publishable key rejected

Edge case: anon/service_role JWT rejected

Edge case: anon/service_role JWT rejected

Settings: account named Test, read-only mode, max rows

Settings: account named Test, read-only mode, max rows

Agent scenarios (16)

Before connecting: agent asks how to connect

Before connecting: agent asks how to connect

Before connecting: explains only a token works

Before connecting: explains only a token works

1. Projects and health

1. Projects and health

2. Table with RLS and three rows

2. Table with RLS and three rows

3. Update and filtered read

3. Update and filtered read

4. Bulk delete asks first

4. Bulk delete asks first

4. Cancel respected

4. Cancel respected

5. Bucket, upload, signed link

5. Bucket, upload, signed link

6. Secret, deploy, invoke

6. Secret, deploy, invoke

7. Auth user and security advisors

7. Auth user and security advisors

8–10. Missing table, invalid project, failing logs

8–10. Missing table, invalid project, failing logs

11. Read-only mode refuses the write

11. Read-only mode refuses the write

11. Reads still work

11. Reads still work

12. Cleanup asks for confirmation

12. Cleanup asks for confirmation

12. Bug 7 before the fix

12. Bug 7 before the fix

12. After the fix

12. After the fix

Re-tests after fixes (2)

13. Create a table: recorded as a migration

13. Create a table: recorded as a migration

14. Create project: 403 explained

14. Create project: 403 explained

How to try it

  1. Settings → Integrations → Supabase → paste a token from supabase.com/dashboard/account/tokens.
  2. In chat: "list my Supabase projects", "create a table … in ", "upload to a bucket".
  3. Live checks without the UI: python scripts/smoke_supabase.py --connected [--project-ref <ref>] [--write]. Use a scratch project for --write.

@ahmad-ajmal
ahmad-ajmal requested a review from zfoong October 5, 2026 10:50
@ahmad-ajmal ahmad-ajmal self-assigned this Oct 5, 2026
@ahmad-ajmal ahmad-ajmal added the feature New feature or request label Oct 5, 2026
@zfoong
zfoong merged commit bba760c into V1.4.4 Oct 5, 2026
@ahmad-ajmal
ahmad-ajmal deleted the ahmad/supabase-integration branch October 12, 2026 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants