[CXH-2723] - Add Unity Catalog table and volume sync - Databricks Connector

[mateoHernandez123]

Description

• Bug fix
• New feature

Last of four stacked PRs for Unity Catalog (CXH-2723). Tables and volumes are the data assets an access review is about. They are siblings under a schema.

flowchart TD
  account[account]
  metastore[metastore]
  catalog[catalog]
  schema[schema]
  table[table]
  volume[volume]
  account --> metastore --> catalog --> schema
  schema --> table
  schema --> volume
  style table fill:#1f6feb,color:#fff,stroke:#1f6feb
  style volume fill:#1f6feb,color:#fff,stroke:#1f6feb

Loading

This PR delivers the blue nodes. #70, #71, then #72 merge first.

Sync:

• Tables (table, NEW) — child of schema. Unity Catalog uses one table securable for tables, views, materialized views, metric views and streaming tables; table_type picks the privilege set. Entitlements are per instance.
• Volumes (volume, NEW) — child of schema, GET /api/2.1/unity-catalog/volumes. Static privilege set plus a read-only owner.
• Types above — unchanged from [CXH-2720] - Add Unity Catalog client, access-path routing, and metastore sync - Databricks Connector #70–[CXH-2722] - Add Unity Catalog schema sync - Databricks Connector #72

flowchart TD
  schema[schema]
  table[table securable]
  volume[volume]
  schema --> table
  schema --> volume
  table --> managed[MANAGED / EXTERNAL]
  table --> view[VIEW / METRIC_VIEW]
  table --> mv[MATERIALIZED_VIEW]
  table --> st[STREAMING_TABLE]
  table --> empty[missing or unrecognised table_type]
  managed --> tpriv[table privileges]
  view --> vpriv["view set; ALL_PRIVILEGES is not requestable"]
  mv --> mvpriv[view set + REFRESH]
  st --> stpriv["SELECT, READ_METADATA, APPLY_TAG, MANAGE, ALL_PRIVILEGES, REFRESH, MODIFY"]
  empty --> anypriv[union of every variant]

Loading

Provisioning:

• Grant/Revoke table privileges — the requestable set follows table_type, so ConductorOne does not offer an entitlement the grants API rejects.
  • VIEW / METRIC_VIEW: the view set. ALL_PRIVILEGES is emitted when Databricks already has it and is not requestable. Grant rejects that write before any PATCH. Revoke still removes it.
  • MATERIALIZED_VIEW: the view set plus REFRESH.
  • STREAMING_TABLE: SELECT, READ_METADATA, APPLY_TAG, MANAGE, ALL_PRIVILEGES, REFRESH and MODIFY. Not the table DML trio (DELETE / INSERT / UPDATE).
  • MANAGED / EXTERNAL: the table set, which does not include REFRESH.
  • Missing or unrecognised table_type: union of every variant, so a held REFRESH is not dropped.
• Grant/Revoke volume privileges — same PATCH path and echo-confirmed idempotency (GrantAlreadyExists / GrantAlreadyRevoked).
• Owner is read-only. A revoke whose table, volume, schema or catalog Databricks reports as gone (TABLE_DOES_NOT_EXIST, VOLUME_DOES_NOT_EXIST, SCHEMA_DOES_NOT_EXIST, CATALOG_DOES_NOT_EXIST) is GrantAlreadyRevoked. Any other 404 still fails.

Auth:

Unchanged from #71 and #72. A catalog without full grant visibility is skipped, and a schema is listed only when that listing is complete. Tables and volumes are listed through that same workspace. information_schema is skipped.

Architecture highlights:

• Table listing uses omit_columns and omit_properties. omit_username is not sent: it also drops owner.
• The tables endpoint rejects max_results above 50; the client clamps.
• Table and volume Grants use the same principal path as [CXH-2722] - Add Unity Catalog schema sync - Databricks Connector #72: full account SCIM only when assignments exist. An owner-only page uses quoted Find* filters. A 400 on that filter leaves the owner unresolved and does not fail the sync.

Useful links:

• Linear ticket CXH-2723 — table and volume sync
• List tables
• List volumes
• Unity Catalog privileges

[linear-code]

CXH-2723

[FeliLucero1]

Did a pass on this one. Main things: the per-table perf (the stack-wide one actually bites here), the List/Revoke 404 handling copied from #72, and the table_type mapping dropping REFRESH / ALL_PRIVILEGES grants from the sync. The rest are nits.

Comments tagged (out of scope) are inherited or stack-wide, so treat them as FYI, not blocking for this PR. Build/vet/tests pass locally.

[FeliLucero1]

[FeliLucero1]

Round 2. Thanks for the fixes, most threads look good. I double-checked the List 404: it keeps codes.NotFound through the wrap, so the SDK warns and skips instead of failing the sync. 👍

New: the view ALL_PRIVILEGES entitlement is grantable but always fails (:136), plus a couple of small ones. I replied on the perf, revoke and streaming threads too.

[sergiocorral-conductorone]

Automated review

drafted 2 → confirmed 1, dropped 1, added 0 (verifier: gpt-5.3-codex-high)

[FeliLucero1]

Round 3. The perf change looks good overall; one regression in the new owner lookup below. The round-2 comments (view ALL_PRIVILEGES, parent codes, default branch, EqualFold, streaming MODIFY) are still open.