Antalya 26.6: Expose IcebergS3 partition_key / sorting_key in system.tables

[zvonand]

Changelog category (leave one):

• Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Expose the Iceberg partition key and sorting key in system.tables for IcebergS3 and catalog-backed Iceberg tables, so a plain SELECT reveals how a table is partitioned and sorted. These columns are now populated for tables that have no data snapshot yet, including empty tables, where they previously came back as empty strings. (#1662 by @zvonand, #1874 by @il9ue).

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All with Aarch64
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

Combined port of 2 PR(s) (group auto-grp-pr-1662). Cherry-picked from #1662, #1874.

• Antalya 26.3 forward-port: Expose IcebergS3 partition_key and sorting_key in system.tables #1662 — Antalya 26.3 forward-port: Expose IcebergS3 partition_key and sorting_key in system.tables
• Antalya 26.3: Fix empty partition_key and sorting_key in system.table… #1874 — Antalya 26.3: Fix empty partition_key and sorting_key in system.table…

[github-actions]

Workflow [PR], commit [a21520f]

[zvonand]

@blau-ai

[blau-ai]

CI triage for #2141

Verdict: 0 of the failures are caused by this PR. Everything red is either a pre-existing regression-suite/build mismatch, a known-flaky stateless/stress case, or a non-blocking image-scan policy gate. The diff only touches the partition_key/sorting_key columns of system.tables (datalake path) plus Iceberg metadata helpers, and none of the failing checks trace back to that code.

Check	Class	PR-caused?
Regression iceberg_1 / iceberg_2 (aarch64 + release)	suite/build mismatch	❌ No
Regression s3_export_partition (aarch64 + release)	pre-existing (red on base)	❌ No
Stateless (amd_debug parallel / distributed-plan-s3 / arm_binary seq)	flaky	❌ No
Stress test (arm_debug)	flaky hung-check	❌ No
Grype Scan	policy gate, 0 high/critical	❌ No
PR	aggregate gate (rolls up the above)	❌ No

---

1. Regression — Iceberg (1)/(2), aarch64 + release → suite/build mismatch

Failing scenarios: swarm/swarm examples, iceberg iterator race condition, and sort key timezone/*. The decisive server-side error, seen in both iceberg_1 and iceberg_2:

Code: 115. DB::Exception: Received from localhost:9000.
DB::Exception: Unknown setting 'object_storage_cluster'. (UNKNOWN_SETTING)
(query: SELECT hostName() ... FROM s3(...) SETTINGS use_hive_partitioning=1, object_storage_cluster=...)

The clickhouse-regression suite issues queries with object_storage_cluster, a setting this 26.6.1 build doesn't have — a suite-vs-build version gap, not a code defect. Failures go through the suite's generic "unexpected exception" helper (assert False, error(r.output)), and each scenario dies in ~2s at setup, i.e. before any sorting_key value is ever compared. The Iceberg suite is also red on the base branch antalya-26.6.

⚠️ One thing worth a 30-second sanity check by you: the sort key timezone/* scenarios exercise the exact feature this PR adds. The evidence above (shared setup exception, 2s aborts, unrelated swarm/iterator scenarios failing in the same run, base-branch redness) says this is the same object_storage_cluster setup failure rather than a wrong-format regression from your diff. I couldn't pull the per-scenario expected/actual diff because it lives in the S3 report artifact, which I can't fetch from this container. If you want certainty, open the report and confirm the sort key timezone rows show the object_storage_cluster exception and not a sorting_key string mismatch:
https://altinity-build-artifacts.s3.amazonaws.com/REFs/2141/merge/a21520f0d0a2e0edfeb7b41f663dd8b674e3a04f/regression/aarch64/with_analyzer/zookeeper/without_thread_fuzzer/iceberg1/report.html

2. Regression — S3Export (partition), aarch64 + release → pre-existing

5/7 scenarios fail (sanity/basic table, export partition all, replicated partition exports local mode peer replica, no partition by, mismatched columns), all on assert r.exitcode == exitcode. These are plain MergeTree/S3 export tests — they don't touch Iceberg or the system.tables display path. The same suite is failing on the base branch antalya-26.6 head (74d55656…), so it's pre-existing. Not this PR.

3. Stateless tests → flaky

Actual FAILs across the three jobs:

• amd_debug (parallel): 00146_aggregate_function_uniq, 04033_tpc_ds_q10
• amd_debug (distributed plan, s3 storage): 01509_check_parallel_quorum_inserts_long, 02377_majority_insert_quorum_zookeeper_long
• arm_binary (sequential): 00024_random_counters

None read partition_key/sorting_key from system.tables or touch Iceberg. They're textbook-flaky categories: approximate uniq, a heavy TPC-DS query, ZooKeeper quorum-insert timing, and a random test. (The 3 BROKEN entries — 03441_deltalake_*, 04070_url_base_setting — are known-broken, listed in broken_tests.yaml.) Safe to re-run.

4. Stress test (arm_debug) → flaky hung-check

Fail [Hung check failed, possible deadlock found]
Found hung queries in processlist: {'Query':3,'InsertQuery':3,...}

The hung queries are INSERTs; the captured stacks are generic pthread_cond_wait / background-pool sleeps — not the system.tables read path this PR modifies. Classic flaky stress hung-check. Safe to re-run.

5. Grype Scan → non-blocking policy gate

Reports 0 high/critical vulnerabilities; the fail status is the image-scan policy gate on the built server image. This PR is C++ source with no new dependencies, so it can't introduce CVEs. Not PR-related.

---

Suggested next step

Nothing to fix in the diff based on CI evidence. The regression Iceberg/S3Export reds are a base-branch/suite-compat issue (object_storage_cluster setting missing from this build), and the stateless/stress reds are flaky — re-running those jobs should clear them. The only item I'd ask you to eyeball is the sort key timezone report rows (link above), just to be 100% sure they're the setup exception and not a sorting_key format mismatch, since I couldn't reach the S3 report from here.

_{🤖 Triage by @blau-ai — evidence-first from CI logs; I can't build/run ClickHouse in this container, so classifications are based on the linked run logs and base-branch comparison.}