RANGER-5720: DB patch 078 — x_audit_config table and audit server bootstrap by ramackri · Pull Request #1138 · apache/ranger

ramackri · 2026-08-06T02:23:39Z

Summary

RANGER-5720 — DB patch 078 for RANGER-5655. Adds the x_audit_config table for audit runtime settings, seeds default config rows, and creates the rangerauditserver machine user. DB-only (no Java); pairs with #1137.

Per review feedback, audit configuration lives in x_audit_config, not x_ranger_global_state. Patch 078 does not widen app_data and does not insert a RangerAuditPartitionPlan global-state row.

Changes

Patch file (all 5 backends): 078-add-x_audit_config.sql (renamed from 078-audit-partition-plan-global-state.sql)

Schema

New table x_audit_config (cfg_name, cfg_value, version; unique on cfg_name)
Matching optimized-schema DDL + seeds for fresh installs

Seed rows in x_audit_config

`cfg_name`	`cfg_value`
`ingestor.url`	`https://ranger-audit-ingestor:8765`
`service.hive.allowed.users`	`hive`
`audit.partition.plan`	`{"topic":"ranger_audits","plugins":{},"buffer":{"partitions":[1,2,3,4,5,6,7,8,9]}}`

User bootstrap

rangerauditserver portal user + x_user row
ROLE_ADMIN_AUDITOR, status=0, empty/disabled JDBC password (Oracle uses ' ' placeholder on NOT NULL column)

Backend-specific notes

Oracle upgrade: uses EXECUTE IMMEDIATE for all x_audit_config DML so PL/SQL compile succeeds before the table exists (avoids ORA-00942 on 077→078 upgrade)
SQL Server optimized schema: removes orphaned CHECK CONSTRAINT x_policy_ref_user_FK_upd_by left after patch 077 dropped audit columns on policy-ref tables

Testing

CI

GitHub Actions build-17 / docker builds: pass on branch

Docker DB E2E (local, `dev-support/ranger-docker`)

Scripts under scripts/db-test/:

Script	Purpose
`verify-patch-078.sh`	Query DB directly; 10 automated checks (see below)
`test-patch-078-docker.sh`	Fresh install only (branch admin tarball)
`test-patch-078-e2e-docker.sh`	Fresh install + 077→078 upgrade (builds master@077 and branch@078 tarballs)

How we ran upgrade tests

cd dev-support/ranger-docker
# Upgrade only (faster re-run):
SKIP_FRESH=1 ./scripts/db-test/test-patch-078-e2e-docker.sh
# Full matrix (fresh + upgrade, all docker DBs):
./scripts/db-test/test-patch-078-e2e-docker.sh

Upgrade path (077→078) simulates production upgrade:

Build admin tarball from master DB scripts @ patch 077 (pre-078 baseline)
Fresh docker stack + db_setup.py → DB at patch 077
Copy branch patch 078-add-x_audit_config.sql into container; clear patch markers
Re-run db_setup.py → applies patch 078 only
Run verify-patch-078.sh against live DB

Results (all docker backends, Aug 2026)

Backend	Fresh install	Upgrade 077→078
PostgreSQL	10/10 pass	10/10 pass
MySQL	10/10 pass	10/10 pass
Oracle	10/10 pass	10/10 pass
SQL Server	10/10 pass	10/10 pass
SQL Anywhere	SQL review only	SQL review only (no docker DB)

What 10/10 means — each check in verify-patch-078.sh:

#	Check
1	`rangerauditserver` status = `0` (JDBC disabled)
2	`rangerauditserver` password empty (Oracle: `' '` counts as empty)
3	Role = `ROLE_ADMIN_AUDITOR`
4	`x_audit_config.ingestor.url` seeded correctly
5	`x_audit_config.service.hive.allowed.users` = `hive`
6	`x_audit_config.audit.partition.plan` JSON contains `ranger_audits`
7	Partition plan JSON contains `buffer` section
8	No `RangerAuditPartitionPlan` row in `x_ranger_global_state`
9	Patch 078 marked active in `x_db_version_h`
10	`x_ranger_global_state.app_data` unchanged (`varchar`/`varchar2`/etc. — not widened)

Tests validate database state after db_setup.py; they do not start Admin UI, ingestor, or plugins.

Harness note: SQL Server upgrade baseline uses master optimized schema with a one-line fix for the orphaned FK CHECK (master bug from patch 077); branch optimized schema includes the same fix.

Pull request overview

Adds DB patch 078 across all supported Ranger databases for audit partition-plan state and machine-user setup.

Changes:

Widens app_data for partition-plan growth.
Seeds the initial audit partition plan.
Creates rangerauditserver with ROLE_ADMIN_AUDITOR.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
security-admin/db/sqlserver/patches/078-audit-partition-plan-global-state.sql	Updated as part of this pull request.
security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql	Updated as part of this pull request.
security-admin/db/sqlanywhere/patches/078-audit-partition-plan-global-state.sql	Updated as part of this pull request.
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql	Updated as part of this pull request.
security-admin/db/postgres/patches/078-audit-partition-plan-global-state.sql	Updated as part of this pull request.
security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql	Updated as part of this pull request.
security-admin/db/oracle/patches/078-audit-partition-plan-global-state.sql	Updated as part of this pull request.
security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql	Updated as part of this pull request.
security-admin/db/mysql/patches/078-audit-partition-plan-global-state.sql	Updated as part of this pull request.
security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql	Updated as part of this pull request.

Suppressed comments (2)

security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql:600

The plan is not bounded to this seed: it grows with plugin/service entries and each service's policy.download.auth.users. MySQL TEXT is limited to 65,535 bytes, so a sufficiently populated plan will fail to update even though the other backends use LOB/max types. Use at least MEDIUMTEXT and keep the upgrade and fresh-install definitions consistent.

`app_data` TEXT NULL DEFAULT NULL,

security-admin/db/mysql/patches/078-audit-partition-plan-global-state.sql:44

The plan is not bounded to this seed: it grows with plugin/service entries and each service's policy.download.auth.users. MySQL TEXT is limited to 65,535 bytes, so a sufficiently populated plan will fail to update even though the other backends use LOB/max types. Use at least MEDIUMTEXT and keep the upgrade and fresh-install definitions consistent.

            ALTER TABLE x_ranger_global_state MODIFY app_data TEXT DEFAULT NULL;

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Disable JDBC login for rangerauditserver (status=0, empty password) for SPIFFE-only Admin access; fix Oracle app_data ALTER guard; use SQL Anywhere ALTER COLUMN syntax.

… 078. Oracle stores empty string as NULL on NOT NULL PASSWORD; use a space placeholder for rangerauditserver. Remove stale x_policy_ref_user_FK_upd_by CHECK from SQL Server optimized schema (patch 077 regression).

Use LONGTEXT for app_data on MySQL so the partition plan can grow beyond TEXT limits. On Oracle upgrade, migrate VARCHAR2 app_data via add/copy/drop/ rename instead of MODIFY to avoid ORA-22858. Co-authored-by: Cursor <cursoragent@cursor.com>

mneethiraj

@ramackri - in addition to topic partition details, it might be necessary to store additional details, such as audit ingestor endpoint, allowed users per service. How about introducing a new table:

x_audit_config(cfg_name string  cfg_value string, version long);

insert into x_audit_config(cfg_name, cfg_value) values ("ingestor.url, "https://ranger-audit-ingestor:8765");
insert into x_audit_config(cfg_name, cfg_value) values ("service.hive.allowed.users", "hive");
insert into x_audit_config(cfg_name, cfg_value) values ("topic-partitions", 30);

Per review feedback, store ingestor URL, per-service allowed users, and topic partition count in x_audit_config (patch 078 + optimized schema on all DB backends). Seed defaults idempotently on fresh install and upgrade. Co-authored-by: Cursor <cursoragent@cursor.com>

Keep ingestor URL and per-service allow-list in x_audit_config; partition routing (topic, topicPartitionCount, plugins, buffer) stays in RangerAuditPartitionPlan JSON only. Drop topic-partitions from x_audit_config. Co-authored-by: Cursor <cursoragent@cursor.com>

This reverts commit 3b33734.

Store ingestor URL, topic, partition plan JSON, and per-service allow-list in x_audit_config. Revert patch 078 changes to x_ranger_global_state (no app_data widen, no RangerAuditPartitionPlan row). Keep rangerauditserver machine user setup in patch 078. Co-authored-by: Cursor <cursoragent@cursor.com>

mneethiraj

rename file 078-audit-partition-plan-global-state.sql as 078-add-x_audit_config.sql
is it necessary to add rangeradminserver user in user tables?

ramackri · 2026-08-27T05:49:03Z

rename file 078-audit-partition-plan-global-state.sql as 078-add-x_audit_config.sql

is it necessary to add rangeradminserver user in user tables?

rename file 078-audit-partition-plan-global-state.sql as 078-add-x_audit_config.sql ---> make sense we will do it
rangerauditserver will be used to invoke from Ranger audit ingestor to Ranger admin
Ingestor → Admin — X-Spiffe-Id as rangerauditserver for partition-plan download (no password).

ramackri · 2026-08-27T13:40:40Z

@ramackri - in addition to topic partition details, it might be necessary to store additional details, such as audit ingestor endpoint, allowed users per service. How about introducing a new table:
x_audit_config(cfg_name string  cfg_value string, version long);

insert into x_audit_config(cfg_name, cfg_value) values ("ingestor.url, "https://ranger-audit-ingestor:8765");
insert into x_audit_config(cfg_name, cfg_value) values ("service.hive.allowed.users", "hive");
insert into x_audit_config(cfg_name, cfg_value) values ("topic-partitions", 30);

Now we moved to x_audit_config

RANGER-5720: DB patch 078 for audit partition plan global state

90ced3a

Add patch 078 and optimized schema updates for all DB backends: seed rangerauditserver with ROLE_ADMIN_AUDITOR and register RangerAuditPartitionPlan global state name for RANGER-5655 admin-managed partition plans.

This was referenced Aug 6, 2026

RANGER-5655: Shared partition plan library, DB patch 078, plugin SPIFFE auth #1135

Closed

RANGER-5719: Shared partition-plan library and SPIFFE header auth utilities #1137

Open

RANGER-5723: Plugin SPIFFE outbound auth for audit-server destination #1139

Merged

ramackri requested review from kumaab, mneethiraj and rameeshm August 6, 2026 02:32

rameeshm requested a lite review from Copilot August 24, 2026 18:05

Copilot started reviewing on behalf of rameeshm August 24, 2026 18:05 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

ramk added 3 commits August 25, 2026 16:53

RANGER-5720: Address Copilot review on patch 078 audit DB changes.

7092596

Disable JDBC login for rangerauditserver (status=0, empty password) for SPIFFE-only Admin access; fix Oracle app_data ALTER guard; use SQL Anywhere ALTER COLUMN syntax.

Merge branch 'master' into RANGER-5720-patch

e2b8e72

rameeshm reviewed Aug 26, 2026

View reviewed changes

Comment thread security-admin/db/oracle/patches/078-audit-partition-plan-global-state.sql Outdated

Comment thread security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql Outdated

mneethiraj reviewed Aug 26, 2026

View reviewed changes

mneethiraj reviewed Aug 27, 2026

View reviewed changes

Comment thread security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql Outdated

ramk and others added 3 commits August 27, 2026 08:55

Revert "RANGER-5720: Align x_audit_config with Option A config split."

44cd426

This reverts commit 3b33734.

mneethiraj reviewed Aug 27, 2026

View reviewed changes

ramackri changed the title ~~RANGER-5720: DB patch 078 for audit partition plan global state~~ RANGER-5720: DB patch 078 — x_audit_config table and audit server bootstrap Aug 27, 2026

Navigation Menu

Uh oh!

RANGER-5720: DB patch 078 — x_audit_config table and audit server bootstrap - #1138

RANGER-5720: DB patch 078 — x_audit_config table and audit server bootstrap#1138
ramackri wants to merge 9 commits into
apache:masterfrom
ramackri:RANGER-5720-patch

ramackri commented Aug 6, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mneethiraj left a comment

Uh oh!

Uh oh!

mneethiraj left a comment

Uh oh!

ramackri commented Aug 27, 2026

Uh oh!

ramackri commented Aug 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

ramackri commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Testing

CI

Docker DB E2E (local, dev-support/ranger-docker)

Related

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mneethiraj left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mneethiraj left a comment

Choose a reason for hiding this comment

Uh oh!

ramackri commented Aug 27, 2026

Uh oh!

ramackri commented Aug 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ramackri commented Aug 6, 2026 •

edited

Loading

Docker DB E2E (local, `dev-support/ranger-docker`)