[feat][pip] PIP-469: Legacy-aware topic policies backend routing and metadata-store topic policies

[BewareMyPower]

No description provided.

[BewareMyPower]

I've tested the impact on topic loading latency with this tool: https://github.com/BewareMyPower/pulsar-stress

Testing

Cluster config: 2 brokers with 2 CPUs for each broker. With only 1 CPU, the CPU throttling could affect the results.

"defaultNumberOfNamespaceBundles":"1"
"loadBalancerAutoBundleSplitEnabled":"false"
"loadBalancerSheddingEnabled":"false"
"metadataStoreBatchingEnabled":"false"

The default bundle count is 1, so I can use the admin namespaces unload <ns> -d <broker> 0x00000000_0xffffffff command to transfer the bundle ownership from broker-1 to broker-0.

Namespaces:

• With SystemTopicBasedTopicPoliciesService: public/ns1,public/ns2,public/ns3,public/ns4
• With MetadataStoreTopicPoliciesService: public/ns5,public/ns6,public/ns7,public/ns8

Each namespace has 5 topics, I ran producers for two namespace sets with the same steps:

1. Use the unload commands to transfer all bundles' owner to broker-1
2. Start the application with 100 messages per second on all topics
3. Use the unload commands to transfer all bundles' owner to broker-0
4. Observe the Created Topic logs from broker-0 for topic loading latencies

Analysis

SystemTopicBasedTopicPoliciesService:

MetadataStoreTopicPoliciesService:

Check
latencies-tracing.patch for the specific steps above.

It's not very fair because the metadata store latency contributes the most, with system topics based policies service, if the __change_event topic has not been loaded, all topics in the same namespace need to wait for the managed ledger of __change_event topic to be loaded.

Full artifacts:

2026-04-14.tar.gz

[Technoboy-]

Please go ahead for the vote thread