Skip to content

Fix lazy-singleton double init and publish registries as immutable snapshots (#16) - #21

Merged
kchiang6997 merged 1 commit into
IABTechLab:mainfrom
mpc00620:fix-16-singleton-init
Oct 27, 2026
Merged

kchiang6997 merged 1 commit into
IABTechLab:mainfrom
mpc00620:fix-16-singleton-init

Conversation

@mpc00620

@mpc00620 mpc00620 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #16.

Two fixes:

  1. Re-check lazy singletons under the lock. ExtractorRegistryFactory, TransformerRegistryFactory, LocalCacheRegistryFactory and ExperimentManagerFactory checked a non-volatile field outside the lock and never re-checked it inside, so concurrent first callers each built an instance and the last one won. A component holding an earlier instance then read a registry the setup task never filled. The fields are now volatile and re-checked inside the synchronized block, the same pattern AWSServiceClientFactory already uses.

  2. Publish registries as immutable snapshots. The extractor and transformer registries are filled by ModelFeatureOperatorRegistrySetupTask on a scheduler thread up to a second after init() returns, while request threads already read them through a plain HashMap. ConcurrentHashMap would fix that but adds a few ns to every lookup on the request path, so instead Registry keeps its records as an unmodifiable snapshot in a volatile field. register() is synchronized, copies the snapshot, adds the entry and publishes the copy. get() does one lookup instead of containsKey plus get (containsKey runs only on a miss, so a name registered with a null value still returns null). The factories are back on HashMap.

No public or protected signature changes. One behavior change for code outside this repo: getRecords() used to return the live backing map. It now returns the current unmodifiable snapshot, as its javadoc already said, so mutating it throws UnsupportedOperationException and a kept reference does not see later registrations. The constructor copies the map it is given. Nothing in this repo relied on either. The snapshot uses Collections.unmodifiableMap over a private HashMap rather than Map.copyOf, because Map.copyOf rejects nulls and randomizes iteration order per JVM.

Tests

  • LazySingletonConcurrencyTest releases 16 threads together on a fresh factory, 50 rounds per getter. On the old code it failed in 174-200 of 200 runs per getter. With the fix it passed 20 of 20 runs.
  • RegistryTest (new) checks the snapshot contract and runs register() while other threads call get() and iterate getRecords(). Against the old Registry, 5 of its 8 tests fail. Four are deterministic: getRecords() is mutable, an earlier snapshot changes after register(), the constructor shares the caller's map, and iterating getRecords() while registering throws ConcurrentModificationException. The threaded test also fails, with ConcurrentModificationException, in every run we observed. All 8 pass with the fix, 10 of 10 repeated runs. The visibility race itself (a reader seeing a stale or partly built map) cannot be reproduced deterministically. The fix relies on publishing a map that is never mutated through a volatile field.
  • ExtractorRegistryTest read the map it passed to the constructor to check register(). It now asserts through the registry.
  • ./gradlew clean build is green: 554 tests, checkstyle and spotbugs clean apart from existing warnings.

Benchmark (JMH 1.37, JDK 17, Graviton2 aarch64)

main ConcurrentHashMap (earlier version) snapshot (this PR)
Registry.get, 1 thread (ns) 20.5 +-0.3 26.5 +-0.7 14.1 +-0.2
Registry.get, 8 threads (ns) 23.7 +-0.9 28.3 +-1.1 15.3 +-0.7
Registry calls of one 2-model evaluate (27 gets, ns) 509 +-94 696 +-9 364 +-10
evaluate() map input, 1 model (us) 4.25 +-0.17 4.34 +-0.14 4.11 +-0.16
evaluate() map input, 2 models (us) 8.58 +-0.16 8.56 +-0.12 8.71 +-0.17

Per lookup the snapshot is about 30% cheaper than main. For evaluate() all three are within noise of each other. For 2 models the snapshot's point estimate is slightly above main in two separate runs (+1.5% and +3.8%), but both confidence intervals include zero. The micro saving is under 2% of an evaluate, which is smaller than the run-to-run spread on this host.

@mpc00620 mpc00620 changed the title Re-check lazy singletons under the lock and use ConcurrentHashMap registries Fix lazy-singleton double init and publish registries as immutable snapshots (#16) Oct 20, 2026
…apshots (IABTechLab#16)

ExtractorRegistryFactory, TransformerRegistryFactory,
LocalCacheRegistryFactory and ExperimentManagerFactory checked a
non-volatile field outside the lock and never re-checked it inside, so
concurrent first callers each built an instance and the last one won. A
component holding an earlier instance then read a registry the setup
task never filled. Make the fields volatile and re-check inside the
synchronized block, matching AWSServiceClientFactory.

The extractor and transformer registries are filled by
ModelFeatureOperatorRegistrySetupTask on a scheduler thread after init()
returns, while request threads already read them through a plain
HashMap. Registry now holds its records as an unmodifiable snapshot in a
volatile field. register() is synchronized, copies the current snapshot,
adds the entry and publishes the copy, so readers always see a complete
map without a lock. get() does one lookup on a hit instead of
containsKey plus get; containsKey runs only on a miss, so a name
registered with a null value still returns null as before. This avoids
the per-lookup cost of ConcurrentHashMap on the request path.

Behavior change for code outside this repo: getRecords() used to return
the live backing map; it now returns the current unmodifiable snapshot,
as its javadoc already said, so mutating it throws
UnsupportedOperationException and a kept reference does not see later
registrations. The constructor copies the map it is given. Nothing in
this repo relied on either. Snapshots use Collections.unmodifiableMap
over a private HashMap rather than Map.copyOf, which rejects null keys
and values and randomizes iteration order per JVM.

Add LazySingletonConcurrencyTest for each lazy getter and RegistryTest
for the snapshot contract and for register() racing get() and
iteration. ExtractorRegistryTest read the map it passed to the
constructor; it now asserts through the registry.

Fixes IABTechLab#16
@mpc00620
mpc00620 force-pushed the fix-16-singleton-init branch from 8585bd1 to 5d1fb97 Compare October 20, 2026 21:41
@kchiang6997
kchiang6997 merged commit 9d95dac into IABTechLab:main Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Java] Unsafe lazy initialization of registry singletons can lose registrations at startup

2 participants