Repository navigation
Conversation
IndexedLayers.getLayer() walked every entry of layers.idx for each jar entry being extracted, making extraction O(entries x index size). The dependency entries are listed first while the application classes are covered by a single directory entry near the end, so the most numerous entries consistently scanned the longest. Index the candidates once when the index file is read, keeping the directory candidates in a separate map that preserves index order. A file name is then resolved with a single map lookup and only the handful of directory candidates has to be scanned. For a jar with 24,823 entries and 700 dependencies this reduces the number of string comparisons from 17,179,582 to 72,946 (106.9ms to 2.5ms). With 4,173 entries it goes from 1,019,857 to 11,896 (5.4ms to 0.1ms) and with 903 entries from 70,612 to 2,426 (0.8ms to 0.1ms). Lookups resolve to the same layer as before. LayersIndex only writes a name once its whole subtree belongs to a single layer, so no indexed name is a prefix of another and at most one candidate can match, which makes the order the two kinds of candidate are consulted in immaterial. See gh-51653 Signed-off-by: Junggi Kim <kimjg2477@gmail.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
snicoll
force-pushed
the
speed-up-layer-lookup-when-extracting
branch
from
September 10, 2026 14:27
ec10e7d to
a0f0cf0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
IndexedLayers.getLayer()is called once for every entry of a layered jar whilejava -Djarmode=tools extractunpacks it, and each call walkslayers.idxfrom the top:That makes extraction
O(entries x index size). The layout of the index makes it worse than the formula suggests: dependencies are listed one line per jar and come first, while the application classes are covered by a singleBOOT-INF/classes/entry near the end. The most numerous entries therefore consistently scan the longest — for a 700 dependency jar every application class walks past all 700 dependency lines before reaching the line that matches it.This indexes the candidates once, when the index file is read, splitting them by kind:
/go into aHashMap, so a file name is resolved with a single lookup/go into aLinkedHashMapthat preserves index order, so only the handful of directory candidates has to be scannedcandidate.endsWith("/")is now evaluated once per index line instead of once per lookup.Measurements
Realistic indexes and entry lists, median of 15 runs after 40 warm-up runs, JDK 25. Comparison counts are deterministic.
The absolute saving is around 100ms on the largest jar, so this is not something most builds will notice. The point is that the cost stops growing with the product of jar size and index size.
Behaviour
Lookups resolve to the same layer as before.
The only reordering is that exact file names are now consulted before directory prefixes, where previously both kinds were interleaved in index order. That cannot change an answer, because
LayersIndex.Node.buildIndexonly writes a name once its whole subtree belongs to a single layer and otherwise recurses into the children. No indexed name can therefore be a prefix of another indexed name, and at most one candidate can match any given entry.putIfAbsentkeeps the first occurrence of a repeated name, matching the first-match-wins behaviour of the loop it replaces.Verified with:
IndexedLayersTests, unchanged, plus one added test covering a name that appears in more than one layer. That test passes against the current implementation as well, so it is a regression guard rather than a new expectationIllegalStateExceptionmessages, with no differences./gradlew :loader:spring-boot-jarmode-tools:test checkFormatMain checkFormatTest checkstyleMain checkstyleTest— 95 tests, no format or Checkstyle violations