Skip to content

Speed up layer lookups when extracting a layered jar - #51653

Merged
snicoll merged 1 commit into
spring-projects:mainfrom
JunggiKim:speed-up-layer-lookup-when-extracting
Sep 10, 2026
Merged

snicoll merged 1 commit into
spring-projects:mainfrom
JunggiKim:speed-up-layer-lookup-when-extracting

Conversation

@JunggiKim

Copy link
Copy Markdown
Contributor

IndexedLayers.getLayer() is called once for every entry of a layered jar while java -Djarmode=tools extract unpacks it, and each call walks layers.idx from the top:

for (Map.Entry<String, List<String>> entry : this.layers.entrySet()) {
	for (String candidate : entry.getValue()) {
		if (candidate.equals(name) || (candidate.endsWith("/") && name.startsWith(candidate))) {
			return entry.getKey();
		}
	}
}

That makes extraction O(entries x index size). The layout of the index makes it worse than the formula suggests: dependencies are listed one line per jar and come first, while the application classes are covered by a single BOOT-INF/classes/ entry near the end. The most numerous entries therefore consistently scan the longest — for a 700 dependency jar every application class walks past all 700 dependency lines before reaching the line that matches it.

This indexes the candidates once, when the index file is read, splitting them by kind:

  • names that do not end in / go into a HashMap, so a file name is resolved with a single lookup
  • names that end in / go into a LinkedHashMap that preserves index order, so only the handful of directory candidates has to be scanned

candidate.endsWith("/") is now evaluated once per index line instead of once per lookup.

Measurements

Realistic indexes and entry lists, median of 15 runs after 40 warm-up runs, JDK 25. Comparison counts are deterministic.

Entries Dependencies Before After String comparisons
903 80 0.76ms 0.05ms 70,612 → 2,426
4,173 250 5.36ms 0.13ms 1,019,857 → 11,896
24,823 700 106.92ms 2.51ms 17,179,582 → 72,946

The absolute saving is around 100ms on the largest jar, so this is not something most builds will notice. The point is that the cost stops growing with the product of jar size and index size.

Behaviour

Lookups resolve to the same layer as before.

The only reordering is that exact file names are now consulted before directory prefixes, where previously both kinds were interleaved in index order. That cannot change an answer, because LayersIndex.Node.buildIndex only writes a name once its whole subtree belongs to a single layer and otherwise recurses into the children. No indexed name can therefore be a prefix of another indexed name, and at most one candidate can match any given entry.

putIfAbsent keeps the first occurrence of a repeated name, matching the first-match-wins behaviour of the loop it replaces.

Verified with:

  • the existing IndexedLayersTests, unchanged, plus one added test covering a name that appears in more than one layer. That test passes against the current implementation as well, so it is a regression guard rather than a new expectation
  • a differential harness running both implementations side by side over randomly generated indexes — 218,963 lookups, including the thrown IllegalStateException messages, with no differences
  • ./gradlew :loader:spring-boot-jarmode-tools:test checkFormatMain checkFormatTest checkstyleMain checkstyleTest — 95 tests, no format or Checkstyle violations

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Sep 10, 2026
@wilkinsona wilkinsona added type: enhancement A general enhancement and removed status: waiting-for-triage An issue we've not yet triaged labels Sep 10, 2026
@wilkinsona wilkinsona added this to the 4.2.x milestone Sep 10, 2026
@snicoll snicoll assigned snicoll and unassigned snicoll Sep 10, 2026
IndexedLayers.getLayer() walked every entry of layers.idx for each jar
entry being extracted, making extraction O(entries x index size). The
dependency entries are listed first while the application classes are
covered by a single directory entry near the end, so the most numerous
entries consistently scanned the longest.

Index the candidates once when the index file is read, keeping the
directory candidates in a separate map that preserves index order. A
file name is then resolved with a single map lookup and only the
handful of directory candidates has to be scanned.

For a jar with 24,823 entries and 700 dependencies this reduces the
number of string comparisons from 17,179,582 to 72,946 (106.9ms to
2.5ms). With 4,173 entries it goes from 1,019,857 to 11,896 (5.4ms to
0.1ms) and with 903 entries from 70,612 to 2,426 (0.8ms to 0.1ms).

Lookups resolve to the same layer as before. LayersIndex only writes a
name once its whole subtree belongs to a single layer, so no indexed
name is a prefix of another and at most one candidate can match, which
makes the order the two kinds of candidate are consulted in immaterial.

See gh-51653

Signed-off-by: Junggi Kim <kimjg2477@gmail.com>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@snicoll
snicoll force-pushed the speed-up-layer-lookup-when-extracting branch from ec10e7d to a0f0cf0 Compare September 10, 2026 14:27
@snicoll snicoll modified the milestones: 4.2.x, 4.2.0-M2 Sep 10, 2026
@snicoll
snicoll merged commit c966eca into spring-projects:main Sep 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: enhancement A general enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants