Skip to content

[improve][client] Deduplicate in-progress lookup requests also for HttpLookupService - #25017

Merged
lhotari merged 3 commits into
apache:masterfrom
lhotari:lh-perform-lookup-deduplication-also-for-http-lookups
Dec 2, 2025
Merged

[improve][client] Deduplicate in-progress lookup requests also for HttpLookupService#25017
lhotari merged 3 commits into
apache:masterfrom
lhotari:lh-perform-lookup-deduplication-also-for-http-lookups

Conversation

@lhotari

@lhotari lhotari commented Nov 25, 2025

Copy link
Copy Markdown
Member

Motivation

There's currently de-duplication logic for in-progress lookup requests in the Pulsar client when the Pulsar binary protocol is used for the serviceUrl. When using HTTP based lookup, the de-duplication is currently missing. This results in significant difference in behavior between the binary lookup and http based lookup.

Modifications

  • Add a decorator class that contains the logic for performing in-progress deduplication
  • Remove the deduplication logic from BinaryProtoLookupService
  • Use the decorator class to wrap both HttpProtoLookupService and BinaryProtoLookupService
  • Add deduplication logic for schema lookups
  • Perform schema lookups for the partitioned topic

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@lhotari lhotari added this to the 4.2.0 milestone Nov 25, 2025
@lhotari lhotari self-assigned this Nov 25, 2025
@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Nov 25, 2025
@lhotari
lhotari requested a review from Technoboy- November 25, 2025 14:14
@codecov-commenter

codecov-commenter commented Nov 25, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.32%. Comparing base (67dafa1) to head (1c78215).
⚠️ Report is 52 commits behind head on master.

Files with missing lines Patch % Lines
...InProgressDeduplicationDecoratorLookupService.java 76.40% 8 Missing and 13 partials ⚠️
...g/apache/pulsar/client/impl/HttpLookupService.java 60.00% 1 Missing and 1 partial ⚠️
...rg/apache/pulsar/client/impl/PulsarClientImpl.java 75.00% 0 Missing and 2 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##             master   #25017       +/-   ##
=============================================
+ Coverage     38.55%   74.32%   +35.76%     
- Complexity    13323    34078    +20755     
=============================================
  Files          1863     1921       +58     
  Lines        146140   150337     +4197     
  Branches      16971    17461      +490     
=============================================
+ Hits          56345   111738    +55393     
+ Misses        82155    29687    -52468     
- Partials       7640     8912     +1272     
Flag Coverage Δ
inttests 26.37% <54.40%> (-0.08%) ⬇️
systests 22.86% <64.80%> (-0.01%) ⬇️
unittests 73.84% <80.00%> (+39.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...e/pulsar/client/impl/BinaryProtoLookupService.java 79.61% <100.00%> (+10.67%) ⬆️
...a/org/apache/pulsar/client/impl/LookupService.java 75.00% <100.00%> (+25.00%) ⬆️
...g/apache/pulsar/client/impl/HttpLookupService.java 82.30% <60.00%> (+18.99%) ⬆️
...rg/apache/pulsar/client/impl/PulsarClientImpl.java 75.43% <75.00%> (+18.02%) ⬆️
...InProgressDeduplicationDecoratorLookupService.java 76.40% <76.40%> (ø)

... and 1422 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@lhotari
lhotari merged commit b37b5b3 into apache:master Dec 2, 2025
53 checks passed
lhotari added a commit that referenced this pull request Apr 17, 2026
lhotari added a commit that referenced this pull request Apr 17, 2026
… also for HttpLookupService (#25017)

(cherry picked from commit b37b5b3)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Apr 23, 2026
… also for HttpLookupService (apache#25017)

(cherry picked from commit b37b5b3)
(cherry picked from commit 470b0e6)
nodece pushed a commit to ascentstream/pulsar that referenced this pull request May 27, 2026
… also for HttpLookupService (apache#25017)

(cherry picked from commit b37b5b3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants