Skip to content

SOLR-18051 Fix double registration of phaser and semaphore in HttpJettySolrClient#3992

Merged
dsmiley merged 3 commits intoapache:mainfrom
jvanneman:SOLR-18051-fix-double-registration-bug
Jan 25, 2026
Merged

SOLR-18051 Fix double registration of phaser and semaphore in HttpJettySolrClient#3992
dsmiley merged 3 commits intoapache:mainfrom
jvanneman:SOLR-18051-fix-double-registration-bug

Conversation

@jvanneman
Copy link
Contributor

@jvanneman jvanneman commented Dec 30, 2025

https://issues.apache.org/jira/browse/SOLR-18051

Description

PR 2402 added support for CompletableFuture which accidentally introduced a double registration bug in HttpJettySolrClient by calling onRequestQueued/onComplete twice with asyncListener.queuedListener/completeListener. The registration is added once in the makeRequest helper inside of decorateRequest:

    if (isAsync) {
      req.onRequestQueued(asyncTracker.queuedListener);
      req.onComplete(asyncTracker.completeListener);
    }

and a second time below that during the following sequence:

    mrrv.request
        .onRequestQueued(asyncTracker.queuedListener)
        .onComplete(asyncTracker.completeListener)

This introduces three undesirable scenarios: 

  1. Potential deadlock due to a surge in requests that all acquire 1 permit and cannot acquire a second
  2. Unnecessary system throttling when a request acquires a single permit but can't acquire a second due to in flight requests and new requests that arrived after the first token acquisition but before the second.
  3. Half the number of permits available

Solution

Remove the duplicate registration.

Tests

This is a difficult bug to reproduce in a test. One option that I tested and works is to expose the permits as a configurable value via system properties and make this an instance level configuration. If that's something we'd want to do it's straightforward to write a test that sets this value with the builderwithMaxAsyncRequests(1) and causes a deadlock.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide
  • I have added a changelog entry for my change

@jvanneman jvanneman force-pushed the SOLR-18051-fix-double-registration-bug branch from 13299a5 to 12f24da Compare December 30, 2025 16:04
@dsmiley dsmiley requested a review from jdyer1 December 31, 2025 04:19
@dsmiley
Copy link
Contributor

dsmiley commented Dec 31, 2025

No need for a test but I appreciate you looked into the matter!

@jvanneman jvanneman force-pushed the SOLR-18051-fix-double-registration-bug branch from 12f24da to af6b33f Compare December 31, 2025 16:48
@@ -0,0 +1,8 @@
# See https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc
title: Fix phaser/semaphore double registration bug in HttpJettySolrClient
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we speak in words end-users might understand (who don't know the internals of these classes). Like what impact does this have on people. The commit message can get into nitty gritty.

Copy link
Contributor Author

@jvanneman jvanneman Jan 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated! Github complained about merge conflicts so i pulled in the latest changes with a rebase, this seemed better for a clean git history but does require a force push which you mentioned in my other PR not to do. please let me know if i should do a merge commit next time, the clean git history seemed better

One of the tests was failing in the last build but passes locally for me, i don't believe these are related to my change

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please merge not rebase (thus avoid force-push).
We want our primary branches (e.g. main, branch_10x, ...) to have linear history in general, but in-progress feature branches -- no (doesn't matter). But I'd like to hear your point of view if you see that there's some value in it. I think there's value in the feature branch reflecting the reality of literally what you did (thus the merge). And of course, not resetting the PR review state.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense, i think as long as the project is consistent either are fine. i have a slight preference towards a rebase because when reviewing PRs myself i will pull down the branch and run a git log to see what the author has done and it's a little easier for me when all of their commits are stacked on top. but both are workable, will merge next time

…uced a double registration bug in HttpJettySolrClient by calling onRequestQueued/onComplete twice with asyncListener.queuedListener/completeListener

This introduces three undesirable scenarios:

Potential deadlock due to a surge in requests that all acquire 1 permit and cannot acquire a second
 Unnecessary system throttling when a request acquires a single permit but can't acquire a second due to in flight requests and new requests that arrived after the first token acquisition but before the second.
Half the number of permits available
@jvanneman jvanneman force-pushed the SOLR-18051-fix-double-registration-bug branch from 48606cf to 83fa10d Compare January 24, 2026 17:48
Co-authored-by: David Smiley <dsmiley@apache.org>
@dsmiley dsmiley merged commit 8ec69db into apache:main Jan 25, 2026
4 of 5 checks passed
dsmiley pushed a commit that referenced this pull request Jan 25, 2026
…3992)

Improve HttpJettySolrClient.requestAsync (used in sharded/distributed-search and more) to increase throughput and prevent a rare deadlock.

Fix double registration of phaser and semaphore in HttpJettySolrClient, which had detrimental effects.
dsmiley pushed a commit that referenced this pull request Jan 25, 2026
…3992)

Improve HttpJettySolrClient.requestAsync (used in sharded/distributed-search and more) to increase throughput and prevent a rare deadlock.

Fix double registration of phaser and semaphore in HttpJettySolrClient, which had detrimental effects.

(cherry picked from commit 8ec69db)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants