Thread Pinning The Hidden Threat in HttpClient’s

Java 21 introduced a new built-in HTTP client API called HttpClient to replace the older HttpURLConnection. HttpClient uses an asynchronous, non-blocking approach for better performance. However, its threading model includes one controversial design choice – thread pinning. In this comprehensive guide, we’ll dig into how it works, the issues it can cause, and techniques to handle it effectively.

The Basics of HttpClient

First, a quick overview of how HttpClient works:

• HttpClient manages an internal thread pool for executing requests asynchronously.
• When you call send(), it dispatches the request to a thread from this pool:

HttpClient client = HttpClient.newHttpClient();

HttpResponse<String> response = client.send(request, BodyHandlers.ofString());

• The thread executes the request in the background while your main code continues.
• HttpClient is non-blocking – your thread is not tied up waiting for the response.

So far, this asynchronous, non-blocking usage is great for performance. Where it gets interesting is how HttpClient handles multiple requests…

The Pinning Effect Explained

Here is the key behavior to understand:

HttpClient will “pin” each request to the thread that starts executing it. This means:

• The same thread will be used for the entire lifetime of the request.
• All event callbacks will happen on that original thread.

For example:

response.onRequestSent(r -> {
  // Original thread  
});

response.bodySubscriber().onComplete(() -> {
  // Also original thread
});

No other threads will handle events for that request. This enables certain optimizations internally, but has downsides.

Why Pinning Causes Problems

Pinning requests to threads can cause several issues:

• Thread starvation – Too many slow requests could occupy all available threads, starving other requests of resources.
• Hanging – A buggy server or network issue could hang forever the thread assigned to that request.
• Reduced flexibility – Limits opportunities to optimize thread usage for event handling.

These problems become more prominent at high request volumes and with complex response processing.

Let’s look at some examples…

Code Example – Thread Starvation

Say your app needs to call multiple backend services to assemble a response. The downstream services are slow and each call takes 100-200ms.

With a small thread pool, just a few concurrent requests could occupy all available threads:

// Downstream call taking 100ms  
void slowBackendCall() {
  sleep(100); // Block thread for 100ms
}

// HttpClient with 5 thread pool
HttpClient client = HttpClient.newBuilder()
                                .executor(Executors.newFixedThreadPool(5))
                                .build();

// 3 concurrent requests
client.sendAsync(req1, resp1 -> {
  slowBackendCall();  
});

client.sendAsync(req2, resp2 -> {
  slowBackendCall(); 
});

client.sendAsync(req3, resp3 -> {
  slowBackendCall();
});

// Try sending 4th request - will be starved of threads! 
client.sendAsync(req4);

This demonstrates how pinning limits concurrency when handling slow backends.

Code Example – Hanging Requests

Now imagine one of those backend services has a bug that causes it to hang indefinitely:

// Buggy downstream call
void faultyBackendCall() {
  while (true) { 
    // Infinite loop, hangs forever
  }
}

HttpClient client = ...;

// Request that will hang
client.sendAsync(req1, resp -> {
  faultyBackendCall();
});

// Try sending another request - will hang due to pinned thread!
client.sendAsync(req2); 

No other requests can use that hung thread. This progressively blocks more of the pool over time as hanging calls accumulate.

These examples show how pinning can exacerbate resource starvation and hanging threads.

Strategies for Mitigation

There are ways to work around the limitations of thread pinning:

• Use judicious timeouts – Prevents any one call from occupying a thread for too long:

client.sendAsync(req, resp -> {
  // Limit backend call timeout 
  resp.timeout(Duration.ofSeconds(1));
});

• Increase pool size – Adds thread capacity to compensate for pinning.
• Create multiple HttpClient instances – Partitions work across multiple pools.
• Use CompletableFuture chaining – Avoids blocking threads.

However, these add complexity and don’t fully resolve the underlying design constraints.

The Future of HttpClient Threading

The Java team acknowledges thread pinning in HttpClient is not ideal. It may very well change in future versions once they find the sweet spot in the threading model.

For now, just be aware of this behavior and design around it. Stress testing will help uncover issues early. As HttpClient evolves, the threading should become more robust.

Key Takeaways

• HttpClient pins each request to a single thread for its lifetime.
• This can cause thread starvation, hanging, and reduced flexibility.
• There are mitigation strategies but they add complexity.
• The threading behavior may improve in future Java releases.
• When migrating to HttpClient, design for this model until it changes.

FAQs

Why did Java introduce this thread pinning behavior?

It enables certain internal optimizations around event handling and request processing. However, it seems the drawbacks outweigh the benefits.

How big of an issue is this in practice?

It depends on your request load, but it can become a bottleneck in high-volume applications. Stress testing helps uncover these types of threading issues.

Should I avoid HttpClient due to this?

No, it’s still the recommended modern HTTP client for Java. Just be aware of the threading behavior until it hopefully changes in the future.

What is the best workaround?

Creating multiple HttpClient instances helps partition requests across multiple pools. But increased complexity is the tradeoff.

I hope this deep dive gives you a solid understanding of HttpClient threading!

For more insightful articles and in-depth guides on JAVA, Spring, Spring Boot and related topics, visit our homepage today.