Bound Java Harness Fan-Out Before Virtual Threads Overload Tools

Start with the constrained dependency

A Java agent harness receives one synthetic request that expands into twelve independent tool calls. The developer replaces a small executor with a virtual-thread-per-task executor. Submission becomes easy, but the downstream resource still accepts only four calls at a time. The failure moved from a visible local queue into time spent waiting for a connection, rejection by a tool, or growth in outstanding request state. A cheaper thread is not additional dependency capacity.

Treat scheduling and admission as separate decisions. The scheduler answers where a task can wait or execute. Admission answers whether the harness is willing to own another outstanding operation. Define the resource first: active HTTP requests, database connections, open files or a synthetic tool fixture. Choose a limit for that resource and record whether the limit applies per process, per dependency or across a whole deployment. These scopes are not interchangeable.

Place the permit before the expensive allocation

The design in this guide tries to acquire a permit before submitting a task. An immediate failure becomes an explicit rejected outcome. This bounds both the admitted dependency work and the number of virtual threads created by this particular gate. Acquiring inside a submitted task can still protect the dependency, but a large number of waiting tasks may accumulate outside that boundary. Both designs need a separate bound on accepted requests and retained input bytes.

A semaphore counts permissions rather than owning a thread. Release exactly once after successful acquisition. Submission failure must return a permit because no worker took responsibility for it. Once the worker owns the permit, its finally block returns it after the protected operation and cleanup. Do not release merely because the caller stopped waiting: the operation may still occupy the resource. The lab holds all admitted calls open so these ownership rules are easy to inspect.

Use virtual threads for waiting, not as a CPU multiplier

The official virtual-thread guide describes their suitability for workloads dominated by blocking waits. They do not make a CPU-bound computation execute faster. If a tool performs substantial local parsing, compression or inference, consider that CPU budget separately. Adding more runnable computations can increase contention while making a dependency-concurrency graph look healthy.

Version matters when discussing pinning. Older advice about blocking inside synchronized code should not be repeated as a universal rule: JDK 24 changed monitor-related virtual-thread behavior. Native calls and implementation details still require investigation on the selected runtime. This article does not change synchronization primitives on a guess. It uses Java 21 APIs, compiled with a Java 21 release target, and keeps the admission argument independent of carrier-thread scheduling.

Give rejection a useful harness meaning

An admission failure is not the same as a tool exception. Record a distinct terminal state such as ADMISSION_REJECTED, with the relevant limit and queue policy. Decide whether the parent can return a partial result, defer the request or fail the operation. The harness should not immediately retry every rejected child; that can turn a protective limit into repeated pressure on the same boundary.

If waiting is required, give admission a bounded waiting policy that consumes the original request deadline. A semaphore with fairness enabled does not make all acquisition paths fair; in particular, untimed tryAcquire can barge. No fairness claim is made by the immediate-admission example. A local gate also does not constrain other replicas. A deployment-wide limit needs coordination at a boundary shared by those callers, or a conservative allocation across instances.

Test ownership instead of publishing a thread-count boast

The downloadable Java example uses a latch to keep four admitted tasks active while twelve submissions are attempted. It checks the rejected count, the observed in-flight maximum in this controlled exercise, and restoration of all permits. There are no API keys, model requests or sleeps used to simulate a performance result. A failed submission path returns its acquired permit explicitly.

In a real acceptance test, also inject interruption, a tool exception, executor shutdown and cancellation while a call is active. Check that the gate never admits more than its limit and that permits recover only after the protected work exits. Measure queue delay, accepted work, rejected work and dependency activity separately. The number of created threads is not a service-quality result. Use the browser lab to find policy mistakes, then validate the actual client implementation.

Run the Java boundary check

Use a JDK with Java 21 APIs or later. Save the original FanoutDemo.java into an empty local directory. The example uses only the JDK and synthetic fixtures; no model, network service or credentials are required.

javac --release 21 FanoutDemo.java
java -ea FanoutDemo

// Author: Sandeep Belgavi. Synthetic admission check; no tool or network calls.
import java.util.ArrayList;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

public class FanoutDemo {
    public static void main(String[] args) throws Exception {
        var permits = new Semaphore(4);
        var release = new CountDownLatch(1);
        var entered = new CountDownLatch(4);
        var active = new AtomicInteger();
        var peak = new AtomicInteger();
        var handles = new ArrayList<Future<?>>();
        int rejected = 0;
        try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
            try {
                for (int i = 0; i < 12; i++) {
                    if (!permits.tryAcquire()) { rejected++; continue; }
                    try {
                        handles.add(executor.submit(() -> {
                            int n = active.incrementAndGet();
                            peak.accumulateAndGet(n, Math::max);
                            entered.countDown();
                            try { release.await(); }
                            catch (InterruptedException e) { Thread.currentThread().interrupt(); }
                            finally { active.decrementAndGet(); permits.release(); }
                        }));
                    } catch (RejectedExecutionException e) {
                        permits.release(); // Submission failed before a worker owned it.
                        throw e;
                    }
                }
                if (!entered.await(5, TimeUnit.SECONDS)) throw new AssertionError("Workers stalled");
            } finally { release.countDown(); }
            for (var handle : handles) handle.get(5, TimeUnit.SECONDS);
        }
        if (peak.get() != 4 || rejected != 8 || permits.availablePermits() != 4)
            throw new AssertionError("Admission invariant failed");
        System.out.println("admitted=4 rejected=8 peak=4 permits_restored=4");
    }
}

Compatibility check: the examples were compiled with --release 21 and executed on local HotSpot JDK 23.0.2. This checks the demonstrated invariants on that runtime; it is not a JDK performance comparison. APIs and diagnostic descriptions are linked to the Java 25 documentation, accessed 30 October 2026. No preview API is required.

Try the companion browser lab

Open Virtual-thread admission: bound a synthetic tool burst. Predict the rejected count for a burst of twelve with four permits. Then set the dependency capacity to two without changing the permits. Explain why a semaphore can enforce the wrong limit perfectly.

Expected outcome: Twelve submissions, four permits and dependency capacity four admit four and reject eight. Unrestricted mode admits twelve and marks eight as exceeding the declared dependency capacity.

Scope: The burst is simultaneous and admitted calls remain active until submission ends. There is no waiting queue, fairness model, distributed limit, connection pool or actual Java scheduling. Dependency over-limit labels are model checks, not observed server errors.

Primary sources and evidence limits

• Java virtual-thread guide
• Semaphore API
• Executor factories
• JDK 24 monitor-pinning change

The scenarios, diagrams, arithmetic and code are original educational examples. Browser ticks are synthetic. Java fixture success is not evidence of production throughput, latency, memory use or model quality.

Views are my own and do not represent any employer, past or present.