Keep Bounded Java Executor Queues From Hiding Harness Overload

A fixed worker count is not a bound on accepted work

A harness submits small parsing and validation jobs to a fixed executor. Under ordinary load the queue is short. During a burst the worker count stays stable, but accepted tasks keep accumulating. Each queued job retains a request, inputs and a completion path. A stable thread graph can therefore coexist with rising memory use and worsening response time.

Executors.newFixedThreadPool uses an unbounded shared queue. That choice limits active workers, not the number of waiting tasks. The first design question is how much work the harness is willing to retain, and for how long. A bounded queue makes one part of that policy explicit. It still needs limits on each task's retained payload, because four large jobs can consume more memory than forty small ones.

Understand the executor admission order

ThreadPoolExecutor distinguishes core size, maximum size and queue behavior. Broadly, it creates workers up to the core size, then tries to queue work, and only expands toward the maximum when queueing cannot accept the task. A large or unbounded queue can prevent the maximum pool size from providing the expansion a developer expected. Read the chosen constructor as a policy, not just a collection of numbers.

The downloadable exercise deliberately sets core and maximum to two and uses an ArrayBlockingQueue of capacity two. Two workers are held by a latch, two more jobs are queued, and the fifth submission is rejected. This controlled state makes the bound observable without relying on arbitrary sleeps or arrival timing. It does not model an elastic pool or prove a throughput target.

Choose rejection as part of the harness contract

AbortPolicy throws a RejectedExecutionException at submission. The caller can translate that into a visible overload outcome before a child Future is returned. CallerRunsPolicy can slow submission by running rejected work in the submitting thread, but that also changes where the work executes. It may block an event loop, request thread or lock owner. It is not a universal backpressure solution.

Silent discard is especially dangerous for a harness expecting a terminal result from every child. Depending on how a task is submitted, discarding it can leave a caller waiting on a completion that never arrives. The example uses explicit rejection and checks every accepted task at shutdown. Production code must also define behavior when submission races with executor shutdown.

Relate queue depth to the original deadline

Queue capacity is not a latency guarantee. In a simple fixed-service-time picture, a queue of eight tasks behind two workers represents several service waves before the last task even starts. Variability, retries and nonuniform work make the actual delay harder to predict. Record queue delay separately from execution duration so the harness does not call a slow queue a slow tool.

If a queued task is already outside its parent deadline, starting it wastes capacity and can create a late effect. Check the remaining budget before submission and again when execution begins. An executor queue alone does not remove expired tasks automatically. Cancellation, queue removal and transport timeouts are distinct implementation responsibilities. Keep overload rejection separate from deadline expiration in both logs and evaluation.

Run a bounded overload experiment

The browser lab supplies arrivals every tick for twenty ticks. Each tick first retires completed work, then starts queued jobs in available slots, then accepts or rejects new arrivals. This ordering is declared because it affects boundary counts. The queue is FIFO and all jobs share the selected service time. Unbounded mode is a comparison that shows retained backlog, not a recommended configuration.

Use conservation as a test oracle: submitted tasks equal completed, running, queued and rejected tasks at the observation boundary. For an accepted-only view, omit rejected tasks from both sides. Then inspect sustained overload, an almost-empty queue and the smallest queue size. The Java fixture checks real submission/rejection behavior; the browser model teaches how the chosen policy behaves over time. Neither result is a production benchmark.

Run the Java boundary check

Use a JDK with Java 21 APIs or later. Save the original BoundedQueueDemo.java into an empty local directory. The example uses only the JDK and synthetic fixtures; no model, network service or credentials are required.

javac --release 21 BoundedQueueDemo.java
java -ea BoundedQueueDemo

// Author: Sandeep Belgavi. Deterministic bounded-queue rejection exercise.
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

public class BoundedQueueDemo {
    public static void main(String[] args) throws Exception {
        var release = new CountDownLatch(1);
        var started = new CountDownLatch(2);
        var completed = new AtomicInteger();
        var pool = new ThreadPoolExecutor(2, 2, 0, TimeUnit.MILLISECONDS,
                new ArrayBlockingQueue<>(2), new ThreadPoolExecutor.AbortPolicy());
        Runnable task = () -> {
            started.countDown();
            try { release.await(); completed.incrementAndGet(); }
            catch (InterruptedException e) { Thread.currentThread().interrupt(); }
        };
        boolean rejected = false;
        try {
            pool.execute(task); pool.execute(task);
            if (!started.await(5, TimeUnit.SECONDS)) throw new AssertionError("Workers stalled");
            pool.execute(task); pool.execute(task); // Both workers remain held.
            try { pool.execute(task); }
            catch (RejectedExecutionException expected) { rejected = true; }
            if (!rejected || pool.getQueue().size() != 2)
                throw new AssertionError("Queue bound not enforced");
        } finally {
            release.countDown(); pool.shutdown();
            if (!pool.awaitTermination(5, TimeUnit.SECONDS)) {
                pool.shutdownNow(); throw new AssertionError("Pool failed to drain");
            }
        }
        if (completed.get() != 4) throw new AssertionError("Accepted work missing");
        System.out.println("workers=2 queue=2 fifth_rejected=true completed=4");
    }
}

Compatibility check: the examples were compiled with --release 21 and executed on local HotSpot JDK 23.0.2. This checks the demonstrated invariants on that runtime; it is not a JDK performance comparison. APIs and diagnostic descriptions are linked to the Java 25 documentation, accessed 30 October 2026. No preview API is required.

Try the companion browser lab

Open Executor overload: inspect queue growth and rejection. With two workers and service time four, predict whether three arrivals per tick can be sustained. Run with queue capacity four, then compare the unbounded queue without changing service capacity.

Expected outcome: The bounded queue never exceeds its configured capacity and excess arrivals are rejected. The unbounded model keeps accepting while backlog grows. Accepted work equals completed plus running plus queued work at the observation boundary.

Scope: The model has a fixed number of workers, FIFO order and exact service times. It omits thread creation, task deadlines, retries, caller-runs execution and queue-node overhead. A real ThreadPoolExecutor with different core and maximum sizes follows additional admission rules.

Primary sources and evidence limits

• ThreadPoolExecutor admission and rejection
• Executor factories
• ExecutorService lifecycle

The scenarios, diagrams, arithmetic and code are original educational examples. Browser ticks are synthetic. Java fixture success is not evidence of production throughput, latency, memory use or model quality.

Views are my own and do not represent any employer, past or present.