Java ExecutorService and Thread Pool Tutorial

Creating a thread per task works until the task rate exceeds what the machine can schedule, at which point you have thousands of threads, most of them waiting, and a megabyte of stack reserved for each. An ExecutorService separates what runs from the threads that run it.

The API is small. Two of its convenience factories have defaults that fail badly under load, and one of its methods swallows exceptions. Those are the parts worth reading carefully.

Written against Java 17, with notes on Java 21.

Submitting work

try (ExecutorService pool = Executors.newFixedThreadPool(4)) {

    pool.execute(() -> log.info("fire and forget"));         // Runnable, no result

    Future<Integer> total = pool.submit(() -> expensiveSum()); // Callable, has a result
    Integer result = total.get();                              // blocks
}

ExecutorService implements AutoCloseable from Java 19, so try-with-resources shuts it down and waits for running tasks. Before that, the shutdown had to be written out, see below.

execute takes a Runnable and returns nothing. submit takes a Callable and returns a Future. That difference is not just about return values, and it is the first trap.

An exception in submit() disappears

pool.execute(() -> { throw new IllegalStateException("visible"); });
// -> handled by the thread's UncaughtExceptionHandler, printed to stderr

pool.submit(() -> { throw new IllegalStateException("silent"); });
// -> captured in the Future. If nobody calls get(), it is never seen.

submit wraps the task so any throwable is stored in the Future and rethrown as ExecutionException when you call get(). Discard the Future, which is exactly what you do for fire-and-forget work, and the failure is invisible. No log line, no stack trace, and a task that silently stopped doing its job.

Two defences:

// 1. use execute() for work with no result
pool.execute(this::reconcile);

// 2. or catch inside the task
pool.submit(() -> {
    try {
        reconcile();
    } catch (Exception e) {
        log.error("reconcile failed", e);
    }
});

For a ThreadPoolExecutor you can also override afterExecute to log both cases centrally. Whichever you choose, decide deliberately. This is the most common way scheduled or background work fails without anyone noticing.

Futures

Future<String> f = pool.submit(() -> fetch(url));

f.isDone();
f.get();                                  // blocks indefinitely
f.get(2, TimeUnit.SECONDS);               // TimeoutException
f.cancel(true);                           // interrupt if running
f.isCancelled();

Always prefer the timed get. The untimed version blocks forever, so one hung task takes the calling thread with it, and if that thread belongs to another pool, the failure spreads.

cancel(true) interrupts the thread. Interruption is cooperative: it sets a flag and throws InterruptedException from blocking calls that support it. A task in a tight computational loop that never checks Thread.currentThread().isInterrupted() cannot be cancelled at all.

pool.submit(() -> {
    while (!Thread.currentThread().isInterrupted()) {
        step();
    }
});

get() after a TimeoutException still leaves the task running. Cancel it explicitly if you no longer want the result.

invokeAll and invokeAny

List<Callable<String>> tasks = urls.stream()
        .map(u -> (Callable<String>) () -> fetch(u))
        .toList();

// waits for every task, returns Futures in the same order
List<Future<String>> all = pool.invokeAll(tasks, 10, TimeUnit.SECONDS);

// returns the first successful result, cancels the rest
String fastest = pool.invokeAny(tasks, 5, TimeUnit.SECONDS);

invokeAll blocks until all tasks finish, are cancelled, or the timeout expires. With a timeout, unfinished tasks are cancelled and their Future.get() throws CancellationException, so iterate with that in mind rather than assuming every future holds a value.

invokeAny is the hedged-request pattern: ask three replicas, take whichever answers first.

Sizing the pool

int cores = Runtime.getRuntime().availableProcessors();

Executors.newFixedThreadPool(cores);       // CPU-bound
Executors.newFixedThreadPool(cores * 8);   // I/O-bound, a starting point

For CPU-bound work, more threads than cores adds context switching and no throughput. For I/O-bound work, threads spend most of their time waiting, so more of them is useful, the theoretical figure is cores × (1 + wait/compute), which is a way of saying measure it.

One pool per kind of work. A single shared pool means a slow batch job starves request handling, and sizing it correctly for both is impossible.

The two Executors factories to avoid

Executors.newFixedThreadPool(n);    // unbounded LinkedBlockingQueue
Executors.newCachedThreadPool();    // unbounded thread count

Both are unbounded, in different directions, and both convert overload into an OutOfMemoryError rather than backpressure.

newFixedThreadPool queues submitted tasks in an unbounded LinkedBlockingQueue. When producers outpace consumers the queue grows until the heap is exhausted, and the failure surfaces far from the cause, as an OOM in unrelated code.

newCachedThreadPool creates a thread per task when none is idle, with no ceiling. A burst of ten thousand tasks attempts ten thousand threads.

Construct the executor instead, and state both bounds:

ExecutorService pool = new ThreadPoolExecutor(
        4, 8,                                       // core and max threads
        60L, TimeUnit.SECONDS,                      // idle timeout for non-core threads
        new ArrayBlockingQueue<>(500),               // BOUNDED queue
        new ThreadFactoryBuilder().setNameFormat("worker-%d").build(),
        new ThreadPoolExecutor.CallerRunsPolicy());  // what happens when full

CallerRunsPolicy is usually the right rejection handler: the submitting thread executes the task itself, which slows the producer down naturally instead of queueing more work. The alternatives are AbortPolicy (throws RejectedExecutionException, the default), DiscardPolicy and DiscardOldestPolicy, the last two lose work silently and are rarely what anyone wants.

Name the threads. A stack trace from worker-3 tells you which pool; one from pool-2-thread-1 does not. It costs one line and saves an incident.

One counter-intuitive detail of ThreadPoolExecutor: it only creates threads beyond corePoolSize when the queue is full. With an unbounded queue, maximumPoolSize is never reached, which is another reason the queue bound is the important number.

Shutting down

pool.shutdown();                                   // stop accepting; finish what is queued
if (!pool.awaitTermination(30, TimeUnit.SECONDS)) {
    pool.shutdownNow();                            // interrupt running tasks
    if (!pool.awaitTermination(10, TimeUnit.SECONDS)) {
        log.error("pool did not terminate");
    }
}

shutdown is graceful; shutdownNow interrupts and returns the tasks that never started. The two-phase pattern above is what close() does in Java 19+.

A pool that is never shut down keeps non-daemon threads alive and the JVM will not exit. That is the classic “my program finished but the process is still running”.

In Spring, prefer a managed executor so the container handles this:

@Bean(destroyMethod = "shutdown")
ThreadPoolTaskExecutor taskExecutor() {
    ThreadPoolTaskExecutor ex = new ThreadPoolTaskExecutor();
    ex.setCorePoolSize(4);
    ex.setMaxPoolSize(8);
    ex.setQueueCapacity(500);
    ex.setThreadNamePrefix("task-");
    ex.setWaitForTasksToCompleteOnShutdown(true);
    ex.setAwaitTerminationSeconds(30);
    ex.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
    return ex;
}

Scheduled execution

ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);

scheduler.schedule(this::once, 5, TimeUnit.SECONDS);
scheduler.scheduleAtFixedRate(this::sample, 0, 1, TimeUnit.MINUTES);      // start to start
scheduler.scheduleWithFixedDelay(this::poll, 0, 1, TimeUnit.MINUTES);     // end to start

scheduleWithFixedDelay for anything whose duration varies. scheduleAtFixedRate when the cadence itself matters and you have confirmed the task finishes inside the interval.

The trap here is severe: an uncaught exception silently cancels the repeating task. Not the next run, all of them. The scheduler stops, no exception is logged, and a job simply never runs again.

scheduler.scheduleWithFixedDelay(() -> {
    try {
        poll();
    } catch (Exception e) {
        log.error("poll failed", e);     // without this, one failure ends the schedule
    }
}, 0, 1, TimeUnit.MINUTES);

Wrap every scheduled task body. This is not optional defensive coding. It is the difference between a job that recovers and one that stops permanently on its first bad response.

Java 21: virtual threads

try (var pool = Executors.newVirtualThreadPerTaskExecutor()) {
    for (Request r : requests) {
        pool.submit(() -> handle(r));      // a million of these is fine
    }
}

A virtual thread costs hundreds of bytes rather than a megabyte, and blocking on I/O unmounts it from its carrier. For I/O-bound work this removes pool sizing as a concern: create one per task.

Two rules that come with them. Do not pool virtual threads: newVirtualThreadPerTaskExecutor creates one per task, which is the point, and a fixed pool of them defeats it. And they change nothing about correctness: races, deadlocks and the swallowed-exception behaviour of submit are all exactly as before.

For CPU-bound work, a bounded platform-thread pool is still the right tool.

Frequently asked questions

What is the difference between execute and submit?

execute takes a Runnable and returns nothing; exceptions reach the uncaught handler. submit returns a Future and captures any exception inside it, invisible unless you call get().

Why did my exception disappear?

You used submit and discarded the Future. Use execute for fire-and-forget, or catch inside the task.

How many threads should a pool have?

Around the core count for CPU-bound work, considerably more for I/O-bound. Measure; the ratio depends on how long tasks wait.

Why avoid Executors.newFixedThreadPool?

Its task queue is unbounded, so a slow consumer grows the queue until the heap is exhausted. Construct a ThreadPoolExecutor with a bounded queue.

Why avoid newCachedThreadPool?

It creates a thread per task with no maximum. A burst of tasks attempts a burst of threads.

Which rejection policy should I use?

CallerRunsPolicy usually, the submitter runs the task, which applies backpressure. AbortPolicy when a rejection should be an error. Avoid the discard policies, which lose work silently.

Why is maximumPoolSize being ignored?

ThreadPoolExecutor only creates threads past corePoolSize when the queue is full. With an unbounded queue that never happens.

Why does my JVM not exit?

A pool that was never shut down keeps non-daemon threads alive. Call shutdown and awaitTermination, or use try-with-resources on Java 19+.

Why did my scheduled task stop running?

An uncaught exception cancels a repeating task permanently, with nothing logged. Wrap the body in try/catch.

Can I cancel a running task?

future.cancel(true) interrupts the thread, but interruption is cooperative, a loop that never checks isInterrupted() cannot be stopped.

Should I use virtual threads instead?

For I/O-bound work on Java 21, yes, one per task, no pooling. For CPU-bound work, keep a bounded platform-thread pool. Neither changes anything about thread safety.

Where should I go next?

CompletableFuture composes the results these pools produce, and Java concurrency basics covers the memory model underneath.