Hire a Crew — Thread Pools

Priya’s sale-day fix — one new thread per order — helped for a day. Then traffic kept growing. Creating and destroying threads for every click costs time and memory. It is like hiring a new employee, training them for one task, and firing them immediately. Priya needs a crew on standby, ready to take the next job.

Why not new Thread() every time?

Raw threads have four big problems:

1. Expensive — creating a thread takes real work from the JVM.
2. No limit — nothing stops you from creating thousands and crashing the server.
3. No queue — if all workers are busy, new work has nowhere to wait.
4. Hard to get results back — you need extra code to collect return values.

ExecutorService fixes this. It keeps a pool of reusable threads, queues extra tasks, and can return a Future when you need a result.

Creating a thread pool

Method 1: Executors factory (simple)

// Fixed thread pool — most common
ExecutorService executor = Executors.newFixedThreadPool(10);

// Cached thread pool — creates threads as needed
ExecutorService executor = Executors.newCachedThreadPool();

// Single thread executor — sequential execution
ExecutorService executor = Executors.newSingleThreadExecutor();

// Scheduled executor — for delayed or periodic tasks
ScheduledExecutorService executor = Executors.newScheduledThreadPool(5);

Method 2: ThreadPoolExecutor (advanced, more control)

When you need bounded queues, custom thread names, or a rejection policy:

ThreadPoolExecutor executor = new ThreadPoolExecutor(
    5,                          // corePoolSize
    10,                         // maximumPoolSize
    60L,                        // keepAliveTime
    TimeUnit.SECONDS,           // unit
    new LinkedBlockingQueue<>(100), // workQueue — bounded!
    new ThreadFactory() {
        private int counter = 0;
        @Override
        public Thread newThread(Runnable r) {
            Thread t = new Thread(r, "QuickCart-Worker-" + counter++);
            t.setDaemon(false);
            return t;
        }
    },
    new ThreadPoolExecutor.CallerRunsPolicy() // rejectionHandler
);

ThreadPoolExecutor parameters

ThreadPoolExecutor executor = new ThreadPoolExecutor(
    corePoolSize,        // Minimum threads to keep alive
    maximumPoolSize,     // Maximum threads allowed
    keepAliveTime,       // Time idle threads wait before termination
    unit,                // Time unit for keepAliveTime
    workQueue,           // Queue for holding tasks
    threadFactory,       // Factory for creating threads
    rejectionHandler     // Handler for rejected tasks
);

Core pool size vs maximum pool size

Example: corePoolSize=5, maximumPoolSize=10

// Scenario 1: 3 tasks submitted
// → 3 threads created (within core pool)

// Scenario 2: 7 tasks submitted
// → 5 threads created (core pool), 2 queued

// Scenario 3: 12 tasks submitted, queue full
// → 5 core threads + 5 extra threads (max=10)
// → 2 tasks rejected (or handled by rejection policy)

Work queue types

// Unbounded queue — can grow indefinitely (risky in production)
new LinkedBlockingQueue<>()

// Bounded queue — fixed size (safer)
new LinkedBlockingQueue<>(100)

// Synchronous queue — no storage, direct handoff to a thread
new SynchronousQueue<>()

// Priority queue — tasks executed by priority
new PriorityBlockingQueue<>()

Rejection policies

When the pool is full and the queue is full, what happens?

// 1. AbortPolicy (default) — throws RejectedExecutionException
new ThreadPoolExecutor.AbortPolicy()

// 2. CallerRunsPolicy — runs task in the caller's thread (slows the caller)
new ThreadPoolExecutor.CallerRunsPolicy()

// 3. DiscardPolicy — silently drops the task
new ThreadPoolExecutor.DiscardPolicy()

// 4. DiscardOldestPolicy — drops oldest queued task, tries again
new ThreadPoolExecutor.DiscardOldestPolicy()

Types of thread pools

Fixed thread pool

Ten workers, always. Extra tasks wait in a queue.

ExecutorService executor = Executors.newFixedThreadPool(10);

Characteristics: fixed number of threads, unbounded queue, threads never terminate.

Use when: you know roughly how much work you have and want predictable resource use. This is the most common choice.

Cached thread pool

Creates a new thread when all are busy. Removes idle threads after 60 seconds.

ExecutorService executor = Executors.newCachedThreadPool();

Characteristics: creates threads as needed, no queue (uses SynchronousQueue), idle threads die after 60 seconds.

Use when: many short tasks with unknown load.

Warning: long-running tasks can spawn unlimited threads and overwhelm the server.

Single thread executor

One worker. Tasks run one after another, in order.

ExecutorService executor = Executors.newSingleThreadExecutor();

Use when: background jobs that must not overlap — like writing audit logs.

Scheduled thread pool

Run tasks after a delay or on a schedule:

ScheduledExecutorService executor = Executors.newScheduledThreadPool(5);

// Execute once after 5 seconds
executor.schedule(() -> {
    System.out.println("Delayed task");
}, 5, TimeUnit.SECONDS);

// Execute every 1 second, starting now
executor.scheduleAtFixedRate(() -> {
    System.out.println("Periodic inventory sync");
}, 0, 1, TimeUnit.SECONDS);

Use when: delayed execution, periodic tasks, or cron-like scheduling.

execute vs submit

execute — fire and forget

No return value, no easy way to track completion:

executor.execute(() -> {
    System.out.println("Task executing");
});
// No way to track completion or get a result

submit — returns a Future

You can check if the task is done or get a result later:

Future<?> future = executor.submit(() -> {
    System.out.println("Task executing");
});

if (future.isDone()) {
    System.out.println("Task completed");
}

Submit also accepts a Callable (returns a value):

Future<String> future = executor.submit(() -> {
    Thread.sleep(1000);
    return "Order confirmed";
});

String result = future.get(); // Blocks until ready

Use submit when you need to track or collect results. Use execute for simple background work where you do not care about the outcome.

QuickCart order pool

Priya sets up a fixed pool for order processing:

public class QuickCartOrderService {
    private final ExecutorService orderPool =
        Executors.newFixedThreadPool(10);

    public void submitOrder(String orderId) {
        orderPool.submit(() -> {
            System.out.println("Processing order: " + orderId
                + " on " + Thread.currentThread().getName());
            // charge card, update inventory, send email...
        });
    }
}

Ten orders can run at once. The rest wait in the queue. No new thread is created per order.

How big should the pool be?

There is no magic number, but this intuition helps:

CPU-bound work — heavy calculation, no waiting on network or disk. Use about as many threads as CPU cores:

int cores = Runtime.getRuntime().availableProcessors();
ExecutorService executor = Executors.newFixedThreadPool(cores);

I/O-bound work — calling payment APIs, reading databases, sending emails. Threads spend most of their time waiting. You can use many more threads than cores — often 50 to 200:

ExecutorService executor = Executors.newFixedThreadPool(200);

Mixed workload — start with cores * 2 and tune based on metrics.

QuickCart does a lot of I/O — payment checks, inventory lookups, email. Priya starts with a larger pool and watches how the server behaves.

Monitor the pool

If you have a ThreadPoolExecutor, you can inspect it:

ThreadPoolExecutor pool = (ThreadPoolExecutor) orderPool;
System.out.println("Active threads: " + pool.getActiveCount());
System.out.println("Queue size: " + pool.getQueue().size());
System.out.println("Completed tasks: " + pool.getCompletedTaskCount());

Shutting down properly

Never leave an executor running after your app stops. Always shut it down:

ExecutorService executor = Executors.newFixedThreadPool(10);

// Submit tasks...
for (int i = 0; i < 100; i++) {
    executor.submit(() -> {
        // task logic
    });
}

// Stop accepting new tasks
executor.shutdown();

try {
    // Wait for existing tasks to complete
    if (!executor.awaitTermination(60, TimeUnit.SECONDS)) {
        // Tasks still running after 60 seconds — force stop
        executor.shutdownNow();

        if (!executor.awaitTermination(60, TimeUnit.SECONDS)) {
            System.err.println("Pool did not terminate");
        }
    }
} catch (InterruptedException e) {
    executor.shutdownNow();
    Thread.currentThread().interrupt();
}

Shutdown methods

executor.shutdown();     // Graceful — no new tasks, running tasks finish
executor.shutdownNow();  // Force — interrupts running tasks

if (executor.isShutdown()) {
    // Already shut down, not accepting new tasks
}

if (executor.isTerminated()) {
    // All tasks completed
}

• shutdown() — no new tasks, but running tasks finish.
• shutdownNow() — tries to stop running tasks immediately (interrupts them).
• awaitTermination() — waits until everything is done (or times out).

Skipping shutdown leaves threads alive and can prevent your app from exiting cleanly.

What to remember

• ExecutorService reuses threads instead of creating new ones every time.
• Fixed pool for steady, predictable load. Cached pool for bursts of short tasks. Single thread for ordered background work. Scheduled pool for delayed or periodic jobs.
• ThreadPoolExecutor gives you core/max size, queue type, and rejection policy.
• submit when you need a Future. execute for fire-and-forget.
• Size pools by work type: cores for CPU work, many threads for I/O work.
• Use bounded queues in production to avoid out-of-memory errors.
• Always shutdown when you are done.

What Priya does next: some order steps need to return a value — like a price from an external API — and she learns how to wait for those results with Callable and Future.