跳到主要内容

8 篇博文 含有标签「multi-threading」

查看所有标签

Deep Dive into Executors and ThreadPoolExecutor

· 阅读需 15 分钟

Scaling a thread pool efficiently is essential to balance resource usage with workload demands, ensuring performance without over-provisioning or exhausting system resources. Below are common scaling strategies, including scenarios for using each and detailed production code examples with comments.

1. Overview of Thread Pool Scaling Strategies​

In thread pool management, the primary goal is to allocate just enough threads to handle tasks effectively without overloading the CPU or running into resource constraints. The Executor framework in Java provides tools for dynamic scaling via multiple built-in pool types and strategies, such as FixedThreadPool for predictable workloads and CachedThreadPool for on-demand scaling.


2. FixedThreadPool - Predictable Task Loads​

For workloads where the number of concurrent tasks is consistent or predictable, using a FixedThreadPool is efficient. The pool is created with a predefined number of threads, ensuring no new threads are added or removed during runtime. This strategy is suitable for batch processing or tasks with steady arrival rates.

Sample Code:

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class FixedThreadPoolExample {
// Define a pool with a fixed number of threads for predictable task load
private static final int THREAD_POOL_SIZE = 5;
private static final ExecutorService executorService = Executors.newFixedThreadPool(THREAD_POOL_SIZE);

public static void main(String[] args) {
// Simulate task submission to the fixed thread pool
for (int i = 0; i < 10; i++) {
final int taskId = i;
executorService.submit(() -> processTask(taskId));
}

// Shutdown the executor service to release resources after processing
executorService.shutdown();
}

private static void processTask(int taskId) {
System.out.println("Processing Task ID: " + taskId + " with thread " + Thread.currentThread().getName());
try {
// Simulating task processing duration
Thread.sleep(1000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}

Output:

Processing Task ID: 0 with thread pool-1-thread-1
Processing Task ID: 1 with thread pool-1-thread-2
Processing Task ID: 2 with thread pool-1-thread-3
Processing Task ID: 3 with thread pool-1-thread-4
Processing Task ID: 4 with thread pool-1-thread-5
Processing Task ID: 5 with thread pool-1-thread-1
Processing Task ID: 6 with thread pool-1-thread-2
Processing Task ID: 7 with thread pool-1-thread-3
Processing Task ID: 8 with thread pool-1-thread-4
Processing Task ID: 9 with thread pool-1-thread-5

Explanation:

  1. Thread Pool Initialization:

    • A fixed number of threads (5) handle the tasks sequentially.
    • The thread pool is initialized with 5 threads, named pool-1-thread-1 to pool-1-thread-5.
  2. Task Submission:

    • The for loop submits 10 tasks to the thread pool.
    • Each task is assigned a unique taskId from 0 to 9.
  3. Task Execution:

    • The tasks are executed by the available threads in the thread pool.
    • Since there are 5 threads, the first 5 tasks (0 to 4) are executed concurrently by the 5 threads.
    • After the first 5 tasks are completed, the next 5 tasks (5 to 9) are executed by the same threads, but the order of execution may vary.
  4. Thread Reuse:

    • The executor will reuse threads, limiting thread creation overhead.
    • The threads in the thread pool are reused to execute subsequent tasks. For example, pool-1-thread-1 might execute task 0 and then task 5.
  5. The shutdown method releases resources once all tasks complete.

Advantages: Predictable memory usage and better performance control for tasks with a steady rate.


3. CachedThreadPool - Unpredictable, Short-Lived Tasks​

A CachedThreadPool dynamically adjusts the pool size depending on task demand. This pool is ideal for handling bursty or sporadic loads. Threads are created as needed and, if idle, removed after a timeout period, allowing the system to scale back.

Sample Code:

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class CachedThreadPoolExample {
// Using CachedThreadPool for sporadic and short-lived tasks
private static final ExecutorService executorService = Executors.newCachedThreadPool();

public static void main(String[] args) {
// Simulate submitting bursty tasks
for (int i = 0; i < 10; i++) {
final int taskId = i;
executorService.submit(() -> handleBurstTask(taskId));
}

// Shutdown executor once all tasks are completed
executorService.shutdown();
}

private static void handleBurstTask(int taskId) {
System.out.println("Handling Burst Task ID: " + taskId + " with thread " + Thread.currentThread().getName());
try {
// Simulating brief processing
Thread.sleep(500);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}

Output:

Handling Burst Task ID: 0 with thread pool-1-thread-1
Handling Burst Task ID: 1 with thread pool-1-thread-2
Handling Burst Task ID: 2 with thread pool-1-thread-3
Handling Burst Task ID: 3 with thread pool-1-thread-4
Handling Burst Task ID: 4 with thread pool-1-thread-5
Handling Burst Task ID: 5 with thread pool-1-thread-6
Handling Burst Task ID: 6 with thread pool-1-thread-7
Handling Burst Task ID: 7 with thread pool-1-thread-8
Handling Burst Task ID: 8 with thread pool-1-thread-9
Handling Burst Task ID: 9 with thread pool-1-thread-10

Explanation:

  1. Thread Pool Initialization:

    • The CachedThreadPool adds threads on demand up to Integer.MAX_VALUE, suitable for short-lived tasks.
    • The CachedThreadPool dynamically creates new threads as needed and reuses previously constructed threads when they are available.
  2. Task Submission:

    • The for loop submits 10 tasks to the thread pool.
    • Each task is assigned a unique taskId from 0 to 9.
  3. Task Execution:

    • The tasks are executed by the available threads in the thread pool.
    • Since the CachedThreadPool creates new threads as needed, it will create up to 10 threads to handle the 10 tasks.
  4. Thread Creation and Reuse:

    • Threads that are idle for 60 seconds are removed, preventing memory buildup.
    • Ideal for systems that handle irregular spikes without knowing task demand in advance.
    • The CachedThreadPool creates new threads for each task initially because no threads are available.
    • The threads are named pool-1-thread-1 to pool-1-thread-10.

Advantages: Scalability and flexibility for dynamic task arrival rates.


4. Dynamic Scaling with Flexible Configuration​

In a production environment, scaling thread pools dynamically may involve using the ThreadPoolExecutor with customized parameters, such as adjustable core pool size and task queue types. This allows for on-the-fly adjustments based on real-time performance metrics.

Sample Code:

import java.util.concurrent.*;

public class DynamicThreadPoolScaling {
public static void main(String[] args) {
// Custom ThreadPoolExecutor with flexible parameters for dynamic scaling
ThreadPoolExecutor executor = new ThreadPoolExecutor(
2, // corePoolSize
10, // maximumPoolSize
60L, // keepAliveTime
TimeUnit.SECONDS,
new LinkedBlockingQueue<>(5), // workQueue
Executors.defaultThreadFactory(),
new ThreadPoolExecutor.CallerRunsPolicy() // Rejection policy
);

for (int i = 0; i < 20; i++) {
final int taskId = i;
executor.submit(() -> performDynamicTask(taskId));
}

// Shut down the executor after tasks completion
executor.shutdown();
}

private static void performDynamicTask(int taskId) {
System.out.println("Performing Dynamic Task ID: " + taskId + " with thread " + Thread.currentThread().getName());
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}

Explanation of Parameters:

  • corePoolSize: Defines the minimum number of threads.
  • maximumPoolSize: Sets a scalable limit for threads.
  • LinkedBlockingQueue: Limits task queue to prevent memory overflows.
  • CallerRunsPolicy: Ensures tasks run in the caller's thread if limits are reached, providing backpressure control.

Output:

Performing Dynamic Task ID: 0 with thread pool-1-thread-1
Performing Dynamic Task ID: 1 with thread pool-1-thread-2
Performing Dynamic Task ID: 2 with thread pool-1-thread-1
Performing Dynamic Task ID: 3 with thread pool-1-thread-2
Performing Dynamic Task ID: 4 with thread pool-1-thread-1
Performing Dynamic Task ID: 5 with thread pool-1-thread-2
Performing Dynamic Task ID: 6 with thread pool-1-thread-1
Performing Dynamic Task ID: 7 with thread pool-1-thread-2
Performing Dynamic Task ID: 8 with thread pool-1-thread-1
Performing Dynamic Task ID: 9 with thread pool-1-thread-2
Performing Dynamic Task ID: 10 with thread pool-1-thread-3
Performing Dynamic Task ID: 11 with thread pool-1-thread-4
Performing Dynamic Task ID: 12 with thread pool-1-thread-5
Performing Dynamic Task ID: 13 with thread pool-1-thread-6
Performing Dynamic Task ID: 14 with thread pool-1-thread-7
Performing Dynamic Task ID: 15 with thread pool-1-thread-8
Performing Dynamic Task ID: 16 with thread pool-1-thread-9
Performing Dynamic Task ID: 17 with thread pool-1-thread-10
Performing Dynamic Task ID: 18 with thread main
Performing Dynamic Task ID: 19 with thread main

Explaination of Output:

  1. ThreadPoolExecutor Initialization:

    • The ThreadPoolExecutor is initialized with:
      • corePoolSize: 2
      • maximumPoolSize: 10
      • keepAliveTime: 60 seconds
      • workQueue: LinkedBlockingQueue with a capacity of 5
      • threadFactory: Default thread factory
      • rejectionPolicy: CallerRunsPolicy
  2. Task Submission:

    • The for loop submits 20 tasks to the thread pool.
    • Each task is assigned a unique taskId from 0 to 19.
  3. Task Execution:

    • Initially, the core threads (pool-1-thread-1 and pool-1-thread-2) handle the first 2 tasks.
    • The next 5 tasks are queued in the LinkedBlockingQueue.
    • As the core threads finish their tasks, they pick up tasks from the queue.
    • When the queue is full (5 tasks), new threads are created up to the maximumPoolSize (10 threads) to handle the remaining tasks.
    • When all 10 threads are busy and the queue is full, the CallerRunsPolicy kicks in, and the remaining tasks are executed by the main thread (main).

Detailed Steps​

  1. Task 0 to 1 Execution:

    • Task 0 is executed by pool-1-thread-1.
    • Task 1 is executed by pool-1-thread-2.
  2. Task 2 to 6 Queued:

    • Tasks 2 to 6 are queued in the LinkedBlockingQueue.
  3. Task 2 to 6 Execution:

    • Tasks 2 to 6 are executed by pool-1-thread-1 and pool-1-thread-2 as they finish their initial tasks.
  4. Task 7 to 11 Queued:

    • Tasks 7 to 11 are queued in the LinkedBlockingQueue.
  5. Task 7 to 11 Execution:

    • Tasks 7 to 11 are executed by pool-1-thread-1 and pool-1-thread-2 as they finish their previous tasks.
  6. Task 12 to 16 Execution:

    • New threads (pool-1-thread-3 to pool-1-thread-7) are created to handle tasks 12 to 16.
  7. Task 17 to 19 Execution:

    • New threads (pool-1-thread-8 to pool-1-thread-10) are created to handle tasks 17 to 19.
  8. Task 18 and 19 Execution by Main Thread:

    • Since all threads are busy and the queue is full, the CallerRunsPolicy causes the main thread to execute tasks 18 and 19.

5. Thread Pool Strategy Decision​

+-------------------------+
| Task Load Predictable? |
+-----------+-------------+
|
|
Yes | No
+----------+------------+
| |
+-------v----------+ +------v------------+
| FixedThreadPool | | CachedThreadPool |
+-------+----------+ +-----+-------------+
| |
| |
Use if tasks are steady Use if tasks are bursty or
and concurrent unpredictable

Using Executors to create thread pools (e.g., via Executors.newFixedThreadPool() or Executors.newCachedThreadPool()) is convenient but introduces hidden risks and drawbacks in certain production scenarios. Let's look at the specific drawbacks and explore best practices for mitigating these issues.


6. Drawbacks of Using Executors to Create Thread Pools​

  1. Unbounded Queue with newFixedThreadPool:

    • The newFixedThreadPool(int nThreads) method uses a LinkedBlockingQueue without a maximum size limit for its task queue. This unbounded queue can lead to memory exhaustion if tasks are submitted faster than they can be processed.
    • When a large number of tasks are added, the queue will keep growing, which can lead to OutOfMemoryError and system instability.
  2. Unbounded Thread Creation in newCachedThreadPool:

    • The newCachedThreadPool() uses a SynchronousQueue that directly hands off tasks to threads. If no threads are idle to handle new tasks, it creates new threads up to Integer.MAX_VALUE.
    • With a high load, this pool may create excessive threads, leading to CPU saturation and potential memory exhaustion, especially in long-running applications.
  3. Lack of Control Over Rejection Policy:

    • The methods in Executors do not allow explicit configuration of rejection policies for when tasks exceed pool capacity. Default policies can fail to handle high-load scenarios gracefully, often leading to performance degradation and unhandled task rejections.
  4. Harder to Manage Idle Threads:

    • For pools like newCachedThreadPool, idle threads remain for a default 60 seconds before being terminated. While this may be appropriate for some workloads, it can lead to inefficiencies in scenarios where shorter idle times would conserve resources.

7. Best Practices Solution: Using ThreadPoolExecutor Directly​

To avoid these pitfalls, it's best to directly instantiate a ThreadPoolExecutor where you can explicitly configure the task queue size, rejection policies, and timeout values. Here’s how to address each issue with recommended configurations:

A. Configure Bounded Queues to Prevent Memory Leaks:

  • Use a bounded task queue, like ArrayBlockingQueue or a sized LinkedBlockingQueue, to control the number of waiting tasks and prevent out-of-memory errors.
  • This limits the number of tasks waiting in the queue, ensuring system stability by capping memory use.
new ThreadPoolExecutor(
corePoolSize,
maxPoolSize,
keepAliveTime,
TimeUnit.SECONDS,
new ArrayBlockingQueue<>(queueSize)
);

B. Limit Thread Growth to Prevent Resource Exhaustion:

  • Use a capped maximumPoolSize when tasks can surge unexpectedly. Setting a limit, e.g., based on the number of available CPU cores, helps prevent CPU saturation.

    When tasks can surge unexpectedly, it's crucial to set a capped maximumPoolSize to prevent the thread pool from creating an excessive number of threads. An unbounded maximumPoolSize can lead to CPU saturation, excessive memory usage, and degraded performance due to excessive context switching.

    CPU Saturation CPU saturation occurs when the CPU is overwhelmed with too many threads, leading to:

    • High CPU Utilization: The CPU spends most of its time executing threads, leaving little time for other system processes.
    • Context Switching Overhead: Frequent context switching between threads consumes CPU cycles, reducing overall efficiency.
    • Memory Pressure: Each thread consumes memory, and a large number of threads can lead to memory exhaustion.

    To prevent CPU saturation, it's recommended to set a maximumPoolSize based on the number of available CPU cores. This ensures that the thread pool can handle a reasonable number of concurrent tasks without overwhelming the CPU.

    A common approach is to set the maximumPoolSize to a value slightly higher than the number of CPU cores to account for both CPU-bound and I/O-bound tasks. A simple formula is maximumPoolSize = N * U

    Where:

    • N: Number of available CPU cores.
    • U: Utilization factor (typically between 1.0 and 2.0).
      • U = 1.0: For CPU-bound tasks where each thread is expected to fully utilize a CPU core.
      • U = 1.5 to 2.0: For mixed CPU-bound and I/O-bound tasks, where some threads may be waiting on I/O operations.

    Example Calculation Suppose you have a system with 8 CPU cores and you expect a mix of CPU-bound and I/O-bound tasks. You can set the maximumPoolSize as follows maximumPoolSize = 8 * 1.5 = 12

  • For highly concurrent applications, you can use dynamic sizing strategies with monitoring metrics (like CPU and memory usage) to inform adjustments to pool parameters.

C. Explicitly Set Rejection Policies:

  • When tasks exceed the pool’s capacity, control the pool’s behavior with rejection policies. ThreadPoolExecutor provides options like CallerRunsPolicy, DiscardPolicy, and DiscardOldestPolicy:
    • CallerRunsPolicy: Executes tasks in the caller thread, which helps slow down task submissions under heavy load.
    • DiscardOldestPolicy: Removes the oldest queued task, making room for newer, high-priority tasks.
  • This choice depends on your application’s tolerance for unprocessed tasks.
new ThreadPoolExecutor(
corePoolSize,
maxPoolSize,
keepAliveTime,
TimeUnit.SECONDS,
new LinkedBlockingQueue<>(queueSize),
new ThreadPoolExecutor.CallerRunsPolicy()
);

Adjust Idle Timeout for Cached Threads:

  • For pools similar to CachedThreadPool, customize the idle timeout to conserve resources by releasing threads sooner when they are not needed. ThreadPoolExecutor allows control over idle time with setKeepAliveTime.
ThreadPoolExecutor cachedPool = new ThreadPoolExecutor(
0, Integer.MAX_VALUE,
30L, TimeUnit.SECONDS, // Reduced keep-alive time
new SynchronousQueue<>(),
Executors.defaultThreadFactory()
);

8. ThreadPoolExecutor with Best Practice Configurations​

Below is a best practice setup using ThreadPoolExecutor that incorporates bounded queues, a limited thread count, a suitable rejection policy, and a controlled idle timeout:

import java.util.concurrent.*;

public class BestPracticeThreadPool {

public static ExecutorService createThreadPool() {
int corePoolSize = 4;
int maxPoolSize = 10;
long keepAliveTime = 30L;
int queueCapacity = 50;

return new ThreadPoolExecutor(
corePoolSize,
maxPoolSize,
keepAliveTime,
TimeUnit.SECONDS,
new ArrayBlockingQueue<>(queueCapacity),
Executors.defaultThreadFactory(),
new ThreadPoolExecutor.CallerRunsPolicy() // Rejection policy to handle overloads
);
}

public static void main(String[] args) {
ExecutorService threadPool = createThreadPool();

// Submitting example tasks
for (int i = 0; i < 100; i++) {
final int taskId = i;
threadPool.submit(() -> processTask(taskId));
}

// Properly shutdown to free resources
threadPool.shutdown();
}

private static void processTask(int taskId) {
System.out.println("Processing Task ID: " + taskId + " with thread " + Thread.currentThread().getName());
try {
Thread.sleep(500); // Simulate work
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}

Explanation of Configurations:

  • ArrayBlockingQueue limits queue size to 50, managing memory consumption.
  • CallerRunsPolicy applies backpressure to slow down submissions.
  • corePoolSize and maxPoolSize control CPU utilization.
  • keepAliveTime of 30 seconds clears idle threads efficiently.

9. Summary​

Using ThreadPoolExecutor directly provides complete control over pool configurations:

  • Bounded queues prevent memory overflows.
  • Limited threads avoid CPU exhaustion.
  • Rejection policies ensure controlled behavior under load.
  • Idle timeouts conserve system resources.

These adjustments create a resilient and scalable thread pool setup suitable for high-performance, resource-intensive applications.

Deep Dive into Spring Boot Multi-Threading

· 阅读需 7 分钟

Introduction to Spring Boot Multi-Threading

Spring Boot provides robust support for multi-threading, enabling developers to build highly concurrent and scalable applications. Leveraging multi-threading in Spring Boot can significantly improve the performance and responsiveness of your application. This article delves into best practices, annotation-based configuration, and real-life production scenarios to help you master multi-threading in Spring Boot.


Best Practices for Spring Boot Multi-Threading

  1. Use @Async for Asynchronous Processing: The @Async annotation allows methods to run in a separate thread, making it easy to implement asynchronous processing.
  2. Configure Thread Pools: Use TaskExecutor to configure thread pools with specific characteristics like core pool size, max pool size, and queue capacity.
  3. Handle Exceptions Gracefully: Implement exception handling mechanisms to manage exceptions that occur in asynchronous tasks.
  4. Monitor Thread Pools: Use Spring Boot Actuator to monitor thread pool metrics and ensure optimal performance.
  5. Avoid Blocking Operations: Ensure that asynchronous methods do not perform blocking operations, which can degrade performance.

Annotation-Based Configuration

Spring Boot provides several annotations to configure and manage multi-threading:

  • @EnableAsync: Enables asynchronous processing in the application.
  • @Async: Marks a method as asynchronous, meaning it will run in a separate thread.
  • @Configuration: Indicates that a class declares one or more @Bean methods and may be processed by the Spring container to generate bean definitions.
  • @Bean: Indicates that a method produces a bean to be managed by the Spring container.

Diagram of Multi-Threading in Spring Boot

+-------------------------+
| Spring Boot Application |
| - @EnableAsync |
| - @Configuration |
| - @Bean |
+-------------------------+
|
v
+-------------------+
| TaskExecutor |
| - corePoolSize |
| - maxPoolSize |
| - queueCapacity |
+-------------------+
|
v
+-------------------+
| @Async Methods |
| - Method1 |
| - Method2 |
+-------------------+
|
v
+-------------------+
| Thread Pool |
| - Thread 1 |
| - Thread 2 |
| - Thread 3 |
+-------------------+
|
v
+-------------------+
| Task Execution |
| - Task 1 |
| - Task 2 |
| - Task 3 |
+-------------------+

Sample Code with Detailed Comments

Configuration Class​

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.util.concurrent.Executor;

@Configuration
@EnableAsync // Enable asynchronous processing
public class AsyncConfig {

@Bean(name = "taskExecutor")
public Executor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(10); // Core pool size
executor.setMaxPoolSize(20); // Maximum pool size
executor.setQueueCapacity(100); // Queue capacity
executor.setThreadNamePrefix("AsyncThread-"); // Thread name prefix
executor.initialize();
return executor;
}
}

Service Class​

import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;

@Service
public class AsyncService {

@Async("taskExecutor") // Use the taskExecutor bean for this method
public void performAsyncTask(String taskName) {
System.out.println("Starting task: " + taskName + " on thread: " + Thread.currentThread().getName());
try {
// Simulate task execution time
Thread.sleep(2000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("Completed task: " + taskName + " on thread: " + Thread.currentThread().getName());
}
}

Controller Class​

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class AsyncController {

@Autowired
private AsyncService asyncService;

@GetMapping("/performTask")
public String performTask(@RequestParam String taskName) {
asyncService.performAsyncTask(taskName);
return "Task " + taskName + " has been submitted for asynchronous processing.";
}
}

Application Class​

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableAsync;

@SpringBootApplication
@EnableAsync // Enable asynchronous processing
public class AsyncApplication {

public static void main(String[] args) {
SpringApplication.run(AsyncApplication.class, args);
}
}

Complex Scenario 1: Flow Control and Backpressure Handling

In high-throughput systems, it's crucial to manage the flow of tasks to avoid overwhelming the system. Backpressure handling ensures that the system can gracefully handle a large number of incoming tasks.

Configuration Class​

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.util.concurrent.Executor;

@Configuration
@EnableAsync
public class BackpressureConfig {

@Bean(name = "backpressureExecutor")
public Executor backpressureExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5); // Core pool size
executor.setMaxPoolSize(10); // Maximum pool size
executor.setQueueCapacity(50); // Queue capacity
executor.setThreadNamePrefix("BackpressureThread-"); // Thread name prefix
executor.initialize();
return executor;
}
}

Service Class​

import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;

@Service
public class BackpressureService {

@Async("backpressureExecutor")
public void handleTask(String taskName) {
System.out.println("Handling task: " + taskName + " on thread: " + Thread.currentThread().getName());
try {
// Simulate task execution time
Thread.sleep(3000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("Completed handling task: " + taskName + " on thread: " + Thread.currentThread().getName());
}
}

Controller Class​

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class BackpressureController {

@Autowired
private BackpressureService backpressureService;

@GetMapping("/handleTask")
public String handleTask(@RequestParam String taskName) {
backpressureService.handleTask(taskName);
return "Task " + taskName + " has been submitted for backpressure handling.";
}
}

Complex Scenario 2: Task Prioritization

In some scenarios, certain tasks may need to be prioritized over others. Spring Boot allows you to configure task executors with priority queues to handle such cases.

Configuration Class​

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.util.concurrent.Executor;
import java.util.concurrent.PriorityBlockingQueue;

@Configuration
@EnableAsync
public class PriorityConfig {

@Bean(name = "priorityExecutor")
public Executor priorityExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5); // Core pool size
executor.setMaxPoolSize(10); // Maximum pool size
executor.setQueueCapacity(50); // Queue capacity
executor.setThreadNamePrefix("PriorityThread-"); // Thread name prefix
executor.setTaskDecorator(new PriorityTaskDecorator()); // Set task decorator for priority
executor.initialize();
return executor;
}
}

Task Decorator​

import org.springframework.core.task.TaskDecorator;

public class PriorityTaskDecorator implements TaskDecorator {

@Override
public Runnable decorate(Runnable runnable) {
return () -> {
// Set priority logic here
System.out.println("Setting priority for task: " + runnable);
runnable.run();
};
}
}

Service Class​

import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;

@Service
public class PriorityService {

@Async("priorityExecutor")
public void prioritizeTask(String taskName) {
System.out.println("Prioritizing task: " + taskName + " on thread: " + Thread.currentThread().getName());
try {
// Simulate task execution time
Thread.sleep(3000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("Completed prioritizing task: " + taskName + " on thread: " + Thread.currentThread().getName());
}
}

Controller Class​

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class PriorityController {

@Autowired
private PriorityService priorityService;

@GetMapping("/prioritizeTask")
public String prioritizeTask(@RequestParam String taskName) {
priorityService.prioritizeTask(taskName);
return "Task " + taskName + " has been submitted for prioritization.";
}
}

Complex Scenario 3: Handling Long-Running Tasks

Long-running tasks can block threads and degrade system performance. Spring Boot provides mechanisms to handle such tasks efficiently.

Configuration Class​

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.util.concurrent.Executor;

@Configuration
@EnableAsync
public class LongRunningConfig {

@Bean(name = "longRunningExecutor")
public Executor longRunningExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5); // Core pool size
executor.setMaxPoolSize(10); // Maximum pool size
executor.setQueueCapacity(50); // Queue capacity
executor.setThreadNamePrefix("LongRunningThread-"); // Thread name prefix
executor.initialize();
return executor;
}
}

Service Class​

import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;

@Service
public class LongRunningService {

@Async("longRunningExecutor")
public void handleLongRunningTask(String taskName) {
System.out.println("Handling long-running task: " + taskName + " on thread: " + Thread.currentThread().getName());
try {
// Simulate long-running task execution time
Thread.sleep(10000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("Completed handling long-running task: " + taskName + " on thread: " + Thread.currentThread().getName());
}
}

Controller Class​

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class LongRunningController {

@Autowired
private LongRunningService longRunningService;

@GetMapping("/handleLongRunningTask")
public String handleLongRunningTask(@RequestParam String taskName) {
longRunningService.handleLongRunningTask(taskName);
return "Long-running task " + taskName + " has been submitted for processing.";
}
}

Conclusion

Spring Boot's support for multi-threading allows developers to build highly concurrent and scalable applications. By following best practices and leveraging annotation-based configuration, you can efficiently manage asynchronous tasks, handle backpressure, prioritize tasks, and manage long-running tasks. This article provides a comprehensive guide to mastering multi-threading in Spring Boot, complete with detailed sample code and real-life production scenarios.

Programming Standards of Java Concurrency

· 阅读需 5 分钟

Thread safety is a critical aspect of building robust and scalable Java applications. Ensuring that your application handles concurrent operations correctly can prevent many common issues such as race conditions, deadlocks, and data inconsistencies. This article provides best practices and detailed guidelines for achieving thread safety in Java applications.


1. Thread Safety in Singleton Objects

  • Requirement: Ensure that singleton objects and their methods are thread-safe.
  • Explanation: Resource-driven classes, utility classes, and singleton factory classes must be thread-safe to prevent race conditions.

2. Meaningful Thread Naming

  • Requirement: Assign meaningful names to threads and thread pools for easier debugging.
  • Example: Custom thread factory with grouping based on external features (e.g., datacenter ID).
public class UserThreadFactory implements ThreadFactory {
private final String namePrefix;
private final AtomicInteger nextId = new AtomicInteger(1);

UserThreadFactory(String whatFeaturOfGroup) {
namePrefix = "From UserThreadFactory's " + whatFeaturOfGroup + "-Worker-";
}

@Override
public Thread newThread(Runnable task) {
String name = namePrefix + nextId.getAndIncrement();
Thread thread = new Thread(null, task, name, 0, false);
System.out.println(thread.getName());
return thread;
}
}

3. Thread Pool Usage

  • Requirement: Use thread pools instead of creating threads manually.
  • Explanation: Thread pools reduce the overhead of thread creation and destruction, preventing resource exhaustion and excessive context switching.

4. Explicit ThreadPoolExecutor Configuration

  • Requirement: Use ThreadPoolExecutor instead of Executors for creating thread pools.
  • Explanation: Executors can lead to resource exhaustion due to unbounded queues and thread creation.
ThreadPoolExecutor executor = new ThreadPoolExecutor(
corePoolSize, maxPoolSize, keepAliveTime, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(queueCapacity),
new UserThreadFactory("Datacenter-1")
);

5. Thread-Safe Date Formatting

  • Requirement: Use thread-safe date formatting.
  • Example: Use ThreadLocal for SimpleDateFormat or switch to DateTimeFormatter in JDK 8.
private static final ThreadLocal<DateFormat> df = ThreadLocal.withInitial(() -> new SimpleDateFormat("yyyy-MM-dd"));

6. ThreadLocal Variable Cleanup

  • Requirement: Clean up ThreadLocal variables, especially in thread pools.
  • Example: Use try-finally blocks to ensure cleanup.
objectThreadLocal.set(userInfo);
try {
// ...
} finally {
objectThreadLocal.remove();
}

7. Locking Performance Considerations

  • Requirement: Minimize the performance impact of locks.
  • Explanation: Prefer lock-free data structures, lock only necessary code blocks, and use object-level locks instead of class-level locks.

8. Consistent Locking Order

  • Requirement: Maintain consistent locking order to prevent deadlocks.
  • Explanation: Ensure that all threads acquire locks in the same order to avoid circular wait conditions.

9. Exception Handling in Locking

  • Requirement: Handle exceptions properly to avoid lock leaks.
  • Example: Ensure that locks are released even if exceptions occur.
Lock lock = new ReentrantLock();
boolean isLocked = lock.tryLock();
if (isLocked) {
try {
doSomething();
doOthers();
} finally {
lock.unlock();
}
}

10. Optimistic and Pessimistic Locking

  • Requirement: Use appropriate locking strategies based on conflict probability.
  • Explanation: Use optimistic locking for low conflict scenarios and pessimistic locking for high conflict scenarios.

11. ScheduledExecutorService for Timed Tasks

  • Requirement: Use ScheduledExecutorService instead of Timer for concurrent timed tasks.
  • Explanation: Timer can terminate all tasks if one throws an uncaught exception.

12. CountDownLatch for Synchronization

  • Requirement: Use CountDownLatch for synchronization in multi-threaded tasks.
  • Explanation: Ensure that all threads call countDown and handle exceptions to avoid blocking the main thread.

13. Random Instance Management

  • Requirement: Avoid sharing Random instances across threads.
  • Example: Use ThreadLocalRandom in JDK 7+ or ensure each thread has its own Random instance.

14. Double-Checked Locking

  • Requirement: Use volatile for double-checked locking to avoid initialization issues.
  • Example: Declare the target variable as volatile.
private volatile Helper helper = null;

15. Volatile and Atomic Operations

  • Requirement: Use volatile for one-write-many-read scenarios and Atomic classes for multi-write scenarios.
  • Example: Use AtomicInteger or LongAdder for atomic operations.

16. HashMap Resize Risks

  • Requirement: Be cautious of HashMap resizing in high-concurrency scenarios.
  • Explanation: Resizing can lead to deadlocks and high CPU usage.

17. ThreadLocal Usage

  • Requirement: Use ThreadLocal with static variables carefully.
  • Explanation: ThreadLocal variables are shared within a thread but can cause issues if not managed properly.

By following these best practices, you can ensure that your Java application is thread-safe, scalable, and robust. These guidelines cover a wide range of scenarios, from basic thread safety to advanced concurrency patterns, helping you build high-performance applications.

Best Practices of Java Core Multi-Threading Programming

· 阅读需 9 分钟

Introduction​

Java's multi-threading capabilities are powerful, but they require careful handling to avoid common pitfalls such as race conditions, deadlocks, and excessive synchronization. This article provides a deep dive into best practices for Java core multi-threading programming, including detailed textual diagrams, explanations, and real-life production sample code.


Thread Safety​

Use Synchronization Wisely​

Synchronization ensures that only one thread can execute a synchronized block or method at a time. However, excessive synchronization can lead to performance bottlenecks.

+-------------------+ +-------------------+
| | | |
| Thread 1 |<----->| Shared Resource|
| | | |
+-------------------+ +-------------------+
| |
| |
v v
+-------------------+ +-------------------+
| | | |
| Thread 2 |<----->| Shared Resource|
| | | |
+-------------------+ +-------------------+
class Counter {
private int count = 0;

public synchronized void increment() {
count++;
}

public synchronized int getCount() {
return count;
}
}

public class Main {
public static void main(String[] args) throws InterruptedException {
Counter counter = new Counter();

Thread t1 = new Thread(() -> {
for (int i = 0; i < 1000; i++) {
counter.increment();
}
});

Thread t2 = new Thread(() -> {
for (int i = 0; i < 1000; i++) {
counter.increment();
}
});

t1.start();
t2.start();

t1.join();
t2.join();

System.out.println("Count: " + counter.getCount()); // Count: 2000
}
}

Avoid Excessive Synchronization​

Excessive synchronization can lead to performance issues. Use finer-grained locks or non-blocking algorithms where possible.

class FineGrainedCounter {
private int count = 0;
private final Object lock = new Object();

public void increment() {
synchronized (lock) {
count++;
}
}

public int getCount() {
synchronized (lock) {
return count;
}
}
}

Use Atomic Classes​

Atomic classes provide thread-safe operations on single variables without the need for explicit synchronization.

import java.util.concurrent.atomic.AtomicInteger;

class AtomicCounter {
private AtomicInteger count = new AtomicInteger(0);

public void increment() {
count.incrementAndGet();
}

public int getCount() {
return count.get();
}
}

public class Main {
public static void main(String[] args) throws InterruptedException {
AtomicCounter counter = new AtomicCounter();

Thread t1 = new Thread(() -> {
for (int i = 0; i < 1000; i++) {
counter.increment();
}
});

Thread t2 = new Thread(() -> {
for (int i = 0; i < 1000; i++) {
counter.increment();
}
});

t1.start();
t2.start();

t1.join();
t2.join();

System.out.println("Count: " + counter.getCount()); // Count: 2000
}
}

Thread Communication​

Use wait() and notify()​

Threads can communicate using wait(), notify(), and notifyAll() methods.

+-------------------+ +-------------------+
| | | |
| Producer |<----->| Consumer |
| | | |
+-------------------+ +-------------------+
| |
| |
v v
+-------------------+ +-------------------+
| | | |
| Shared Buffer |<----->| Shared Buffer |
| | | |
+-------------------+ +-------------------+
class SharedBuffer {
private int[] buffer = new int[10];
private int count = 0;

public synchronized void produce(int value) throws InterruptedException {
while (count == buffer.length) {
wait();
}
buffer[count++] = value;
notifyAll();
}

public synchronized int consume() throws InterruptedException {
while (count == 0) {
wait();
}
int value = buffer[--count];
notifyAll();
return value;
}
}

class Producer implements Runnable {
private SharedBuffer buffer;

public Producer(SharedBuffer buffer) {
this.buffer = buffer;
}

@Override
public void run() {
for (int i = 0; i < 100; i++) {
try {
buffer.produce(i);
System.out.println("Produced: " + i);
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}

class Consumer implements Runnable {
private SharedBuffer buffer;

public Consumer(SharedBuffer buffer) {
this.buffer = buffer;
}

@Override
public void run() {
for (int i = 0; i < 100; i++) {
try {
int value = buffer.consume();
System.out.println("Consumed: " + value);
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}

public class Main {
public static void main(String[] args) {
SharedBuffer buffer = new SharedBuffer();
new Thread(new Producer(buffer)).start();
new Thread(new Consumer(buffer)).start();
}
}

Avoid Busy Waiting​

Busy waiting is inefficient and should be avoided. Use wait() and notify() instead.

class BusyWaitExample {
private boolean flag = false;

public synchronized void setFlag(boolean flag) {
this.flag = flag;
notifyAll();
}

public synchronized void waitForFlag() throws InterruptedException {
while (!flag) {
wait();
}
}
}

Thread Pooling​

Use ExecutorService​

Thread pooling manages a pool of worker threads to execute tasks concurrently.

+-------------------+ +-------------------+
| | | |
| Task 1 |<----->| Thread Pool |
| | | |
+-------------------+ +-------------------+
| |
| |
v v
+-------------------+ +-------------------+
| | | |
| Task 2 |<----->| Worker Threads |
| | | |
+-------------------+ +-------------------+
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Main {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(5);

for (int i = 0; i < 10; i++) {
executor.submit(() -> {
System.out.println("Thread " + Thread.currentThread().getName() + " is running");
});
}

executor.shutdown();
}
}

Custom Thread Pools​

Custom thread pools can be created using ThreadPoolExecutor.

import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class Main {
public static void main(String[] args) {
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(5);

for (int i = 0; i < 10; i++) {
executor.submit(() -> {
System.out.println("Thread " + Thread.currentThread().getName() + " is running");
});
}

executor.shutdown();
try {
executor.awaitTermination(1, TimeUnit.MINUTES);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

Deadlock Prevention​

Lock Ordering​

Ensure that locks are acquired in a consistent order to prevent deadlocks.

+-------------------+ +-------------------+
| | | |
| Thread 1 |<----->| Resource A |
| | | |
+-------------------+ +-------------------+
| |
| |
v v
+-------------------+ +-------------------+
| | | |
| Thread 2 |<----->| Resource B |
| | | |
+-------------------+ +-------------------+
public class Main {
public static void main(String[] args) {
Object lock1 = new Object();
Object lock2 = new Object();

Thread t1 = new Thread(() -> {
synchronized (lock1) {
System.out.println("Thread 1: Holding lock 1...");
try { Thread.sleep(10); } catch (InterruptedException e) {}
System.out.println("Thread 1: Waiting for lock 2...");
synchronized (lock2) {
System.out.println("Thread 1: Holding lock 1 & 2...");
}
}
});

Thread t2 = new Thread(() -> {
synchronized (lock1) { // Acquire locks in the same order
System.out.println("Thread 2: Holding lock 1...");
try { Thread.sleep(10); } catch (InterruptedException e) {}
System.out.println("Thread 2: Waiting for lock 2...");
synchronized (lock2) {
System.out.println("Thread 2: Holding lock 1 & 2...");
}
}
});

t1.start();
t2.start();
}
}

Timeout Mechanisms​

Use timeouts to avoid waiting indefinitely for a lock.

import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;

public class Main {
public static void main(String[] args) {
Lock lock1 = new ReentrantLock();
Lock lock2 = new ReentrantLock();

Thread t1 = new Thread(() -> {
try {
if (lock1.tryLock(1, TimeUnit.SECONDS)) {
System.out.println("Thread 1: Holding lock 1...");
try { Thread.sleep(10); } catch (InterruptedException e) {}
if (lock2.tryLock(1, TimeUnit.SECONDS)) {
System.out.println("Thread 1: Holding lock 1 & 2...");
lock2.unlock();
}
lock1.unlock();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
});

Thread t2 = new Thread(() -> {
try {
if (lock1.tryLock(1, TimeUnit.SECONDS)) {
System.out.println("Thread 2: Holding lock 1...");
try { Thread.sleep(10); } catch (InterruptedException e) {}
if (lock2.tryLock(1, TimeUnit.SECONDS)) {
System.out.println("Thread 2: Holding lock 1 & 2...");
lock2.unlock();
}
lock1.unlock();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
});

t1.start();
t2.start();
}
}

Concurrency Utilities​

Use Concurrent Collections​

Concurrent collections provide thread-safe data structures.

import java.util.concurrent.ConcurrentHashMap;

public class Main {
public static void main(String[] args) {
ConcurrentHashMap<String, Integer> map = new ConcurrentHashMap<>();

Thread t1 = new Thread(() -> {
for (int i = 0; i < 1000; i++) {
map.put("Key" + i, i);
}
});

Thread t2 = new Thread(() -> {
for (int i = 0; i < 1000; i++) {
map.put("Key" + i, i);
}
});

t1.start();
t2.start();

try {
t1.join();
t2.join();
} catch (InterruptedException e) {
e.printStackTrace();
}

System.out.println("Map size: " + map.size()); // Map size: 1000
}
}

Fork/Join Framework​

The Fork/Join framework is designed for tasks that can be recursively split into smaller tasks.

import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveTask;

class SumTask extends RecursiveTask<Integer> {
private static final int THRESHOLD = 10;
private int[] array;
private int start;
private int end;

public SumTask(int[] array, int start, int end) {
this.array = array;
this.start = start;
this.end = end;
}

@Override
protected Integer compute() {
if (end - start <= THRESHOLD) {
int sum = 0;
for (int i = start; i < end; i++) {
sum += array[i];
}
return sum;
} else {
int mid = (start + end) / 2;
SumTask leftTask = new SumTask(array, start, mid);
SumTask rightTask = new SumTask(array, mid, end);
leftTask.fork();
int rightResult = rightTask.compute();
int leftResult = leftTask.join();
return leftResult + rightResult;
}
}
}

public class Main {
public static void main(String[] args) {
int[] array = new int[100];
for (int i = 0; i < array.length; i++) {
array[i] = i + 1;
}

ForkJoinPool pool = new ForkJoinPool();
int sum = pool.invoke(new SumTask(array, 0, array.length));
System.out.println("Sum: " + sum); // Sum: 5050
}
}

Thread Local Variables​

ThreadLocal variables allow each thread to have its own copy of a variable.

public class Main {
private static ThreadLocal<Integer> threadLocal = ThreadLocal.withInitial(() -> 0);

public static void main(String[] args) {
Thread t1 = new Thread(() -> {
threadLocal.set(1);
System.out.println("Thread 1: " + threadLocal.get());
});

Thread t2 = new Thread(() -> {
threadLocal.set(2);
System.out.println("Thread 2: " + threadLocal.get());
});

t1.start();
t2.start();
}
}

Example: Concurrent Web Server​

import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

class RequestHandler implements Runnable {
private Socket clientSocket;

public RequestHandler(Socket clientSocket) {
this.clientSocket = clientSocket;
}

@Override
public void run() {
try {
// Handle the client request
System.out.println("Handling request from " + clientSocket.getInetAddress());
// Simulate processing time
Thread.sleep(1000);
clientSocket.getOutputStream().write("HTTP/1.1 200 OK\r\n\r\nHello, World!".getBytes());
clientSocket.close();
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
}
}

public class WebServer {
public static void main(String[] args) throws IOException {
ServerSocket serverSocket = new ServerSocket(8080);
ExecutorService executor = Executors.newFixedThreadPool(10);

while (true) {
Socket clientSocket = serverSocket.accept();
executor.submit(new RequestHandler(clientSocket));
}
}
}

Conclusion​

Java's multi-threading capabilities are powerful, but they require careful handling to avoid common pitfalls such as race conditions, deadlocks, and excessive synchronization. By following the best practices outlined in this article, you can write robust, high-performance Java applications that leverage the full potential of multi-threading. The provided real-life production sample code illustrates how these concepts can be applied in practical scenarios.


References​

  1. Java Concurrency in Practice
  2. Oracle Java Documentation
  3. Java Memory Model
  4. Fork/Join Framework
  5. Concurrent Collections

By mastering these concepts and practices, you can build robust, high-performance Java applications that leverage the full potential of multi-threading.

Best Practices of Java Core ThreadPoolExecutor

· 阅读需 8 分钟

1. Why Executors is Discouraged​

Java’s Executors class provides several factory methods to create various types of thread pools:

public static ExecutorService newSingleThreadExecutor()
public static ExecutorService newFixedThreadPool(int nThreads)
public static ExecutorService newCachedThreadPool()
public static ScheduledExecutorService newSingleThreadScheduledExecutor()
public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize)

These convenience methods simplify thread pool creation, but they do not allow fine-tuning. Using Executors to create thread pools is generally discouraged due to potential risks, such as resource exhaustion. The official guidance is to use ThreadPoolExecutor directly, enabling full control over pool configurations, ensuring stability under varying load conditions.

public class ThreadDemo {
public static void main(String[] args) {
ExecutorService es = Executors.newSingleThreadExecutor();
}
}

Why it's discouraged: Creating a thread pool this way can lead to issues like uncontrolled resource usage. Best practice: use ThreadPoolExecutor instead.

2. ThreadPoolExecutor Introduction​

Java's ThreadPoolExecutor is a cornerstone of concurrent programming. It provides a high-performance, versatile API for managing threads and handling tasks, but it requires careful configuration to prevent resource wastage, improve system responsiveness, and enhance maintainability.


3. Overview of ThreadPoolExecutor​

The ThreadPoolExecutor manages thread allocation, scaling, and task queueing while optimizing thread lifecycle to minimize system overhead. Here is a basic textual model of a ThreadPoolExecutor lifecycle and memory model:

+-----------------------------+
| Task Queue |
| (BlockingQueue<Runnable>) |
+------------+----------------+
|
+-----------------v-----------------+
| |
| ThreadPoolExecutor |
| |
+----------------+------------------+
|
+----------------------+--------------------+
| Worker Threads |
| +-----------------------------------+ |
| | Worker Thread 1 | |
| |-----------------------------------| |
| | Worker Thread 2 | |
| |-----------------------------------| |
| | Worker Thread N | |
+-------------------------------------------+
  • Worker Threads: Execute tasks within a controlled environment.
  • Task Queue: Holds tasks awaiting execution, preventing unnecessary thread creation.
  • ThreadPoolExecutor: Controls the execution strategy, pool scaling, and task rejection policies.

To avoid risks, use ThreadPoolExecutor directly for creating thread pools:

public class ThreadDemo {
public static void main(String[] args) {
ExecutorService es = new ThreadPoolExecutor(1, 1, 0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(10), Executors.defaultThreadFactory(),
new ThreadPoolExecutor.DiscardPolicy());
}
}

5. Core Configuration Parameters​

The ThreadPoolExecutor relies on several key configuration parameters:

  • corePoolSize: Minimum number of threads in the pool.
  • maximumPoolSize: Maximum threads allowed in the pool.
  • keepAliveTime: The time a thread can remain idle before termination if beyond corePoolSize.
  • workQueue: Queue holding tasks before they are assigned to threads.

Best Practice: Choose configuration values based on workload type and system capacity.​


6. Analyzing Executors Source Code​

The methods newSingleThreadExecutor(), newFixedThreadPool(int nThreads), and newCachedThreadPool() all internally use ThreadPoolExecutor. Below are the source code implementations from JDK 8:

public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService(
new ThreadPoolExecutor(1, 1, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>())
);
}

public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>());
}

public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE, 60L, TimeUnit.SECONDS, new SynchronousQueue<Runnable>());
}

All three methods use ThreadPoolExecutor but provide different default configurations. These static methods abstract parameters, which can lead to undesired behaviors under high loads.

7. ThreadPoolExecutor Constructors and Parameters​

When creating thread pools with ThreadPoolExecutor, initializing parameters must be specified. One of its constructors takes the following parameters:

public ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit,
BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory,
RejectedExecutionHandler handler)

Parameter Breakdown​

  • corePoolSize: Number of threads to keep in the pool.
  • maximumPoolSize: Maximum number of threads allowed.
  • keepAliveTime: Time limit for idle threads over corePoolSize before termination.
  • unit: Unit of time for keepAliveTime.
  • workQueue: Queue to hold tasks before execution.
  • threadFactory: Factory for creating threads, usually Executors.defaultThreadFactory().
  • handler: Policy for handling tasks when the pool reaches its limit.

8. Advanced Queue Management​

Choosing the Right Queue:

  1. SynchronousQueue: No internal capacity, ideal for transient spikes.
  2. LinkedBlockingQueue: Unbounded, suitable for long-running processes.
  3. ArrayBlockingQueue: Fixed size, optimal for bounded workloads.

Note: Select a queue type aligned with task load, duration, and resource availability.

Relationships between corePoolSize and maximumPoolSize​

  • If thread count < corePoolSize, a new thread is created for each task.
  • If thread count >= corePoolSize, tasks are added to the task queue.
  • If the queue is full and thread count < maximumPoolSize, a new thread is created.
  • If thread count >= maximumPoolSize, the handler policy determines the behavior.

Note: These relationships vary with the type of queue (bounded or unbounded). Unbounded queues will prevent maximum thread creation from being reached.

Understanding workQueue Types​

The workQueue holds pending tasks that await execution. Different queue types handle tasks differently:

  • SynchronousQueue: A direct hand-off mechanism with no capacity. If no thread is immediately available, a new one is created or the task is rejected based on maximumPoolSize.
  • LinkedBlockingQueue: An unbounded queue. When tasks exceed corePoolSize, they are added to the queue instead of creating new threads. Warning: Unbounded growth may exhaust memory if tasks are generated faster than processed.

9. Thread Pool Scaling Strategies​

Efficient scaling balances resource use against workload demands. Consider using dynamic scaling mechanisms for responsive systems.

  • FixedThreadPool: When task load and timing are predictable.
  • CachedThreadPool: For unpredictable or short-lived task loads.

10. RejectedExecutionHandler: Strategies for Handling Excess Tasks​

When the pool and queue are full, the handler parameter specifies a strategy to handle tasks. Java provides four built-in policies:

  • DiscardOldestPolicy: Removes the oldest task in the queue to accommodate the new task.
  • CallerRunsPolicy: Executes the task in the caller thread if the pool is busy, providing a backpressure mechanism.
  • DiscardPolicy: Silently discards the task without further action.
  • AbortPolicy: Throws an exception to signal that the pool is at capacity.

Note: Customize RejectedExecutionHandler based on workload characteristics. Expecially for applications where resilience is critical, consider implementing a custom rejection handler or using CallerRunsPolicy to limit load. Proper configuration can prevent issues like out-of-memory errors.

RejectedExecutionHandler customHandler = new RejectedExecutionHandler() {
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
System.out.println("Task rejected: " + r.toString());
// custom logic for rejected task
}
};

11. Enhanced Monitoring Techniques​

Monitoring thread and queue activity is essential for real-time scaling and troubleshooting. Use the following tools and approaches:

  • JMX (Java Management Extensions): For real-time metrics on threads, tasks, and system load.
  • Thread Pool Executor's getPoolSize and getQueue().size() Methods: To track runtime capacity and pending task counts.

12. Backpressure Handling in Production​

Backpressure handling in high-demand environments prevents overloads by controlling the flow of tasks:

  • Rejection Policies: Define fallback strategies for rejected tasks.
  • Queue Sizing: Set boundaries on task input rate.

Diagram: Dynamic Task Flow and Backpressure​

Client Requests
|
+----v----+ Backpressure Control
| Task |--------------------+
| Queue | |
+---------+ |
| |
+------v-------+ |
| Thread | |
| Pool | |
+--------------+----------------+

13. Example​

import java.util.concurrent.*;

public class OptimizedThreadPoolExecutor {
public static void main(String[] args) {
// 1. Configure core parameters with scaling strategy
ThreadPoolExecutor executor = new ThreadPoolExecutor(
4, // core pool size
10, // maximum pool size
30, // keep-alive time
TimeUnit.SECONDS, // keep-alive time unit
new ArrayBlockingQueue<>(100), // task queue type
new ThreadPoolExecutor.CallerRunsPolicy() // rejection policy
);

// 2. Simulate task submission
for (int i = 0; i < 200; i++) {
executor.submit(new Task(i));
}

executor.shutdown();
}

static class Task implements Runnable {
private final int taskId;

Task(int taskId) {
this.taskId = taskId;
}

@Override
public void run() {
System.out.println("Executing Task " + taskId);
try {
Thread.sleep(500); // Simulate work
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
}

14. Conclusion​

Summary: Use ThreadPoolExecutor directly with appropriately sized pools, tailored queue types, and effective rejection policies to ensure optimal, safe performance. Mastering ThreadPoolExecutor for complex, production-grade applications requires an understanding of core parameters, workload patterns, and monitoring needs. These best practices are intended to improve performance, efficiency, and system resilience in multithreaded environments.

Create a Java Thread Pool From Scratch

· 阅读需 17 分钟

Custom Java Thread Pool: Proof of Concept

In this article, we will create a simple but complete custom thread pool. We’ll build a SimpleThreadPool class that includes key features such as managing worker threads, a task queue, and shutdown capabilities. We'll break down the code into logical components, explain the design concepts, and provide sample usage.


Implementation of SimpleThreadPool​

The core classes in our implementation are:

  • SimpleThreadPool: Manages the thread pool and task queue.
  • Worker: Represents the worker threads that execute tasks.
import java.util.LinkedList;
import java.util.Queue;

public class SimpleThreadPool {
private final int poolSize;
private final PoolWorker[] workers;
private final Queue<Runnable> taskQueue;
private boolean isShutdown = false;

public SimpleThreadPool(int poolSize) {
this.poolSize = poolSize;
taskQueue = new LinkedList<>();
workers = new PoolWorker[poolSize];

// Start worker threads
for (int i = 0; i < poolSize; i++) {
workers[i] = new PoolWorker();
workers[i].start();
}
}

// Submit a task to the thread pool
public synchronized void submit(Runnable task) {
if (!isShutdown) {
taskQueue.add(task);
notify(); // Notify workers that a task is available
} else {
throw new IllegalStateException("ThreadPool is shut down");
}
}

// Initiates an orderly shutdown
public synchronized void shutdown() {
isShutdown = true;
for (PoolWorker worker : workers) {
worker.interrupt(); // Interrupt each worker to stop waiting
}
}

// Worker class represents threads in the pool
private class PoolWorker extends Thread {
public void run() {
Runnable task;

while (true) {
synchronized (SimpleThreadPool.this) {
while (taskQueue.isEmpty() && !isShutdown) {
try {
SimpleThreadPool.this.wait(); // Wait for a task
} catch (InterruptedException ignored) {
return; // Exit if interrupted during shutdown
}
}
// If shutdown and no tasks, end thread
if (isShutdown && taskQueue.isEmpty()) {
return;
}
task = taskQueue.poll(); // Take task from queue
}
try {
task.run(); // Execute task outside synchronized block
} catch (RuntimeException e) {
System.err.println("Task execution failed: " + e.getMessage());
}
}
}
}
}

Explanation of Key Concepts​

  1. Task Queue: A LinkedList serves as the task queue, holding Runnable tasks for worker threads.
  2. Worker Threads: PoolWorker threads execute tasks. Each worker loops to fetch a task from the queue and processes it if available.
  3. Synchronization with wait and notify:
    • wait(): Called when no tasks are available, putting the thread in a waiting state to save CPU resources.
    • notify(): Called when a new task is added, waking up a worker thread to execute the task.
  4. Graceful Shutdown:
    • shutdown(): Sets the isShutdown flag to stop accepting tasks and interrupts all worker threads, allowing them to exit if they’re waiting for a task.

Sample Code to Use the SimpleThreadPool​

This example demonstrates how to use the custom SimpleThreadPool to run tasks, including graceful shutdown.

public class SimpleThreadPoolExample {
public static void main(String[] args) {
// Create a thread pool with 3 threads
SimpleThreadPool threadPool = new SimpleThreadPool(3);

// Submit tasks to the thread pool
for (int i = 1; i <= 5; i++) {
int taskId = i;
threadPool.submit(() -> {
System.out.println("Task " + taskId + " is starting on thread " + Thread.currentThread().getName());
try {
Thread.sleep(1000); // Simulate task execution time
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("Task " + taskId + " completed.");
});
}

// Shutdown the thread pool after tasks complete
threadPool.shutdown();
}
}

Explanation of Sample Usage Code​

  1. Creating a Thread Pool: A SimpleThreadPool instance with 3 threads is created.
  2. Submitting Tasks: Five tasks are submitted to the pool, each of which simulates a brief work period.
  3. Shutting Down the Pool: After submitting the tasks, the pool is gracefully shut down using shutdown().

Expected Output​

Task 1 is starting on thread Thread-0
Task 2 is starting on thread Thread-1
Task 3 is starting on thread Thread-2
Task 1 completed.
Task 2 completed.
Task 3 completed.
Task 4 is starting on thread Thread-0
Task 5 is starting on thread Thread-1
Task 4 completed.
Task 5 completed.

Conclusion​

This SimpleThreadPool is a demonstration of how a basic thread pool is structured and works:

  • Worker Threads repeatedly pick up and execute tasks from a shared queue.
  • Synchronization between task producers and worker threads prevents busy-waiting.
  • Graceful Shutdown lets worker threads exit safely after processing all queued tasks.

This POC lays the foundation for understanding and extending Java thread pool management by illustrating how resources are managed and synchronized in a concurrent environment.


Custom Java Thread Pool with Separate Task Class

Separating the Task as an individual class will enhance the code's modularity and flexibility, making it easier to reuse or extend for specific task-related features. Let's modify the SimpleThreadPool implementation to include an external Task class.

Here’s the updated implementation:

  1. SimpleThreadPool: Manages worker threads and the task queue.
  2. Task: Represents individual tasks, implementing Runnable.
  3. PoolWorker: Represents the worker threads that pick up tasks from the queue and execute them.

Task Class Implementation​

We’ll define Task as a standalone class that implements Runnable. Each task can have custom logic.

public class Task implements Runnable {
private final int taskId;

public Task(int taskId) {
this.taskId = taskId;
}

@Override
public void run() {
System.out.println("Task " + taskId + " is starting on thread " + Thread.currentThread().getName());
try {
Thread.sleep(1000); // Simulate task execution time
} catch (InterruptedException e) {
Thread.currentThread().interrupt(); // Handle interruption
}
System.out.println("Task " + taskId + " completed.");
}
}
  • taskId: Each task is given an ID to identify it during execution.
  • run() Method: Defines the task's actions, including a simulated delay.

SimpleThreadPool Implementation​

The SimpleThreadPool class manages the task queue and distributes tasks among worker threads.

import java.util.LinkedList;
import java.util.Queue;

public class SimpleThreadPool {
private final int poolSize;
private final PoolWorker[] workers;
private final Queue<Runnable> taskQueue;
private boolean isShutdown = false;

public SimpleThreadPool(int poolSize) {
this.poolSize = poolSize;
taskQueue = new LinkedList<>();
workers = new PoolWorker[poolSize];

// Initialize and start each worker thread
for (int i = 0; i < poolSize; i++) {
workers[i] = new PoolWorker();
workers[i].start();
}
}

// Method to submit a task to the thread pool
public synchronized void submit(Runnable task) {
if (!isShutdown) {
taskQueue.add(task);
notify(); // Notify a waiting worker thread about the new task
} else {
throw new IllegalStateException("ThreadPool is shut down and cannot accept tasks");
}
}

// Initiates shutdown and interrupts all waiting worker threads
public synchronized void shutdown() {
isShutdown = true;
for (PoolWorker worker : workers) {
worker.interrupt(); // Interrupts workers waiting for tasks
}
}

// Worker class that executes tasks from the queue
private class PoolWorker extends Thread {
public void run() {
while (true) {
Runnable task;
synchronized (SimpleThreadPool.this) {
while (taskQueue.isEmpty() && !isShutdown) {
try {
SimpleThreadPool.this.wait(); // Wait for a new task
} catch (InterruptedException e) {
return; // Exit if interrupted during shutdown
}
}
if (isShutdown && taskQueue.isEmpty()) {
return; // Exit if shutdown and no tasks are left
}
task = taskQueue.poll(); // Fetch a task
}
try {
task.run(); // Execute the task outside the synchronized block
} catch (RuntimeException e) {
System.err.println("Task execution failed: " + e.getMessage());
}
}
}
}
}

Sample Code to Use the Custom Thread Pool with Task​

This example demonstrates submitting Task instances to the SimpleThreadPool.

public class SimpleThreadPoolExample {
public static void main(String[] args) {
// Create a thread pool with 3 threads
SimpleThreadPool threadPool = new SimpleThreadPool(3);

// Submit tasks to the thread pool
for (int i = 1; i <= 5; i++) {
Task task = new Task(i); // Create a new Task with a unique ID
threadPool.submit(task); // Submit the Task to the thread pool
}

// Shutdown the thread pool after tasks are complete
threadPool.shutdown();
}
}

Explanation of Sample Usage​

  1. Task Creation: Each Task object has a unique ID, making it easy to track execution progress.
  2. Task Submission: Tasks are added to the SimpleThreadPool’s task queue.
  3. Shutdown: Once all tasks are submitted, shutdown() stops the pool after finishing queued tasks.

Expected Output​

Task 1 is starting on thread Thread-0
Task 2 is starting on thread Thread-1
Task 3 is starting on thread Thread-2
Task 1 completed.
Task 2 completed.
Task 3 completed.
Task 4 is starting on thread Thread-0
Task 5 is starting on thread Thread-1
Task 4 completed.
Task 5 completed.

This implementation improves modularity by allowing different Task implementations and extends the reusability of SimpleThreadPool. Each component (Task, PoolWorker, and SimpleThreadPool) is designed for flexible use in varied production scenarios.


Wait but what happened to the terminated thread?

To demonstrate the behavior where a terminated thread in a thread pool is repurposed and moves back to the runnable state when a new task is added, we need to tweak the SimpleThreadPool to simulate this behavior.

Typically, when threads in a pool complete their tasks, they wait for new tasks rather than fully terminating (they don't reach the "TERMINATED" state in Java's built-in ThreadPoolExecutor). However, for demonstration, I’ll create logic that represents threads reaching a "stopped" state and being reactivated for new tasks.

Let's add a mechanism to track the states and illustrate the thread reusability behavior.


Enhanced SimpleThreadPool Implementation​

Below is an enhanced version of SimpleThreadPool with thread state tracking. We’ll add logging and state simulation to show when threads are "terminated" and later reactivated when new tasks arrive.

import java.util.LinkedList;
import java.util.Queue;

public class SimpleThreadPool {
private final int poolSize;
private final PoolWorker[] workers;
private final Queue<Runnable> taskQueue;
private boolean isShutdown = false;

public SimpleThreadPool(int poolSize) {
this.poolSize = poolSize;
taskQueue = new LinkedList<>();
workers = new PoolWorker[poolSize];

// Initialize and start each worker thread
for (int i = 0; i < poolSize; i++) {
workers[i] = new PoolWorker(i + 1); // Assign a unique ID to each worker
workers[i].start();
}
}

// Method to submit a task to the thread pool
public synchronized void submit(Runnable task) {
if (!isShutdown) {
taskQueue.add(task);
notify(); // Notify a waiting worker thread about the new task
} else {
throw new IllegalStateException("ThreadPool is shut down and cannot accept tasks");
}
}

// Initiates shutdown and interrupts all waiting worker threads
public synchronized void shutdown() {
isShutdown = true;
for (PoolWorker worker : workers) {
worker.interrupt(); // Interrupts workers waiting for tasks
}
}

// Worker class that executes tasks from the queue
private class PoolWorker extends Thread {
private final int workerId;
private boolean isTerminated = false;

public PoolWorker(int id) {
this.workerId = id;
}

public void run() {
while (true) {
Runnable task;
synchronized (SimpleThreadPool.this) {
while (taskQueue.isEmpty() && !isShutdown) {
System.out.println("Worker-" + workerId + " is waiting for a task...");
try {
SimpleThreadPool.this.wait(); // Wait for a new task
} catch (InterruptedException e) {
System.out.println("Worker-" + workerId + " is terminated.");
isTerminated = true;
return; // Exit if interrupted during shutdown
}
}
if (isShutdown && taskQueue.isEmpty()) {
System.out.println("Worker-" + workerId + " is terminated as pool shuts down.");
isTerminated = true;
return; // Exit if shutdown and no tasks are left
}
task = taskQueue.poll(); // Fetch a task
}

try {
isTerminated = false; // Worker is active
System.out.println("Worker-" + workerId + " is executing task.");
task.run(); // Execute the task
System.out.println("Worker-" + workerId + " completed task and goes back to waiting.");
} catch (RuntimeException e) {
System.err.println("Task execution failed in Worker-" + workerId + ": " + e.getMessage());
}
}
}
}
}

Enhanced Task Class Implementation​

The Task class remains mostly the same, but we can add unique identifiers to differentiate tasks.

public class Task implements Runnable {
private final int taskId;

public Task(int taskId) {
this.taskId = taskId;
}

@Override
public void run() {
System.out.println("Task " + taskId + " is starting on thread " + Thread.currentThread().getName());
try {
Thread.sleep(1000); // Simulate task execution time
} catch (InterruptedException e) {
Thread.currentThread().interrupt(); // Handle interruption
}
System.out.println("Task " + taskId + " completed.");
}
}

Usage and State Transition Simulation​

Let's add a usage example showing how to submit tasks, observe workers entering a “terminated” state, and then reactivating to process new tasks.

public class ThreadPoolExample {
public static void main(String[] args) throws InterruptedException {
// Create a thread pool with 2 workers
SimpleThreadPool threadPool = new SimpleThreadPool(2);

// Submit initial tasks to the thread pool
for (int i = 1; i <= 3; i++) {
threadPool.submit(new Task(i));
}

// Wait to let initial tasks complete
Thread.sleep(3000);

// Submit additional tasks to show reactivation
System.out.println("Adding more tasks to test thread reactivation...");
for (int i = 4; i <= 5; i++) {
threadPool.submit(new Task(i));
}

// Give some time for tasks to be picked up
Thread.sleep(3000);

// Shutdown the thread pool
threadPool.shutdown();
}
}

Explanation of the Workflow and State Transitions​

  1. Initial Task Submission: Three tasks are submitted, and two workers pick them up immediately since the pool size is 2.
  2. Worker Reactivation: After the initial tasks complete, we add two more tasks, which are picked up by waiting workers.
  3. Shutdown Process: Finally, we shut down the thread pool, and the workers gracefully terminate after completing their tasks.

Expected Output (Sample)​

Worker-1 is waiting for a task...
Worker-2 is waiting for a task...
Worker-1 is executing task.
Task 1 is starting on thread Worker-1
Worker-2 is executing task.
Task 2 is starting on thread Worker-2
Task 1 completed.
Worker-1 completed task and goes back to waiting.
Task 2 completed.
Worker-2 completed task and goes back to waiting.
Worker-1 is executing task.
Task 3 is starting on thread Worker-1
Task 3 completed.
Worker-1 completed task and goes back to waiting.
Worker-1 is waiting for a task...
Worker-2 is waiting for a task...

Adding more tasks to test thread reactivation...
Worker-1 is executing task.
Task 4 is starting on thread Worker-1
Worker-2 is executing task.
Task 5 is starting on thread Worker-2
Task 4 completed.
Worker-1 completed task and goes back to waiting.
Task 5 completed.
Worker-2 completed task and goes back to waiting.
Worker-1 is terminated as pool shuts down.
Worker-2 is terminated as pool shuts down.

This output demonstrates that the workers remain active, go back to a waiting state after completing tasks, and terminate only after shutdown() is called.

How thread being actually reused?

In a typical thread pool implementation, once a thread completes its task, it does not transition to the "TERMINATED" state and then back to the "RUNNABLE" state. Instead, the thread remains in a waiting state (often referred to as "idle" or "waiting for work") until a new task is available. This waiting state is different from the "TERMINATED" state, which indicates that the thread has finished execution and cannot be reused.

Here’s a more detailed explanation:

Thread States in a Thread Pool​

  1. NEW: The thread has been created but not yet started.
  2. RUNNABLE: The thread is executing in the JVM.
  3. BLOCKED: The thread is blocked waiting for a monitor lock.
  4. WAITING: The thread is waiting indefinitely for another thread to perform a particular action.
  5. TIMED_WAITING: The thread is waiting for another thread to perform an action for up to a specified waiting time.
  6. TERMINATED: The thread has completed execution.

In a thread pool, threads typically cycle between the RUNNABLE and WAITING states:

  • RUNNABLE: When a thread is actively executing a task.
  • WAITING: When a thread is idle, waiting for a new task to be added to the queue.

How Threads Are Reused​

When a thread completes its task, it checks the task queue for new tasks. If no tasks are available, the thread waits (using wait() or a similar mechanism) until a new task is submitted. This waiting state is not the same as the "TERMINATED" state. A thread in the "TERMINATED" state cannot be reused; it must be recreated, which is inefficient.

Example of Thread Reuse in SimpleThreadPool​

In the SimpleThreadPool example, threads are reused as follows:

  1. Thread Creation: When the thread pool is created, a fixed number of worker threads are started.
  2. Task Execution: Each worker thread fetches a task from the queue and executes it.
  3. Waiting for New Tasks: After completing a task, the worker thread checks the queue for new tasks. If no tasks are available, it waits (wait()) until a new task is submitted.
  4. Reactivation: When a new task is submitted, the waiting threads are notified (notify()), and one of them wakes up to execute the new task.

Code Example​

Here’s a simplified version of the SimpleThreadPool to illustrate this:

import java.util.LinkedList;
import java.util.Queue;

public class SimpleThreadPool {
private final int poolSize;
private final PoolWorker[] workers;
private final Queue<Runnable> taskQueue;
private boolean isShutdown = false;

public SimpleThreadPool(int poolSize) {
this.poolSize = poolSize;
taskQueue = new LinkedList<>();
workers = new PoolWorker[poolSize];

// Start worker threads
for (int i = 0; i < poolSize; i++) {
workers[i] = new PoolWorker();
workers[i].start();
}
}

public synchronized void submit(Runnable task) {
if (!isShutdown) {
taskQueue.add(task);
notify(); // Notify a waiting worker thread
} else {
throw new IllegalStateException("ThreadPool is shut down");
}
}

public synchronized void shutdown() {
isShutdown = true;
for (PoolWorker worker : workers) {
worker.interrupt(); // Interrupt each worker to stop waiting
}
}

private class PoolWorker extends Thread {
public void run() {
while (true) {
Runnable task;
synchronized (SimpleThreadPool.this) {
while (taskQueue.isEmpty() && !isShutdown) {
try {
SimpleThreadPool.this.wait(); // Wait for a task
} catch (InterruptedException ignored) {
return; // Exit if interrupted during shutdown
}
}
if (isShutdown && taskQueue.isEmpty()) {
return; // Exit if shutdown and no tasks are left
}
task = taskQueue.poll(); // Take task from queue
}
try {
task.run(); // Execute task
} catch (RuntimeException e) {
System.err.println("Task execution failed: " + e.getMessage());
}
}
}
}
}

Key Points​

  • Thread Reuse: Threads do not transition to the "TERMINATED" state after completing a task. Instead, they wait for new tasks.
  • Task Queue: The task queue holds tasks that are waiting to be executed. When a new task is submitted, a waiting thread is notified and picks up the task.
  • Graceful Shutdown: The shutdown() method sets the isShutdown flag to stop accepting new tasks and interrupts all worker threads to ensure they exit if they are waiting.

Summary​

In Java’s ThreadPoolExecutor (or our custom pool example), the worker threads do not actually terminate after completing a task. Instead, they enter a waiting or idle state and wait to pick up the next task. The main goal of a thread pool is to reuse threads efficiently, keeping them alive for as long as the pool is running and ready to transition back to the runnable state whenever a new task becomes available.

Here's how it works:

  1. Runnable → Running: A thread in the pool becomes running when it starts executing a task.
  2. Running → Waiting/Idle: Once it finishes the task, it doesn't terminate. Instead, it returns to an idle state within the pool, waiting for more tasks to be added to the queue.
  3. Waiting → Runnable: When a new task is submitted, an idle worker transitions to runnable and executes the task.

The threads only reach a terminated state when the pool itself is shut down (e.g., calling shutdown() in ThreadPoolExecutor). Only then will the threads stop, exit the idle/waiting state, and go to terminated. This approach keeps the pool flexible and efficient, reducing the overhead of creating and destroying threads for every task.

So in short: threads in a thread pool do not reach terminated after completing tasks. They go back to idle/waiting and can accept new tasks. And threads in a thread pool do not go back to the "RUNNABLE" state from the "TERMINATED" state neither. Instead, they remain in a waiting state until new tasks are available, ensuring efficient reuse of thread resources.

Deep Dive into Java Core Multi-Threading Concepts

· 阅读需 5 分钟

Java multithreading is a fundamental part of the language that allows concurrent execution of multiple parts of a program to achieve greater efficiency and responsiveness. This article explores the latest Java version's core multithreading concepts, the Java Memory Model (JMM), thread lifecycle, key classes, and real-life examples.

1. Java Thread Model​

Java's thread model is based on the OS thread model, providing abstraction to manage and create threads directly. Java threads allow programs to perform multiple tasks concurrently, improving resource utilization. Thread priorities help in determining the order of thread execution, though scheduling ultimately depends on the underlying OS.

2. Java Memory Model (JMM)​

Java Thread Memory Model Diagram​

+-------------------+ +-------------------+
| | | |
| Main Memory |<----->| Thread 1 |
| | | |
+-------------------+ +-------------------+
| |
| |
v v
+-------------------+ +-------------------+
| | | |
| Thread 2 |<----->| Thread 3 |
| | | |
+-------------------+ +-------------------+

The Java Memory Model (JMM) defines how threads interact through memory and what values can be read or written by threads. The JMM provides:

  1. Visibility: Changes made by one thread are visible to other threads.
  2. Atomicity: Certain actions are indivisible or atomic.
  3. Ordering: The sequence in which instructions are executed.

JMM Key Concepts:​

  • Happens-Before Relationship: Ensures that memory writes by one specific action are visible to another specific action.
  • Volatile Keyword: Variables declared as volatile ensure visibility of changes across threads.
  • Synchronized Blocks: Using synchronized enforces an order of access to variables between threads.

3. Thread Lifecycle and State Machine​

Java threads go through several states:

StateDescription
NewThe thread is created but not yet started.
RunnableThe thread is eligible to run but may be waiting for a CPU cycle.
BlockedThe thread is waiting to acquire a lock to enter a synchronized block.
WaitingThe thread is waiting indefinitely for another thread to perform an action.
Timed WaitingThe thread is waiting for another thread for a specified period.
TerminatedThe thread has completed execution.

Thread State Machine Diagram​

NEW -> Runnable -> BLOCKED -> WAITING <-> TIMED_WAITING -> TERMINATED
+-------------------+ +-------------------+
| | | |
| NEW |------>| RUNNABLE |
| | | |
+-------------------+ +-------------------+
| |
| |
v v
+-------------------+ +-------------------+
| | | |
| BLOCKED |<----->| WAITING |
| | | |
+-------------------+ +-------------------+
| |
| |
v v
+-------------------+ +-------------------+
| | | |
| TIMED_WAITING |<----->| TERMINATED |
| | | |
+-------------------+ +-------------------+

4. Key Threading Classes in Java​

4.1 Thread Class​

The Thread class provides methods to create, start, and manage threads. You can define a custom thread by extending the Thread class and overriding its run() method.

Example:

public class MyThread extends Thread {
@Override
public void run() {
System.out.println("Thread is running");
}
}

MyThread thread = new MyThread();
thread.start();

4.2 Runnable Interface​

The Runnable interface is a functional interface representing a task to be executed on a thread. It separates the task from the thread itself, allowing reusability and flexibility.

Example:

public class MyRunnable implements Runnable {
@Override
public void run() {
System.out.println("Runnable is running");
}
}

Thread thread = new Thread(new MyRunnable());
thread.start();

4.3 Callable and Future​

Callable allows tasks to return results or throw checked exceptions, while Future represents the result of an asynchronous computation.

Example:

Callable<Integer> task = () -> {
Thread.sleep(1000);
return 123;
};

ExecutorService executor = Executors.newFixedThreadPool(1);
Future<Integer> future = executor.submit(task);

Integer result = future.get(); // blocks until the result is available
executor.shutdown();

4.4 Executors Framework​

The Executors framework provides a higher-level API for thread management, thread pooling, and scheduling, abstracting the creation and management of threads.

Example:

ExecutorService executor = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
executor.submit(() -> {
System.out.println("Task running");
});
}
executor.shutdown();

5. Synchronization and Locks​

Java provides several ways to control access to shared resources, including synchronized methods and blocks, the ReentrantLock class, and the ReadWriteLock.

Synchronized​

The synchronized keyword ensures that only one thread can access a method or block at a time.

public synchronized void increment() {
count++;
}

ReentrantLock​

ReentrantLock provides explicit lock mechanisms, giving more control over thread coordination.

Lock lock = new ReentrantLock();
lock.lock();
try {
// critical section
} finally {
lock.unlock();
}

6. Real-Life Production Example​

Consider a production scenario where we have multiple tasks needing parallel execution but require control over access to shared resources. Here’s a multithreaded solution using a ScheduledExecutorService in a banking transaction processing system.

import java.util.concurrent.*;

public class TransactionProcessor {
private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(3);

public void startProcessing() {
Runnable task = () -> {
// Simulate fetching and processing a transaction
System.out.println("Processing transaction: " + Thread.currentThread().getName());
try {
Thread.sleep(2000); // Simulating a transaction processing delay
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
};

// Schedule at fixed rate
scheduler.scheduleAtFixedRate(task, 0, 5, TimeUnit.SECONDS);
}

public static void main(String[] args) {
TransactionProcessor processor = new TransactionProcessor();
processor.startProcessing();
}
}

Explanation​

  • We use ScheduledExecutorService for periodic transaction processing.
  • The thread pool’s scheduleAtFixedRate method ensures that the task runs every 5 seconds, enabling controlled execution with set intervals.
  • This design is beneficial in systems where tasks need continuous or scheduled processing but must prevent overloading resources.

This article provides an overview of multithreading concepts, memory models, core classes, and real-world examples. Java’s concurrency mechanisms offer flexibility and control, making it essential to use these constructs carefully to manage resources efficiently and avoid pitfalls such as deadlocks and race conditions.

Deep Dive into Java Core Multi-Threading

· 阅读需 6 分钟

This article explores Java’s thread pool concept, detailing each component and state involved. We’ll cover tasks, threads, workers, thread pools, and the state transitions that occur during task execution. Detailed diagrams and annotated code examples demonstrate how each part interacts, making this a practical reference for developing multithreaded Java applications.


1. Introduction​

Java’s ThreadPoolExecutor provides a robust framework for managing a pool of worker threads that can efficiently process a series of asynchronous tasks. It’s designed to handle tasks without creating new threads for every request, conserving resources and reducing overhead. This article breaks down each term and component involved in a Java thread pool to provide a clear, practical understanding.


2. Java Thread Pool: Overview and Components​

A thread pool is a collection of threads that are managed to execute tasks concurrently. Rather than creating a new thread for every task, a pool of threads is reused, allowing efficient execution without constant thread creation and destruction.

Here’s a textual diagram illustrating a typical Java thread pool architecture:

+-------------------------------------------+
| Thread Pool |
|-------------------------------------------|
| |
| Task Queue --> [Task1] [Task2] [TaskN] |
| |
| +-------------------------------+ |
| | Worker Thread | |
| |-------------------------------| |
| | |--Thread-1--|--Running--| | |
| | |--Thread-2--|--Runnable-| | |
| | |--Thread-N--|--Waiting--| | |
| +-------------------------------+ |
+-------------------------------------------+

3. Core Terms Explained​

Task​

  • Definition: A Runnable or Callable object representing a unit of work.
  • Purpose: The task contains logic that needs to be executed by a thread in the pool.

Thread​

  • Definition: An individual worker capable of executing a task.
  • States: Threads can be in various states, including NEW, RUNNABLE, WAITING, TIMED_WAITING, BLOCKED, and TERMINATED.

Worker​

  • Definition: An entity within the thread pool that wraps a thread, providing task management.
  • Role: Handles the execution of tasks by submitting them to a thread in the pool.

Thread Pool​

  • Definition: A collection of threads managed by an executor that oversees thread lifecycle and task allocation.
  • Purpose: To improve performance by reusing threads for multiple tasks, avoiding the cost of creating and destroying threads frequently.

4. Thread Lifecycle in a Thread Pool​

A thread in a pool goes through multiple states throughout its lifecycle. Let’s illustrate these states and transitions with a state diagram.

+-----------+
| NEW | -- Thread created
+-----------+
|
v
+-----------+
| RUNNABLE | -- Thread ready to execute
+-----------+
|
v
+-----------+
| RUNNING | -- Thread actively executing a task
+-----------+
|
v
+-----------+
| BLOCKED | -- Thread blocked, waiting for a resource
+-----------+
|
v
+-----------+
| WAITING | -- Thread waiting indefinitely for a condition
+-----------+
|
v
+-----------+
| TERMINATED| -- Thread completed execution or terminated
+-----------+

5. Thread State Transitions​

Each thread’s state changes as it progresses through its lifecycle:

  1. NEW: The thread is created but has not yet started.
  2. RUNNABLE: The thread is ready to run but is waiting for CPU availability.
  3. RUNNING: The thread is actively running its assigned task.
  4. BLOCKED: The thread is waiting to acquire a lock to access a synchronized resource.
  5. WAITING/TIMED_WAITING: The thread is waiting indefinitely or for a specified time, typically for task completion or resource availability.
  6. TERMINATED: The thread has completed its task and will either be recycled for a new task or discarded if no longer needed.

6. Sample Code and Explanation​

Below is a sample code demonstrating a thread pool with comments explaining each part of the process.

import java.util.concurrent.*;

public class ThreadPoolExample {
public static void main(String[] args) {
// Create a thread pool with a fixed number of threads
ExecutorService threadPool = Executors.newFixedThreadPool(3);

// Submit multiple tasks to the thread pool
for (int i = 1; i <= 5; i++) {
threadPool.submit(new Task(i));
}

// Shutdown the pool after tasks are completed
threadPool.shutdown();
}
}

// Task class implementing Runnable, representing a unit of work for the pool
class Task implements Runnable {
private final int taskId;

public Task(int taskId) {
this.taskId = taskId;
}

@Override
public void run() {
System.out.println("Executing Task " + taskId + " with " + Thread.currentThread().getName());
try {
Thread.sleep(2000); // Simulate task duration
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
System.out.println("Task " + taskId + " completed.");
}
}

In this example:

  • ThreadPoolExecutor: Manages a pool of three threads.
  • Task: Each task simulates a job that runs for two seconds, allowing us to observe thread reuse.
  • Thread: Each task is assigned a thread by the thread pool, which either executes it immediately or queues it until a thread becomes available.

7. Handling TERMINATED Threads​

What Happens When a Thread Reaches TERMINATED?​

Once a thread in the pool completes its task and reaches the TERMINATED state, it does not automatically return to RUNNABLE. In a fixed pool, the terminated thread is replaced by a new one, ensuring the pool maintains its core size. In cached pools, the thread may be recycled if the task load increases, maintaining efficiency.

Reuse in Fixed Pools​

In fixed-size pools, a thread that terminates due to completion is usually retained within the pool as a worker for future tasks.

State Machine Transition​

The Java thread pool manages thread lifecycles by detecting TERMINATED threads and maintaining the required pool size by creating new threads as needed. This automatic management ensures the pool maintains availability and task-handling capacity.


8. Conclusion​

Java’s thread pool provides an efficient way to handle concurrent tasks by reusing threads and managing their lifecycles. Key concepts like tasks, threads, and workers work together to ensure a scalable, resource-efficient application structure. By understanding thread pool states, configurations, and transition mechanisms, developers can design more resilient and optimized concurrent applications.