Java Virtual Threads: A Deep Dive into Project Loom and Structured Concurrency

• 40 min read
Programming Concepts

Java Virtual Threads, introduced in Java 19 as a preview feature and stabilized in Java 21, represent a fundamental shift in how Java handles concurrency. Unlike traditional platform threads, virtual threads are lightweight, managed entirely by the JVM, enabling the creation of millions of threads without exhausting system resources. This comprehensive guide explores virtual threads in depth, covering their implementation, the M:N threading model, structured concurrency, and how they revolutionize concurrent programming in Java.

The Problem with Platform Threads

Limitations of Traditional Threading

Resource Overhead:

  • Each platform thread requires ~1-2MB of stack memory
  • OS thread creation is expensive (~1ms)
  • Context switching between threads is costly
  • Limited scalability: typically thousands of threads max

Blocking I/O Problem:

// Traditional blocking I/O
try (ServerSocket server = new ServerSocket(8080)) {
    while (true) {
        Socket socket = server.accept(); // Blocks thread
        new Thread(() -> {
            // Handle request - thread blocked during I/O
            processRequest(socket);
        }).start();
    }
}

Issues:

  • One thread per connection
  • Thread blocked during I/O operations
  • Cannot scale beyond thread limit
  • Wastes resources (thread waiting for I/O)

Why Virtual Threads?

Goals:

  • Enable millions of concurrent threads
  • Eliminate need for thread pools
  • Simplify concurrent programming
  • Maintain compatibility with existing code

Understanding Virtual Threads

What Are Virtual Threads?

Virtual threads are lightweight threads managed by the Java Virtual Machine, not the operating system. They are implemented in Java code and scheduled by the JVM onto a small number of platform threads (carrier threads).

Key Characteristics:

  • Lightweight: ~few KB memory per thread
  • Cheap to create: Millions can be created
  • Non-blocking: Suspended during blocking operations
  • Compatible: Work with existing Java code

M:N Threading Model

Traditional (1:1 Model):

Java Thread 1 → OS Thread 1
Java Thread 2 → OS Thread 2
Java Thread 3 → OS Thread 3
...

Virtual Threads (M:N Model):

Virtual Thread 1 ┐
Virtual Thread 2 ├─→ Carrier Thread 1 (OS Thread)
Virtual Thread 3 ┘
Virtual Thread 4 ┐
Virtual Thread 5 ├─→ Carrier Thread 2 (OS Thread)
Virtual Thread 6 ┘
...

Benefits:

  • M virtual threads mapped to N platform threads (M >> N)
  • Efficient resource utilization
  • Scales to millions of virtual threads

Creating and Using Virtual Threads

Creating Virtual Threads

Method 1: Thread.ofVirtual()

Thread virtualThread = Thread.ofVirtual()
    .name("virtual-thread-", 0)
    .start(() -> {
        System.out.println("Running on: " + Thread.currentThread());
    });

virtualThread.join();

Method 2: Executors.newVirtualThreadPerTaskExecutor()

try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 10_000; i++) {
        executor.submit(() -> {
            System.out.println("Task: " + Thread.currentThread());
        });
    }
}

Method 3: StructuredTaskScope (Structured Concurrency)

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Future<String> future1 = scope.fork(() -> task1());
    Future<String> future2 = scope.fork(() -> task2());
    
    scope.join();
    scope.throwIfFailed();
    
    String result1 = future1.resultNow();
    String result2 = future2.resultNow();
}

Virtual Thread Lifecycle

States:

  1. NEW: Created but not started
  2. RUNNABLE: Can be scheduled
  3. BLOCKED: Blocked on monitor lock
  4. WAITING: Waiting indefinitely
  5. TIMED_WAITING: Waiting with timeout
  6. TERMINATED: Completed execution

Key Difference: Virtual threads are unmounted when blocked, allowing carrier thread to run other virtual threads.

How Virtual Threads Work

Mounting and Unmounting

Mounting: Virtual thread attached to carrier thread for execution

Unmounting: Virtual thread detached from carrier thread when blocked

Virtual Thread Execution:
1. Virtual thread mounted on carrier thread
2. Virtual thread runs (CPU-bound work)
3. Virtual thread blocks on I/O → UNMOUNTED
4. Carrier thread picks another virtual thread
5. I/O completes → Virtual thread remounted
6. Virtual thread continues execution

When Are Virtual Threads Unmounted?

Unmounted (Non-blocking):

  • Blocking I/O operations (when using NIO)
  • Lock acquisition (ReentrantLock, synchronized)
  • Blocking queue operations
  • Sleep operations

Not Unmounted (Pinned):

  • Native method calls
  • Synchronized blocks on objects
  • Some JNI operations

Pinning Problem: Virtual thread pinned to carrier thread, blocking other virtual threads.

Carrier Threads

Default Carrier Thread Pool: ForkJoinPool with parallelism = number of CPU cores

Customization:

System.setProperty("jdk.virtualThreadScheduler.parallelism", "100");
System.setProperty("jdk.virtualThreadScheduler.maxPoolSize", "200");

Best Practice: Usually don’t need to customize - defaults are good.

Structured Concurrency

The Problem with Unstructured Concurrency

Traditional Approach:

ExecutorService executor = Executors.newCachedThreadPool();
Future<String> future1 = executor.submit(() -> task1());
Future<String> future2 = executor.submit(() -> task2());

// What if task1 fails? task2 still running!
// No way to cancel task2 automatically

Issues:

  • No relationship between tasks
  • Hard to cancel related tasks
  • Difficult to handle errors
  • Resource leaks possible

Structured Concurrency Solution

StructuredTaskScope: Ensures tasks complete together

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Future<String> future1 = scope.fork(() -> task1());
    Future<String> future2 = scope.fork(() -> task2());
    
    scope.join(); // Wait for both
    
    // If either fails, both are cancelled
    scope.throwIfFailed();
    
    return combine(future1.resultNow(), future2.resultNow());
}

Benefits:

  • Automatic cancellation: If one fails, others cancelled
  • Error propagation: Errors handled together
  • Resource management: Scope ensures cleanup
  • Observability: Better thread dumps and debugging

StructuredTaskScope Variants

ShutdownOnFailure: Cancel all if any fails

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Future<String> f1 = scope.fork(() -> task1());
    Future<String> f2 = scope.fork(() -> task2());
    
    scope.join();
    scope.throwIfFailed(); // Throws if any failed
    
    return f1.resultNow() + f2.resultNow();
}

ShutdownOnSuccess: Cancel all if any succeeds

try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
    Future<String> f1 = scope.fork(() -> slowTask1());
    Future<String> f2 = scope.fork(() -> slowTask2());
    
    scope.join();
    return scope.result(); // Returns first successful result
}

Virtual Threads vs Platform Threads

Performance Comparison

Thread Creation:

// Platform threads: ~1ms per thread
long start = System.currentTimeMillis();
for (int i = 0; i < 10_000; i++) {
    new Thread(() -> {}).start();
}
// Takes ~10 seconds

// Virtual threads: ~microseconds per thread
long start = System.currentTimeMillis();
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 10_000; i++) {
        executor.submit(() -> {});
    }
}
// Takes ~milliseconds

Memory Usage:

Platform Thread: ~1-2MB stack
Virtual Thread: ~few KB

Scalability:

Platform Threads: Thousands (limited by OS)
Virtual Threads: Millions (limited by heap)

When to Use Each

Use Virtual Threads For:

  • I/O-bound tasks
  • High concurrency requirements
  • Simple concurrent code
  • Replacing thread pools

Use Platform Threads For:

  • CPU-bound tasks
  • Long-running background tasks
  • Tasks requiring OS thread features
  • Native code integration

Best Practices

1. Don’t Pool Virtual Threads

Wrong:

// DON'T DO THIS!
ExecutorService pool = Executors.newVirtualThreadPerTaskExecutor();
// Reusing pool defeats the purpose

Right:

// Create new executor per use case
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    executor.submit(() -> task());
}

2. Avoid Thread-Local Variables

Problem: Thread-local variables don’t work well with virtual threads (can be expensive)

Solution: Use scoped values (Java 20+)

// Instead of ThreadLocal
ScopedValue<String> CONTEXT = ScopedValue.newInstance();

ScopedValue.runWhere(CONTEXT, "value", () -> {
    // Access CONTEXT.get() here
});

3. Be Careful with Synchronized

Problem: Synchronized blocks can pin virtual threads

Solution: Use ReentrantLock instead

// BAD: Can pin virtual thread
synchronized (lock) {
    // Blocking operation
}

// GOOD: Unmounts virtual thread
ReentrantLock lock = new ReentrantLock();
lock.lock();
try {
    // Blocking operation
} finally {
    lock.unlock();
}

4. Use Structured Concurrency

Always use StructuredTaskScope for related tasks:

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    // Fork related tasks
    // Automatic cleanup and cancellation
}

Real-World Examples

HTTP Server with Virtual Threads

Traditional (Platform Threads):

try (ServerSocket server = new ServerSocket(8080)) {
    ExecutorService pool = Executors.newFixedThreadPool(100);
    while (true) {
        Socket socket = server.accept();
        pool.submit(() -> handleRequest(socket));
    }
}
// Limited to 100 concurrent connections

With Virtual Threads:

try (ServerSocket server = new ServerSocket(8080)) {
    while (true) {
        Socket socket = server.accept();
        Thread.ofVirtual().start(() -> handleRequest(socket));
    }
}
// Can handle millions of concurrent connections

Database Connection Pooling

Traditional Approach:

// Thread pool size = connection pool size
ExecutorService executor = Executors.newFixedThreadPool(10);
ConnectionPool pool = new ConnectionPool(10);

executor.submit(() -> {
    Connection conn = pool.getConnection();
    try {
        // Use connection
    } finally {
        pool.release(connection);
    }
});

With Virtual Threads:

// Can have more virtual threads than connections
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    executor.submit(() -> {
        Connection conn = pool.getConnection();
        try {
            // Use connection - virtual thread unmounts if connection busy
        } finally {
            pool.release(connection);
        }
    });
}

Migration Strategies

Migrating from Thread Pools

Step 1: Identify I/O-bound tasks

Step 2: Replace ExecutorService:

// Before
ExecutorService executor = Executors.newFixedThreadPool(100);

// After
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();

Step 3: Remove thread pool size limits

Step 4: Test thoroughly

Migrating from CompletableFuture

Before:

CompletableFuture<String> future1 = CompletableFuture.supplyAsync(() -> task1());
CompletableFuture<String> future2 = CompletableFuture.supplyAsync(() -> task2());
CompletableFuture<String> combined = future1.thenCombine(future2, (a, b) -> a + b);

After:

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Future<String> future1 = scope.fork(() -> task1());
    Future<String> future2 = scope.fork(() -> task2());
    
    scope.join();
    scope.throwIfFailed();
    
    String combined = future1.resultNow() + future2.resultNow();
}

Performance Considerations

CPU-Bound Tasks

Virtual threads are NOT for CPU-bound tasks:

// BAD: CPU-bound work on virtual threads
Thread.ofVirtual().start(() -> {
    for (int i = 0; i < 1_000_000; i++) {
        // CPU-intensive computation
    }
});

Use platform threads or ForkJoinPool for CPU-bound work:

// GOOD: CPU-bound work on platform threads
ForkJoinPool pool = ForkJoinPool.commonPool();
pool.submit(() -> {
    for (int i = 0; i < 1_000_000; i++) {
        // CPU-intensive computation
    }
});

Monitoring Virtual Threads

Thread Dumps: Virtual threads appear in thread dumps

JFR (Java Flight Recorder): Track virtual thread creation, mounting, unmounting

Metrics: Monitor virtual thread count, carrier thread utilization

Limitations and Considerations

Pinning Scenarios

Virtual threads can be pinned (not unmounted) in:

  • Synchronized blocks
  • Native method calls
  • Some JNI operations

Impact: Pinned virtual threads block carrier threads

Solution: Minimize synchronized blocks, use ReentrantLock

Thread-Local Variables

Problem: Expensive with virtual threads (many threads)

Solution: Use ScopedValue (Java 20+) or avoid thread-locals

Debugging

Challenge: Many threads make debugging harder

Solution: Use structured concurrency, better observability tools

Comparison with Other Approaches

Virtual Threads vs Reactive Programming

Virtual Threads:

  • Synchronous code style
  • Easy to understand
  • Good for I/O-bound tasks
  • Millions of threads

Reactive Programming:

  • Asynchronous code style
  • Steeper learning curve
  • Good for streaming data
  • Backpressure handling

When to Use:

  • Virtual Threads: Simple I/O-bound applications
  • Reactive: Streaming, backpressure needs, event-driven

Virtual Threads vs Async/Await (Other Languages)

Similarities:

  • Both enable high concurrency
  • Both handle I/O efficiently
  • Both simplify concurrent code

Differences:

  • Virtual threads: JVM-managed, work with existing code
  • Async/await: Language feature, requires async/await syntax

Conclusion

Java Virtual Threads represent a paradigm shift in Java concurrency, enabling developers to write simple, synchronous code that scales to millions of concurrent operations. By understanding how virtual threads work, when to use them, and best practices, developers can build highly scalable applications without the complexity of reactive programming or thread pool management.

Key takeaways:

  1. Virtual threads are lightweight - millions can be created
  2. Use for I/O-bound tasks - not CPU-bound
  3. Don’t pool virtual threads - create as needed
  4. Use structured concurrency - StructuredTaskScope
  5. Avoid synchronized blocks - use ReentrantLock
  6. Monitor for pinning - ensure virtual threads unmount

Virtual threads make concurrent programming in Java simpler and more scalable, bringing Java’s concurrency model into the modern era while maintaining compatibility with existing code.