1. Introduction

In this tutorial, we’ll discuss virtual threads.

2. Motivation Behind virtual threads

Server applications typically manage concurrent user requests that operate independently. Therefore, it is logical for an application to handle each request by assigning a dedicated thread for its entire duration. This thread-per-request approach is straightforward to understand, program, debug, and profile because it aligns the application’s unit of concurrency with the platform’s unit of concurrency.

The scalability of server applications is determined by Little’s Law, which connects latency, concurrency, and throughput. According to this principle, for a given request-processing time (latency), the number of concurrent requests (concurrency) must increase in proportion to the request arrival rate (throughput). For instance, if an application with an average latency of 50ms processes 10 requests concurrently to achieve a throughput of 200 requests per second, it will need to handle 100 requests concurrently to scale to a throughput of 2000 requests per second. Consequently, if each request is managed by a dedicated thread throughout its duration, the number of threads must increase as the throughput increases to maintain performance.

Unfortunately, the number of available threads is limited because the JDK implements threads as wrappers around operating system (OS) threads. OS threads are expensive in terms of resources, so having too many of them is impractical. This makes the thread-per-request model inefficient. When each request uses a separate thread, and thus an OS thread, the number of threads often becomes a bottleneck before other resources, like CPU or network connections, are fully utilized. Consequently, the JDK’s current threading model restricts the application’s throughput to a level significantly below the hardware’s capabilities. Even using thread pooling doesn’t resolve this limitation; it merely reduces the overhead of creating new threads without increasing the total number of threads available.

To make best use of available threads, programmers create a pool of threads and also use asynchronous programming. Asynchronous programs are not straightforward to understand, write and debug.

To address these issues, virtual threads were introduced.

3. Platform Threads

A platform thread is implemented as a wrapper around an operating system (OS) thread. A platform thread runs Java code on its underlying OS thread, and the platform thread captures its OS thread for the platform thread’s entire lifetime. The number of available platform threads is limited to the number of OS threads.

Platform threads usually come with a sizable thread stack and other resources managed by the operating system. While they are suitable for executing all kinds of tasks, their availability may be limited due to their reliance on OS resources.

4. Virtual Thread

Similar to that of a platform thread, a virtual thread is also an instance of java.lang.Thread. However, a virtual thread isn’t tied to a specific OS thread. Unlike platform threads, virtual threads typically have a shallow call stack. Virtual threads are ideal for tasks that are frequently blocked, such as waiting for I/O operations to finish. However, they are not designed for long-running, CPU-intensive tasks.

A platform thread is an instance of java.lang.Thread implemented traditionally, functioning as a lightweight wrapper around an OS thread. Virtual thread consumes an OS thread only while it performs calculations on the CPU. When code running in a virtual thread calls a blocking I/O operation in the java.* API, the runtime performs a non-blocking OS call and automatically suspends the virtual thread until it can be resumed later. For Java developers, virtual threads are simply threads that are cheap to create and almost infinitely plentiful. The number of virtual threads can be much larger than the number of OS threads. Virtual threads use M:N scheduling, meaning a large number (M) of virtual threads are managed to run on a smaller number (N) of OS threads.

Virtual thread does not capture the OS thread for the code’s entire lifetime This means that many virtual threads can run their Java code on the same OS thread, effectively sharing it.

Virtual threads can significantly improve application throughput when:

• The number of concurrent tasks is high (more than a few thousand), and
• The workload is not CPU-bound, as having many more threads than processor cores cannot improve throughput in such cases.

Just like platform threads, virtual threads support thread-local variables and thread interruptions.

Virtual threads are economical and abundant, making pooling unnecessary. Each application task should spawn a new virtual thread. Consequently, most virtual threads will be short-lived with shallow call stacks, handling tasks as minimal as a single HTTP client call or JDBC query.

Java debuggers can debug virtual threads, inspect variables in stack frames, show call stacks. JDK Flight Recorder can associate application events with the correct virtual threads. Traditional thread dump can work with hundreds of threads but not thousands of threads. Therefore in jcmd a new kind of thread dump was introduced to work with virtual threads.

5. Scheduling Virtual Threads

In case of platform threads, JDK relies on the scheduler of OS for scheduling. For virtual threads, JDK has its own scheduler. JDK assignes virtual threads to platform threads and then platform threads scheduled by the OS.

The JDK’s virtual thread scheduler is a work-stealing ForkJoinPool that operates in FIFO mode. The number of platform threads available for the purpose of scheduling virtual threads determines the parallelism of the scheduler. By default it is equal to the number of available processors, but it can be configured with the system property jdk.virtualThreadScheduler.parallelism. This ForkJoinPool is different from the common pool, which is used for tasks such as implementing parallel streams and operates in LIFO mode.

The platform thread that the scheduler assigns to a virtual thread is known as the virtual thread’s carrier. Throughout its lifetime, a virtual thread can be scheduled on various carriers.

• The virtual thread cannot access the identity of its carrier. The value returned by Thread.currentThread() is always the virtual thread itself.
• The stack traces of the carrier and the virtual thread are distinct. An exception thrown in the virtual thread will not contain the carrier’s stack frames. Similarly, thread dumps will not display the carrier’s stack frames within the virtual thread’s stack, and vice versa.
• Thread-local variables of the carrier are unavailable to the virtual thread, and vice-versa.

6. Executing virtual threads

To execute code in a virtual thread, the JDK’s virtual thread scheduler mounts the virtual thread onto a platform thread, designating it as the carrier. After some execution, the virtual thread can unmount from its carrier, freeing the platform thread. The scheduler can then mount another virtual thread onto this now-available platform thread, making it a carrier once more. A virtual thread typically unmounts when it encounters blocking operations like I/O. Once the blocking operation is ready to complete, it resubmits the virtual thread to the scheduler. The scheduler then mounts the virtual thread onto a carrier to continue its execution.

The mounting and unmounting of virtual threads occur frequently and seamlessly, without blocking any OS threads.

The maximum number of platform threads available to the scheduler can be configured with the system property jdk.virtualThreadScheduler.maxPoolSize.

There are two scenarios in which a virtual thread cannot be unmounted during blocking operations because it is pinned to its carrier:

• When it executes code inside a synchronized block or method, or
• When it executes a native method or a foreign function.

Pinning can impact the scalability of the application if blocking operation is called from a pinned thread.

7. API differences between virtual threads and platform threads

• The public Thread constructors cannot create virtual threads.
• Virtual threads are always daemon threads. The Thread.setDaemon(boolean) method cannot change a virtual thread to be a non-daemon thread.
• Virtual threads have a fixed priority of Thread.NORM_PRIORITY. The Thread.setPriority(int) method has no effect on virtual threads.
• Virtual threads are not formally part of thread groups. When you invoke Thread.getThreadGroup() on a virtual thread, it returns a placeholder thread group named “VirtualThreads”. Additionally, the Thread.Builder API does not include a method to specify the thread group for a virtual thread.
• Virtual threads have no permissions when running with a SecurityManager set.

8. Thread-local variables

Virtual threads support thread-local variables (ThreadLocal) and inheritable thread-local variables (InheritableThreadLocal), similar to platform threads. This capability allows them to seamlessly execute existing code that relies on thread locals.

9. Creating a virtual thread

9.1 Thread.ofVirtual()

Call the Thread.ofVirtual() method to create an instance of Thread.Builder for creating virtual threads. Following example creates and starts a virtual Thread and prints a message “Hello”. It calls the join method to wait for the virtual thread to terminate.

public class Main {
    public static void main(String[] args) throws InterruptedException {
        Thread thread = Thread.ofVirtual().start(() -> System.out.println("Hello"));
        thread.join();
    }
}

9.2 Thread.Builder interface

Thread.Builder.OfVirtual creates virtual threads. Here is an example:

public class Main {
    public static void main(String[] args) throws InterruptedException {
        Thread.Builder builder = Thread.ofVirtual().name("MyThread");
        Runnable task = () -> {
            System.out.println("Running thread");
        };
        Thread t = builder.start(task);
        System.out.println("Thread t name: " + t.getName());
        t.join();
    }
}

10. Using Executors.newVirtualThreadPerTaskExecutor() Method

The following example creates an ExecutorService with the Executors.newVirtualThreadPerTaskExecutor() method. Whenever ExecutorService.submit(Runnable) is called, a new virtual thread is created and started to run the task.

public class Main {
    public static void main(String[] args) throws InterruptedException {
        try (ExecutorService myExecutor = Executors.newVirtualThreadPerTaskExecutor()) {
            Future<?> future = myExecutor.submit(() -> System.out.println("Running thread"));
            future.get();
            System.out.println("Task completed");
        } catch (ExecutionException e) {
            throw new RuntimeException(e);
        }
    }
}

11. Conclusion

Virtual threads in Java represent a significant advancement in concurrent programming, offering a lightweight and efficient solution for managing numerous tasks. They are particularly well-suited for tasks that are frequently blocked by I/O operations, providing a more efficient alternative to traditional platform threads for such scenarios.

While virtual threads are not intended for CPU-intensive operations, their seamless integration with existing thread-local variables and their transparent mounting and unmounting process make them a powerful tool for modern Java applications.