Virtual threads: blocking is cheap again

Final since Java 21 (JEP 444). A virtual thread that blocks in sleep, a socket read or a java.util.concurrent lock unmounts and frees its carrier thread, so plain blocking code, one thread per request, scales.

Transcript

Eight virtual threads take turns on four carrier threads: each one parks on the heap when it sleeps, freeing its carrier for the next, and resumes later on whichever carrier is free.

import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;

public class Main {
  public static void main(String[] args) {
    Set<String> carriers =
        ConcurrentHashMap.newKeySet();
    var done = new AtomicInteger();
    long t0 = System.nanoTime();
    try (var ex = Executors
        .newVirtualThreadPerTaskExecutor()) {
      for (int i = 0; i < 10_000; i++) {
        ex.submit(() -> {
          Thread.sleep(1000); // parks, carrier freed
          // demo only: toString format is unspecified
          String t =
              Thread.currentThread().toString();
          carriers.add(t.substring(t.indexOf('@')));
          done.incrementAndGet();
          return null;
        });
      }
    } // close() waits for all 10,000
    long ms = (System.nanoTime() - t0) / 1_000_000;
    System.out.println(done + " tasks, " + ms
        + " ms, " + carriers.size() + " carriers");
  }
}

On a 14 core laptop, three runs printed 10000 tasks in 1073 to 1082 ms on 14 carriers: 10,000 one second sleeps in about one second. The carrier is the tail of toString (@ForkJoinPool-1-worker-3), a format that is unspecified, so use it to look, never in real logic.

Why it works#

A virtual thread is a Java object, not an OS thread. To run, it is mounted on a carrier: a platform thread in a ForkJoinPool sized by default to the core count. When it blocks in sleep, a socket read or BlockingQueue.take, the JDK copies its stack to the heap and unmounts it, and the carrier runs another virtual thread. When the wait ends it is scheduled again, possibly on a different carrier.

What it does not buy#

CPU throughput. 14 carriers compute about as fast as 14 platform threads. Virtual threads win when tasks mostly wait.

Where it breaks on 21#

Some blocking captures the carrier instead. File I/O does (the scheduler may add a carrier to compensate), and so does blocking inside synchronized, which pins. 28 tasks, each sleeping 1 s inside synchronized on its own object, took 2.0 s here, not 1.0 s: every pinned sleeper held on to a carrier. -Djdk.tracePinnedThreads=short names the culprit. Use ReentrantLock on 21; JDK 24 (JEP 491) lets synchronized unmount too.

Do not pool them#

A pool rations expensive threads, and these are cheap. If the database takes 50 connections, guard it with a Semaphore(50).

Rule of thumb#

Write blocking code, one virtual thread per task, and put the limit on the scarce resource, not on threads. Pooling virtual threads is organizing a carpool for people who work from home.