async and await in C#: Task, the thread pool and races

Problem

The last item, sold twice

The sale is in full swing. One pair of headphones is left. checkout checks the stock, waits for the bank to confirm the payment and only then decreases the stock. Two buyers press "Buy" at the same moment:

int stock = 1;
int sold = 0;

async Task Checkout(string buyer)
{
    int left = stock;
    if (left == 0) return;
    await Pay(buyer);
    stock = left - 1;
    Interlocked.Increment(ref sold);
}

await Task.WhenAll(Checkout("Ann"), Checkout("Bob"));
Console.WriteLine($"Sold: {sold}, stock: {stock}");

// The bank confirms the payment in 100 ms.
async Task Pay(string buyer)
{
    await Task.Delay(100);
}

Sold: 2, stock: 0

Two are sold, and there is one item. One buyer will get an e-mail "Your order has been shipped" for headphones that do not exist.

There is no Thread and no Parallel here. In the lesson on race conditions two threads collided on count++. Here the two orders are not even in the processor at the same time: both just wait for the bank.

The output is the same on every run: this is not bad luck, it is the order in which the code runs. To see it, we need to know what await does.

The idea

One thread and many waits

Most of the work in an online shop is work that waits (see work that waits and work that computes): the payment waits for the bank, the e-mail waits for the mail server, the stock waits for the warehouse. A thread that sits and waits is wasted.

An event loop is one thread and a queue of pieces of code that are ready. The thread takes the next piece and runs it to the end. When the code reaches a wait, it does not hold the thread: it says "wake me up when the bank answers" and lets go. The thread takes the next ready piece. When the bank answers, the rest of the function (the continuation) goes into the queue and waits for its turn.

In C# await also lets go of the thread while the task waits. But in a console program and in ASP.NET Core there is no single event loop thread. The continuation after await runs in the thread pool (the lesson on the thread pool): in whichever thread is free. So two continuations can really run at the same time. WPF and WinForms are different: there the window has its own thread with a message queue, and the continuation comes back to it, as with an event loop.

Three requests to three services, started at once:

int waiting = 0;
int most = 0;

async Task<string> Call(string name, int ms)
{
    int now = Interlocked.Increment(ref waiting);
    most = Math.Max(most, now);
    await Task.Delay(ms);
    Interlocked.Decrement(ref waiting);
    return $"{name}: done after {ms} ms";
}

string[] done = await Task.WhenAll(
    Call("payment", 300),
    Call("e-mail", 100),
    Call("warehouse", 200));
foreach (string line in done)
{
    Console.WriteLine(line);
}
Console.WriteLine($"Waiting at once: {most}");

payment: done after 300 ms
e-mail: done after 100 ms
warehouse: done after 200 ms
Waiting at once: 3

Each Call runs up to await Task.Delay(ms) and returns an unfinished task. Task.WhenAll waits for all three and gives the results in the order of the tasks, so the output is in that order. waiting is decreased with Interlocked, because the continuations are in the pool and may run at the same time.

"Waiting at once: 3": all three requests wait at the same time, and there is only one thread. Everything takes about 300 ms (the longest), not 600 (the sum). The thread did not wait for any of them.

Try it

How many wait at once

The same call, but the first request is awaited on its own:

What will the program print?

string first = await Call("payment", 300);
string[] rest = await Task.WhenAll(Call("e-mail", 100), Call("warehouse", 200));
Console.WriteLine($"Waiting at once: {most}");

Answer

Waiting at once: 2

Waiting at once: 1

Waiting at once: 3

The first await Call(...) waits for the payment to the end: by then waiting is 0 again. Only after that does Task.WhenAll start the other two at once, so the most is 2. The whole thing takes about 500 ms: 300, then 200.

await right after the call means "wait for this before you go on". For them to wait together, start them all first, then wait.

See it

A race between two awaits

Back to the last item. This is how the two orders interleave:

int stock = 1;
int sold = 0;

async Task Checkout(string buyer)
{
    int left = stock;
    if (left == 0) return;
    await Pay(buyer);
    stock = left - 1;
    Interlocked.Increment(ref sold);
}

Step by step, in words

One item is left. Two orders arrive at the same moment: Ann and Bob.
Ann's checkout reads the stock: left is 1. At await it lets go of the thread while the bank answers.
The thread is free, so Bob's checkout runs. It reads 1 as well: Ann has not written anything yet.
The bank answers Ann. Her code goes on after the await: stock = left - 1 = 0, sold = 1. Correct so far.
Bob goes on with his old left = 1: stock = 0 again, sold = 2. One item, two sales, and the gap is the await, not a second thread.

Between two awaits the code runs without interruption. But at an await the function lets go of the thread, and meanwhile anything else can run. When it goes on, the world may be different: left is a value read before the wait.

This is the same "read, change, write" as with threads, only the interruption is not just anywhere, it is exactly at the await. That is also why the output is the same on every run: the interruption is not random.

C# has one more thing: Ann's and Bob's continuations run in the pool and may run at the same time in two threads. Here the bank answers them at different times, so they do not collide. But sold++ in two threads at once loses increments, as in the lesson on race conditions. That is why sold is increased with Interlocked.Increment.

The idea

The fix: reserve the item before the await

The check and the decrease must be one piece of code with no await between them. The order reserves the item at once and only then waits for the bank:

int stock = 1;
int sold = 0;
object sync = new object();

async Task Checkout(string buyer)
{
    lock (sync)
    {
        if (stock == 0) return;
        stock--;   // reserve the item before the await
    }
    await Pay(buyer);
    Interlocked.Increment(ref sold);
}

Sold: 1, stock: 0

The check and stock-- are inside a lock, because Checkout can be called from several threads at once: the requests in a web server come from the pool. C# does not allow await inside a lock (error CS1996): the lock belongs to a thread, and after await the continuation may be in another one. So the lock guards only the piece without await.

If the payment fails, the item must be returned. That is the exercise at the end.

More precisely, if you are curious

Sometimes the whole piece must be locked together with the wait: for example, only one order at a time may write to an old warehouse system that cannot take two. Then you need a lock that understands await:

SemaphoreSlim(1, 1): await gate.WaitAsync() waits without holding a thread, and gate.Release() goes in finally. That way a piece with await inside can be guarded too.

How it is in the other languages: in JavaScript and in asyncio there is one thread, and the piece between two awaits runs alone. In C# the continuations are in the thread pool, so a lock is needed there too. Java has no await: there the same thing is done with CompletableFuture, and since Java 21 also with virtual threads, where you write ordinary blocking code and the JVM unmounts the thread by itself while it waits.

Try it

Find the bug

The cart adds up the prices of three orders. The price comes from the warehouse with await. The three orders cost €25 each, and the total comes out as 25:

int total = 0;

async Task AddOrder(int id)
{
    total += await PriceOf(id);
}

await Task.WhenAll(AddOrder(1), AddOrder(2), AddOrder(3));
Console.WriteLine($"Total: €{total}");

Total: €25

Pick the line with the bug.

total += await PriceOf(id) is total = total + await PriceOf(id): C# reads total before the await. All three read 0 and write 0 + 25. Wait first, then add: int price = await PriceOf(id); total += price;. (Here the continuations are also in the pool, so in real code use Interlocked.Add as well.)

The bug is not in the threads but in the order: a value read before an await is stale after it.

The idea

Do not block the event loop

The event loop works only if no piece of code holds the thread for long. The sales report is made with Thread.Sleep(300): it sleeps without await. At the same time a buyer opens a product page, which waits 50 ms for the warehouse:

var watch = Stopwatch.StartNew();
await Task.WhenAll(Report(), ProductPage("Ann"));

async Task Report()
{
    Thread.Sleep(300);   // blocks: waits 300 ms without await
    Console.WriteLine("The report is ready");
    await Task.Delay(10);
}

async Task ProductPage(string buyer)
{
    await Task.Delay(50);
    Console.WriteLine($"{buyer} waits over 0.25 s: {watch.ElapsedMilliseconds > 250}");
}

The report is ready
Ann waits over 0.25 s: True

The page waited over 0.25 s instead of 50 ms. While the report holds the thread, no other continuation can run: not this page, not payments, not timers. On a server with thousands of buyers, they all freeze at once.

In C# Thread.Sleep in an async method blocks the thread that runs it. Here that is the main thread, before the first await, so ProductPage has not even started. Waiting synchronously (Thread.Sleep, .Result, .Wait()) in async code is the same mistake: use await Task.Delay and await. In a WPF or WinForms application the same thing freezes the window: the buttons do not respond while the window's thread is busy.

The fix is to keep the heavy work out of the event loop's thread. It goes to a thread pool, and the event loop only waits for the result:

await Task.WhenAll(Report(), ProductPage("Ann"));

async Task Report()
{
    await Task.Run(() => Thread.Sleep(300));
    Console.WriteLine("The report is ready");
}

async Task ProductPage(string buyer)
{
    await Task.Delay(50);
    Console.WriteLine($"{buyer}: the page is ready");
}

Ann: the page is ready
The report is ready

await Task.Run(...) sends the work to the pool and lets go of the thread. The page is ready after 50 ms, the report after 300.

Try it

Fill in

Start the three requests at once and wait for them together:

string[] done = await (
    Call("payment", 300),
    Call("e-mail", 100),
    Call("warehouse", 200));
foreach (string line in done)
{
    Console.WriteLine(line);
}
Console.WriteLine($"Waiting at once: {most}");

Exercise

Reserve, pay, return

Solve it in your own editor.

Two items are left. Four buyers order at once: Ann, Bob, Vera and Gina. The payment takes 100 ms. Bob's card is declined: the payment throws an exception with the text "card declined".

checkout reserves the item before the await, then pays. If the payment fails, the item is returned. Print each buyer's result in the order of the orders, and at the end the stock and the number sold:

Ann: paid
Bob: card declined, the item is returned
Vera: out of stock
Gina: out of stock
Stock: 1, sold: 1

Vera and Gina see "out of stock", because Bob's item is still reserved until the bank answers. That is fair: the returned item goes to the next buyer.

More tasks

Try again. If a buyer finds no stock but another one may return an item, let them wait up to 200 ms and try again.
Timeout. The bank sometimes does not answer. If the payment takes over 1 s, decline the order and return the item. (Task.WaitAsync)
Report. Work out the total of the sales (€25 each) and print it at the end.

Test

Check yourself

1. In how many threads does your code run in JavaScript (Node) and in Python asyncio?In one: the event loop runs the pieces of code one by oneIn one thread per awaitIn as many as there are cores

2. Three requests wait 100, 200 and 300 ms. How long in total if you await them one by one, and how long with Promise.all, gather, Task.WhenAll or allOf?About 300 ms both waysAbout 600 ms and about 300 msAbout 600 ms both ways

3. In JavaScript and Python an async function reads shared data, then awaits, then writes. When can other code change the data?Only while it waits at the awaitNever: there is one threadBetween any two lines

4. Handling a request includes a computation that takes 2 seconds, without a single await. What happens to the other requests in Node or asyncio?Nothing: the async function runs separatelyThey move to another threadThey wait 2 seconds: the event loop is busy

5. How do you avoid selling the last item twice?Run the orders in two threadsCheck and decrease the stock with no await between them, then pay; if the payment fails, return the itemAdd one more await before the write

async is not a thread

Cookies