async is not a thread

Code in:JavaC#PythonJavaScript

In short: asyncio runs coroutines on one thread with an event loop: await releases the thread while waiting, so a race between two waits is possible even without threads.

Problem

The last item, sold twice

The sale is in full swing. One pair of headphones is left. checkout checks the stock, waits for the bank to confirm the payment and only then decreases the stock. Two buyers press "Buy" at the same moment:

# The bank confirms the payment in 100 ms.
async def pay(buyer):
    await asyncio.sleep(0.1)


class Shop:
    def __init__(self):
        self.stock = 1
        self.sold = 0

    async def checkout(self, buyer):
        left = self.stock
        if left == 0:
            return
        await pay(buyer)
        self.stock = left - 1
        self.sold += 1


async def main():
    shop = Shop()
    await asyncio.gather(shop.checkout("Ann"), shop.checkout("Bob"))
    print(f"Sold: {shop.sold}, stock: {shop.stock}")


asyncio.run(main())

Output

Sold: 2, stock: 0

Two are sold, and there is one item. One buyer will get an e-mail "Your order has been shipped" for headphones that do not exist.

In asyncio this looks impossible: the code runs in one thread. In the lesson on race conditions two threads collided on count += 1. Here there is no second thread.

The output is the same on every run: this is not bad luck, it is the order in which the code runs. To see it, we need to know what await does.

The idea

One thread and many waits

Most of the work in an online shop is work that waits (see work that waits and work that computes): the payment waits for the bank, the e-mail waits for the mail server, the stock waits for the warehouse. A thread that sits and waits is wasted.

An event loop is one thread and a queue of pieces of code that are ready. The thread takes the next piece and runs it to the end. When the code reaches a wait, it does not hold the thread: it says "wake me up when the bank answers" and lets go. The thread takes the next ready piece. When the bank answers, the rest of the function (the continuation) goes into the queue and waits for its turn.

In Python the event loop is in asyncio. async def makes a coroutine: a function that can stop in the middle. await is the place where it stops and lets go of the thread. asyncio.run(main()) starts the event loop and the coroutine main in it.

Three requests to three services, started at once:

class Calls:
    def __init__(self):
        self.waiting = 0
        self.most = 0

    async def call(self, name, ms):
        self.waiting += 1
        self.most = max(self.most, self.waiting)
        await asyncio.sleep(ms / 1000)
        self.waiting -= 1
        print(f"{name}: done after {ms} ms")


async def main():
    calls = Calls()
    await asyncio.gather(
        calls.call("payment", 300),
        calls.call("e-mail", 100),
        calls.call("warehouse", 200),
    )
    print(f"Waiting at once: {calls.most}")


asyncio.run(main())

Output

e-mail: done after 100 ms
warehouse: done after 200 ms
payment: done after 300 ms
Waiting at once: 3

asyncio.gather starts the three coroutines and waits for all of them. Each runs up to await asyncio.sleep(...) and lets go of the thread, so all three enter call before any of them finishes.

"Waiting at once: 3": all three requests wait at the same time, and there is only one thread. Everything takes about 300 ms (the longest), not 600 (the sum). The thread did not wait for any of them.

Try it

How many wait at once

The same call, but the first request is awaited on its own:

What will the program print?

async def main():
    calls = Calls()
    await calls.call("payment", 300)
    await asyncio.gather(calls.call("e-mail", 100), calls.call("warehouse", 200))
    print(f"Waiting at once: {calls.most}")


asyncio.run(main())

Answer

e-mail: done after 100 ms
warehouse: done after 200 ms
payment: done after 300 ms
Waiting at once: 3

payment: done after 300 ms
e-mail: done after 100 ms
warehouse: done after 200 ms
Waiting at once: 3

payment: done after 300 ms
e-mail: done after 100 ms
warehouse: done after 200 ms
Waiting at once: 2

Check

See it

A race between two awaits

Back to the last item. This is how the two orders interleave:

class Shop:
    def __init__(self):
        self.stock = 1
        self.sold = 0

    async def checkout(self, buyer):
        left = self.stock
        if left == 0:
            return
        await pay(buyer)
        self.stock = left - 1
        self.sold += 1

Step by step, in words

1. One item is left. Two orders arrive at the same moment: Ann and Bob.
2. Ann's checkout reads the stock: left is 1. At await it lets go of the thread while the bank answers.
3. The thread is free, so Bob's checkout runs. It reads 1 as well: Ann has not written anything yet.
4. The bank answers Ann. Her code goes on after the await: stock = left - 1 = 0, sold = 1. Correct so far.
5. Bob goes on with his old left = 1: stock = 0 again, sold = 2. One item, two sales, with only one thread.

Between two awaits the code runs without interruption. But at an await the function lets go of the thread, and meanwhile anything else can run. When it goes on, the world may be different: left is a value read before the wait.

This is the same "read, change, write" as with threads, only the interruption is not just anywhere, it is exactly at the await. That is also why the output is the same on every run: the interruption is not random.

The idea

The fix: reserve the item before the await

The check and the decrease must be one piece of code with no await between them. The order reserves the item at once and only then waits for the bank:

class Shop:
    def __init__(self):
        self.stock = 1
        self.sold = 0

    async def checkout(self, buyer):
        if self.stock == 0:
            return
        self.stock -= 1  # reserve the item before the await
        await pay(buyer)
        self.sold += 1

Output

Sold: 1, stock: 0

There is no await between if self.stock == 0 and self.stock -= 1, so no other coroutine can get in between. Bob sees 0 and leaves.

If the payment fails, the item must be returned. That is the exercise at the end.

More precisely, if you are curious

Sometimes the whole piece must be locked together with the wait: for example, only one order at a time may write to an old warehouse system that cannot take two. Then you need a lock that understands await:

asyncio.Lock(): async with lock: lets go of the thread while it waits for the lock. threading.Lock does not work here: it blocks the whole thread, and the event loop with it.

How it is in the other languages: in JavaScript and in asyncio there is one thread, and the piece between two awaits runs alone. In C# the continuations are in the thread pool, so a lock is needed there too. Java has no await: there the same thing is done with CompletableFuture, and since Java 21 also with virtual threads, where you write ordinary blocking code and the JVM unmounts the thread by itself while it waits.

Try it

Find the bug

The cart adds up the prices of three orders. The price comes from the warehouse with await. The three orders cost €25 each, and the total comes out as 25:

class Cart:
    def __init__(self):
        self.total = 0

    async def add_order(self, order_id):
        self.total += await price_of(order_id)


async def main():
    cart = Cart()
    await asyncio.gather(cart.add_order(1), cart.add_order(2), cart.add_order(3))
    print(f"Total: €{cart.total}")


asyncio.run(main())

Output

Total: €25

Pick the line with the bug.

1class Cart:2 def __init__(self):3 self.total = 045 async def add_order(self, order_id):6 self.total += await price_of(order_id)

Check

The idea

Do not block the event loop

The event loop works only if no piece of code holds the thread for long. The sales report is made with time.sleep(0.3): it sleeps without await. At the same time a buyer opens a product page, which waits 50 ms for the warehouse:

async def report():
    time.sleep(0.3)  # blocks: waits 300 ms without await
    print("The report is ready")


async def product_page(buyer, start):
    await asyncio.sleep(0.05)
    print(f"{buyer} waits over 0.25 s: {time.monotonic() - start > 0.25}")


async def main():
    start = time.monotonic()
    await asyncio.gather(report(), product_page("Ann", start))


asyncio.run(main())

Output

The report is ready
Ann waits over 0.25 s: True

The page waited over 0.25 s instead of 50 ms. While the report holds the thread, no other continuation can run: not this page, not payments, not timers. On a server with thousands of buyers, they all freeze at once.

The fix is to keep the heavy work out of the event loop's thread. It goes to a thread pool, and the event loop only waits for the result:

def build_report():
    time.sleep(0.3)


async def report():
    await asyncio.to_thread(build_report)
    print("The report is ready")


async def product_page(buyer):
    await asyncio.sleep(0.05)
    print(f"{buyer}: the page is ready")


async def main():
    await asyncio.gather(report(), product_page("Ann"))


asyncio.run(main())

Output

Ann: the page is ready
The report is ready

await asyncio.to_thread(build_report) runs the function in a thread pool and waits for the result without holding the event loop. If the work computes (rather than sleeps), the GIL means you need a pool of processes: await asyncio.get_running_loop().run_in_executor(processes, build_report), where processes is a ProcessPoolExecutor.

Try it

Fill in

Start the three requests at once and wait for them together:

async def main():
    calls = Calls()
    await (
        calls.call("payment", 300),
        calls.call("e-mail", 100),
        calls.call("warehouse", 200),
    )
    print(f"Waiting at once: {calls.most}")


asyncio.run(main())

Check

Exercise

Reserve, pay, return

Solve it in your own editor.

Two items are left. Four buyers order at once: Ann, Bob, Vera and Gina. The payment takes 100 ms. Bob's card is declined: the payment throws an exception with the text "card declined".

checkout reserves the item before the await, then pays. If the payment fails, the item is returned. Print each buyer's result in the order of the orders, and at the end the stock and the number sold:

Output

Ann: paid
Bob: card declined, the item is returned
Vera: out of stock
Gina: out of stock
Stock: 1, sold: 1

Vera and Gina see "out of stock", because Bob's item is still reserved until the bank answers. That is fair: the returned item goes to the next buyer.

More tasks

• Try again. If a buyer finds no stock but another one may return an item, let them wait up to 200 ms and try again.
• Timeout. The bank sometimes does not answer. If the payment takes over 1 s, decline the order and return the item. (asyncio.wait_for)
• Report. Work out the total of the sales (€25 each) and print it at the end.

Test

Check yourself

1. In how many threads does your code run in JavaScript (Node) and in Python asyncio?

In one: the event loop runs the pieces of code one by oneIn one thread per awaitIn as many as there are cores

2. Three requests wait 100, 200 and 300 ms. How long in total if you await them one by one, and how long with Promise.all, gather, Task.WhenAll or allOf?

About 300 ms both waysAbout 600 ms and about 300 msAbout 600 ms both ways

3. In JavaScript and Python an async function reads shared data, then awaits, then writes. When can other code change the data?

Only while it waits at the awaitNever: there is one threadBetween any two lines

4. Handling a request includes a computation that takes 2 seconds, without a single await. What happens to the other requests in Node or asyncio?

Nothing: the async function runs separatelyThey move to another threadThey wait 2 seconds: the event loop is busy

5. How do you avoid selling the last item twice?

Run the orders in two threadsCheck and decrease the stock with no await between them, then pay; if the payment fails, return the itemAdd one more await before the write

Check

Start the lesson over

Related lessons

• Builds on: Why threads · Race condition · Locking · A pool of threads and tasks
• Continues in: Testing concurrent code
• Tasks on this topic: Concurrency and race conditions