Async, await and threads

Problem

Three reports, one at a time

The program downloads three monthly reports from the server. Here the download is a pause: await sleep(ms) waits for that many milliseconds. sleep is setTimeout from node:timers/promises, and async in front of the function allows await inside it.

import { setTimeout as sleep } from "node:timers/promises";

async function download(file, ms) {
  await sleep(ms);
  console.log(`Downloaded ${file}`);
}

await download("march.pdf", 600);
await download("april.pdf", 200);
await download("may.pdf", 400);
console.log("Done");

Downloaded march.pdf
Downloaded april.pdf
Downloaded may.pdf
Done

The pauses are 0.6, 0.2 and 0.4 seconds, 1.2 seconds in total. While it waits, Node is not locked up: you will see that it can do other things too. But each await waits for the previous download to finish before the next one starts. And the three reports do not depend on each other, so they could easily be downloaded at the same time.

The idea

Process and thread

When you start the program, the operating system gives it a process: its own memory, its own open files. Inside the process, one or more threads run. A thread is a sequence of commands that run one after another. So far, each of your programs had one thread: the one that runs the file from top to bottom.

The threads in one process share their memory: they see the same objects. That makes them handy (they can easily pass data to each other) and dangerous (they can mess each other up, as you will see in a moment).

Parallel means really at the same time, on different cores of the processor.
Concurrent means on one core, taking turns so fast that it looks like the same time.

For the programmer the difference rarely matters: in both cases you do not know which thread will go on when.

JavaScript is different. Node runs your code in one thread. When the code waits (a timer, the network, a file), Node does not stand still: it leaves the waiting to the system and goes on with other code. When the result is ready, the function that waits for it goes into a queue. This is the event loop: concurrent, in one thread.

The price: while your code is calculating without waiting, nothing else can start. The timer here is for 0 ms, but it comes out only after the loop:

setTimeout(() => console.log("the timer"), 0);
const end = Date.now() + 200;
while (Date.now() < end) {
  // 0.2 s of computing, without a single await
}
console.log("the loop is over");

the loop is over
the timer

For really parallel calculations Node has Worker threads (node:worker_threads): each one with its own memory. You will see them in the middle of the lesson.

More precisely, if you are curious

The Chrome browser starts a separate process for each tab. That is why one frozen site does not stop the others, and also why the browser uses a lot of memory.
Every program that does several things at once uses threads: a game plays the sound in one thread and runs the logic in another; a web server serves each client in its own thread.
The price: each thread has its own stack, and the memory for it is set aside separately. Thousands of threads cost a lot. That is why real programs use a thread pool (the last idea in the lesson).

See it

The first promise

download is an async function. When you call it without await, it starts and right away returns a promise (Promise): an object that says "the result will come". The program goes on:

const worker = download("april.pdf", 200);
console.log("The program does not wait");
await worker;
console.log("The report is downloaded");

The program does not wait
Downloaded april.pdf
The report is downloaded

The call without await starts the download and returns right away, so "The program does not wait" is printed first.
await worker waits for the promise to be fulfilled. This is like join() with threads in other languages.

There is no thread here: the download is a timer, and Node calls the rest of download when the timer runs out.

More precisely, if you are curious

await is the newest way to say "do this when it is ready". In older code you will come across the other two:

setTimeout(() => {
  console.log("1. a function that Node calls later (callback)");
}, 0);
await sleep(50);

sleep(50).then(() => console.log("2. then on a promise"));
await sleep(100);

await sleep(50);
console.log("3. await");

1. a function that Node calls later (callback)
2. then on a promise
3. await

A callback is a function you pass so that Node calls it later. then attaches such a function to a promise. await does the same, but the code reads from top to bottom, which is why code is written this way today.

See it

Three at once

Now we start the three downloads without waiting, and wait for the three promises together with Promise.all:

const start = Date.now();
await Promise.all([
  download("march.pdf", 600),
  download("april.pdf", 200),
  download("may.pdf", 400),
]);
const ms = Date.now() - start;
console.log(`Done in under a second: ${ms < 1000}`);

Downloaded april.pdf
Downloaded may.pdf
Downloaded march.pdf
Done in under a second: true

The downloads start in the order March, April, May, but they finish in the order of the pauses: April (0.2 s), May (0.4 s), March (0.6 s). The whole wait is as long as the longest one, 0.6 seconds instead of 1.2.

Promise.all waits for all the promises and gives you an array with their results. There is one thread: the timers run out in their order, and Node calls the continuations one by one.

Try it

await in a loop

What will the program print?

const start = Date.now();
for (const [file, ms] of [["march.pdf", 600], ["april.pdf", 200]]) {
  await download(file, ms);
}
console.log(`Took over 0.7 s: ${Date.now() - start > 700}`);

Answer

Downloaded april.pdf
Downloaded march.pdf
Took over 0.7 s: false

Downloaded march.pdf
Downloaded april.pdf
Took over 0.7 s: true

Downloaded april.pdf
Downloaded march.pdf
Took over 0.7 s: true

await in the loop waits for each download to finish before it starts the next one. So March (0.6 s) is first, and everything takes 0.8 s. This is a common mistake: the program works, only nothing runs at the same time. To run them at the same time, you start them all and wait with Promise.all.

The idea

The life of a promise

A promise is in one of three states. You cannot ask which one in code, but inspect from node:util shows it, and so does console.log:

const report = sleep(500, "may.pdf");
console.log(inspect(report));
await report;
console.log(inspect(report));

const broken = Promise.reject(new Error("no connection"));
try {
  await broken;
} catch (error) {
  console.log(`Error: ${error.message}`);
}

Promise { <pending> }
Promise { 'may.pdf' }
Error: no connection

State	When
`pending`	the work is not done yet
`fulfilled`	there is a result: `await` returns it
`rejected`	there is an error: `await` throws it, and `try`/`catch` catches it

A promise changes its state only once. After that, await on it returns the same result right away.

More precisely, if you are curious

You cancel a timer or a request with AbortController: you pass a signal to sleep or to fetch, and abort() rejects the promise with AbortError.
Promise.allSettled waits for all the promises, even if one is rejected, and Promise.race gives you the first one that is ready.

Problem

The lost visits

The site counts its visits. The counter is in a SharedArrayBuffer: memory that several Workers can see. An Int32Array reads and writes whole numbers in it. Two Workers serve visitors, and each one increases the counter 1,000,000 times. new Worker(new URL(import.meta.url)) runs the same file in a new thread, and there isMainThread is false:

import { Worker, isMainThread, workerData } from "node:worker_threads";
import { once } from "node:events";

if (isMainThread) {
  const visits = new Int32Array(new SharedArrayBuffer(4));
  const pages = [1, 2].map(() => new Worker(new URL(import.meta.url), { workerData: visits }));
  await Promise.all(pages.map((page) => once(page, "exit")));
  console.log(`Visits: ${visits[0]}`);
} else {
  for (let i = 0; i < 1_000_000; i++) {
    workerData[0]++;
  }
}

You expect 2,000,000. Five runs:

Visits: 1091916
Visits: 1063068
Visits: 1072106
Visits: 1119773
Visits: 1080911

Less every time, and different every time. The reason is that workerData[0]++ is not one step, but three:

let value = workerData[0];   // 1. read
value = value + 1;           // 2. add 1
workerData[0] = value;       // 3. write

// counts is an Int32Array in a SharedArrayBuffer: memory shared by the two Workers.
function increment(counts) {
  counts[0]++;   // read, add 1, write
}

function safeIncrement(counts) {
  Atomics.add(counts, 0, 1);
}

Step by step, in words

counts[0] is 5. Worker A and Worker B call increment() at the same time.
A reads count: 5.
B reads count before A has written: also 5.
A adds 1 in its own copy and writes 6.
B writes 6 too. Two calls, and count is 6, not 7: one of the additions is lost.
Once more, now with Atomics.add: A reads 5 and writes 6 in one indivisible step.
B reaches Atomics.add and waits until the step of A is done.
B reads 6 and writes 7: nothing is lost.

When both threads read the same value before either of them has written, one of the additions is lost. This is called a race condition. Whether it happens depends on when the operating system switches the threads, so the result is different on every run.

The idea

Atomics: the three steps at once

The solution is to turn the three steps into one that nobody can interrupt. That is what Atomics.add (an atomic, indivisible addition) does:

import { Worker, isMainThread, workerData } from "node:worker_threads";
import { once } from "node:events";

if (isMainThread) {
  const visits = new Int32Array(new SharedArrayBuffer(4));
  const pages = [1, 2].map(() => new Worker(new URL(import.meta.url), { workerData: visits }));
  await Promise.all(pages.map((page) => once(page, "exit")));
  console.log(`Visits: ${visits[0]}`);
} else {
  for (let i = 0; i < 1_000_000; i++) {
    Atomics.add(workerData, 0, 1);
  }
}

Visits: 2000000

Atomics.add(array, index, amount) reads, adds and writes as one step. The two Workers cannot split it, so the result is 2,000,000 on every run. Atomics.load reads a value in the same safe way. There are no locks like in Java: ordinary objects and arrays are not shared between Workers, they are copied.

More precisely, if you are curious

Producer and consumer in one thread does not need a lock: the kitchen waits for an order with await, and put fulfills its promise:

class OrderBox {
  #orders = [];
  #waiting = [];

  put(order) {
    const kitchen = this.#waiting.shift();
    if (kitchen) {
      kitchen(order);
    } else {
      this.#orders.push(order);
    }
  }

  take() {
    if (this.#orders.length > 0) {
      return Promise.resolve(this.#orders.shift());
    }
    return new Promise((resolve) => this.#waiting.push(resolve));
  }
}

const box = new OrderBox();
async function kitchen() {
  for (let i = 0; i < 3; i++) {
    console.log(`The kitchen took ${await box.take()}`);
  }
}
const cook = kitchen();
box.put("pizza");
box.put("salad");
box.put("soup");
await cook;

The kitchen took pizza
The kitchen took salad
The kitchen took soup

There is no deadlock between locks in one thread, because there are no locks. Workers can wait for each other with Atomics.wait, and a deadlock is possible there, but you rarely need it.
Workers have no priorities.

Try it

Two ATMs

Two people each withdraw €80 from one account with €100 at the same moment. The bank confirms each withdrawal in 0.1 s. The program runs in one thread, yet both of them get the money and the balance becomes -€60. Which line is the bug on?

Pick the line with the bug.

await is between the check and the withdrawal. While the first ATM waits for the bank, the second one passes the check: the balance is still 100. There are no threads, yet the bug is the same as with them: the check and the change are not one whole. The fix is to withdraw first, then wait:

class Account {
  balance = 100;

  async withdraw(amount) {
    if (this.balance >= amount) {
      this.balance -= amount;
      await sleep(100);   // the bank confirms
    }
  }
}

Then the second ATM sees €20 and refuses. The balance stays €20. In one thread, other code can step in only at an await.

The idea

Task pool and Promise

A thousand downloads at once overload the server. A pool limits how many tasks run at the same time: as soon as one finishes, the next one starts. Node has no ready-made pool for promises, but it takes only a few lines: runLimited starts limit "lanes", and each one takes the next task as soon as it finishes its own:

async function sizeOf(file, ms) {
  await sleep(ms);
  return file.length * 100;
}

async function runLimited(limit, jobs) {
  const results = [];
  let next = 0;
  async function lane() {
    while (next < jobs.length) {
      const i = next++;
      results[i] = await jobs[i]();
    }
  }
  await Promise.all(Array.from({ length: limit }, lane));
  return results;
}

const files = ["march.pdf", "april.pdf", "may.pdf", "june.pdf"];
const sizes = await runLimited(2, files.map((file) => () => sizeOf(file, 300)));
files.forEach((file, i) => console.log(`${file}: ${sizes[i]} KB`));

march.pdf: 900 KB
april.pdf: 900 KB
may.pdf: 700 KB
june.pdf: 800 KB

The task is a function that returns a promise: () => sizeOf(file, 300). That way it starts only when a lane takes it.
A promise is JavaScript's Future: the result will come, and await waits for it.
runLimited returns an array with the results in the order of the tasks, not in the order they finish.

For heavy calculations, the pool is made of Workers. Node does not have one ready-made either, but libraries such as Piscina do.

More precisely, if you are curious

If the task does not return anything, you only wait for runLimited:

const reports = [1, 2, 3, 4, 5].map((i) => `report-${i}.pdf`);
await runLimited(2, reports.map((file) => () => download(file, 100)));
console.log("All done.");

Downloaded report-1.pdf
Downloaded report-2.pdf
Downloaded report-3.pdf
Downloaded report-4.pdf
Downloaded report-5.pdf
All done.

The first two reports run at the same time, but in one thread: their timers run out in the order they were started, so the order is the same on every run.

Try it

Which result comes first?

"archive.zip" takes 0.6 s to download, and "note.txt" takes 0.1 s. What will the program print?

const big = sizeOf("archive.zip", 600);
const small = sizeOf("note.txt", 100);
console.log(`Big: ${await big}`);
console.log(`Small: ${await small}`);

Answer

It depends on which task finishes first

Big: 1100
Small: 800

Small: 800
Big: 1100

Exercise

Programs with async and Worker

Solve these in your editor.

1. Parallel sum. The array sales has 1,000,000 sales, sales[i] = i % 100, in an Int32Array on a SharedArrayBuffer. Split it into 4 equal parts and sum each one in its own Worker. In workerData, pass each one the array, the number of its part and the shared Float64Array partial. Each Worker writes to its own place in partial, so you do not need Atomics. Wait for them with once(worker, "exit") and add up the parts:

Parts: [12375000, 12375000, 12375000, 12375000]
Total: 49500000

2. Downloading with a pool. Download six reports (march.pdf, april.pdf, may.pdf, june.pdf, july.pdf, august.pdf) with runLimited, 3 at a time. Use sizeOf(file, 300) from the pool card. Print the sizes in the order of the files, the total, and whether it all took under 1.5 seconds (one after another would be 1.8 s):

march.pdf: 900 KB
april.pdf: 900 KB
may.pdf: 700 KB
june.pdf: 800 KB
july.pdf: 800 KB
august.pdf: 1000 KB
Total: 5100 KB
Under 1.5 seconds: true

3. A counter without losses. Four Workers, each with 50,000 visits, increase a shared counter in a SharedArrayBuffer. Write it so that the result is always 200,000. First try it with ++ and see how much is lost (it is easier to see with 1,000,000 per Worker), then fix it with Atomics.add.

Visits: 200000

More tasks

A test for the counter. Write a node:test test that starts 8 Workers with 100,000 increases each and checks for 800,000 with assert.equal. Run it against the counter with ++ a few times. A test that fails only sometimes is the typical sign of a race condition.
The ATMs. In the program with the two ATMs, put the check and the withdrawal in a separate function with no await between them, and check that the balance stays €20 even with five ATMs.

Test

Check yourself

1. What does const report = download("april.pdf", 200); do without await?It starts the download and right away gives you a promise (Promise) for itIt waits for the download to finishNothing, until you write await

2. Three downloads are in a for loop, each with await. How long do they take?Zero: await does not waitAs long as the longest oneAs long as all three together: each one starts after the previous one finishes

3. Two Workers each do workerData[0]++ 1,000,000 times on a shared SharedArrayBuffer. What will it be at the end?Always 2,000,000Always 1,000,000Often less than 2,000,000, and different on every run

4. Why does JavaScript code in one thread not need locks for its ordinary variables?Because only one function runs at a time, and another one can step in only at an awaitBecause it is fasterBecause the variables cannot change

5. You start two tasks and wait first with await big, then with await small. The small one finishes first. What is printed first?It depends on which task finishes firstThe result of the big oneThe result of the small one

Async, await and threads

Cookies