Threads in JavaScript: why and when

Problem

The customer waits three times

The customer clicks "Order". Before they see "Thank you", the server has to ask three services over the network: the payment (does the bank accept the card), the stock (is it in stock) and the e-mail (has the confirmation gone out). Each one answers in about 200 ms.

Here the network is Network. call waits 200 ms and returns the service's answer. The program asks the three one after another:

const net = new Network();
const payment = await net.call("payment", "accepted");
const stock = await net.call("stock", "available");
const email = await net.call("e-mail", "sent");
console.log(payment);
console.log(stock);
console.log(email);
console.log(`Rounds of waiting: ${net.rounds}`);

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 3

Each question starts after the previous one has finished. The customer waits for the sum: about 600 ms. And the whole time the processor does nothing: it only waits for an answer.

Network also counts the rounds of waiting: a new round starts when the program asks and no other answer is being waited for. Here there are three, one for each question. In this lesson speed is measured with this count and not in milliseconds: time depends on the machine, while the number of rounds is the same on every run.

More precisely, if you are curious

This is how Network works. sleep is setTimeout from node:timers/promises, as in the lesson "Async, await and threads". There is one thread, so the counters need no protection.

class Network {
  inFlight = 0;
  rounds = 0;

  // Asks a service over the network: the answer comes after ~200 ms.
  async call(service, answer) {
    if (this.inFlight === 0) this.rounds++;   // nobody is waiting: a new round
    this.inFlight++;
    await sleep(200);
    this.inFlight--;
    return `${service}: ${answer}`;
  }
}

The idea

Work that waits and work that computes

A program's work comes in two kinds.

Work that waits (I/O-bound): a request to the bank, a query to the database, reading a file, sending an e-mail. The program asks and stands still until the answer comes. Meanwhile the processor is free.
Work that computes (CPU-bound): 1000 invoices as PDF, product thumbnails, a report over a million orders. Here the processor is busy the whole time.

In JavaScript, work that waits needs no threads at all. Node leaves the waiting to the system and goes on with other code (the lesson "Async, await and threads"). Work that computes holds the only thread. It needs real threads, Workers, on different cores.

So the first question to ask about a slow program is "what is it waiting for?". The answer decides what helps.

See it

All three at once

You send the three questions without waiting and wait for the three promises together with Promise.all (the lesson "Async, await and threads"). No thread is needed:

const net = new Network();
const [payment, stock, email] = await Promise.all([
  net.call("payment", "accepted"),
  net.call("stock", "available"),
  net.call("e-mail", "sent"),
]);
console.log(payment);
console.log(stock);
console.log(email);
console.log(`Rounds of waiting: ${net.rounds}`);

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 1

const net = new Network();
const [payment, stock, email] = await Promise.all([
  net.call("payment", "accepted"),
  net.call("stock", "available"),
  net.call("e-mail", "sent"),
]);
console.log(payment);
console.log(stock);
console.log(email);
console.log(`Rounds of waiting: ${net.rounds}`);

const net = new Network();
const payment = await net.call("payment", "accepted");
const stock = await net.call("stock", "available");
const email = await net.call("e-mail", "sent");
console.log(payment);
console.log(stock);
console.log(email);
console.log(`Rounds of waiting: ${net.rounds}`);

Step by step, in words

An order needs three answers over the network: the payment, the stock check and the confirmation e-mail. Each one waits about 200 ms.
One after another: the thread asks for the payment and does nothing until the answer comes.
Only then does it ask for the stock check. Another 200 ms of waiting.
Only then does it ask for the e-mail. Another 200 ms of waiting.
3 rounds of waiting: the customer waits about 600 ms, and the processor stands idle the whole time.
Side by side: the three questions go out at once, each on its own thread, and the three waits overlap.
The program waits for the answers. By the time the first one comes, the other two are ready too.
1 round of waiting: about 200 ms, as long as the slowest answer. Same answers, same order, a third of the wait.

The answers are the same and in the same order. There is one round of waiting: all three questions go out before any answer has come. The customer waits about 200 ms, as long as the slowest service, instead of 600.

The three timers fire in the order they were set, and Node calls the continuations one by one. Everything is on one thread.

More precisely, if you are curious

How exactly Node juggles the waits and where the trap between two awaits is, is the lesson "async is not a thread".

Try it

The e-mail waits for the payment

A colleague changes the order: the e-mail has to say whether it is paid and in stock, so it goes out only after the payment and the stock have answered.

const net = new Network();
const [payment, stock] = await Promise.all([
  net.call("payment", "accepted"),
  net.call("stock", "available"),
]);
// the e-mail says whether it is paid and in stock
console.log(payment);
console.log(stock);
console.log(await net.call("e-mail", "sent"));
console.log(`Rounds of waiting: ${net.rounds}`);

Answer

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 2

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 3

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 1

Try it

Wait for the answer

What waits for the payment promise?

const payment = net.call("payment", "accepted");
console.log( payment);

await waits for the promise and gives the result. Without it console.log prints the promise itself: Promise { <pending> }.

Problem

A thousand invoices at the end of the month

At the end of the month the shop makes a PDF invoice for each of 1000 orders. Nothing waits for the network here: the processor computes the whole time. render stands in for drawing the PDF with a computation that only loads the processor and returns a checksum:

// Renders the invoice (here: a calculation that only loads the CPU).
function render(invoice) {
  let h = invoice;
  for (let i = 0; i < 2000; i++) {
    h = (h * 31 + i) % 1_000_003;
  }
  return h;
}

// The invoices from from to to, inclusive.
function renderRange(from, to) {
  let sum = 0;
  for (let n = from; n <= to; n++) {
    sum += render(n);
  }
  return sum;
}

const total = renderRange(1, 1000);
console.log("Invoices: 1000");
console.log(`Checksum: ${total}`);

Invoices: 1000
Checksum: 499409728

One core computes everything while the others stand idle. There are no rounds of waiting here: nothing waits.

See it

Four Workers, four cores

The invoices are split into 4 parts of 250, and each part goes to its own Worker: a real thread on its own core. The Worker sends back the sum of its part with postMessage, and the main thread receives it with once(worker, "message"):

if (isMainThread) {
  const workers = [0, 1, 2, 3].map((p) => {
    const from = p * 250 + 1;
    const range = { from, to: from + 249 };
    return new Worker(new URL(import.meta.url), { workerData: range });
  });
  const parts = await Promise.all(
    workers.map(async (worker) => (await once(worker, "message"))[0]),
  );
  const total = parts.reduce((sum, part) => sum + part, 0);
  console.log("Invoices: 1000");
  console.log(`Checksum: ${total}`);
} else {
  parentPort.postMessage(renderRange(workerData.from, workerData.to));
}

Invoices: 1000
Checksum: 499409728

The checksum is the same: splitting does not change the result, only who computes it.

How much faster it is depends on the machine. On my laptop, with a computation 250 times heavier per invoice (500,000 steps instead of 2000), one core took about 1.9 s and four about 0.5 s.

These are example times, not a promise. More parts than cores do not help: they just take turns on the same cores.

More precisely, if you are curious

How many threads. For work that computes, as many as the cores: os.availableParallelism() tells you how many. For work that waits, many more is fine: they only sleep.
Not everything splits. If a tenth of the work cannot be split (for example adding up the parts or writing one file), no number of cores makes it more than 10 times faster. That is Amdahl's law.

Workers do not share ordinary objects: workerData and postMessage copy them. Shared memory exists only in SharedArrayBuffer, and that is where the problems of the next lesson come in.

The idea

What helps

	Work that waits	Work that computes
What slows it down	the answer from the network, the disk, the database	the computing itself
What helps	waiting for several things at once	more cores
In JavaScript	Promise.all, no threads	Worker threads
On one core	helps	does not help

Threads come at a price. As long as each one works with its own data (its own service, its own 250 invoices), all is well. As soon as two threads touch the same thing (a shared counter, the stock in the warehouse), errors start that do not show on every run. That is the next lesson: "Race condition".

Exercise

The product page

Solve the task in your editor.

The product page. Before it shows the page, the server asks three services: the price, the reviews and the delivery. Each answers in about 200 ms and does not depend on the others. Use Network from the first card and ask them all at once, so that there is one round of waiting:

price: 89 €
reviews: 4.6 of 5
delivery: tomorrow
Rounds of waiting: 1

More tasks

Five services. Add stock and a discount. How many rounds are there if the discount depends on the price?
How many cores. Run the invoices with 1, 2, 4 and 8 parts and time them on your machine. When does it stop getting faster? Compare with the number of cores on your machine. - The wrong tool. Run the 1000 invoices with Promise.all and an async function instead of Workers. Does it get faster? Why?

Test

Test yourself

1. An order waits for three services of 200 ms each that do not depend on each other. How long does the customer wait if you ask them all at once?About 600 ms: the sumAbout 200 ms: as long as the slowest oneAbout 67 ms: a third

2. How does JavaScript ask three services at once?With Promise.all on one thread: while the answers are on their way, the event loop is freeIt cannot: JavaScript has only one threadWith three Worker threads: otherwise they wait one after another

3. You have to compute 1000 invoices in Node on 4 cores. What speeds it up?await in a loopPromise.all with 1000 promises4 Worker threads, each with a quarter

4. Why does the lesson count rounds of waiting and not milliseconds?Because rounds are smaller numbersThe time depends on the machine and its load, while the number of rounds is the same on every runBecause milliseconds cannot be measured

Why threads

Cookies