Threads in C#: why and when

Problem

The customer waits three times

The customer clicks "Order". Before they see "Thank you", the server has to ask three services over the network: the payment (does the bank accept the card), the stock (is it in stock) and the e-mail (has the confirmation gone out). Each one answers in about 200 ms.

Here the network is Network. Call waits 200 ms and returns the service's answer. The program asks the three one after another:

Network net = new Network();
string payment = net.Call("payment", "accepted");
string stock = net.Call("stock", "available");
string email = net.Call("e-mail", "sent");
Console.WriteLine(payment);
Console.WriteLine(stock);
Console.WriteLine(email);
Console.WriteLine($"Rounds of waiting: {net.Rounds}");

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 3

Each question starts after the previous one has finished. The customer waits for the sum: about 600 ms. And the whole time the processor does nothing: it only waits for an answer.

Network also counts the rounds of waiting: a new round starts when the program asks and no other answer is being waited for. Here there are three, one for each question. In this lesson speed is measured with this count and not in milliseconds: time depends on the machine, while the number of rounds is the same on every run.

More precisely, if you are curious

This is how Network works. lock is from the lesson "Threads": the two counters are touched by several threads at once. Why numbers get lost without it is the next lesson, "Race condition".

class Network
{
    private readonly object sync = new object();
    private int inFlight = 0;
    private int rounds = 0;

    // Asks a service over the network: the answer comes after ~200 ms.
    public string Call(string service, string answer)
    {
        lock (sync)
        {
            if (inFlight == 0) rounds++;   // nobody is waiting: a new round
            inFlight++;
        }
        Thread.Sleep(200);
        lock (sync)
        {
            inFlight--;
        }
        return $"{service}: {answer}";
    }

    public int Rounds
    {
        get { lock (sync) return rounds; }
    }
}

The idea

Work that waits and work that computes

A program's work comes in two kinds.

Work that waits (I/O-bound): a request to the bank, a query to the database, reading a file, sending an e-mail. The program asks and stands still until the answer comes. Meanwhile the processor is free.
Work that computes (CPU-bound): 1000 invoices as PDF, product thumbnails, a report over a million orders. Here the processor is busy the whole time.

A thread that waits does not use the processor. That is why threads help work that waits even on one core: the first thread asks and goes to sleep, the second asks and goes to sleep, so does the third, and the three waits run together. Work that computes needs a core the whole time. It only gets faster with more cores, one per thread.

So the first question to ask about a slow program is "what is it waiting for?". The answer decides what helps.

See it

All three at once

Each question gets its own thread. First you start all three with Start, then you wait for all three with Join (the lesson "Threads"). The lambda writes the answer into a variable outside:

Network net = new Network();
string payment = "", stock = "", email = "";
Thread[] calls =
[
    new(() => payment = net.Call("payment", "accepted")),
    new(() => stock = net.Call("stock", "available")),
    new(() => email = net.Call("e-mail", "sent")),
];
foreach (Thread t in calls) t.Start();
foreach (Thread t in calls) t.Join();
Console.WriteLine(payment);
Console.WriteLine(stock);
Console.WriteLine(email);
Console.WriteLine($"Rounds of waiting: {net.Rounds}");

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 1

Network net = new Network();
string payment = "", stock = "", email = "";
Thread[] calls =
[
    new(() => payment = net.Call("payment", "accepted")),
    new(() => stock = net.Call("stock", "available")),
    new(() => email = net.Call("e-mail", "sent")),
];
foreach (Thread t in calls) t.Start();
foreach (Thread t in calls) t.Join();
Console.WriteLine(payment);
Console.WriteLine(stock);
Console.WriteLine(email);
Console.WriteLine($"Rounds of waiting: {net.Rounds}");

Network net = new Network();
string payment = net.Call("payment", "accepted");
string stock = net.Call("stock", "available");
string email = net.Call("e-mail", "sent");
Console.WriteLine(payment);
Console.WriteLine(stock);
Console.WriteLine(email);
Console.WriteLine($"Rounds of waiting: {net.Rounds}");

Step by step, in words

An order needs three answers over the network: the payment, the stock check and the confirmation e-mail. Each one waits about 200 ms.
One after another: the thread asks for the payment and does nothing until the answer comes.
Only then does it ask for the stock check. Another 200 ms of waiting.
Only then does it ask for the e-mail. Another 200 ms of waiting.
3 rounds of waiting: the customer waits about 600 ms, and the processor stands idle the whole time.
Side by side: the three questions go out at once, each on its own thread, and the three waits overlap.
The program waits for the answers. By the time the first one comes, the other two are ready too.
1 round of waiting: about 200 ms, as long as the slowest answer. Same answers, same order, a third of the wait.

The answers are the same and in the same order. There is one round of waiting: all three questions go out before any answer has come. The customer waits about 200 ms, as long as the slowest service, instead of 600.

The threads here hardly use the processor: three short runs of call and three sleeps. That is why this works on a server with one core too.

More precisely, if you are curious

Three threads for three questions are clear but expensive: each has its own stack, and while it waits, it just stands there. In C# the usual way to do work that waits is async/await: Task.Delay and real requests hold no thread while they wait. That is the lesson "async is not a thread". Task.Run from the .NET pool would also work here, but the pool adds new threads slowly when all of its threads are waiting.

Try it

The e-mail waits for the payment

A colleague changes the order: the e-mail has to say whether it is paid and in stock, so it goes out only after the payment and the stock have answered.

Network net = new Network();
string payment = "", stock = "";
Thread pay = new(() => payment = net.Call("payment", "accepted"));
Thread check = new(() => stock = net.Call("stock", "available"));
pay.Start();
check.Start();
pay.Join();
check.Join();
// the e-mail says whether it is paid and in stock
Console.WriteLine(payment);
Console.WriteLine(stock);
Console.WriteLine(net.Call("e-mail", "sent"));
Console.WriteLine($"Rounds of waiting: {net.Rounds}");

Answer

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 1

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 3

payment: accepted
stock: available
e-mail: sent
Rounds of waiting: 2

Try it

Wait for the answer

What waits for the payment thread to finish?

Thread pay = new Thread(() => payment = net.Call("payment", "accepted"));
pay.Start();
pay.;
Console.WriteLine(payment);

Join() waits for the thread to finish. Only then is payment written.

Problem

A thousand invoices at the end of the month

At the end of the month the shop makes a PDF invoice for each of 1000 orders. Nothing waits for the network here: the processor computes the whole time. render stands in for drawing the PDF with a computation that only loads the processor and returns a checksum:

// Renders the invoice (here: a calculation that only loads the CPU).
static int Render(int invoice)
{
    int h = invoice;
    for (int i = 0; i < 2000; i++)
    {
        h = (h * 31 + i) % 1_000_003;
    }
    return h;
}

// The invoices from from to to, inclusive.
static long RenderRange(int from, int to)
{
    long sum = 0;
    for (int n = from; n <= to; n++)
    {
        sum += Render(n);
    }
    return sum;
}

long total = RenderRange(1, 1000);
Console.WriteLine("Invoices: 1000");
Console.WriteLine($"Checksum: {total}");

Invoices: 1000
Checksum: 499409728

One core computes everything while the others stand idle. There are no rounds of waiting here: nothing waits.

See it

Four tasks, four cores

The invoices are split into 4 parts of 250, and each part is a task in the .NET pool (Task.Run). Each thread computes on its own core:

List<Task<long>> parts = [];
for (int p = 0; p < 4; p++)
{
    int from = p * 250 + 1;
    parts.Add(Task.Run(() => RenderRange(from, from + 249)));
}
long total = 0;
foreach (Task<long> part in parts)
{
    total += part.Result;
}
Console.WriteLine("Invoices: 1000");
Console.WriteLine($"Checksum: {total}");

Invoices: 1000
Checksum: 499409728

The checksum is the same: splitting does not change the result, only who computes it.

How much faster it is depends on the machine: with four free cores up to about four times, and on one core not faster at all.

These are example times, not a promise. More parts than cores do not help: they just take turns on the same cores.

More precisely, if you are curious

How many threads. For work that computes, as many as the cores: Environment.ProcessorCount tells you how many. For work that waits, many more is fine: they only sleep.
Not everything splits. If a tenth of the work cannot be split (for example adding up the parts or writing one file), no number of cores makes it more than 10 times faster. That is Amdahl's law.

Parallel.For from the lesson "Threads" does the same with less code: it splits the work and waits for all the parts.

The idea

What helps

	Work that waits	Work that computes
What slows it down	the answer from the network, the disk, the database	the computing itself
What helps	waiting for several things at once	more cores
In C#	async and await, threads	Task.Run and Parallel.For
On one core	helps	does not help

Threads come at a price. As long as each one works with its own data (its own service, its own 250 invoices), all is well. As soon as two threads touch the same thing (a shared counter, the stock in the warehouse), errors start that do not show on every run. That is the next lesson: "Race condition".

Exercise

The product page

Solve the task in your editor.

The product page. Before it shows the page, the server asks three services: the price, the reviews and the delivery. Each answers in about 200 ms and does not depend on the others. Use Network from the first card and ask them all at once, so that there is one round of waiting:

price: 89 €
reviews: 4.6 of 5
delivery: tomorrow
Rounds of waiting: 1

More tasks

Five services. Add stock and a discount. How many rounds are there if the discount depends on the price?
How many cores. Run the invoices with 1, 2, 4 and 8 parts and time them on your machine. When does it stop getting faster? Compare with the number of cores on your machine. - The wrong tool. Put a 200 ms pause/sleep in render for every tenth invoice, as if it waits for the disk. What helps more now: more threads or more cores?

Test

Test yourself

1. An order waits for three services of 200 ms each that do not depend on each other. How long does the customer wait if you ask them all at once?About 600 ms: the sumAbout 200 ms: as long as the slowest oneAbout 67 ms: a third

2. The server has one core. Do threads help work that waits for the network?Only if there are at least as many threads as coresYes: a waiting thread does not use the processor, so the others run while it waitsNo: on one core the threads take turns and it all takes as long as one after another

3. You have to compute 1000 invoices and the processor has 4 cores. How many threads help the most?1000: one per invoiceAbout 4: as many as the coresOne: threads do not help with computing

4. Why does the lesson count rounds of waiting and not milliseconds?Because rounds are smaller numbersThe time depends on the machine and its load, while the number of rounds is the same on every runBecause milliseconds cannot be measured

Why threads

Cookies