Threads

Problem

Three reports, one at a time

The program downloads three monthly reports from the server. Here the download is a pause: time.sleep(seconds) stops the program for that many seconds.

reports.py

import time


def download(file, seconds):
    time.sleep(seconds)
    print(f"Downloaded {file}")


download("march.pdf", 0.6)
download("april.pdf", 0.2)
download("may.pdf", 0.4)
print("Done")

Output

Downloaded march.pdf
Downloaded april.pdf
Downloaded may.pdf
Done

The pauses are 0.6, 0.2 and 0.4 seconds, 1.2 seconds in total. The whole time, the program only waits for the network. If there were a "Cancel" button, it would not respond. And the three reports do not depend on each other, so they could easily be downloaded at the same time.

The idea

Process and thread

When you start the program, the operating system gives it a process: its own memory, its own open files. Inside the process, one or more threads run. A thread is a sequence of commands that run one after another. So far, each of your programs had one thread: the one that runs the file from top to bottom.

The threads in one process share their memory: they see the same objects. That makes them handy (they can easily pass data to each other) and dangerous (they can mess each other up, as you will see in a moment).

• Parallel means really at the same time, on different cores of the processor.
• Concurrent means on one core, taking turns so fast that it looks like the same time.

For the programmer the difference rarely matters: in both cases you do not know which thread will go on when.

Python (CPython, the one from python.org) has a catch: the GIL (global interpreter lock). Only one thread at a time runs Python code, even if the processor has many cores. So threads help when the program waits (network, files, sleep): while one waits, another one works. For heavy calculations threads do not speed things up. There you use processes, which you will see in the exercise.

More precisely, if you are curious

• The Chrome browser starts a separate process for each tab. That is why one frozen site does not stop the others, and also why the browser uses a lot of memory.
• Every program that does several things at once uses threads: a game plays the sound in one thread and runs the logic in another; a web server serves each client in its own thread.
• The price: each thread has its own stack, and the memory for it is set aside separately. Thousands of threads cost a lot. That is why real programs use a thread pool (the last idea in the lesson). - Python 3.13 also has a separate build without the GIL (free-threaded). It is still experimental, so here we write about ordinary Python.

See it

The first thread

The Thread class from the threading module is a thread. With target you tell it which function to run, and with args which arguments to pass (a tuple):

worker = threading.Thread(target=download, args=("april.pdf", 0.2))
worker.start()
print("The program does not wait")
worker.join()
print("The report is downloaded")

Output

The program does not wait
Downloaded april.pdf
The report is downloaded

• start() starts the new thread and returns right away. The program goes on without waiting, so "The program does not wait" is printed first.
• join() makes the program wait for the thread to finish.

download is the function from the first card. Python has no checked exceptions, so there is nothing to declare here.

More precisely, if you are curious

target is the shortest way, but not the only one. You will also come across a subclass of Thread that writes its own run(), and a lambda:

class ReportThread(threading.Thread):
    def run(self):
        print("1. a subclass of Thread")


def report_job():
    print("2. a function as target")


a = ReportThread()
b = threading.Thread(target=report_job)
c = threading.Thread(target=lambda: print("3. a lambda"))
for t in [a, b, c]:
    t.start()
    t.join()

Output

1. a subclass of Thread
2. a function as target
3. a lambda

Python has no anonymous classes. It is more flexible to keep the task (the function) apart from the thread: that way the same function can be given to a thread pool.

See it

Three at once

Now the three reports, each in its own thread:

start = time.time()
march = threading.Thread(target=download, args=("march.pdf", 0.6))
april = threading.Thread(target=download, args=("april.pdf", 0.2))
may = threading.Thread(target=download, args=("may.pdf", 0.4))
march.start()
april.start()
may.start()
march.join()
april.join()
may.join()
seconds = time.time() - start
print(f"Done in under a second: {seconds < 1}")

Output

Downloaded april.pdf
Downloaded may.pdf
Downloaded march.pdf
Done in under a second: True

The threads start in the order March, April, May, but they finish in the order of the pauses: April (0.2 s), May (0.4 s), March (0.6 s). The whole wait is as long as the longest one, 0.6 seconds instead of 1.2.

The operating system decides the order in which the threads reach print. Here the pauses are different enough, so the order is the same. If they were equal, it could be different on every run.

Try it

start or run

What will the program print? (name is the name of the thread)

class Report(threading.Thread):
    def run(self):
        print(f"Running in: {threading.current_thread().name}")


t = Report(name="downloader")
t.run()
t.start()
t.join()

Answer

Running in: MainThread
Running in: downloader

Running in: downloader
Running in: downloader

Running in: downloader
Running in: MainThread

Check

The idea

The life of a thread

is_alive() tells you whether the thread is running right now: from start() to the end of its function:

t = threading.Thread(target=download, args=("may.pdf", 0.5))
print(t.is_alive())
t.start()
time.sleep(0.1)
print(t.is_alive())
t.join()
print(t.is_alive())
try:
    t.start()
except RuntimeError as e:
    print(f"RuntimeError: {e}")

Output

False
True
Downloaded may.pdf
False
RuntimeError: threads can only be started once

Python does not tell you more than this: you cannot see whether it is waiting or calculating. A thread can be started only once. A second start() on the same thread raises RuntimeError.

More precisely, if you are curious

• A thread in Python cannot be stopped from outside. If it has to stop, it checks a signal itself, for example a threading.Event that another thread raises with set().
• name and native_id are the name and the number of the thread. threading.current_thread() gives you the thread the code is running in.
• time.sleep(0) hints that the thread can give way to another one. We will use it in a moment.

Problem

The lost visits

The site counts its visits. Two threads serve visitors, and each one calls increment() 10,000 times. Between reading and writing the counter, the thread does something else. Here that is time.sleep(0): "let the other threads run for a moment":

visits.py

class Counter:
    def __init__(self):
        self.count = 0

    def increment(self):
        value = self.count
        time.sleep(0)  # here the thread gives way to the other one
        self.count = value + 1


visits = Counter()


def work():
    for i in range(10_000):
        visits.increment()


first = threading.Thread(target=work)
second = threading.Thread(target=work)
first.start()
second.start()
first.join()
second.join()
print(f"Visits: {visits.count}")

You expect 20,000. Five runs:

Output (different every time)

Visits: 10000
Visits: 10001
Visits: 10001
Visits: 10000
Visits: 10000

Less every time, almost by half. The reason is that the increase is not one step, but three:

value = self.count       # 1. read
value = value + 1        # 2. add 1
self.count = value       # 3. write

class Counter:
    def __init__(self):
        self.count = 5
        self.lock = threading.Lock()

    def increment(self):
        self.count += 1  # read, add 1, write

    def safe_increment(self):
        with self.lock:
            self.count += 1

Step by step, in words

1. count is 5. Threads A and B call increment() at the same time.
2. A reads count: 5.
3. B reads count before A has written: also 5.
4. A adds 1 in its own copy and writes 6.
5. B writes 6 too. Two calls, and count is 6, not 7: one of the additions is lost.
6. Once more, now using with self.lock: A takes the lock, reads 5 and writes 6.
7. B reaches with self.lock and waits until A releases the lock.
8. B takes the lock, reads 6 and writes 7: nothing is lost.

When both threads read the same value before either of them has written, one of the additions is lost. This is called a race condition. Whether it happens depends on when the threads are switched, so the result is different on every run.

More precisely, if you are curious

Without time.sleep(0), with self.count += 1 and 100,000 increases per thread, five runs in Python 3.12 gave 200,000 five times:

class Counter:
    def __init__(self):
        self.count = 0

    def increment(self):
        self.count += 1

Output

Visits: 200000
Visits: 200000
Visits: 200000
Visits: 200000
Visits: 200000

This is not because it is safe. The GIL switches threads only at certain points in the execution, and here it happens not to be between the read and the write. Python does not promise this: another version, other code in the method or Python without the GIL will lose visits. So shared data is always protected with a lock.

The idea

Lock: one at a time

The solution is for only one thread at a time to do these three steps. That is what a lock, threading.Lock, does: the block with self.lock: is for only one thread at a time:

class Counter:
    def __init__(self):
        self.count = 0
        self.lock = threading.Lock()

    def increment(self):
        with self.lock:
            value = self.count
            time.sleep(0)
            self.count = value + 1

Output

Visits: 20000

The thread takes the lock when it enters with and releases it when it leaves, even on an exception. The other thread waits in front of with. Now the result is 20,000 on every run. Python has no ready-made atomic counter like Java: the lock is the way.

More precisely, if you are curious

• Condition: wait() and notify_all(): a thread in the with block of a Condition can release the lock and go to sleep until another thread wakes it up. That way the kitchen waits for an order, and the checkout waits for the kitchen to take the previous one (the "producer and consumer" problem):

class OrderBox:
    def __init__(self):
        self.order = None
        self.changed = threading.Condition()

    def put(self, new_order):
        with self.changed:
            while self.order is not None:
                self.changed.wait()
            self.order = new_order
            self.changed.notify_all()

    def take(self):
        with self.changed:
            while self.order is None:
                self.changed.wait()
            result = self.order
            self.order = None
            self.changed.notify_all()
            return result


box = OrderBox()


def kitchen():
    for i in range(3):
        print(f"The kitchen took {box.take()}")


cook = threading.Thread(target=kitchen)
cook.start()
box.put("pizza")
box.put("salad")
box.put("soup")
cook.join()

Output

The kitchen took pizza
The kitchen took salad
The kitchen took soup

wait() is in a while, not in an if: the thread can wake up for no reason, so it checks the condition again. queue.Queue(maxsize=1) does the same out of the box: put waits if the queue is full, and get waits if it is empty.

• Deadlock: two threads wait for each other forever. The office holds the printer and waits for the scanner, and the front desk holds the scanner and waits for the printer:

printer = threading.Lock()
scanner = threading.Lock()


def office():
    with printer:
        time.sleep(0.1)
        with scanner:
            print("The office is copying")


def reception():
    with scanner:
        time.sleep(0.1)
        with printer:
            print("The front desk is copying")

The program hangs and never prints anything. The way to avoid it is for all the threads to take the locks in the same order.

• Threads in Python have no priorities. There are daemon threads (daemon=True): the program ends without waiting for them:

report = threading.Thread(target=download, args=("annual.pdf", 0.1), daemon=True)
print(report.name, report.daemon)

Output

Thread-1 (download) True

Try it

Two ATMs

Two people each withdraw €80 from one account with €100 at the same moment. Sometimes both of them get the money, and the balance becomes -€60. Which line is the bug on?

Pick the line with the bug.

1class Account:2 def __init__(self):3 self.balance = 10045 def withdraw(self, amount):6 if self.balance >= amount:7 self.balance -= amount

Check

The idea

Thread pool and Future

Making a thread for every task is expensive, and with a thousand tasks it is dangerous too. A thread pool is a few threads that take tasks from a queue. In Python it is ThreadPoolExecutor (from concurrent.futures):

def size_of(file, seconds):
    time.sleep(seconds)
    return len(file) * 100

files = ["march.pdf", "april.pdf", "may.pdf", "june.pdf"]
with ThreadPoolExecutor(max_workers=2) as pool:
    sizes = [pool.submit(size_of, file, 0.3) for file in files]
    for file, size in zip(files, sizes):
        print(f"{file}: {size.result()} KB")

Output

march.pdf: 900 KB
april.pdf: 900 KB
may.pdf: 700 KB
june.pdf: 800 KB

• ThreadPoolExecutor(max_workers=2) makes a pool of 2 threads. The four tasks take turns in them.
• submit(function, arguments) takes a task and right away gives you a Future: a promise that the result will come.
• result() waits for the task to finish and returns the result. If the task raised an exception, result() raises it again.
• with shuts the pool down at the end: it waits for the tasks to finish and stops its threads.

More precisely, if you are curious

If the task does not return anything, you just do not keep the Future. The end of with waits for all of them:

with ThreadPoolExecutor(max_workers=2) as pool:
    for i in range(1, 6):
        pool.submit(download, f"report-{i}.pdf", 0.1)
print("All done.")

Output of one run

Downloaded report-2.pdf
Downloaded report-1.pdf
Downloaded report-3.pdf
Downloaded report-4.pdf
Downloaded report-5.pdf
All done.

The first two reports run at the same time, so their order changes from run to run. Without with, you write pool.shutdown(wait=True) at the end.

Try it

Which result comes first?

"archive.zip" takes 0.6 s to download, and "note.txt" takes 0.1 s. What will the program print?

with ThreadPoolExecutor(max_workers=2) as pool:
    big = pool.submit(size_of, "archive.zip", 0.6)
    small = pool.submit(size_of, "note.txt", 0.1)
    print(f"Big: {big.result()}")
    print(f"Small: {small.result()}")

Answer

Small: 800
Big: 1100

It depends on which task finishes first

Big: 1100
Small: 800

Check

Exercise

Programs with threads

Solve these in your editor.

1. Parallel sum. The list sales has 1,000,000 sales, sales[i] = i % 100. Split it into 4 equal parts and sum each one in its own thread. Each thread writes to its own place in the list partial, so you do not need synchronization. Wait for them with join and add up the parts:

Output

Parts: [12375000, 12375000, 12375000, 12375000]
Total: 49500000

More precisely, if you are curious

Because of the GIL, the four threads do not calculate faster than one. Really parallel is with processes: ProcessPoolExecutor starts separate copies of Python, each with its own GIL. The parts are passed to them, and the result comes back:

from concurrent.futures import ProcessPoolExecutor


if __name__ == "__main__":
    sales = [i % 100 for i in range(1_000_000)]
    size = len(sales) // 4
    chunks = [sales[p * size:(p + 1) * size] for p in range(4)]
    with ProcessPoolExecutor(max_workers=4) as pool:
        partial = list(pool.map(sum, chunks))
    print(f"Parts: {partial}")
    print(f"Total: {sum(partial)}")

Output

Parts: [12375000, 12375000, 12375000, 12375000]
Total: 49500000

if __name__ == "__main__": keeps the main code from running again in each process: on Windows and macOS a process starts by loading the file from the beginning.

2. Downloading with a pool. Download six reports (march.pdf, april.pdf, may.pdf, june.pdf, july.pdf, august.pdf) with a pool of 3 threads. Use size_of(file, 0.3) from the pool card. Print the sizes in the order of the files, the total, and whether it all took under 1.5 seconds (one after another would be 1.8 s):

Output

march.pdf: 900 KB
april.pdf: 900 KB
may.pdf: 700 KB
june.pdf: 800 KB
july.pdf: 800 KB
august.pdf: 1000 KB
Total: 5100 KB
Under 1.5 seconds: True

3. A counter without losses. Four threads, each with 50,000 visits, increase a shared counter. Write the counter so that the result is always 200,000. First try it without a Lock and with time.sleep(0) between the read and the write and see how much is lost, then fix it with a Lock.

Output

Visits: 200000

More tasks

• A test for the counter. Write a unittest test that starts 8 threads with 10,000 increases each and checks for 80,000 with assertEqual. Run it a few times against the counter with time.sleep(0) and without a Lock. A test that fails only sometimes is the typical sign of a race condition.
• The deadlock. Run the example with the printer and the scanner from "More precisely" (with two threads and join) and watch it hang. Fix it so that both threads take the printer first.

Test

Check yourself

1. What is the difference between t.start() and t.run()?

start() runs run() in a new thread, and run() runs it right away in the current onerun() starts a new thread, and start() does notThere is no difference

2. Why does the program call join() before it prints the result?

To wait for the thread to finishTo start the threadTo stop the thread right away

3. Two threads increase a shared counter without a Lock. What does Python promise about the result?

That it is always halfThat it is always right, because of the GILNothing: if the thread is switched between the read and the write, one increase is lost

4. What does with self.lock: around a few lines guarantee?

That the threads run in the order they were startedThat the lines are fasterThat only one thread at a time is inside the block with this lock

5. You start three tasks with pool.submit and keep a Future for each one. What does future.result() do?

It returns the result right away, even if it is not readyIt runs the task againIt waits for the task to finish and returns its result

Check

Start the lesson over