Skip to content
wannacode

Programs on the network

Problem

The checkout and the warehouse

The checkout in the shop has to ask the warehouse how much coffee is left, and the warehouse program runs on another computer. A file does not help: the two programs cannot see each other's disks. One of them has to ask and the other one has to answer, right away, over the network.

That is how almost everything around you works: the browser asks the server for a page, the bank app asks the bank for your balance, a game sends your moves to the other players.

The idea

Address, port and two ways to send

For data to reach a program on another computer, you need two things:

  • An IP address: the number of the computer on the network, for example 192.168.1.20 or 142.250.185.78. Your own computer is always 127.0.0.1, with the name localhost.
  • A port: a number from 0 to 65,535 for the program on that computer. The address is the building, the port is the apartment. HTTPS uses 443, and we will use 5401 and up.

The program that waits for someone to reach it is the server. The one that does the reaching is the client. How the data travels is decided by the protocol:

TCP UDP
Connection they connect first, then they talk no connection, every packet is a separate letter
Delivery everything arrives, in the same order, or you find out it did not a packet can get lost or come in a different order
Speed slower faster
Used for the web, email, payments, files live voice and video, games, sensors
The end points of a connection are called sockets. In Python they are made by the socket module.
More precisely, if you are curious
  • The network is described in layers. The OSI model has seven: physical (bits over the cable or the air), data link (the connection to the next device), network (IP addresses and routes), transport (TCP, UDP), session, presentation and application (HTTP, email). In practice the internet uses the simpler TCP/IP model with four layers. Your program works on the top one: Python and the operating system take care of everything below.
  • IP delivers separate packets to an address, but it does not promise that they will arrive. TCP is built on top of IP and numbers the packets, so it can put them in order and ask again for the lost ones.
  • Protocols you will come across: HTTP and HTTPS (the web; HTTPS is HTTP encrypted with TLS, whose older name is SSL), WebSocket (a two-way connection for chat and notifications in the browser), FTP (files), SSH (secure access to a remote computer).

See it

The server

The warehouse server gets the name of a product and answers with its price:

PRICES = {"coffee": 7.50, "tea": 3.20}


def serve(server):
    client, address = server.accept()
    with client, client.makefile(encoding="utf-8") as reader:
        product = reader.readline().rstrip("\n")
        print("Server received:", product)
        price = PRICES.get(product)
        answer = "no such product" if price is None else f"{product}: €{price}"
        client.sendall(f"{answer}\n".encode())
  • server is a socket that listens on a port (we will open it in the next card). server.accept() waits until a client connects and returns a pair: a socket for the connection with it, and its address.
  • client.makefile(encoding="utf-8") gives you a file for reading from the socket, as with files: readline() reads a line, together with the "\n" at the end. rstrip("\n") removes it.
  • client.sendall(bytes) sends. Bytes travel over the network, so the text is turned into bytes with .encode() (in UTF-8). The "\n" at the end tells the other side that the line is over. You will see why that matters.
  • with closes the connection and the file at the end.

See it

The client

The client connects with socket.create_connection((address, port)) and uses the same makefile and sendall. Here the server and the client are in one program: the server runs in its own thread, and the main program is the client. That way you can try them without a second computer:
server = socket.create_server(("localhost", 5401))
threading.Thread(target=serve, args=(server,)).start()


with (
    socket.create_connection(("localhost", 5401)) as sock,
    sock.makefile(encoding="utf-8") as reader,
):
    sock.sendall("coffee\n".encode())
    print("Client received:", reader.readline().rstrip("\n"))
server.close()
Output
Server received: coffee
Client received: coffee: €7.5
The port is opened with socket.create_server(("localhost", 5401)) in the main program, before the thread starts. That way, when the client connects, the server is already listening. The parentheses after with allow two resources on separate lines.

The two sides take turns: the client writes a line and waits for an answer with readline(), and the server reads the line and answers. If they swap places and both wait to read, they will wait forever.

server = socket.create_server(("localhost", 5401))
threading.Thread(target=serve, args=(server,)).start()


with (
    socket.create_connection(("localhost", 5401)) as sock,
    sock.makefile(encoding="utf-8") as reader,
):
    sock.sendall("coffee\n".encode())
    print("Client received:", reader.readline().rstrip("\n"))
server.close()
Step by step, in words
  1. The server opens port 5401 and starts listening.
  2. In its own thread, the server calls accept() and waits for a client.
  3. The client connects. accept() returns a socket, and both sides can write and read.
  4. The client writes a line: "coffee". It travels over the connection to the server.
  5. The client waits in readline(). The server reads the line and prints it.
  6. The server finds the price and writes back: "coffee: €7.5".
  7. The client's readline() returns the answer, and the client prints it.
  8. with closes the socket, and server.close() closes the port.
More precisely, if you are curious
  • The whole program, with the imports:
prices.py
import socket
import threading




PRICES = {"coffee": 7.50, "tea": 3.20}




def serve(server):
    client, address = server.accept()
    with client, client.makefile(encoding="utf-8") as reader:
        product = reader.readline().rstrip("\n")
        print("Server received:", product)
        price = PRICES.get(product)
        answer = "no such product" if price is None else f"{product}: €{price}"
        client.sendall(f"{answer}\n".encode())




server = socket.create_server(("localhost", 5401))
threading.Thread(target=serve, args=(server,)).start()


with (
    socket.create_connection(("localhost", 5401)) as sock,
    sock.makefile(encoding="utf-8") as reader,
):
    sock.sendall("coffee\n".encode())
    print("Client received:", reader.readline().rstrip("\n"))
server.close()
  • In two separate files and two terminals it is the same: you start the server first, then the client. If there is no server on the port, create_connection raises ConnectionRefusedError. If the port is already taken by another program, create_server raises OSError: [Errno 98] Address already in use.
  • Text travels over the network as bytes. .encode() and makefile(encoding="utf-8") use UTF-8, so non-ASCII characters such as € arrive intact.

Try it

Wait for a client

Which method waits until a client connects?

server = socket.create_server(("localhost", 5401))
client, address = server.()

The idea

Many clients at once

While the server talks to one client, accept() is not called and the others wait. So a real server runs an endless loop: it accepts a client, gives it to a new thread and goes right back to accept():
def accept_all(server):
    number = 0
    while True:
        try:
            client, address = server.accept()
        except OSError:
            return
        number += 1
        threading.Thread(target=handle, args=(client, number)).start()


def handle(client, number):
    with client, client.makefile(encoding="utf-8") as reader:
        for line in reader:
            line = line.rstrip("\n")
            print(f"Client {number}: {line}")
            client.sendall(f"Echo: {line}\n".encode())
handle answers every line until the client closes the connection: then for line in reader ends. Two clients, one after the other:
server = socket.create_server(("localhost", 5402))
threading.Thread(target=accept_all, args=(server,), daemon=True).start()


for order in ["table 4: two coffees", "table 7: the check"]:
    with (
        socket.create_connection(("localhost", 5402)) as sock,
        sock.makefile(encoding="utf-8") as reader,
    ):
        sock.sendall(f"{order}\n".encode())
        print(reader.readline().rstrip("\n"))
server.close()
Output
Client 1: table 4: two coffees
Echo: table 4: two coffees
Client 2: table 7: the check
Echo: table 7: the check
More precisely, if you are curious
  • daemon=True makes the thread a "background" thread: the program ends when the main program ends, without waiting for the endless loop. A real server does not make it a background thread: it has to run until you stop it.
  • with client, … closes the socket that already exists at the end.
  • With thousands of clients, a thread for each one is expensive. Then the accept loop gives the clients to a thread pool from the previous lesson: pool.submit(handle, client, number).

Try it

Both of them wait

The client sends "coffee" and waits for an answer. The server is the same, but the program hangs and prints nothing. Which line is the bug on?

Pick the line with the bug.

The idea

UDP: letters without a connection

The sensors in the shop send the temperature every minute. If one reading gets lost, the next one comes a minute later, so UDP is handy here. There is no connection and no makefile: you send bytes to an address and a port through a socket of the type SOCK_DGRAM (datagram, a separate packet):
receiver = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
receiver.bind(("localhost", 5403))


with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sensor:
    for reading in ["kitchen 21.5", "storeroom 4.0", "fridge 3.2"]:
        sensor.sendto(reading.encode(), ("localhost", 5403))


for i in range(3):
    data, address = receiver.recvfrom(256)
    print(f"Temperature: {data.decode()} °C")
receiver.close()
Output
Temperature: kitchen 21.5 °C
Temperature: storeroom 4.0 °C
Temperature: fridge 3.2 °C
  • The receiver listens on port 5403 with bind(("localhost", 5403)). The sender does not call bind: the system gives it a free port.
  • sendto(bytes, (address, port)) sends one packet. recvfrom(256) waits for a packet of up to 256 bytes and returns the bytes and the address of the sender.
  • Nobody confirms that the packet arrived. On localhost they do not get lost, but on a real network they can.
More precisely, if you are curious

If the receiver has to answer, it sends a new packet to the address that recvfrom returned: receiver.sendto(answer, address). A packet has a size limit (under 64 KB, and over the internet it is sensible to stay at about 1,400 bytes), so UDP is no good for files.

The idea

HTTP: the language of the web

When you open a page, the browser makes a TCP connection to the server and sends a text HTTP request: "give me /". The server sends back a response: a status, headers and a body (HTML, JSON, an image). In Python this is done by urllib.request:

import urllib.request


with urllib.request.urlopen("https://example.com") as response:
    print("Status:", response.status)
    print("Type:", response.headers["content-type"])
    body = response.read().decode()
    print("Title:", body[body.index("<title>") + 7 : body.index("</title>")])
Output (with internet)
Status: 200
Type: text/html; charset=utf-8
Title: Example Domain
  • urlopen(address) sends a GET request and waits for the response. with closes the connection at the end.
  • response.read() gives you the body as bytes, and .decode() turns it into text.
  • The status tells you how it went: 200 is success, 404 means there is no such page, 500 is an error on the server. On a 4xx or 5xx status, urlopen raises urllib.error.HTTPError.

The site's code check runs without internet, so this program is shown with the output from a run with a network. Without a network, urlopen raises urllib.error.URLError.

More precisely, if you are curious
Python itself has an HTTP server too (http.server). That way you can try urlopen without internet, on localhost:
class Stock(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path != "/stock":
            self.send_error(404)
            return
        answer = "coffee: 12 in stock".encode()
        self.send_response(200)
        self.send_header("Content-Length", str(len(answer)))
        self.end_headers()
        self.wfile.write(answer)


    def log_message(self, format, *args):
        pass  # no line in the console for each request




server = HTTPServer(("localhost", 5404), Stock)
threading.Thread(target=server.serve_forever, daemon=True).start()


for path in ["/stock", "/prices"]:
    try:
        with urllib.request.urlopen(f"http://localhost:5404{path}") as response:
            print(path, "->", response.status, response.read().decode())
    except urllib.error.HTTPError as e:
        print(path, "->", e.code, e.reason)
server.shutdown()
Output
/stock -> 200 coffee: 12 in stock
/prices -> 404 Not Found
For every GET request the server calls do_GET. For /prices it returns 404 with send_error, and urlopen raises it as an HTTPError. log_message is empty, so the server does not print a line for every request. In real projects, web servers are built with libraries such as Flask, Django or FastAPI. They are built on the same sockets.

Exercise

Servers on localhost

Solve these in your editor. The server and the client can be in one program (the server in a thread) or in two separate ones, run in two terminals.

1. Echo server. The server sends back every line with "Echo: " in front, until it gets "end". Then it answers "Goodbye" and closes the connection. The client sends "hello", "how are you" and "end", and prints the answers:

Output
Echo: hello
Echo: how are you
Goodbye

2. A server with commands. UPCASE text returns the text in uppercase, TIME returns the current time as 14:05 (datetime.now().strftime("%H:%M")), and anything else returns "Unknown command: " and the command. The time changes, so the client checks only its shape with a regular expression:

Output
ORDER 42 IS READY
The time looks like HH:MM: True
Unknown command: HOUR
3. A chat for two. Ana opens the server on port 5407, Boris connects. Each one is in their own thread and sends their prepared lines, waiting for an answer after each one:
Output
Boris received: Hi, Boris! Are you free at 6 PM?
Ana received: Yes, I will be there.
Boris received: Great, see you soon.
Ana received: Bye!
Then make it real: two programs, each with two threads. One thread reads from the keyboard and sends, and the other one reads from the socket and prints. That way you can type while you wait for an answer.

Test

Check yourself

1. What does the port point to?
2. When is UDP the better choice over TCP?
3. What does server.accept() do?
4. The client sends sock.sendall("coffee".encode()), and the server reads with readline(). Both of them wait. Why?
5. urlopen raises HTTPError, and e.code is 404. What does it mean?

Made by Ivo Gergov as part of Evo Tech's campaign for accessible programming education ·