Network programming

In short: Network programming is two programs talking over the network: the server waits on an IP address and port, the client connects, and sockets send the data over TCP or UDP.

Problem

The checkout and the warehouse

The checkout in the shop has to ask the warehouse how much coffee is left, and the warehouse program runs on another computer. A file does not help: the two programs cannot see each other's disks. One of them has to ask and the other one has to answer, right away, over the network.

That is how almost everything around you works: the browser asks the server for a page, the bank app asks the bank for your balance, a game sends your moves to the other players.

The idea

Address, port and two ways to send

For data to reach a program on another computer, you need two things:

• An IP address: the number of the computer on the network, for example 192.168.1.20 or 142.250.185.78. Your own computer is always 127.0.0.1, with the name localhost.
• A port: a number from 0 to 65,535 for the program on that computer. The address is the building, the port is the apartment. HTTPS uses 443, and we will use 5601 and up.

The program that waits for someone to reach it is the server. The one that does the reaching is the client. How the data travels is decided by the protocol:

	TCP	UDP
Connection	they connect first, then they talk	no connection, every packet is a separate letter
Delivery	everything arrives, in the same order, or you find out it did not	a packet can get lost or come in a different order
Speed	slower	faster
Used for	the web, email, payments, files	live voice and video, games, sensors

The end points of a connection are called sockets. In C# their classes are in System.Net.Sockets.

More precisely, if you are curious

• The network is described in layers. The OSI model has seven: physical (bits over the cable or the air), data link (the connection to the next device), network (IP addresses and routes), transport (TCP, UDP), session, presentation and application (HTTP, email). In practice the internet uses the simpler TCP/IP model with four layers. Your program works on the top one: .NET and the operating system take care of everything below.
• IP delivers separate packets to an address, but it does not promise that they will arrive. TCP is built on top of IP and numbers the packets, so it can put them in order and ask again for the lost ones.
• Protocols you will come across: HTTP and HTTPS (the web; HTTPS is HTTP encrypted with TLS, whose older name is SSL), WebSocket (a two-way connection for chat and notifications in the browser), FTP (files), SSH (secure access to a remote computer).

See it

The server

The warehouse server gets the name of a product and answers with its price:

static readonly Dictionary<string, double> Prices = new Dictionary<string, double> { ["coffee"] = 7.50, ["tea"] = 3.20 };

static void Serve(TcpListener server)
{
    try
    {
        using (TcpClient client = server.AcceptTcpClient())
        using (StreamReader reader = new StreamReader(client.GetStream()))
        using (StreamWriter writer = new StreamWriter(client.GetStream()) { AutoFlush = true })
        {
            string product = reader.ReadLine();
            Console.WriteLine("Server received: " + product);
            writer.WriteLine(Prices.TryGetValue(product, out double price) ? product + ": €" + price : "no such product");
        }
    }
    catch (IOException e)
    {
        Console.WriteLine("Server: " + e.Message);
    }
}

• TcpListener listens on a port. server.AcceptTcpClient() waits until a client connects and returns a TcpClient for the connection with it: the same type the client uses to connect.
• client.GetStream() gives you the stream of the connection: bytes flow over it in both directions. We wrap it in two classes: StreamReader reads text line by line with ReadLine(), and StreamWriter writes with WriteLine(). The same classes read and write files too.
• AutoFlush = true means "send after every WriteLine". You will see why that matters.
• using (…) closes the connection and the streams at the end of the block, even if an exception happens in the middle: as if a finally called Dispose(). Several using lines one under another share one block and are closed in reverse order.
• TryGetValue looks up the product and gives you the price through out.

See it

The client

The client connects with new TcpClient(address, port) and uses the same StreamReader and StreamWriter. Here the server and the client are in one program: the server runs in its own thread, and Main is the client. That way you can try them without a second computer:

TcpListener server = new TcpListener(IPAddress.Loopback, 5601);
server.Start();
new Thread(() => Serve(server)).Start();

using (TcpClient client = new TcpClient("localhost", 5601))
using (StreamReader reader = new StreamReader(client.GetStream()))
using (StreamWriter writer = new StreamWriter(client.GetStream()) { AutoFlush = true })
{
    writer.WriteLine("coffee");
    Console.WriteLine("Client received: " + reader.ReadLine());
}
server.Stop();

Output

Server received: coffee
Client received: coffee: €7.5

new TcpListener(IPAddress.Loopback, 5601) does not open the port yet. server.Start() opens it, in Main, before the thread starts. That way, when the client connects, the server is already listening. IPAddress.Loopback is 127.0.0.1: the server accepts only programs from the same computer.

The two sides take turns: the client writes a line and waits for an answer with ReadLine(), and the server reads the line and answers. If they swap places and both wait to read, they will wait forever.

TcpListener server = new TcpListener(IPAddress.Loopback, 5601);
server.Start();
new Thread(() => Serve(server)).Start();

using (TcpClient client = new TcpClient("localhost", 5601))
using (StreamReader reader = new StreamReader(client.GetStream()))
using (StreamWriter writer = new StreamWriter(client.GetStream()) { AutoFlush = true })
{
    writer.WriteLine("coffee");
    Console.WriteLine("Client received: " + reader.ReadLine());
}
server.Stop();

Step by step, in words

1. The server opens port 5601 and starts listening.
2. In its own thread, the server calls AcceptTcpClient() and waits for a client.
3. The client connects. AcceptTcpClient() returns a TcpClient, and each side has a writer for writing and a reader for reading.
4. The client writes a line: "coffee". It travels over the connection to the server.
5. The client waits in ReadLine(). The server reads the line and prints it.
6. The server finds the price and writes back: "coffee: €7.5".
7. The client's ReadLine() returns the answer, and the client prints it.
8. using closes the connection, and server.Stop() closes the port.

More precisely, if you are curious

• The whole program, with the imports:

Program.cs

using System.Net;
using System.Net.Sockets;

class Program
{
    static readonly Dictionary<string, double> Prices = new Dictionary<string, double> { ["coffee"] = 7.50, ["tea"] = 3.20 };

    static void Serve(TcpListener server)
    {
        try
        {
            using (TcpClient client = server.AcceptTcpClient())
            using (StreamReader reader = new StreamReader(client.GetStream()))
            using (StreamWriter writer = new StreamWriter(client.GetStream()) { AutoFlush = true })
            {
                string product = reader.ReadLine();
                Console.WriteLine("Server received: " + product);
                writer.WriteLine(Prices.TryGetValue(product, out double price) ? product + ": €" + price : "no such product");
            }
        }
        catch (IOException e)
        {
            Console.WriteLine("Server: " + e.Message);
        }
    }

    static void Main()
    {
        TcpListener server = new TcpListener(IPAddress.Loopback, 5601);
        server.Start();
        new Thread(() => Serve(server)).Start();

        using (TcpClient client = new TcpClient("localhost", 5601))
        using (StreamReader reader = new StreamReader(client.GetStream()))
        using (StreamWriter writer = new StreamWriter(client.GetStream()) { AutoFlush = true })
        {
            writer.WriteLine("coffee");
            Console.WriteLine("Client received: " + reader.ReadLine());
        }
        server.Stop();
    }
}

• In two separate files and two terminals it is the same: you start the server first, then the client. If there is no server on the port, new TcpClient throws a SocketException (Connection refused). If the port is already taken by another program, server.Start() throws a SocketException (Address already in use). In .NET a SocketException is not an IOException, so catch (IOException) does not catch it.
• Text travels over the network as bytes. StreamReader and StreamWriter use UTF-8 by default, so non-ASCII characters such as € arrive intact.
• using also has a short form, with no parentheses and no block: using TcpClient client = new TcpClient("localhost", 5601); closes the connection at the end of the block that the line is in.

Try it

Wait for a client

Which method waits until a client connects?

TcpListener server = new TcpListener(IPAddress.Loopback, 5601);
server.Start();
TcpClient client = server.();

Check

The idea

Many clients at once

While the server talks to one client, AcceptTcpClient() is not called and the others wait. So a real server runs an endless loop: it accepts a client, gives it to a new thread and goes right back to AcceptTcpClient():

static void AcceptAll(TcpListener server)
{
    int number = 0;
    while (true)
    {
        try
        {
            TcpClient client = server.AcceptTcpClient();
            number++;
            int n = number;
            new Thread(() => Handle(client, n)).Start();
        }
        catch (SocketException)
        {
            return;
        }
    }
}

static void Handle(TcpClient client, int number)
{
    try
    {
        using (client)
        using (StreamReader reader = new StreamReader(client.GetStream()))
        using (StreamWriter writer = new StreamWriter(client.GetStream()) { AutoFlush = true })
        {
            string line;
            while ((line = reader.ReadLine()) != null)
            {
                Console.WriteLine("Client " + number + ": " + line);
                writer.WriteLine("Echo: " + line);
            }
        }
    }
    catch (IOException e)
    {
        Console.WriteLine("Client " + number + " disconnected: " + e.Message);
    }
}

Handle answers every line until the client closes the connection: then ReadLine() returns null. Two clients, one after the other:

TcpListener server = new TcpListener(IPAddress.Loopback, 5602);
server.Start();
Thread acceptor = new Thread(() => AcceptAll(server));
acceptor.IsBackground = true;
acceptor.Start();

foreach (string order in new string[] { "table 4: two coffees", "table 7: the check" })
{
    using (TcpClient client = new TcpClient("localhost", 5602))
    using (StreamReader reader = new StreamReader(client.GetStream()))
    using (StreamWriter writer = new StreamWriter(client.GetStream()) { AutoFlush = true })
    {
        writer.WriteLine(order);
        Console.WriteLine(reader.ReadLine());
    }
}
server.Stop();

Output

Client 1: table 4: two coffees
Echo: table 4: two coffees
Client 2: table 7: the check
Echo: table 7: the check

More precisely, if you are curious

• IsBackground = true makes the thread a "background" thread: the program ends when Main ends, without waiting for the endless loop. A thread from new Thread is not a background thread by default. A real server does not make it a background thread: it has to run until you stop it.
• server.Stop() closes the port. The waiting AcceptTcpClient() throws a SocketException, and the loop ends with return.
• using (client) puts the TcpClient that already exists into a using, so it is closed at the end.
• int n = number; is needed: the lambda does not remember the value of number, it remembers the variable itself. Without the copy, the first client's thread could read number after the loop has already increased it for the second one.
• With thousands of clients, a thread for each one is expensive. Then the accept loop gives the clients to the thread pool from the previous lesson: Task.Run(() => Handle(client, n)).

Try it

Both of them wait

The client sends "coffee" and waits for an answer. The server is the same, but the program hangs and prints nothing. Which line is the bug on?

Pick the line with the bug.

1using (TcpClient client = new TcpClient("localhost", 5601))2using (StreamReader reader = new StreamReader(client.GetStream()))3using (StreamWriter writer = new StreamWriter(client.GetStream()))4{5 writer.WriteLine("coffee");6 Console.WriteLine("Client received: " + reader.ReadLine());7}

Check

The idea

UDP: letters without a connection

The sensors in the shop send the temperature every minute. If one reading gets lost, the next one comes a minute later, so UDP is handy here. There is no connection and there are no streams: you send bytes to an address and a port through a UdpClient:

UdpClient receiver = new UdpClient(new IPEndPoint(IPAddress.Loopback, 5603));

using (UdpClient sensor = new UdpClient())
{
    foreach (string reading in new string[] { "kitchen 21.5", "storeroom 4.0", "fridge 3.2" })
    {
        byte[] data = Encoding.UTF8.GetBytes(reading);
        sensor.Send(data, data.Length, "localhost", 5603);
    }
}

IPEndPoint sender = new IPEndPoint(IPAddress.Any, 0);
for (int i = 0; i < 3; i++)
{
    byte[] data = receiver.Receive(ref sender);
    string text = Encoding.UTF8.GetString(data);
    Console.WriteLine("Temperature: " + text + " °C");
}
receiver.Close();

Output

Temperature: kitchen 21.5 °C
Temperature: storeroom 4.0 °C
Temperature: fridge 3.2 °C

• The receiver listens on port 5603: new UdpClient(new IPEndPoint(IPAddress.Loopback, 5603)). An IPEndPoint is an address plus a port. The sender is new UdpClient() without a port: the system gives it a free one.
• Bytes travel over the network. Encoding.UTF8.GetBytes turns the text into bytes, and Encoding.UTF8.GetString turns them back (Encoding is in System.Text).
• Send(bytes, count, address, port) sends one packet. Receive(ref sender) waits for a packet, returns its bytes and writes into sender where it came from: that is why it is ref.
• Nobody confirms that the packet arrived. On localhost they do not get lost, but on a real network they can.

More precisely, if you are curious

If the receiver has to answer, it sends a new packet to the address that Receive wrote into sender: receiver.Send(answer, answer.Length, sender). A packet has a size limit (under 64 KB, and over the internet it is sensible to stay at about 1,400 bytes), so UDP is no good for files.

The idea

HTTP: the language of the web

When you open a page, the browser makes a TCP connection to the server and sends a text HTTP request: "give me /". The server sends back a response: a status, headers and a body (HTML, JSON, an image). In C# this is done by HttpClient, but it works with async and await, which come in the "Web and desktop" module. Here the request is written by hand and sent over a TcpClient. That way you can see that HTTP is just text over a TCP connection:

using System.Net.Sockets;

using (TcpClient client = new TcpClient("example.com", 80))
using (StreamReader reader = new StreamReader(client.GetStream()))
using (StreamWriter writer = new StreamWriter(client.GetStream()) { AutoFlush = true })
{
    writer.Write("GET / HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n");

    string status = reader.ReadLine();
    Console.WriteLine("Status: " + status.Split(' ')[1]);
    string line;
    while ((line = reader.ReadLine()) != "")
    {
        if (line.StartsWith("Content-Type: "))
        {
            Console.WriteLine("Type: " + line.Substring(14));
        }
    }
    string body = reader.ReadToEnd();
    int start = body.IndexOf("<title>") + 7;
    Console.WriteLine("Title: " + body.Substring(start, body.IndexOf("</title>") - start));
}

Output (with internet)

Status: 200
Type: text/html; charset=utf-8
Title: Example Domain

• The request is a few lines, and each one ends with \r\n: GET / HTTP/1.1 ("give me /"), Host: (which site) and Connection: close ("close after the response"). The empty line at the end says the request is over.
• The response starts with a line like HTTP/1.1 200 OK. Its second part is the status. The headers (Content-Type: …) follow until an empty line, and then comes the body. ReadToEnd() reads it until the server closes the connection.
• The status tells you how it went: 200 is success, 404 means there is no such page, 500 is an error on the server.
• Port 80 is HTTP without encryption. Most sites today want HTTPS on port 443, where the text travels encrypted (TLS). HttpClient does that for you.

The site's code check runs without internet, so this program is shown with the output from a run with a network. Without a network, new TcpClient throws a SocketException. In the "Web and desktop" module the request and the response are taken apart line by line, and then HttpClient does all of this for you.

More precisely, if you are curious

.NET itself has an HTTP server too (HttpListener). That way you can try the same request without internet, on localhost:

static void Answer(HttpListener server)
{
    try
    {
        while (true)
        {
            HttpListenerContext context = server.GetContext();
            bool found = context.Request.Url.AbsolutePath == "/stock";
            byte[] answer = Encoding.UTF8.GetBytes(found ? "coffee: 12 in stock" : "no such address");
            context.Response.StatusCode = found ? 200 : 404;
            context.Response.ContentLength64 = answer.Length;
            context.Response.OutputStream.Write(answer, 0, answer.Length);
            context.Response.Close();
        }
    }
    catch (HttpListenerException)
    {
        return;
    }
}

HttpListener server = new HttpListener();
server.Prefixes.Add("http://localhost:5604/");
server.Start();
Thread answering = new Thread(() => Answer(server));
answering.IsBackground = true;
answering.Start();

foreach (string path in new string[] { "/stock", "/prices" })
{
    using (TcpClient client = new TcpClient("localhost", 5604))
    using (StreamReader reader = new StreamReader(client.GetStream()))
    using (StreamWriter writer = new StreamWriter(client.GetStream()) { AutoFlush = true })
    {
        writer.Write("GET " + path + " HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n");
        string status = reader.ReadLine().Split(' ')[1];
        while (reader.ReadLine() != "")
        {
            // the response headers are not needed here
        }
        Console.WriteLine(path + " -> " + status + " " + reader.ReadToEnd());
    }
}
server.Stop();

Output

/stock -> 200 coffee: 12 in stock
/prices -> 404 no such address

GetContext() waits for a request, the way AcceptTcpClient() waits for a client. For /prices the server sets the status 404 itself. ContentLength64 says how many bytes the body has, so the client knows where it ends. server.Stop() interrupts the waiting GetContext() with an HttpListenerException. In real projects, web servers are written with ASP.NET Core, in the "Web and desktop" module. It is built on the same sockets and threads.

Exercise

Servers on localhost

Solve these in your editor. The server and the client can be in one program (the server in a thread) or in two separate ones, run in two terminals.

1. Echo server. The server sends back every line with "Echo: " in front, until it gets "end". Then it answers "Goodbye" and closes the connection. The client sends "hello", "how are you" and "end", and prints the answers:

Output

Echo: hello
Echo: how are you
Goodbye

2. A server with commands. UPCASE text returns the text in uppercase, TIME returns the current time as 14:05 (DateTime.Now.ToString("HH:mm")), and anything else returns "Unknown command: " and the command. The time changes, so the client checks only its shape with TimeOnly.TryParseExact(time, "HH:mm", out _):

Output

ORDER 42 IS READY
The time looks like HH:MM: True
Unknown command: HOUR

3. A chat for two. Ana opens the server on port 5607, Boris connects. Each one is in their own thread and sends their prepared lines, waiting for an answer after each one:

Output

Boris received: Hi, Boris! Are you free at 6 PM?
Ana received: Yes, I will be there.
Boris received: Great, see you soon.
Ana received: Bye!

Then make it real: two programs, each with two threads. One thread reads from the keyboard and sends, and the other one reads from the socket and prints. That way you can type while you wait for an answer.

Test

Check yourself

1. What does the port point to?

The computer on the networkThe speed of the connectionThe program on the computer that the data goes to

2. When is UDP the better choice over TCP?

NeverWhen speed matters and a lost packet is no big deal: voice, video, sensor readingsWhen every byte has to arrive, for example for a payment

3. What does server.AcceptTcpClient() do?

It waits until a client connects and returns a TcpClient for itIt closes the serverIt sends an answer to the client

4. The client writes with WriteLine, but the server gets nothing, and both of them wait. What is the most common cause?

The wrong portThe server is on another computerStreamWriter without AutoFlush = true: the text sits in the buffer and is not sent

5. The first line of the response is HTTP/1.1 404 Not Found. What does it mean?

Everything is fineThere is no internetThe server has no such address (the page was not found)

Check

Start the lesson over

Related lessons

• Builds on: Lists · Dictionary and HashSet · Lambdas and delegates · Threads
• Continues in: Untrusted input · Sessions, cookies and CSRF · JWT and its traps · API design: contract, versions, retried requests
• Tasks on this topic: Real-world programs