Programs on the network

Problem

The checkout and the warehouse

The checkout in the shop has to ask the warehouse how much coffee is left, and the warehouse program runs on another computer. A file does not help: the two programs cannot see each other's disks. One of them has to ask and the other one has to answer, right away, over the network.

That is how almost everything around you works: the browser asks the server for a page, the bank app asks the bank for your balance, a game sends your moves to the other players.

The idea

Address, port and two ways to send

For data to reach a program on another computer, you need two things:

• An IP address: the number of the computer on the network, for example 192.168.1.20 or 142.250.185.78. Your own computer is always 127.0.0.1, with the name localhost.
• A port: a number from 0 to 65,535 for the program on that computer. The address is the building, the port is the apartment. HTTPS uses 443, and we will use 5401 and up.

The program that waits for someone to reach it is the server. The one that does the reaching is the client. How the data travels is decided by the protocol:

	TCP	UDP
Connection	they connect first, then they talk	no connection, every packet is a separate letter
Delivery	everything arrives, in the same order, or you find out it did not	a packet can get lost or come in a different order
Speed	slower	faster
Used for	the web, email, payments, files	live voice and video, games, sensors

The end points of a connection are called sockets. In Node they are made by the node:net module.

More precisely, if you are curious

• The network is described in layers. The OSI model has seven: physical (bits over the cable or the air), data link (the connection to the next device), network (IP addresses and routes), transport (TCP, UDP), session, presentation and application (HTTP, email). In practice the internet uses the simpler TCP/IP model with four layers. Your program works on the top one: Node and the operating system take care of everything below.
• IP delivers separate packets to an address, but it does not promise that they will arrive. TCP is built on top of IP and numbers the packets, so it can put them in order and ask again for the lost ones.
• Protocols you will come across: HTTP and HTTPS (the web; HTTPS is HTTP encrypted with TLS, whose older name is SSL), WebSocket (a two-way connection for chat and notifications in the browser), FTP (files), SSH (secure access to a remote computer).

See it

The server

The warehouse server gets the name of a product and answers with its price:

// await readLine() waits for the next line from the socket (undefined when the connection is closed).
function lineReader(socket) {
  const lines = readline.createInterface({ input: socket })[Symbol.asyncIterator]();
  return async () => (await lines.next()).value;
}

const PRICES = new Map([["coffee", 7.50], ["tea", 3.20]]);

async function serve(client) {
  const readLine = lineReader(client);
  const product = await readLine();
  console.log(`Server received: ${product}`);
  const price = PRICES.get(product);
  client.end(price === undefined ? "no such product\n" : `${product}: €${price}\n`);
}

• There is no thread and no accept(). Node calls serve for every client that connects and gives it a socket for the connection (client).
• The socket is a stream: the bytes come in pieces, as they arrive. lineReader (above) collects them into lines with node:readline, and await readLine() waits for the next line.
• client.end(text) sends the last text and closes the connection. The text travels over the network in UTF-8. The "\n" at the end tells the other side that the line is over. You will see why that matters.

See it

The client

The client connects with net.connect(port, address) and uses the same lineReader and write. Here the server and the client are in one program: Node serves both sides in one thread, taking turns while each one waits. That way you can try them without a second computer:

const server = net.createServer(serve);
server.listen(5401, "localhost");
await once(server, "listening");

const socket = net.connect(5401, "localhost");
const readLine = lineReader(socket);
socket.write("coffee\n");
console.log(`Client received: ${await readLine()}`);
socket.end();
server.close();

Output

Server received: coffee
Client received: coffee: €7.5

server.listen(5401, "localhost") opens the port, and await once(server, "listening") waits until it is open. That way, when the client connects, the server is already listening.

The two sides take turns: the client writes a line and waits for an answer with await readLine(), and the server reads the line and answers. If they swap places and both wait to read, they will wait forever.

const server = net.createServer(serve);
server.listen(5401, "localhost");
await once(server, "listening");

const socket = net.connect(5401, "localhost");
const readLine = lineReader(socket);
socket.write("coffee\n");
console.log(`Client received: ${await readLine()}`);
socket.end();
server.close();

Step by step, in words

1. The server opens port 5401 and starts listening.
2. There is no thread: Node will call serve when a client comes. Until then the program goes on.
3. The client connects. Node calls serve with a socket for the client, and both sides can write and read.
4. The client writes a line: "coffee". It travels over the connection to the server.
5. The client waits with await readLine(). The server reads the line and prints it.
6. The server finds the price and writes back: "coffee: €7.5".
7. The client's await readLine() returns the answer, and the client prints it.
8. socket.end() closes the connection, and server.close() closes the port.

More precisely, if you are curious

• The whole program, with the imports:

prices.js

import net from "node:net";
import readline from "node:readline";
import { once } from "node:events";

// await readLine() waits for the next line from the socket (undefined when the connection is closed).
function lineReader(socket) {
  const lines = readline.createInterface({ input: socket })[Symbol.asyncIterator]();
  return async () => (await lines.next()).value;
}

const PRICES = new Map([["coffee", 7.50], ["tea", 3.20]]);

async function serve(client) {
  const readLine = lineReader(client);
  const product = await readLine();
  console.log(`Server received: ${product}`);
  const price = PRICES.get(product);
  client.end(price === undefined ? "no such product\n" : `${product}: €${price}\n`);
}

const server = net.createServer(serve);
server.listen(5401, "localhost");
await once(server, "listening");

const socket = net.connect(5401, "localhost");
const readLine = lineReader(socket);
socket.write("coffee\n");
console.log(`Client received: ${await readLine()}`);
socket.end();
server.close();

• In two separate files and two terminals it is the same: you start the server first, then the client. If there is no server on the port, the socket gives an ECONNREFUSED error. If the port is already taken by another program, the server gives an EADDRINUSE error. The errors come as an "error" event: without socket.on("error", …) the program stops with them.
• Text travels over the network as bytes. write and readline use UTF-8 by default, so non-ASCII characters such as € arrive intact.

Try it

Open the port

Which method opens the port, so the server can wait for clients?

const server = net.createServer(serve);
server.(5401, "localhost");

Check

The idea

Many clients at once

Here JavaScript is simpler. There is no loop for accepting: Node calls handle for every new client. While one client waits, Node serves the others, all in one thread:

let number = 0;

async function handle(client) {
  number++;
  const n = number;
  for await (const line of readline.createInterface({ input: client })) {
    console.log(`Client ${n}: ${line}`);
    client.write(`Echo: ${line}\n`);
  }
}

handle answers every line until the client closes the connection: then for await ends. Two clients, one after the other:

const server = net.createServer(handle);
server.listen(5402, "localhost");
await once(server, "listening");

for (const order of ["table 4: two coffees", "table 7: the check"]) {
  const socket = net.connect(5402, "localhost");
  const readLine = lineReader(socket);
  socket.write(`${order}\n`);
  console.log(await readLine());
  socket.end();
}
server.close();

Output

Client 1: table 4: two coffees
Echo: table 4: two coffees
Client 2: table 7: the check
Echo: table 7: the check

More precisely, if you are curious

• number is shared by all the clients, but there is no race condition: only one function runs at a time, and there is no await between number++ and const n = number.
• server.close() stops new connections. Node ends the program when the old ones are closed too.
• This one thread is enough for thousands of clients, because most of the time they are waiting for the network. If the server has to calculate a lot, the heavy work goes to a Worker from the previous lesson.

Try it

Both of them wait

The client sends "coffee" and waits for an answer. The server is the same, but the program hangs and prints nothing. Which line is the bug on?

Pick the line with the bug.

1const socket = net.connect(5401, "localhost");2const readLine = lineReader(socket);3socket.write("coffee");4console.log(`Client received: ${await readLine()}`);

Check

The idea

UDP: letters without a connection

The sensors in the shop send the temperature every minute. If one reading gets lost, the next one comes a minute later, so UDP is handy here. The packets are sent by the node:dgram module (datagram, a separate packet):

const receiver = dgram.createSocket("udp4");
receiver.bind(5403, "localhost");
await once(receiver, "listening");

const sensor = dgram.createSocket("udp4");
sensor.connect(5403, "localhost");
await once(sensor, "connect");
for (const reading of ["kitchen 21.5", "storeroom 4.0", "fridge 3.2"]) {
  sensor.send(reading);
}

let count = 0;
receiver.on("message", (data) => {
  console.log(`Temperature: ${data} °C`);
  count++;
  if (count === 3) {
    receiver.close();
    sensor.close();
  }
});

Output

Temperature: kitchen 21.5 °C
Temperature: storeroom 4.0 °C
Temperature: fridge 3.2 °C

• createSocket("udp4") makes a UDP socket. The receiver listens on port 5403 with bind(5403, "localhost").
• The sender calls connect(5403, "localhost"): with UDP this does not make a connection, it only remembers where the packets go. Then send(text) sends one packet.
• Each packet comes as a "message" event. After the third one the program closes both sockets, otherwise Node waits for more packets forever.
• Nobody confirms that the packet arrived. On localhost they do not get lost, but on a real network they can.

More precisely, if you are curious

If the receiver has to answer, it takes the address and the port of the sender from the second parameter of "message": (data, rinfo) => receiver.send(answer, rinfo.port, rinfo.address). A packet has a size limit (under 64 KB, and over the internet it is sensible to stay at about 1,400 bytes), so UDP is no good for files.

The idea

HTTP: the language of the web

When you open a page, the browser makes a TCP connection to the server and sends a text HTTP request: "give me /". The server sends back a response: a status, headers and a body (HTML, JSON, an image). In Node (and in the browser) this is done by fetch, with no import:

const response = await fetch("https://example.com");
console.log(`Status: ${response.status}`);
console.log(`Type: ${response.headers.get("content-type")}`);
const body = await response.text();
console.log(`Title: ${body.slice(body.indexOf("<title>") + 7, body.indexOf("</title>"))}`);

Output (with internet)

Status: 200
Type: text/html; charset=utf-8
Title: Example Domain

• fetch(address) sends a GET request and gives you a promise for the response. await waits for it.
• response.text() reads the body as text. That is a promise too, so await again.
• The status tells you how it went: 200 is success, 404 means there is no such page, 500 is an error on the server. fetch does not throw on a 404: check response.status or response.ok.

The site's code check runs without internet, so this program is shown with the output from a run with a network. Without a network, fetch throws TypeError: fetch failed.

More precisely, if you are curious

Node itself has an HTTP server too (node:http). That way you can try fetch without internet, on localhost:

const server = http.createServer((request, response) => {
  if (request.url !== "/stock") {
    response.writeHead(404).end("no such address");
    return;
  }
  response.writeHead(200, { "content-type": "text/plain; charset=utf-8" }).end("coffee: 12 in stock");
});
server.listen(5404, "localhost");
await once(server, "listening");

for (const path of ["/stock", "/prices"]) {
  const response = await fetch(`http://localhost:5404${path}`);
  console.log(`${path} -> ${response.status} ${await response.text()}`);
}
server.close();

Output

/stock -> 200 coffee: 12 in stock
/prices -> 404 no such address

Node calls the server's function for every request. For /prices it returns 404, and fetch gives it to you as an ordinary response. In real projects, web servers are built with libraries such as Express. They are built on the same node:http.

Exercise

Servers on localhost

Solve these in your editor. The server and the client can be in one program or in two separate ones, run in two terminals.

1. Echo server. The server sends back every line with "Echo: " in front, until it gets "end". Then it answers "Goodbye" and closes the connection. The client sends "hello", "how are you" and "end", and prints the answers:

Output

Echo: hello
Echo: how are you
Goodbye

2. A server with commands. UPCASE text returns the text in uppercase, TIME returns the current time as 14:05 (new Date().toTimeString().slice(0, 5)), and anything else returns "Unknown command: " and the command. The time changes, so the client checks only its shape with a regular expression:

Output

ORDER 42 IS READY
The time looks like HH:MM: true
Unknown command: HOUR

3. A chat for two. Ana opens the server on port 5407, Boris connects. Each one is an async function, the two are started together with Promise.all, and each one sends their prepared lines, waiting for an answer after each one (await once(server, "connection") waits for the first client):

Output

Boris received: Hi, Boris! Are you free at 6 PM?
Ana received: Yes, I will be there.
Boris received: Great, see you soon.
Ana received: Bye!

Then make it real: two programs. Each one reads lines from the keyboard (readline with input: process.stdin) and sends them, and prints the lines from the socket as soon as they come. You do not need a second thread: both come as events in the event loop. That way you can type while you wait for an answer.

Test

Check yourself

1. What does the port point to?

The computer on the networkThe speed of the connectionThe program on the computer that the data goes to

2. When is UDP the better choice over TCP?

NeverWhen speed matters and a lost packet is no big deal: voice, video, sensor readingsWhen every byte has to arrive, for example for a payment

3. What does net.createServer(serve) do?

It sends an answer to the clientIt makes a server, and Node calls serve with a socket for every client that connectsIt connects to a server

4. The client sends socket.write("coffee"), and the server reads lines. Both of them wait. Why?

The server is on another computerThe wrong portThe "\n" is missing: the server waits for the end of the line, and the client waits for an answer

5. response.status from fetch is 404. What does it mean?

There is no internetThe server has no such address (the page was not found)Everything is fine

Check

Start the lesson over