Monolith, modular monolith, services

Code in:JavaC#PythonJavaScript

In short: A modular monolith is one application with clear modules, each with a public API and its own data; microservices are split out when teams, scale or independent deploys are worth the cost.

Problem

Orders in the warehouse table

The shop in the city reserves items for customers who will come in person. The warehouse keeps the stock and the reservations, and for the kettle KTL-100 the stock is 2. The online orders were written by another team, which "to save time" reads and writes the warehouse table directly:

// Warehouse: stock and the city store's reservations
class Warehouse {
  stock = new Map();
  reserved = new Map();

  reserve(sku, qty) {
    if (this.free(sku) < qty) return false;
    this.reserved.set(sku, (this.reserved.get(sku) ?? 0) + qty);
    return true;
  }

  free(sku) {
    return (this.stock.get(sku) ?? 0) - (this.reserved.get(sku) ?? 0);
  }
}

// Orders: they reach straight into the warehouse's table
class Orders {
  constructor(warehouse) {
    this.warehouse = warehouse;
  }

  place(sku, qty) {
    const left = this.warehouse.stock.get(sku);
    if (left < qty) return "out of stock";
    this.warehouse.stock.set(sku, left - qty);
    return "accepted";
  }
}

const warehouse = new Warehouse();
warehouse.stock.set("KTL-100", 2);
const held = warehouse.reserve("KTL-100", 2);
console.log("Reservation for the store: " + (held ? "yes" : "no"));
const orders = new Orders(warehouse);
console.log("Online order: " + orders.place("KTL-100", 1));
console.log("Free in the warehouse: " + warehouse.free("KTL-100"));

Output

Reservation for the store: yes
Online order: accepted
Free in the warehouse: -1

The shop has reserved both kettles, and the online order is accepted. Free stock is -1: an item that does not exist has been sold. The rule for reservations is in the warehouse, but the orders do not go through it.

The second pain comes later. The warehouse team wants to split stock by warehouse (Sofia, Plovdiv). It cannot: it does not know who else reads and writes its table, and every change breaks someone else's code.

The idea

Modules with an API and their own data

The code is one program that is deployed all at once: a monolith. That alone is not the trouble. The trouble is that there are no borders inside.

A modular monolith is a monolith split into modules. Each module:

• has a public API: a few methods the others call it through (receive, reserve, available);
• has its own data: the tables and classes inside are its own, and nobody else reads or writes them;
• depends only on the modules it is allowed to.

This is the "by feature" layout from the lesson Layers, with a rule for the borders. In domain-driven design such a border is called a bounded context: the part of the system in which the word "stock" has one meaning.

The modules are in one process and often in one database. Calling a module is calling a method: fast, with no network, and in one transaction if needed.

See it

The warehouse as a module

// The "warehouse" module: the data is its own, only the API is outside
class Warehouse {
  #stock = new Map();
  #reserved = new Map();

  receive(sku, qty) {
    this.#stock.set(sku, (this.#stock.get(sku) ?? 0) + qty);
  }

  reserve(sku, qty) {
    if (this.available(sku) < qty) return false;
    this.#reserved.set(sku, (this.#reserved.get(sku) ?? 0) + qty);
    return true;
  }

  available(sku) {
    return (this.#stock.get(sku) ?? 0) - (this.#reserved.get(sku) ?? 0);
  }
}

// The "orders" module: it calls only the API of the warehouse
class Orders {
  constructor(warehouse) {
    this.warehouse = warehouse;
  }

  place(sku, qty) {
    if (!this.warehouse.reserve(sku, qty)) return "out of stock";
    return "accepted";
  }
}

const warehouse = new Warehouse();
warehouse.receive("KTL-100", 2);
const held = warehouse.reserve("KTL-100", 2);
console.log("Reservation for the store: " + (held ? "yes" : "no"));
const orders = new Orders(warehouse);
console.log("Online order: " + orders.place("KTL-100", 1));
console.log("Free in the warehouse: " + warehouse.available("KTL-100"));

Output

Reservation for the store: yes
Online order: out of stock
Free in the warehouse: 0

// Warehouse: stock and the city store's reservations
class Warehouse {
  stock = new Map();
  reserved = new Map();

  reserve(sku, qty) {
    if (this.free(sku) < qty) return false;
    this.reserved.set(sku, (this.reserved.get(sku) ?? 0) + qty);
    return true;
  }

  free(sku) {
    return (this.stock.get(sku) ?? 0) - (this.reserved.get(sku) ?? 0);
  }
}

// Orders: they reach straight into the warehouse's table
class Orders {
  constructor(warehouse) {
    this.warehouse = warehouse;
  }

  place(sku, qty) {
    const left = this.warehouse.stock.get(sku);
    if (left < qty) return "out of stock";
    this.warehouse.stock.set(sku, left - qty);
    return "accepted";
  }
}

Step by step, in words

1. One process, one database. The warehouse keeps stock and reservations, and its tables are open to everyone.
2. The store reserves both kettles. The online order reaches straight into the table, misses the reservation, and free stock becomes -1.
3. A modular monolith: the data of the warehouse becomes its own. Only the API stays outside: receive, reserve, available.
4. Orders go through reserve, and the warehouse keeps its rule: "out of stock", free stays 0. Still one process, no network.
5. The warehouse leaves as a separate service: its own process, deploy and team. A call becomes a message over the network.
6. Three orders, three messages: ok, silence, no. A call that could not fail inside the monolith now fails.
7. Modules keep the borders without a network. A service adds an independent deploy, and pays with the network, split data and monitoring.

The order goes through reserve, and the warehouse keeps its rule: both kettles are reserved, so the online order gets "out of stock". The last steps of the scene show what happens when the warehouse leaves as a separate service. Their code is further down.

In JavaScript #stock is really hidden: it cannot be reached from outside the class. A module is a file or a package that exports (export) only its API.

Try it

Find the bug

The warehouse report wants to show the whole table, and someone adds a method:

Pick the line with the bug.

1// The "warehouse" module: the data is its own, only the API is outside2class Warehouse {3 #stock = new Map();4 #reserved = new Map();56 receive(sku, qty) {7 this.#stock.set(sku, (this.#stock.get(sku) ?? 0) + qty);8 }910 reserve(sku, qty) {11 if (this.available(sku) < qty) return false;12 this.#reserved.set(sku, (this.#reserved.get(sku) ?? 0) + qty);13 return true;14 }1516 stockTable() {17 return this.#stock;18 }1920 available(sku) {21 return (this.#stock.get(sku) ?? 0) - (this.#reserved.get(sku) ?? 0);22 }23}2425// The "orders" module: it calls only the API of the warehouse26class Orders {27 constructor(warehouse) {28 this.warehouse = warehouse;29 }3031 place(sku, qty) {32 if (!this.warehouse.reserve(sku, qty)) return "out of stock";33 return "accepted";34 }35}

Check

See it

Checking the borders

Borders are not kept by good intentions. Here is a small check: what each module offers, which module may depend on which, and what the code really uses. In a real project the last list is read from the imports.

function check(api, allowed, uses) {
  const problems = [];
  for (const [from, to, name] of uses) {
    if (!allowed[from].includes(to))
      problems.push(`✗ ${from} → ${to}: this dependency is not allowed`);
    else if (!api[to].includes(name))
      problems.push(`✗ ${from} → ${to}.${name}: not in the API`);
  }
  return problems;
}

// What each module offers outside
const api = {
  orders: ["place"],
  warehouse: ["receive", "reserve", "available"],
  payments: ["charge"],
  invoices: ["issue"],
};
// Which module may depend on which
const allowed = {
  orders: ["warehouse", "payments"],
  warehouse: [],
  payments: [],
  invoices: ["orders"],
};
// What the code really uses (from the imports)
const uses = [
  ["orders", "warehouse", "reserve"],
  ["orders", "warehouse", "stock"],
  ["orders", "payments", "charge"],
  ["invoices", "orders", "rows"],
  ["warehouse", "orders", "place"],
];

const problems = check(api, allowed, uses);
problems.forEach((problem) => console.log(problem));
console.log("Violations: " + problems.length);

Output

✗ orders → warehouse.stock: not in the API
✗ invoices → orders.rows: not in the API
✗ warehouse → orders: this dependency is not allowed
Violations: 3

Three violations: the orders in the warehouse table, the invoices in the order rows, and a warehouse that calls the orders (while the orders already call the warehouse). Such a check runs on every change and fails as soon as someone goes through the wall. How it becomes part of CI is in the lesson Architecture review.

The idea

When a module becomes a service

A service (microservice) is a module that runs on its own: its own process, its own deploy, its own database. The others call it over the network. It is worth it when:

• teams get in each other's way: the warehouse wants to ship a new version three times a day, and the shop once a week;
• load differs: catalogue search needs ten copies, and invoices need one;
• failure must be isolated: if recommendations go down, payment must keep working.

And it is paid for with:

• the network: a call may be slow, may not arrive, or may arrive twice;
• data: each service has its own database, so there is no JOIN between them and no single transaction. An order and a reservation are no longer saved "together". How to live with that is in Events and queues, The lost event and Saga;
• monitoring: one request goes through five processes, and the error is searched for in five logs.

Servers, containers, Kubernetes and monitoring as infrastructure are the subject of the ZeroToOps DevOps course, not of this lesson.

See it

The warehouse over the network

The warehouse is a separate service. In the program the "network" is an object with a recorded script: to each message it answers ok, no, or stays silent (timeout). The orders see only the API reserve and do not know there is a network behind it:

// The network is a recorded script: "ok", "no" or "timeout"
class Network {
  sent = 0;

  constructor(...script) {
    this.script = script;
  }

  send(message) {
    this.sent++;
    return this.script.shift();
  }
}

// The warehouse as a separate service: every call is a message
class RemoteWarehouse {
  constructor(network) {
    this.network = network;
  }

  reserve(sku, qty) {
    const answer = this.network.send(`reserve ${sku} ${qty}`);
    if (answer === "timeout") throw new Error("the warehouse does not answer");
    return answer === "ok";
  }
}

// Orders do not know if the warehouse is a module or a service
class Orders {
  constructor(warehouse) {
    this.warehouse = warehouse;
  }

  place(sku, qty) {
    try {
      if (!this.warehouse.reserve(sku, qty)) return "out of stock";
      return "accepted";
    } catch (e) {
      return "waiting: " + e.message;
    }
  }
}

const network = new Network("ok", "timeout", "no");
const orders = new Orders(new RemoteWarehouse(network));
for (let i = 1; i <= 3; i++)
  console.log(`Order ${i}: ${orders.place("KTL-100", 1)}`);
console.log("Messages over the network: " + network.sent);

Output

Order 1: accepted
Order 2: waiting: the warehouse does not answer
Order 3: out of stock
Messages over the network: 3

The second order does not know whether the item is reserved: the warehouse may have received the message and answered, and the answer may have been lost. In the module this question did not exist.

Try it

Predict the output

The network stays silent twice, then answers ok. What does the program print?

const network = new Network("timeout", "timeout", "ok");
const orders = new Orders(new RemoteWarehouse(network));
for (let i = 1; i <= 3; i++)
  console.log(`Order ${i}: ${orders.place("KTL-100", 1)}`);
console.log("Messages over the network: " + network.sent);

Answer

Order 1: out of stock
Order 2: out of stock
Order 3: accepted
Messages over the network: 3

Order 1: accepted
Messages over the network: 1

Order 1: waiting: the warehouse does not answer
Order 2: waiting: the warehouse does not answer
Order 3: accepted
Messages over the network: 3

Check

The idea

When services hurt

• Too early. A new shop with one team and ten services pays the whole price of the network without needing independent deploys. The borders are not clear yet, and moving a border between services costs far more than between modules.
• A distributed monolith. Services that must be deployed together and call one another on every request have all the drawbacks of a monolith and of services at once.
• A shared database. Two services that write to the same tables are the problem from the start of the lesson, only over the network.

That is why the common path is modular monolith first: clear modules in one process and a boundary check. A module that really needs to be separate leaves as a service. That is easy if it already has an API and its own data, like the warehouse in the scene.

More precisely, if you are curious

The frameworks. A service is an ordinary application: Spring Boot, ASP.NET Core, FastAPI or Django, Express or Nest. Services talk over HTTP (REST, gRPC) or through a message broker (Kafka, RabbitMQ). In front there is often an API gateway: one entrance that routes requests to the services and checks the input.

Conway's law. Systems resemble the organisation that builds them: three teams build a system of three parts. That is why the borders of services often follow the teams.

Checking the borders. ArchUnit (Java) and ArchUnitNET or NetArchTest (C#) write the check as a test. import-linter (Python) and dependency-cruiser (JavaScript) write it as rules in configuration. Spring Modulith (Java) and Nx (JavaScript) also know about modules.

How big a service is. "Micro" is not a size in lines. A good border is one that few calls go through, and behind which there is data nobody else writes.

Exercise

A cycle between modules

Solve it in your editor.

Someone has also allowed warehouse → orders. Now the two modules depend on each other. Write a function that takes the map "which module may depend on which" and finds every pair of modules that depend on each other. Each pair is printed once, in alphabetical order.

Output

✗ orders ↔ warehouse
Violations: 1

More exercises

• A cycle through three modules. orders → warehouse → invoices → orders is a cycle too. Find it with depth-first search (the lesson DFS finds cycles).
• The tables. Add to the check which module reads whose table (invoices reads orders.rows) and print every table that more than one module uses.

Test

Check yourself

1. What makes a module a module, and not just a folder?

A public API, and data the others cannot reach except through itA separate database on another machineA separate server

2. Orders write straight into the stock table. What is the trouble?

The warehouse's rule (the reservations) is bypassed, and every change to the table breaks the ordersThere is no trouble, since it is all one processIt is slower than going through a method

3. Which one is a good reason to make the warehouse a separate service?

Services are more modern than a monolithA separate team ships it several times a day, and its load differs from that of the shopThe warehouse code has grown large

4. The warehouse is a service and does not answer. Which of these is a new problem the monolith did not have?

The warehouse may refuse because it is out of stockThe order has to call reserveThe call may not get through, and the order must decide what to do then

5. The boundary check allows both orders → warehouse and warehouse → orders. Why is that bad?

Because Java does not compile cyclesIt is not bad: a two-way link is more flexibleThe two modules form a cycle: neither can change or leave without the other

Check

Start the lesson over

Related lessons

• Builds on: Repository · SOLID without dogma · Layers · Hexagonal and clean architecture
• Continues in: Events and queues · Transactional outbox · Scaling · ADR: architecture decision records
• Tasks on this topic: Software architecture