API and user interface

In short: A component test calls the rule but does not see how the server uses it. An API test sends a request like a real client and checks the status code and the body: that is how it catches a server that copied the rule and forgot the refusal. A UI test opens a browser and finds buttons by role and name, the way a person sees them. It is slow and brittle, so there are few of them and they do not run on every change.

Problem

Everything is green, and the empty cart goes through

The scenarios pass. The component tests pass. On Friday the shop releases a new version, and on Monday the database has orders for 0.00 € with paid delivery of 4.90 €.

The rule in the delivery class is correct. But the programmer on the server "optimised" things and computed the fee right in the code that answers requests. He did not copy the refusal. No test went through the server.

The idea

The API test

The shop server answers requests. The customer app asks GET /delivery?total=4999 and gets the fee. For an empty cart it gets the status code 400 and an explanation:

Java

src/main/java/shop/ShopApi.java

private static void delivery(HttpExchange exchange) throws IOException {
    int total = Integer.parseInt(exchange.getRequestURI().getQuery().replace("total=", ""));
    try {
        send(exchange, 200, String.valueOf(Delivery.feeFor(total)));
    } catch (IllegalArgumentException e) {
        send(exchange, 400, e.getMessage());
    }
}

C#

Shop/ShopApi.cs

app.MapGet("/delivery", (int total) =>
{
    try
    {
        return Results.Text(Delivery.FeeFor(total).ToString());
    }
    catch (ArgumentException e)
    {
        return Results.Text(e.Message, statusCode: 400);
    }
});

Python

shop_api.py

def do_GET(self):
    total = int(parse_qs(urlparse(self.path).query)["total"][0])
    try:
        self.send(200, str(delivery_fee(total)))
    except ValueError as e:
        self.send(400, str(e))

JavaScript

shop-api.js

const server = createServer((req, res) => {
  const total = Number(new URL(req.url, "http://localhost").searchParams.get("total"));
  try {
    send(res, 200, String(deliveryFee(total)));
  } catch (e) {
    send(res, 400, e.message);
  }
});

An API test is a client of this server. It starts the server, sends a real HTTP request and checks two things: the status code and the body. It does not see the insides of the server, only its contract with the client:

Java

src/test/java/shop/DeliveryApiTest.java

@Test
@DisplayName("empty cart → 400")
void emptyCartIsRefused() throws Exception {
    HttpResponse<String> response = get(0);
    assertEquals(400, response.statusCode());
    assertEquals("An empty cart cannot be ordered", response.body());
}

C#

Shop.Tests/DeliveryApiTest.cs

[Fact(DisplayName = "empty cart → 400")]
public async Task EmptyCartIsRefused()
{
    var response = await client.GetAsync("/delivery?total=0");
    Assert.Equal(400, (int)response.StatusCode);
    Assert.Equal("An empty cart cannot be ordered", await response.Content.ReadAsStringAsync());
}

Python

test_shop_api.py

def test_empty_cart_is_refused(base_url):
    status, text = get(base_url, 0)
    assert status == 400
    assert text == "An empty cart cannot be ordered"

JavaScript

shop-api.test.js

test("empty cart → 400", async () => {
  const response = await fetch(`${base}/delivery?total=0`);
  assert.equal(response.status, 400);
  assert.equal(await response.text(), "An empty cart cannot be ordered");
});

The server starts in the same process, on a free port the system picks. That way the test depends on nothing outside itself and can run anywhere.

More precisely, if you are curious

The servers here are the smallest each language has built in: com.sun.net.httpserver in Java, ASP.NET Core Minimal API in C#, http.server in Python and node:http in JavaScript. In a real project the server is big, but the test is the same: a request, a status code, a body. Java and C# also have tools that start the application in test mode without a real network (for example WebApplicationFactory in ASP.NET Core). Python and JavaScript have their own (TestClient in FastAPI, supertest).

The idea

The run

The whole API test file, against the server that calls the rule. Everything passes:

Java

src/test/java/shop/DeliveryApiTest.java

package shop;

import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;

import static org.junit.jupiter.api.Assertions.assertEquals;

class DeliveryApiTest {
    private static HttpServer server;
    private static final HttpClient client = HttpClient.newHttpClient();

    @BeforeAll
    static void start() throws Exception {
        server = ShopApi.start(0);
    }

    @AfterAll
    static void stop() {
        server.stop(0);
    }

    private HttpResponse<String> get(int total) throws Exception {
        URI uri = URI.create("http://127.0.0.1:" + server.getAddress().getPort() + "/delivery?total=" + total);
        return client.send(HttpRequest.newBuilder(uri).build(), HttpResponse.BodyHandlers.ofString());
    }

    @Test
    @DisplayName("49.99 € → delivery 4.90 €")
    void paidDelivery() throws Exception {
        HttpResponse<String> response = get(4999);
        assertEquals(200, response.statusCode());
        assertEquals("490", response.body());
    }

    @Test
    @DisplayName("50.00 € → free delivery")
    void freeDelivery() throws Exception {
        HttpResponse<String> response = get(5000);
        assertEquals(200, response.statusCode());
        assertEquals("0", response.body());
    }

    @Test
    @DisplayName("empty cart → 400")
    void emptyCartIsRefused() throws Exception {
        HttpResponse<String> response = get(0);
        assertEquals(400, response.statusCode());
        assertEquals("An empty cart cannot be ordered", response.body());
    }
}

Output

PASS  49.99 € → delivery 4.90 €
PASS  50.00 € → free delivery
PASS  empty cart → 400
Tests: 3, passed: 3, failed: 0

C#

Shop.Tests/DeliveryApiTest.cs

using Microsoft.AspNetCore.Builder;
using Xunit;

namespace Shop;

public class DeliveryApiTest : IAsyncLifetime
{
    private WebApplication app;
    private readonly HttpClient client = new();

    public async Task InitializeAsync()
    {
        app = ShopApi.Create();
        app.Urls.Add("http://127.0.0.1:0");
        await app.StartAsync();
        client.BaseAddress = new Uri(app.Urls.First());
    }

    public async Task DisposeAsync() => await app.StopAsync();

    [Fact(DisplayName = "49.99 € → delivery 4.90 €")]
    public async Task PaidDelivery()
    {
        var response = await client.GetAsync("/delivery?total=4999");
        Assert.Equal(200, (int)response.StatusCode);
        Assert.Equal("490", await response.Content.ReadAsStringAsync());
    }

    [Fact(DisplayName = "50.00 € → free delivery")]
    public async Task FreeDelivery()
    {
        var response = await client.GetAsync("/delivery?total=5000");
        Assert.Equal(200, (int)response.StatusCode);
        Assert.Equal("0", await response.Content.ReadAsStringAsync());
    }

    [Fact(DisplayName = "empty cart → 400")]
    public async Task EmptyCartIsRefused()
    {
        var response = await client.GetAsync("/delivery?total=0");
        Assert.Equal(400, (int)response.StatusCode);
        Assert.Equal("An empty cart cannot be ordered", await response.Content.ReadAsStringAsync());
    }
}

Output

PASS  49.99 € → delivery 4.90 €
PASS  50.00 € → free delivery
PASS  empty cart → 400
Tests: 3, passed: 3, failed: 0

Python

test_shop_api.py

import urllib.request
from urllib.error import HTTPError

import pytest

from shop_api import start


@pytest.fixture(scope="module")
def base_url():
    server = start()
    yield f"http://127.0.0.1:{server.server_address[1]}"
    server.shutdown()


def get(base_url, total):
    try:
        with urllib.request.urlopen(f"{base_url}/delivery?total={total}") as response:
            return response.status, response.read().decode()
    except HTTPError as e:
        return e.code, e.read().decode()


def test_paid_delivery(base_url):
    status, text = get(base_url, 4999)
    assert status == 200
    assert text == "490"


def test_free_delivery(base_url):
    status, text = get(base_url, 5000)
    assert status == 200
    assert text == "0"


def test_empty_cart_is_refused(base_url):
    status, text = get(base_url, 0)
    assert status == 400
    assert text == "An empty cart cannot be ordered"

Output

PASS  test_empty_cart_is_refused
PASS  test_free_delivery
PASS  test_paid_delivery
Tests: 3, passed: 3, failed: 0

JavaScript

shop-api.test.js

import { after, before, test } from "node:test";
import assert from "node:assert/strict";
import { start } from "./shop-api.js";

let server;
let base;

before(async () => {
  server = await start();
  base = `http://127.0.0.1:${server.address().port}`;
});

after(() => server.close());

test("49.99 € → delivery 4.90 €", async () => {
  const response = await fetch(`${base}/delivery?total=4999`);
  assert.equal(response.status, 200);
  assert.equal(await response.text(), "490");
});

test("50.00 € → free delivery", async () => {
  const response = await fetch(`${base}/delivery?total=5000`);
  assert.equal(response.status, 200);
  assert.equal(await response.text(), "0");
});

test("empty cart → 400", async () => {
  const response = await fetch(`${base}/delivery?total=0`);
  assert.equal(response.status, 400);
  assert.equal(await response.text(), "An empty cart cannot be ordered");
});

Output

PASS  49.99 € → delivery 4.90 €
PASS  50.00 € → free delivery
PASS  empty cart → 400
Tests: 3, passed: 3, failed: 0

Now the server from the start of the lesson: the rule is copied, without the refusal.

Java

src/main/java/shop/ShopApi.java

private static void delivery(HttpExchange exchange) throws IOException {
    int total = Integer.parseInt(exchange.getRequestURI().getQuery().replace("total=", ""));
    // The rule is copied here instead of calling Delivery.
    int fee = total >= Delivery.FREE_FROM ? 0 : Delivery.FEE;
    send(exchange, 200, String.valueOf(fee));
}

C#

Shop/ShopApi.cs

app.MapGet("/delivery", (int total) =>
{
    // The rule is copied here instead of calling Delivery.
    int fee = total >= Delivery.FreeFrom ? 0 : Delivery.Fee;
    return Results.Text(fee.ToString());
});

Python

shop_api.py

def do_GET(self):
    total = int(parse_qs(urlparse(self.path).query)["total"][0])
    # The rule is copied here instead of calling Delivery.
    fee = 0 if total >= FREE_FROM else FEE
    self.send(200, str(fee))

JavaScript

shop-api.js

const server = createServer((req, res) => {
  const total = Number(new URL(req.url, "http://localhost").searchParams.get("total"));
  // The rule is copied here instead of calling Delivery.
  const fee = total >= FREE_FROM ? 0 : FEE;
  send(res, 200, String(fee));
});

The same tests fail on the empty cart. The server says 200, and it should say 400:

Java

src/test/java/shop/DeliveryApiTest.java

package shop;

import com.sun.net.httpserver.HttpServer;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;

import static org.junit.jupiter.api.Assertions.assertEquals;

class DeliveryApiTest {
    private static HttpServer server;
    private static final HttpClient client = HttpClient.newHttpClient();

    @BeforeAll
    static void start() throws Exception {
        server = ShopApi.start(0);
    }

    @AfterAll
    static void stop() {
        server.stop(0);
    }

    private HttpResponse<String> get(int total) throws Exception {
        URI uri = URI.create("http://127.0.0.1:" + server.getAddress().getPort() + "/delivery?total=" + total);
        return client.send(HttpRequest.newBuilder(uri).build(), HttpResponse.BodyHandlers.ofString());
    }

    @Test
    @DisplayName("49.99 € → delivery 4.90 €")
    void paidDelivery() throws Exception {
        HttpResponse<String> response = get(4999);
        assertEquals(200, response.statusCode());
        assertEquals("490", response.body());
    }

    @Test
    @DisplayName("50.00 € → free delivery")
    void freeDelivery() throws Exception {
        HttpResponse<String> response = get(5000);
        assertEquals(200, response.statusCode());
        assertEquals("0", response.body());
    }

    @Test
    @DisplayName("empty cart → 400")
    void emptyCartIsRefused() throws Exception {
        HttpResponse<String> response = get(0);
        assertEquals(400, response.statusCode());
        assertEquals("An empty cart cannot be ordered", response.body());
    }
}

Output

FAIL  empty cart → 400: expected: <400> but was: <200>
PASS  49.99 € → delivery 4.90 €
PASS  50.00 € → free delivery
Tests: 3, passed: 2, failed: 1

C#

Shop.Tests/DeliveryApiTest.cs

using Microsoft.AspNetCore.Builder;
using Xunit;

namespace Shop;

public class DeliveryApiTest : IAsyncLifetime
{
    private WebApplication app;
    private readonly HttpClient client = new();

    public async Task InitializeAsync()
    {
        app = ShopApi.Create();
        app.Urls.Add("http://127.0.0.1:0");
        await app.StartAsync();
        client.BaseAddress = new Uri(app.Urls.First());
    }

    public async Task DisposeAsync() => await app.StopAsync();

    [Fact(DisplayName = "49.99 € → delivery 4.90 €")]
    public async Task PaidDelivery()
    {
        var response = await client.GetAsync("/delivery?total=4999");
        Assert.Equal(200, (int)response.StatusCode);
        Assert.Equal("490", await response.Content.ReadAsStringAsync());
    }

    [Fact(DisplayName = "50.00 € → free delivery")]
    public async Task FreeDelivery()
    {
        var response = await client.GetAsync("/delivery?total=5000");
        Assert.Equal(200, (int)response.StatusCode);
        Assert.Equal("0", await response.Content.ReadAsStringAsync());
    }

    [Fact(DisplayName = "empty cart → 400")]
    public async Task EmptyCartIsRefused()
    {
        var response = await client.GetAsync("/delivery?total=0");
        Assert.Equal(400, (int)response.StatusCode);
        Assert.Equal("An empty cart cannot be ordered", await response.Content.ReadAsStringAsync());
    }
}

Output

FAIL  empty cart → 400: Assert.Equal() Failure: Values differ Expected: 400 Actual: 200
PASS  49.99 € → delivery 4.90 €
PASS  50.00 € → free delivery
Tests: 3, passed: 2, failed: 1

Python

test_shop_api.py

import urllib.request
from urllib.error import HTTPError

import pytest

from shop_api import start


@pytest.fixture(scope="module")
def base_url():
    server = start()
    yield f"http://127.0.0.1:{server.server_address[1]}"
    server.shutdown()


def get(base_url, total):
    try:
        with urllib.request.urlopen(f"{base_url}/delivery?total={total}") as response:
            return response.status, response.read().decode()
    except HTTPError as e:
        return e.code, e.read().decode()


def test_paid_delivery(base_url):
    status, text = get(base_url, 4999)
    assert status == 200
    assert text == "490"


def test_free_delivery(base_url):
    status, text = get(base_url, 5000)
    assert status == 200
    assert text == "0"


def test_empty_cart_is_refused(base_url):
    status, text = get(base_url, 0)
    assert status == 400
    assert text == "An empty cart cannot be ordered"

Output

FAIL  test_empty_cart_is_refused: assert 200 == 400
PASS  test_free_delivery
PASS  test_paid_delivery
Tests: 3, passed: 2, failed: 1

JavaScript

shop-api.test.js

import { after, before, test } from "node:test";
import assert from "node:assert/strict";
import { start } from "./shop-api.js";

let server;
let base;

before(async () => {
  server = await start();
  base = `http://127.0.0.1:${server.address().port}`;
});

after(() => server.close());

test("49.99 € → delivery 4.90 €", async () => {
  const response = await fetch(`${base}/delivery?total=4999`);
  assert.equal(response.status, 200);
  assert.equal(await response.text(), "490");
});

test("50.00 € → free delivery", async () => {
  const response = await fetch(`${base}/delivery?total=5000`);
  assert.equal(response.status, 200);
  assert.equal(await response.text(), "0");
});

test("empty cart → 400", async () => {
  const response = await fetch(`${base}/delivery?total=0`);
  assert.equal(response.status, 400);
  assert.equal(await response.text(), "An empty cart cannot be ordered");
});

Output

FAIL  empty cart → 400: 200 !== 400
PASS  49.99 € → delivery 4.90 €
PASS  50.00 € → free delivery
Tests: 3, passed: 2, failed: 1

The component tests from the previous lesson still pass, because the delivery class has not been touched. Every level catches its own bugs.

Try it

What does only the API test see?

1. The component tests pass. What else can be broken that only the API test sees?

The colour of the buttonHow the server uses the rule: whether it calls it, which status code it returns, what is in the bodyThe delivery rule

2. An empty cart has to be refused. The API test checks only the body "An empty cart cannot be ordered". What does it miss?

NothingThe status code: a server that returns 200 with the error text passesThe response time

3. Why does the API test start the server on a free port (port 0) and not on 8080?

So that it does not clash with another process and the tests can run anywhere, and at the same timeIt makes no differenceBecause 8080 is forbidden

Check

The idea

The UI test with Playwright

A UI test opens a real browser and does what the customer does: opens the cart, presses "Place order", reads what it says. The tool here is Playwright. It has versions for all four languages:

Java

src/test/java/shop/CheckoutUiTest.java

package shop;

import com.microsoft.playwright.Browser;
import com.microsoft.playwright.Page;
import com.microsoft.playwright.Playwright;
import com.microsoft.playwright.options.AriaRole;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

import static com.microsoft.playwright.assertions.PlaywrightAssertions.assertThat;

@Tag("ui")
class CheckoutUiTest {
    @Test
    @DisplayName("an order exactly on the boundary")
    void freeDeliveryOnTheBoundary() {
        try (Playwright playwright = Playwright.create()) {
            Browser browser = playwright.chromium().launch();
            Page page = browser.newPage();
            // a cart of 50.00 €
            page.navigate("http://localhost:4100/cart?test-cart=5000");
            page.getByRole(AriaRole.BUTTON, new Page.GetByRoleOptions().setName("Place order")).click();
            assertThat(page.getByRole(AriaRole.HEADING, new Page.GetByRoleOptions().setName("Order accepted"))).isVisible();
            assertThat(page.getByText("Delivery: free")).isVisible();
        }
    }
}

C#

Shop.UiTests/CheckoutUiTest.cs

using Microsoft.Playwright;
using Xunit;

namespace Shop;

public class CheckoutUiTest
{
    [Fact(DisplayName = "an order exactly on the boundary")]
    [Trait("Category", "UI")]
    public async Task FreeDeliveryOnTheBoundary()
    {
        using var playwright = await Playwright.CreateAsync();
        await using var browser = await playwright.Chromium.LaunchAsync();
        var page = await browser.NewPageAsync();
        // a cart of 50.00 €
        await page.GotoAsync("http://localhost:4100/cart?test-cart=5000");
        await page.GetByRole(AriaRole.Button, new() { Name = "Place order" }).ClickAsync();
        await Assertions.Expect(page.GetByRole(AriaRole.Heading, new() { Name = "Order accepted" })).ToBeVisibleAsync();
        await Assertions.Expect(page.GetByText("Delivery: free")).ToBeVisibleAsync();
    }
}

Python

test_checkout_ui.py

import pytest
from playwright.sync_api import Page, expect


@pytest.mark.ui
def test_free_delivery_on_the_boundary(page: Page):
    # a cart of 50.00 €
    page.goto("http://localhost:4100/cart?test-cart=5000")
    page.get_by_role("button", name="Place order").click()
    expect(page.get_by_role("heading", name="Order accepted")).to_be_visible()
    expect(page.get_by_text("Delivery: free")).to_be_visible()

JavaScript

tests/checkout.spec.js

import { test, expect } from "@playwright/test";

test("an order exactly on the boundary", async ({ page }) => {
  // a cart of 50.00 €
  await page.goto("http://localhost:4100/cart?test-cart=5000");
  await page.getByRole("button", { name: "Place order" }).click();
  await expect(page.getByRole("heading", { name: "Order accepted" })).toBeVisible();
  await expect(page.getByText("Delivery: free")).toBeVisible();
});

Notice how the test finds the button: by role and name ("a button named Place order"), not by CSS (#app > div:nth-child(3) > button). Role and name are what the customer sees and what a screen reader says to a blind person. They change when the meaning changes. CSS changes when the designer moves something.

Playwright checks wait by themselves: "is the heading visible" waits for up to a few seconds until it appears. That is why there are no pauses in the test.

More precisely, if you are curious

The test cart (?test-cart=5000) is an arrangement with the programmers: in the test environment the shop can start with a ready cart, so that the UI test does not have to search for and add products. This is a part of the test that lives not in Playwright but in the shop.

Playwright is made by Microsoft and is open source. Other widespread tools are Selenium (the oldest) and Cypress (JavaScript only). Locating by role was made popular by Testing Library, and the roles are those of the ARIA accessibility standard.

The idea

Why UI tests break and why there are few

A UI test is the closest to the customer and the most expensive:

• slow: a browser, a network, a page. Seconds per test, not milliseconds;
• needs everything: a running shop, a database, test data;
• brittle: it fails because of a moved button, a slow network, an animation. A brittle test fails without a defect, and people stop trusting it.

That is why there are few UI tests: only the end-to-end paths that must not break. The rules are checked further down: the amounts at component level, the server contract with an API test.

This UI test does not run in the course: it needs a browser and a running shop, which the course checker does not have. For the same reason, in the CI configuration in the next lesson the UI tests are kept apart and run once a night against the test environment, not on every change. The scenarios, the component tests and the API tests in this module really run.

Try it

Which locator survives?

1. Which locator survives if the designer moves the "Place order" button into another block and changes its classes?

#app > div:nth-child(3) > button.btn-primaryThe button named "Place order" (role and name)The third button on the page

2. The button is <div class="btn" onclick="…">Place order</div>. A test looking for the role "button" does not find it. What does this mean?

That a customer with a screen reader will not find it either: it is an accessibility defect, not a problem of the testThat the test is wrong and needs CSSThat Playwright does not support div

3. After "Place order" the confirmation appears after half a second. How does the test wait for it?

It does not waitWith a 3-second pauseWith a check that waits by itself, up to a set time, until something appears

Check

Exercise

Which level for what

1. API, UI or further down. The shop is sorting out its tests.

1. What is best checked with an API test?

That a request for an empty cart returns 400 and an explanationThat the threshold is 50 € for ten different amountsThat the customer understands the message

2. The shop has 400 UI tests, they run for 2 hours, and every day five or six fail without a defect. What do you suggest?

Move most checks down to the API and component level; keep UI tests for the few paths that must not breakRun them only once a monthMore UI tests

3. Which UI test is worth the most?

The colour of every buttonThe customer orders a cart on the boundary and sees "Delivery: free"Every amount from 1 to 100 € through the screen

Check

Test

Check yourself

1. What does an API test check?

The rule code from the insideThe server response to a real request: the status code and the bodyWhat the page looks like

2. Why does a UI test find elements by role and name and not by CSS?

Role and name change when the meaning changes; CSS changes with every design changeCSS is forbidden in PlaywrightRoles are faster

3. Why are there few UI tests?

They are slow, they need a browser and a running shop, and they break on small changesBecause they do not find defectsBecause browsers cost money

4. The server copied the rule and forgot the refusal. Which test catches it?

NoneThe API test for an empty cartThe component test

Check

Start the lesson over

Related lessons

• Builds on: Usability and accessibility · Why and what to automate · From scenario to steps · Component tests
• Continues in: Tests in CI
• Tasks on this topic: Automation