Template Method and Iterator

Code in:JavaC#PythonJavaScript

In short: Template Method keeps the shared skeleton of an algorithm in a base class while subclasses fill in the steps; Iterator hands out elements one by one without showing how they are stored.

Problem

Two reports with the same skeleton

Accounting wants the orders in CSV, and the website wants the same ones in an HTML table. Cancelled orders don't go into the report. Someone wrote the CSV one, and then copied the method for HTML:

static string ExportCsv(List<Order> orders)
{
    var lines = new List<string>();
    lines.Add("order,total");
    decimal sum = 0;
    foreach (var order in orders)
    {
        if (order.Cancelled)
        {
            continue;
        }
        lines.Add($"{order.Id},{order.Total:F2}");
        sum += order.Total;
    }
    lines.Add($"total,{sum:F2}");
    return string.Join("\n", lines);
}

static string ExportHtml(List<Order> orders)
{
    var lines = new List<string>();
    lines.Add("<table>");
    lines.Add("<tr><th>Order</th><th>Total</th></tr>");
    decimal sum = 0;
    foreach (var order in orders)
    {
        lines.Add($"<tr><td>{order.Id}</td><td>{order.Total:F2}</td></tr>");
        sum += order.Total;
    }
    lines.Add($"<tr><td>Total</td><td>{sum:F2}</td></tr>");
    lines.Add("</table>");
    return string.Join("\n", lines);
}

var orders = new List<Order>
{
    new Order(1001, 80.00m, false),
    new Order(1002, 25.50m, false),
    new Order(1003, 40.00m, true),
};
Console.WriteLine(ExportCsv(orders));
Console.WriteLine(ExportHtml(orders));

Output

order,total
1001,80.00
1002,25.50
total,105.50
<table>
<tr><th>Order</th><th>Total</th></tr>
<tr><td>1001</td><td>80.00</td></tr>
<tr><td>1002</td><td>25.50</td></tr>
<tr><td>1003</td><td>40.00</td></tr>
<tr><td>Total</td><td>145.50</td></tr>
</table>

The two reports show a different total: the CSV says €105.50, and the HTML says €145.50. The rule "cancelled orders don't count" exists only in the first copy. Both copies do the same thing: they open, write a header, go over the orders, write the total and close. Only the lines they write are different. A third format will be a third copy, and the rule will have to be remembered in one more place.

The idea

Template Method: one skeleton, different steps

What doesn't change is the order of the steps and the rules between them: open, header, a row for each order except the cancelled ones, total, close. What changes is what each step looks like.

The skeleton goes into the base class Report, in the method Export. At every place where the formats differ, it calls a step: Header, Row, Footer. The subclass writes only the steps. A step that is optional (Open and Close, which CSV doesn't use) has an empty body in the base class and is called a hook.

This is the Template Method pattern. It has two roles:

• the base class with the skeleton: it decides the order and calls the steps;
• the subclasses: CsvReport and HtmlReport write the steps and call nothing themselves.

See it

The skeleton and the two reports

Export is not virtual, so a subclass can't change it. The steps are protected and abstract, except the two hooks, which are virtual with an empty body:

abstract class Report
{
    public string Export(List<Order> orders)
    {
        var lines = new List<string>();
        Open(lines);
        Header(lines);
        decimal sum = 0;
        foreach (var order in orders)
        {
            if (order.Cancelled)
            {
                continue;
            }
            Row(lines, order);
            sum += order.Total;
        }
        Footer(lines, sum);
        Close(lines);
        return string.Join("\n", lines);
    }

    protected virtual void Open(List<string> lines) { }
    protected abstract void Header(List<string> lines);
    protected abstract void Row(List<string> lines, Order order);
    protected abstract void Footer(List<string> lines, decimal sum);
    protected virtual void Close(List<string> lines) { }
}

The two reports write only their lines. CsvReport doesn't write the hooks, so it uses the empty ones from the base class:

class CsvReport : Report
{
    protected override void Header(List<string> lines) => lines.Add("order,total");
    protected override void Row(List<string> lines, Order order) => lines.Add($"{order.Id},{order.Total:F2}");
    protected override void Footer(List<string> lines, decimal sum) => lines.Add($"total,{sum:F2}");
}

class HtmlReport : Report
{
    protected override void Open(List<string> lines) => lines.Add("<table>");
    protected override void Header(List<string> lines) => lines.Add("<tr><th>Order</th><th>Total</th></tr>");
    protected override void Row(List<string> lines, Order order) => lines.Add($"<tr><td>{order.Id}</td><td>{order.Total:F2}</td></tr>");
    protected override void Footer(List<string> lines, decimal sum) => lines.Add($"<tr><td>Total</td><td>{sum:F2}</td></tr>");
    protected override void Close(List<string> lines) => lines.Add("</table>");
}

var orders = new List<Order>
{
    new Order(1001, 80.00m, false),
    new Order(1002, 25.50m, false),
    new Order(1003, 40.00m, true),
};
Console.WriteLine(new CsvReport().Export(orders));
Console.WriteLine(new HtmlReport().Export(orders));

Output

order,total
1001,80.00
1002,25.50
total,105.50
<table>
<tr><th>Order</th><th>Total</th></tr>
<tr><td>1001</td><td>80.00</td></tr>
<tr><td>1002</td><td>25.50</td></tr>
<tr><td>Total</td><td>105.50</td></tr>
</table>

abstract class Report
{
    public string Export(List<Order> orders)
    {
        var lines = new List<string>();
        Open(lines);
        Header(lines);
        decimal sum = 0;
        foreach (var order in orders)
        {
            if (order.Cancelled)
            {
                continue;
            }
            Row(lines, order);
            sum += order.Total;
        }
        Footer(lines, sum);
        Close(lines);
        return string.Join("\n", lines);
    }

    protected virtual void Open(List<string> lines) { }
    protected abstract void Header(List<string> lines);
    protected abstract void Row(List<string> lines, Order order);
    protected abstract void Footer(List<string> lines, decimal sum);
    protected virtual void Close(List<string> lines) { }
}

Step by step, in words

1. new CsvReport().Export(orders): the skeleton is in Report. It decides the order of the steps.
2. Open: CsvReport doesn't write this step, so the empty body from Report runs. Nothing is added.
3. Header: the skeleton calls the step, and CsvReport writes its line.
4. Row for order 1001: the skeleton calls the step, and CsvReport writes its line.
5. Row for order 1002: the skeleton calls the step, and CsvReport writes its line.
6. Order 1003 is cancelled: the skeleton itself skips it with continue. No report has to remember this rule.
7. Footer: the skeleton calls the step, and CsvReport writes its line.
8. Close: CsvReport doesn't write this step, so the empty body from Report runs. Nothing is added.
9. Now HtmlReport: the same skeleton, other steps.
10. Open: the skeleton calls the step, and HtmlReport writes its line.
11. Header: the skeleton calls the step, and HtmlReport writes its line.
12. Row for order 1001: the skeleton calls the step, and HtmlReport writes its line.
13. Row for order 1002: the skeleton calls the step, and HtmlReport writes its line.
14. Order 1003 is cancelled: the skeleton itself skips it with continue. No report has to remember this rule.
15. Footer: the skeleton calls the step, and HtmlReport writes its line.
16. Close: the skeleton calls the step, and HtmlReport writes its line.
17. Two reports, one skeleton. Both totals are €105.50, because the rule about cancelled orders is in one place.

The total is €105.50 in both reports: the rule about cancelled orders is in the skeleton, in one place.

See it

The skeleton is not touched

A subclass that tries to write its own Export doesn't compile:

class QuickCsvReport : CsvReport
{
    public override string Export(List<Order> orders)
    {
        return "order,total";
    }
}

Error

error CS0506: 'QuickCsvReport.Export(List<Order>)': cannot override inherited member 'Report.Export(List<Order>)' because it is not marked virtual, abstract, or override

In C#, methods aren't overridden unless they are virtual, so the skeleton is protected by itself. If the subclass writes Export without override, it becomes a new method that hides the old one (warning CS0108), and Report r = new QuickCsvReport() still calls the skeleton.

See it

Without inheritance: the steps as functions

The steps don't have to be methods of a subclass. The skeleton can take them as parameters: the header is text, and the row and the total are functions.

static string Export(List<Order> orders, string header, Func<Order, string> row, Func<decimal, string> footer)
{
    var lines = new List<string> { header };
    decimal sum = 0;
    foreach (var order in orders)
    {
        if (order.Cancelled)
        {
            continue;
        }
        lines.Add(row(order));
        sum += order.Total;
    }
    lines.Add(footer(sum));
    return string.Join("\n", lines);
}

Console.WriteLine(Export(orders,
    "order,total",
    order => $"{order.Id},{order.Total:F2}",
    sum => $"total,{sum:F2}"));
Console.WriteLine(Export(orders,
    "<table>\n<tr><th>Order</th><th>Total</th></tr>",
    order => $"<tr><td>{order.Id}</td><td>{order.Total:F2}</td></tr>",
    sum => $"<tr><td>Total</td><td>{sum:F2}</td></tr>\n</table>"));

Output

order,total
1001,80.00
1002,25.50
total,105.50
<table>
<tr><th>Order</th><th>Total</th></tr>
<tr><td>1001</td><td>80.00</td></tr>
<tr><td>1002</td><td>25.50</td></tr>
<tr><td>Total</td><td>105.50</td></tr>
</table>

The skeleton is still in one place, and cancelled orders still don't count.

This is Strategy for every step: Func<Order, string> instead of an abstract method. It is enough for two or three short steps. A subclass pays off when there are many steps, they share fields, or some are optional, like the hooks Open and Close.

Problem

Everyone who wants the orders turns the pages

The orders are in a system that gives them in pages: two per request, because a request is slow. Here the source is a list in memory, and every load prints a line, so you can see when it happens:

class OrderSource
{
    private readonly List<Order> orders;

    public OrderSource(List<Order> orders)
    {
        this.orders = orders;
    }

    public List<Order> FetchPage(int number)
    {
        Console.WriteLine($"loading page {number}");
        int from = Math.Min((number - 1) * 2, orders.Count);
        int count = Math.Min(2, orders.Count - from);
        return orders.GetRange(from, count);
    }
}

The report, the email to customers and the export all want every order, and each one turns the pages itself:

int number = 1;
var page = source.FetchPage(number);
while (page.Count > 0)
{
    foreach (var order in page)
    {
        Console.WriteLine(order.Id);
    }
    number++;
    page = source.FetchPage(number);
}

Output

loading page 1
1001
1002
loading page 2
1003
1004
loading page 3
1005
loading page 4

The code that only wants "each order in turn" knows that there are pages, how they are counted, and that the end is an empty page. The same lines are in three files. When the system moves from page numbers to "a key for the next page", you change all three.

The idea

Iterator: going over things behind a cursor

The caller wants one thing: the next order, while there is one. How to get to it (a list, pages, a file, the network) is someone else's job.

That someone is the cursor: an object that remembers how far it got and answers the question "is there more?" by itself, loading a new page only when the previous one is over. The caller writes an ordinary foreach and doesn't know about the pages.

This is the Iterator pattern. It has two roles:

• the collection: the thing being gone over. It gives a cursor;
• the cursor: remembers the place and gives the next element.

This pattern is built into each of the four languages: foreach works with every object that gives a cursor. In C# the collection is an IEnumerable<T>, and the cursor is an IEnumerator<T> with MoveNext() and Current. You don't write the cursor by hand: yield return writes it for you.

See it

The cursor that loads the pages

yield return gives one order and pauses the function right there until foreach asks for the next one. That's why the next page is loaded only when the current one is over:

static IEnumerable<Order> AllOrders(OrderSource source)
{
    for (int number = 1; ; number++)
    {
        var page = source.FetchPage(number);
        if (page.Count == 0)
        {
            yield break;
        }
        foreach (var order in page)
        {
            yield return order;
        }
    }
}

The caller doesn't know about the pages:

foreach (var order in AllOrders(source))
{
    Console.WriteLine(order.Id);
}

Output

loading page 1
1001
1002
loading page 2
1003
1004
loading page 3
1005
loading page 4

static IEnumerable<Order> AllOrders(OrderSource source)
{
    for (int number = 1; ; number++)
    {
        var page = source.FetchPage(number);
        if (page.Count == 0)
        {
            yield break;
        }
        foreach (var order in page)
        {
            yield return order;
        }
    }
}

foreach (var order in AllOrders(source))
{
    Console.WriteLine(order.Id);
}

Step by step, in words

1. The loop asks for the first order. Nothing has been loaded yet.
2. AllOrders with yield has no page yet, so it loads page 1.
3. 1001: the first order from the page goes to the loop, and the loop prints it.
4. 1002: from the same page. Nothing is loaded.
5. The page is over. Only now does the cursor load page 2.
6. 1003 and 1004 from page 2.
7. The border again: page 3.
8. 1005: page 3 has only one order.
9. Page 4 is empty: there are no more orders.
10. The cursor says "no more", and the loop ends. The loop never saw a page.

The output is the same, and counting pages is in one place. When the system changes the way it gives pages, you change the function with yield, and the three places that go over the orders aren't touched.

See it

What for-each does inside

foreach is a short way to write this loop with the cursor:

using IEnumerator<Order> it = AllOrders(source).GetEnumerator();
while (it.MoveNext())
{
    Order order = it.Current;
    Console.WriteLine(order.Id);
}

Output

loading page 1
1001
1002
loading page 2
1003
1004
loading page 3
1005
loading page 4

foreach calls GetEnumerator() once, and then MoveNext() and Current on every step. yield return made the compiler write this cursor class for you.

Try it

Predict the output

The loop stops after the third order. What does the program print?

int shown = 0;
foreach (var order in AllOrders(source))
{
    Console.WriteLine(order.Id);
    shown++;
    if (shown == 3)
    {
        break;
    }
}

Answer

loading page 1 / 1001 / 1002 / loading page 2 / 1003 / 1004

loading page 1 / loading page 2 / loading page 3 / loading page 4 / 1001 / 1002 / 1003

loading page 1 / 1001 / 1002 / loading page 2 / 1003

Check

Try it

Find the bug

The email to customers first says how many orders there are, and then lists them:

var orders = AllOrders(new OrderSource(all));
int count = 0;
foreach (var order in orders)
{
    count++;
}
Console.WriteLine($"Orders: {count}");
foreach (var order in orders)
{
    Console.WriteLine(order.Id);
}

Output

loading page 1
loading page 2
loading page 3
loading page 4
Orders: 5
loading page 1
1001
1002
loading page 2
1003
1004
loading page 3
1005
loading page 4

The numbers come out, but every page is loaded twice, and the source is slow. Which line do you have to change?

Pick the line with the bug.

1var orders = AllOrders(new OrderSource(all));2int count = 0;3foreach (var order in orders)4{5 count++;6}7Console.WriteLine($"Orders: {count}");8foreach (var order in orders)9{10 Console.WriteLine(order.Id);11}

Check

The idea

When you don't need them

Template Method has a cost: a base class and inheritance, and inheritance is a tie that is hard to undo. It isn't worth it when:

• there is one format. One method without steps is clearer;
• there are one or two steps. Pass them as lambdas, as in the card "Without inheritance";
• the subclasses want to change the order of the steps too. Then the skeleton isn't shared, and forcing it gives you steps with an if inside.

You don't need your own Iterator when the data is already in a list: it can already be gone over. You write your own when there is something behind it that isn't a collection in memory: pages, a file read line by line, a tree, or an endless sequence (invoice numbers).

Exercise

A report in Markdown

Solve the task in your editor.

Markdown. The documentation site wants the report as a Markdown table. Write MarkdownReport, which writes only the steps. The header is two lines. Report and the other reports are not touched. With the orders from the lesson:

Output

| Order | Total |
|---|---|
| 1001 | 80.00 |
| 1002 | 25.50 |
| **Total** | 105.50 |

More tasks

• Row numbers. The HTML report numbers its rows (<td>1</td>, <td>2</td>), and cancelled orders don't count. Where do you keep the counter: in the skeleton or in HtmlReport?
• The first N. Write a method with yield that gives only the first N orders from another cursor. How many pages are loaded for the first three?
• Only the big ones. Go over the orders from the source and give only those over €30, again without the caller knowing about the pages.

Test

Check yourself

1. The shop wants the report in JSON too. What do you write with Template Method?

A new if in export for each formatA copy of the CSV method, and you change itA new subclass of Report that writes the steps; the skeleton isn't touched

2. Why does the open step have an empty body in the base class, while header is abstract?

open is optional (CSV doesn't use it), and a report without a header makes no senseAbstract steps are fasterNo reason, both could be abstract

3. A loop goes over the orders from the slow source and stops with break at the second one. How many pages are loaded?

All of themTwoOne

4. What does for-each call to go over something?

The indexes from 0 to size()The cursor of the object: it asks if there is a next one and takes itA method that returns the whole list at once

Check

Start the lesson over

Related lessons

• Builds on: Lambdas and delegates · record, switch and patterns · Design patterns · Strategy
• Tasks on this topic: Design patterns