Skip to content
wannacode

Tests with xUnit

In short: A unit test is code that checks one small piece, usually one method, on every change; in C# you write it with xUnit, [Fact] and Assert.Equal.

Problem

The discount that broke quietly

The shop's rule is: the discount is in percent, and on a purchase of €100 or more you add another 5%. You write the method and try it by hand:

Program.cs
class Discount
{
    // Discount in percent. From €100 up, add another 5%.
    public static double Apply(double price, int percent)
    {
        if (price > 100)
        {
            percent = percent + 5;
        }
        return price * (100 - percent) / 100;
    }
}


Console.WriteLine(Discount.Apply(50, 10));
Console.WriteLine(Discount.Apply(200, 10));
Output
45
170

€50 with 10% off is €45, and €200 with 15% off is €170. It looks right, so you ship it. A week later a customer with a purchase of exactly €100 pays €90 instead of €85. The condition is price > 100, and it should be price >= 100.

Nobody tried exactly €100. And on every later change to the method, you have to try all the cases by hand again.

The idea

Code that checks code

Instead of reading the output with your own eyes, you write small methods that call your code and compare the result with the expected one. That is a test. You run all the tests with one click, after every change, and they tell you what broke.

Kind What it checks Example
Unit test one small piece, usually one method, apart from the rest Discount.Apply(100, 10) returns 85
Integration how the pieces work together: with a database, a file, another service the order is saved to the database and read back
End-to-end the whole application, the way a person uses it the customer adds to the cart, pays and gets an email
In this lesson we write unit tests. In C# the library most often used for them is xUnit.

Why it is worth it:

  • you see the bug right away, while you still remember what you changed, not when the customer complains;
  • the test shows how the code is used: which case returns what;
  • you can clean up the code boldly. If the tests are still green after the change, the behavior has not changed.
More precisely, if you are curious

There are other kinds of tests too: functional (does the program meet the requirements), performance (how many requests it can handle), security (are there any holes) and user interface (the buttons, the forms). All of them build on unit tests: there are the most of them, and they are the fastest and the cheapest to maintain.

Besides xUnit, in .NET you will see NUnit ([Test], [SetUp]) and MSTest ([TestMethod], [TestInitialize]). The ideas are the same. Here we use xUnit: it is also what the dotnet new xunit template gives you.

See it

The first test

The tests for Discount are in a separate class, DiscountTest, in a separate test project:
DiscountTest.cs
using Xunit;


public class DiscountTest
{
    [Fact]
    public void TenPercentOff()
    {
        Assert.Equal(45.0, Discount.Apply(50, 10));
    }


    [Fact]
    public void ExtraFivePercentFrom100()
    {
        Assert.Equal(85.0, Discount.Apply(100, 10));
    }
}
  • [Fact] in front of the method tells xUnit "this is a test, run it". Tags like this in square brackets are called attributes. How xUnit finds them, you will see in the last lesson of the module.
  • The test class is public: xUnit only runs tests from public classes. using Xunit; at the top gives you Fact and Assert.
  • Assert.Equal(expected, actual) compares the two values. If they are different, the test fails.

We run the two tests against the code from the first card:

Result
FAIL  ExtraFivePercentFrom100
      Assert.Equal() Failure: Values differ
      Expected: 85
      Actual:   90
PASS  TenPercentOff
Tests: 2, passed: 1, failed: 1

The first test passes. The second one fails and says exactly what happened: it expected 85, and the method returned 90. In xUnit's output the expected value is Expected, and the returned one is Actual. You fix the condition to price >= 100, run them again, and both are green:

Result
PASS  ExtraFivePercentFrom100
PASS  TenPercentOff
Tests: 2, passed: 2, failed: 0

Every test does three things: it prepares the values, calls the code and checks the result.

public class DiscountTest
{
    [Fact]
    public void TenPercentOff()
    {
        Assert.Equal(45.0, Discount.Apply(50, 10));
    }


    [Fact]
    public void ExtraFivePercentFrom100()
    {
        Assert.Equal(85.0, Discount.Apply(100, 10));
    }
}
Step by step, in words
  1. DiscountTest has two tests. xUnit runs each one on its own and lights a lamp next to it.
  2. Test 1, prepare: a price of €50 and 10% off.
  3. Act: Discount.Apply(50, 10) returns 45.
  4. Check: Assert.Equal compares the expected 45 with the returned 45. They are the same, so the test passes.
  5. Test 2: exactly €100 and 10%. We expect 85, because from €100 up there is another 5%.
  6. The condition in the method is price > 100, and 100 is not greater than 100. The method returns 90.
  7. Expected 85, returned 90: the test fails and says exactly that.
  8. You fix the condition to price >= 100 and run the tests again.
  9. Now 100 passes the condition and the method returns 85.
  10. Expected and returned match. Both tests are green.
More precisely, if you are curious
  • In a real project the tests are in a separate project next to the code, for example Shop.Tests next to Shop. Visual Studio and Rider show all the tests in Test Explorer: you click "Run" and see the result.
Layout
Shop/Discount.cs                  the code
Shop/Shop.csproj
Shop.Tests/DiscountTest.cs        the tests for it
Shop.Tests/Shop.Tests.csproj      xUnit is added here
  • You make the test project with dotnet new xunit, and dotnet test builds it and runs the tests:
Commands
dotnet new xunit -o Shop.Tests                 a new test project
dotnet add Shop.Tests reference Shop           the tests can see the code
dotnet test                                    builds and runs all the tests
dotnet test --filter "FullyQualifiedName~Discount"
                                               only the tests with Discount in the name
  • xUnit is a package in the .csproj file of the test project. The template adds the packages, and dotnet add reference adds the link to the code:
Shop.Tests.csproj
<ItemGroup>
  <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1" />
  <PackageReference Include="xunit" Version="2.9.3" />
  <PackageReference Include="xunit.runner.visualstudio" Version="3.1.4" />
</ItemGroup>
<ItemGroup>
  <ProjectReference Include="../Shop/Shop.csproj" />
</ItemGroup>
  • Here, on the site, the tests and Main are in one program. Main runs the tests through xUnit's programming interface (AssemblyRunner from the xunit.runner.utility package), one at a time and with the method name without the class, and prints the results sorted by name. They are sorted because xUnit does not promise the order of the tests. The failure messages are xUnit's own, indented under the name of the test:
The runner
using System.Reflection;
using Xunit;
using Xunit.Runners;


class Program
{
    static void Main()
    {
        RunTests();
    }


    // Runs the tests in this program one by one and prints the results, sorted by name.
    static void RunTests(string category = null)
    {
        var results = new List<string>();
        int failed = 0;
        using var runner = AssemblyRunner.WithoutAppDomain(Assembly.GetExecutingAssembly().Location);
        if (category != null)
        {
            runner.TestCaseFilter = test => test.Traits.TryGetValue("Category", out var values) && values.Contains(category);
        }
        runner.OnTestPassed = test => results.Add("PASS  " + test.TestDisplayName);
        runner.OnTestFailed = test =>
        {
            failed++;
            results.Add("FAIL  " + test.TestDisplayName + "\n      " + test.ExceptionMessage.ReplaceLineEndings("\n      "));
        };
        runner.Start(new AssemblyRunnerStartOptions { Parallel = false, MethodDisplay = TestMethodDisplay.Method });
        while (runner.Status != AssemblyRunnerStatus.Idle)
        {
            Thread.Sleep(10);
        }
        results.Sort(string.CompareOrdinal);
        foreach (string result in results)
        {
            Console.WriteLine(result);
        }
        Console.WriteLine("Tests: " + results.Count + ", passed: " + (results.Count - failed) + ", failed: " + failed);
    }
}

Try it

VAT with double

The price with VAT is the price times 1.2. Two tests:

What will the tests show?

class Vat
{
    public static double WithVat(double price)
    {
        return price * 1.2;
    }
}


using Xunit;


public class VatTest
{
    [Fact]
    public void Coffee()
    {
        Assert.Equal(12.0, Vat.WithVat(10));
    }


    [Fact]
    public void Tea()
    {
        Assert.Equal(3.6, Vat.WithVat(3));
    }
}
Answer

The idea

Yes or no, and an expected exception

Besides equality, you often check a condition, or that the code refuses. Here is Account from the module on exceptions, with a new method, CanPay:

Account.cs
class InsufficientFundsException : Exception
{
    public InsufficientFundsException(int shortfall) : base("Short by €" + shortfall)
    {
    }
}


class Account
{
    public int Balance { get; private set; }


    public Account(int balance)
    {
        Balance = balance;
    }


    public bool CanPay(int amount)
    {
        return amount <= Balance;
    }


    public void Withdraw(int amount)
    {
        if (amount <= 0)
        {
            throw new ArgumentException("The amount must be positive");
        }
        if (amount > Balance)
        {
            throw new InsufficientFundsException(amount - Balance);
        }
        Balance -= amount;
    }
}
AccountTest.cs
using Xunit;


public class AccountTest
{
    [Fact]
    public void CanPayWhatIsThere()
    {
        var account = new Account(100);
        Assert.True(account.CanPay(100));
        Assert.False(account.CanPay(101));
    }


    [Fact]
    public void WithdrawTooMuch()
    {
        var account = new Account(100);
        var e = Assert.Throws<InsufficientFundsException>(() => account.Withdraw(130));
        Assert.Equal("Short by €30", e.Message);
        Assert.Equal(100, account.Balance);
    }
}
Output
PASS  CanPayWhatIsThere
PASS  WithdrawTooMuch
Tests: 2, passed: 2, failed: 0
  • Assert.True and Assert.False check a condition.
  • Assert.Throws<Type>(code) fails if the code does not throw an exception of that type. The type goes in < >, as in List<string>. You pass the code as a lambda: Assert.Throws runs it itself and catches the exception. Then it returns it, so you can check the message too (e.Message).

The second test also checks something that is easy to forget: after the refusal the balance is the same.

Assert.Throws<T> wants an exception of exactly the type T, not a subclass of it. InsufficientFundsException inherits from Exception, but Assert.Throws<Exception> fails. For "this type or a subclass of it" there is Assert.ThrowsAny<T>:

AccountTest.cs
public class AccountTest
{
    [Fact]
    public void ExactType()
    {
        var account = new Account(100);
        Assert.Throws<Exception>(() => account.Withdraw(130));
    }


    [Fact]
    public void TypeOrDerived()
    {
        var account = new Account(100);
        Assert.ThrowsAny<Exception>(() => account.Withdraw(130));
    }
}
Output
FAIL  ExactType
      Assert.Throws() Failure: Exception type was not an exact match
      Expected: typeof(System.Exception)
      Actual:   typeof(InsufficientFundsException)
PASS  TypeOrDerived
Tests: 2, passed: 1, failed: 1

The idea

Every test starts clean

Here the two tests share one account with €100:

AccountTest.cs
public class AccountTest
{
    private static Account account = new Account(100);


    [Fact(DisplayName = "withdrawing €40 leaves €60")]
    public void Withdraw40()
    {
        account.Withdraw(40);
        Assert.Equal(60, account.Balance);
    }


    [Fact(DisplayName = "withdrawing €70 leaves €30")]
    public void Withdraw70()
    {
        account.Withdraw(70);
        Assert.Equal(30, account.Balance);
    }
}
Output
FAIL  withdrawing €70 leaves €30
      Short by €10
PASS  withdrawing €40 leaves €60
Tests: 2, passed: 1, failed: 1

account is a static field: one account for all the tests. xUnit does not promise the order of the tests, and here it ran the test for €40 first. The second one has €60 left, and it withdraws €70. If the order were the other way around, the other one would fail. A test that depends on another test is a trap. xUnit has no separate attribute for setup: for each test it makes a new object of the test class. So the account is a plain field, and it is created in the constructor:

AccountTest.cs
using Xunit;


public class AccountTest
{
    private readonly Account account;


    public AccountTest()
    {
        account = new Account(100);
    }


    [Fact(DisplayName = "withdrawing €40 leaves €60")]
    public void Withdraw40()
    {
        account.Withdraw(40);
        Assert.Equal(60, account.Balance);
    }


    [Fact(DisplayName = "withdrawing €70 leaves €30")]
    public void Withdraw70()
    {
        account.Withdraw(70);
        Assert.Equal(30, account.Balance);
    }
}
Output
PASS  withdrawing €40 leaves €60
PASS  withdrawing €70 leaves €30
Tests: 2, passed: 2, failed: 0
The constructor is called before each test, because each test gets a new object. DisplayName in [Fact] gives the test a name in words. It is shown in the result instead of the method name.
More precisely, if you are curious
More things from xUnit you will come across:
xUnit
constructor                     before each test (a new object for each test)
Dispose() from IDisposable      after each test (cleanup)
IClassFixture<T>                one shared T object for all the tests in the class
[Fact(Skip = "...")]            the test is skipped for now
[Trait("Category", "fast")]     a tag, so you can run only some of the tests
[MemberData(nameof(Prices))]    the rows come from a static property or method
[ClassData(typeof(PriceRows))]  the rows come from a separate class
[Collection("Database")]        classes in one collection do not run at the same time
                                (xUnit runs different test classes in parallel)
ITestOutputHelper               printing from a test that shows up in the results

The idea

One check, many cases

Four cases for the discount mean four almost identical methods. [Theory] is a test with parameters, and each [InlineData] is one row of "price, percent, expected" that xUnit runs the method with:

DiscountTest.cs
using Xunit;


public class DiscountTest
{
    [Theory]
    [InlineData(50, 10, 45.0)]
    [InlineData(99, 0, 99.0)]
    [InlineData(100, 10, 85.0)]
    [InlineData(200, 20, 150.0)]
    public void Apply(double price, int percent, double expected)
    {
        Assert.Equal(expected, Discount.Apply(price, percent));
    }
}
Output
PASS  Apply(price: 100, percent: 10, expected: 85)
PASS  Apply(price: 200, percent: 20, expected: 150)
PASS  Apply(price: 50, percent: 10, expected: 45)
PASS  Apply(price: 99, percent: 0, expected: 99)
Tests: 4, passed: 4, failed: 0

Each row is a separate test with a separate result. xUnit builds its name from the method and the values of the parameters. A new case is a new row, not a new method. The important cases are at the edges: exactly €100, €99, 0%.

Try it

Find the bug from the test

Delivery is €4, and free from €50 up. The test:

DeliveryTest.cs
public class DeliveryTest
{
    [Theory]
    [InlineData(20, 4)]
    [InlineData(50, 0)]
    [InlineData(80, 0)]
    public void Price(int total, int expected)
    {
        Assert.Equal(expected, Delivery.Price(total));
    }
}
Output
FAIL  Price(total: 50, expected: 0)
      Assert.Equal() Failure: Values differ
      Expected: 0
      Actual:   4
PASS  Price(total: 20, expected: 4)
PASS  Price(total: 80, expected: 0)
Tests: 3, passed: 2, failed: 1

Which line of the code is the bug on?

Pick the line with the bug.

The idea

The test first

The checkout has to accept the promo code FALL10 for 10% off, whether it is in lowercase or uppercase. This time you write the tests first:

PromoTest.cs
public class PromoTest
{
    [Fact]
    public void KnownCode()
    {
        Assert.Equal(10, Promo.Percent("FALL10"));
    }


    [Fact]
    public void CodeIsCaseInsensitive()
    {
        Assert.Equal(10, Promo.Percent("fall10"));
    }


    [Fact]
    public void UnknownCode()
    {
        Assert.Equal(0, Promo.Percent("NOPE"));
    }
}
  1. Red. For now the method returns 0. The tests fail, and that is good: it means they really check something.
class Promo
{
    public static int Percent(string code)
    {
        return 0;
    }
}
Output
FAIL  CodeIsCaseInsensitive
      Assert.Equal() Failure: Values differ
      Expected: 10
      Actual:   0
FAIL  KnownCode
      Assert.Equal() Failure: Values differ
      Expected: 10
      Actual:   0
PASS  UnknownCode
Tests: 3, passed: 1, failed: 2
  1. Green. You write the simplest code that makes them pass:
class Promo
{
    public static int Percent(string code)
    {
        if (code.ToUpper() == "FALL10")
        {
            return 10;
        }
        return 0;
    }
}
Output
PASS  CodeIsCaseInsensitive
PASS  KnownCode
PASS  UnknownCode
Tests: 3, passed: 3, failed: 0
  1. Clean up. Now you can move the codes into a Dictionary, add new ones and clean up the method. The tests will tell you if you break something.

This is TDD (test-driven development). Writing the test first makes you decide what exactly the method has to do, before you think about how.

More precisely

Groups of tests and fake objects

xUnit has no suites. A group of tests is a tag ([Trait]) on the classes, and running the group is a filter. The delivery tests, tagged "Checkout":

DeliveryTest.cs
[Trait("Category", "Checkout")]
public class DeliveryTest
{
    [Theory]
    [InlineData(20, 4)]
    [InlineData(50, 0)]
    [InlineData(80, 0)]
    public void Price(int total, int expected)
    {
        Assert.Equal(expected, Delivery.Price(total));
    }
}

DiscountTest and AccountTest have the same tag. The command runs only the tests with the tag. The output is from the site's runner, which filters by the same tag (TestCaseFilter). VatTest from the VAT card is in the same program, but it has no tag and does not run:

dotnet test --filter "Category=Checkout"
Output
PASS  ExtraFivePercentFrom100
PASS  Price(total: 20, expected: 4)
PASS  Price(total: 50, expected: 0)
PASS  Price(total: 80, expected: 0)
PASS  TenPercentOff
PASS  withdrawing €40 leaves €60
PASS  withdrawing €70 leaves €30
Tests: 7, passed: 7, failed: 0
More precisely, if you are curious
  • Sometimes the code depends on something you do not want to call in a test: an exchange rate from a bank, a payment, an email. Then you pass in a fake object (a mock) that returns whatever you tell it to. The best known library for this in .NET is Moq:
PriceInUsdTest.cs
using Moq;
using Xunit;


public interface IExchangeRates
{
    double EurTo(string currency);
}


class Converter
{
    private readonly IExchangeRates rates;


    public Converter(IExchangeRates rates)
    {
        this.rates = rates;
    }


    public double ToUsd(double eur)
    {
        return eur * rates.EurTo("USD");
    }
}


public class PriceInUsdTest
{
    [Fact]
    public void ConvertsWithTodaysRate()
    {
        var rates = new Mock<IExchangeRates>();
        rates.Setup(r => r.EurTo("USD")).Returns(1.10);


        var converter = new Converter(rates.Object);
        Assert.Equal(11.0, converter.ToUsd(10), 0.001);
        rates.Verify(r => r.EurTo("USD"), Times.Once());
    }
}
Output
PASS  ConvertsWithTodaysRate
Tests: 1, passed: 1, failed: 0

new Mock<IExchangeRates>() makes an object that implements the interface, Setup(...).Returns(...) says what to return, and Verify checks that the method was called exactly once (Times.Once()). rates.Object is the fake object itself, which you pass to Converter. The interface is public, because Moq makes a class that implements it on its own. Here on the site the test really runs.

  • More checks besides Assert.Equal, Assert.True and Assert.Throws:
Checks
Assert.NotEqual(a, b)              are they different
Assert.Same(a, b)                  is it the same object, not just equal
Assert.NotSame(a, b)
Assert.Null(x), Assert.NotNull(x)
Assert.Equal(list1, list2)         two lists (or arrays) with the same elements in the same order
Assert.Contains(x, items), Assert.DoesNotContain(x, items)
Assert.Empty(items), Assert.Single(items)
Assert.InRange(x, low, high)       x is between low and high
Assert.Matches(pattern, text)      the text matches a regular expression
Assert.IsType<T>(x)                the object is exactly of type T
Assert.ThrowsAny<T>(() => ...)     an exception of type T or a subclass of it
Assert.Fail("message")             the test fails right away

Exercise

Write the tests

Solve these in your editor, with xUnit in a test project (dotnet new xunit).

1. The discount calculator. Write DiscountTest for Discount.Apply from the first card. Cover these cases: €80 with 10% off is €72, €100 with 0% off is €95, €150 with 20% off is €112.50, €0 with 30% off is €0, and exactly €100 with 10% off is €85. Run them against the broken version with price > 100 too. Exactly the two cases with €100 should fail:

Result against the broken version
FAIL  Apply(price: 100, percent: 0, expected: 95)
      Assert.Equal() Failure: Values are not within tolerance 0.001
      Expected: 95
      Actual:   100
FAIL  Exactly100GetsTheExtra
      Assert.Equal() Failure: Values are not within tolerance 0.001
      Expected: 85
      Actual:   90
PASS  Apply(price: 0, percent: 30, expected: 0)
PASS  Apply(price: 150, percent: 20, expected: 112.5)
PASS  Apply(price: 80, percent: 10, expected: 72)
Tests: 5, passed: 3, failed: 2

2. The account. Write AccountTest for Account from the card "Yes or no, and an expected exception". Create the account with €100 in AccountTest(). Check that:

  • withdrawing €30 leaves €70;
  • you can withdraw everything, and 0 is left;
  • withdrawing €150 throws InsufficientFundsException with the message "Short by €50", and the balance stays €100;
  • 0 and -20 throw ArgumentException (one [Theory] with [InlineData(0)] and [InlineData(-20)]).

3. The bug in the average grade. The method looks fine:

Grades.cs
class Grades
{
    public static double Average(int[] grades)
    {
        int sum = 0;
        foreach (int g in grades)
        {
            sum += g;
        }
        return sum / grades.Length;
    }
}
Write two tests: the average of 4 and 6 is 5.0, and the average of 5 and 6 is 5.5 (with an allowed difference of 0.001). One of them fails:
Result
FAIL  AverageWithAFraction
      Assert.Equal() Failure: Values are not within tolerance 0.001
      Expected: 5.5
      Actual:   5
PASS  WholeAverage
Tests: 2, passed: 1, failed: 1

Find the cause and fix the method, so that both pass.

Test

Check yourself

1. What does a unit test check?
2. The test fails with "Expected: 85" and "Actual: 90". What did the method return?
3. Why is the account created in the constructor of the test class and not once in a static field?
4. How do you check that Withdraw(130) throws InsufficientFundsException?
5. What is the order in TDD?

Made by Ivo Gergov as part of Evo Tech's campaign for accessible programming education ·