LINQ

In short: LINQ processes a collection as a series of steps: Where sifts, Select transforms, OrderBy orders, and Sum and Average do the math.

Problem

A report in a loop

Orders now have a city too:

Order

class Order
{
    public string Customer { get; }
    public string City { get; }
    public decimal Total { get; }

    public Order(string customer, string city, decimal total)
    {
        Customer = customer;
        City = city;
        Total = total;
    }
}

List<Order> orders = new List<Order>
{
    new Order("Maria", "Sofia", 75.50m),
    new Order("Georgi", "Varna", 12.90m),
    new Order("Elena", "Sofia", 40.00m),
    new Order("Dimitar", "Plovdiv", 120.00m),
    new Order("Ana", "Varna", 64.00m),
};

You want the customers with orders of at least €50, in alphabetical order:

List<string> result = new List<string>();
foreach (Order o in orders)
{
    if (o.Total >= 50)
    {
        result.Add(o.Customer);
    }
}
result.Sort();
Console.WriteLine(string.Join(", ", result));

Output

Ana, Dimitar, Maria

Ten lines, and the question is just one. The code says how: make an empty list, loop, check, add, sort. To understand what it does, you have to run it in your head. Add "only the first three" or "no duplicates", and the loop will grow.

The idea

LINQ: a series of steps

The same report, written as steps:

List<string> result = orders
    .Where(o => o.Total >= 50)
    .Select(o => o.Customer)
    .Order()
    .ToList();
Console.WriteLine(string.Join(", ", result));

Output

Ana, Dimitar, Maria

You read it from top to bottom: take the orders, keep the ones of at least €50, take the customer, sort, collect into a list.

These methods are LINQ (Language Integrated Query). They live in System.Linq and work on anything you can go through with foreach: a list, an array, a dictionary. What they return is a query (IEnumerable<T>). A query does not store data. It is a conveyor belt that the elements go along one at a time. Each step takes a lambda from the previous lesson:

• Where(condition) lets through only the elements for which the condition is true. The lambda is a Func<Order, bool>.
• Select(function) turns each element into another one: it makes a string from an Order. That is a Func<Order, string>.
• Order() sorts by the natural order, and OrderBy(o => o.Total) by a key you choose.
• ToList() collects whatever got to the end into a new List<string>.

The list orders does not change. The query only reads it.

More precisely, if you are curious

• Where, Select, Order, OrderBy and Take return a new query and do nothing yet. ToList, Count, Sum, First and foreach ask for a result and start the conveyor belt.
• ToList() returns an ordinary List<T> that you can change. ToArray() gives an array.
• Order() is from .NET 7. In older code you will see OrderBy(name => name): it does the same.
• There is also a second way to write it, which looks like SQL. The compiler turns it into the same calls. This course writes the methods:

List<string> result = (from o in orders
                       where o.Total >= 50
                       orderby o.Customer
                       select o.Customer).ToList();
Console.WriteLine(string.Join(", ", result));

Output

Ana, Dimitar, Maria

See it

What gets through the filter

List<int> prices = new List<int> { 12, 75, 40, 130, 58 };
List<string> labels = prices
    .Where(p => p >= 50)
    .Select(p => "€" + p)
    .ToList();
Console.WriteLine(string.Join(", ", labels));

Step by step, in words

1. Five prices. The query only sets up the path: nothing has gone through yet.
2. 12 does not pass p >= 50 and drops out. It never gets to Select.
3. 75 passes Where, Select turns it into the text "€75", ToList() collects it.
4. 40 drops out too. Each element goes the whole way before the next one starts.
5. 130 and 58 pass. The list prices does not change.
6. The result is a new list with three strings.

The five prices go along the conveyor belt one at a time. 12 gets to Where and drops out. 75 passes, Select turns it into the text "€75", and it goes into the list. Then 40 drops out, and 130 and 58 pass.

Each element goes the whole way before the next one starts. And nothing starts until ToList() asks for a result.

More precisely, if you are curious

This is called lazy evaluation. The query methods only record what needs to be done. See it with a print in every step:

IEnumerable<int> query = new List<int> { 12, 75, 40 }
    .Where(p =>
    {
        Console.WriteLine("Where " + p);
        return p >= 50;
    })
    .Select(p =>
    {
        Console.WriteLine("Select " + p);
        return p * 2;
    });
Console.WriteLine("The query is ready, nothing has gone through it yet.");
Console.WriteLine(string.Join(", ", query.ToList()));

Output

The query is ready, nothing has gone through it yet.
Where 12
Where 75
Select 75
Where 40
150

• The first line of the output is printed before a single element has gone through the query. The C# documentation calls this deferred execution.
• Select 75 comes right after Where 75: the elements do not wait for all of them to pass Where.
• That is why Take(2) stops the query as soon as two elements get to the end. The rest are not even checked:

IEnumerable<int> firstTwo = new List<int> { 75, 130, 58, 12, 99 }
    .Where(p =>
    {
        Console.WriteLine("Where " + p);
        return p >= 50;
    })
    .Take(2);
Console.WriteLine(string.Join(", ", firstTwo));

Output

Where 75
Where 130
75, 130

• A query does not get used up. Every time you go through it, it runs from the start: the two Count() calls both go over the four prices, so there are 8 checks. If you need the result several times, collect it once with ToList().

int checks = 0;
IEnumerable<int> expensive = new List<int> { 12, 75, 40, 130 }.Where(p =>
{
    checks++;
    return p >= 50;
});
Console.WriteLine(expensive.Count());
Console.WriteLine(expensive.Count());
Console.WriteLine("Checks: " + checks);

Output

2
2
Checks: 8

• AsParallel() sends the elements through several threads at once (PLINQ). It helps with lots of data and heavy calculations. With small lists it is slower, and the lambdas must not touch shared data.

The idea

Sum, count, average

The result can be a number instead of a list. The sum of the even numbers:

List<int> numbers = new List<int> { 1, 2, 3, 4, 5 };
int sum = numbers.Where(n => n % 2 == 0).Sum();
Console.WriteLine("Sum of the even numbers: " + sum);

Output

Sum of the even numbers: 6

Sum() adds up the numbers that got to the end. With a selector, Sum(o => o.Total), it first takes the number from each element.

The same for the orders:

int big = orders.Count(o => o.Total >= 50);
decimal revenue = orders.Sum(o => o.Total);
decimal average = orders.Average(o => o.Total);
decimal maxTotal = orders.Max(o => o.Total);

Console.WriteLine("Orders €50 or more: " + big);
Console.WriteLine($"Revenue: €{revenue:F2}");
Console.WriteLine($"Average: €{average:F2}");
Console.WriteLine($"Largest: €{maxTotal:F2}");

Output

Orders €50 or more: 3
Revenue: €312.40
Average: €62.48
Largest: €120.00

• Count(condition) counts the elements for which the condition is true. The type is int.
• Average and Max cannot give an answer for an empty list: there is no average of nothing. Then they throw an InvalidOperationException. Sum of an empty list is 0. What to do with a result that may be missing is the next idea.

More precisely, if you are curious

With an empty list. DefaultIfEmpty(0) gives a fallback: if there are no elements, the query gives one element, 0.

List<decimal> prices = new List<decimal>();
Console.WriteLine(prices.Sum());
Console.WriteLine(prices.DefaultIfEmpty(0).Max());
try
{
    Console.WriteLine(prices.Max());
}
catch (InvalidOperationException e)
{
    Console.WriteLine("InvalidOperationException: " + e.Message);
}

Output

0
0
InvalidOperationException: Sequence contains no elements

The idea

Sort, cut, no duplicates

List<int> ratings = new List<int> { 5, 3, 5, 4, 2, 4, 5 };
Console.WriteLine(string.Join(", ", ratings.Distinct().Order()));

List<string> top = orders
    .OrderByDescending(o => o.Total)
    .Take(3)
    .Select(o => o.Customer)
    .ToList();
Console.WriteLine(string.Join(", ", top));

foreach (int i in Enumerable.Range(1, 3))
{
    Console.WriteLine("Table " + i);
}

Output

2, 3, 4, 5
Dimitar, Maria, Ana
Table 1
Table 2
Table 3

• Distinct() removes the duplicates. It compares with Equals and GetHashCode.
• OrderByDescending(o => o.Total) sorts from the largest order.
• Take(3) lets through only the first three. Together with the sorting, that is a "top 3".
• Enumerable.Range(1, 3) gives the numbers 1, 2, 3. The second parameter is how many numbers, not where to stop.
• Only List<T> has ForEach. You go through a query with an ordinary foreach.

Try it

What gets to the end?

What will the program print?

List<string> drinks = new List<string> { "coffee", "tea", "coffee", "water", "tea", "juice" };
List<string> result = drinks
    .Distinct()
    .Order()
    .Take(3)
    .Select(d => d.ToUpper())
    .ToList();
Console.WriteLine(string.Join(", ", result));

Answer

COFFEE, COFFEE, JUICE

COFFEE, TEA, COFFEE

COFFEE, TEA, WATER

COFFEE, JUICE, TEA

Check

The idea

null: a result that may be missing

Which order is the largest? If the list is empty, there is no answer. Then the result is null: "no object". But the caller has to remember to check. If they forget, the program stops with a NullReferenceException somewhere later.

Java has Optional for this: a box that either holds a value or is empty. C# has no such box. The LINQ methods that look for one element return null if there is none, and the check is your job:

Order biggest = orders.MaxBy(o => o.Total);
if (biggest != null)
{
    Console.WriteLine("Largest: " + biggest.Customer);
}

Order huge = orders.FirstOrDefault(o => o.Total > 500);
Console.WriteLine(huge != null);
Console.WriteLine(huge?.Customer ?? "no such order");

Output

Largest: Dimitar
False
no such order

• MaxBy(o => o.Total) gives the order with the largest total, not the total itself. For an empty list of objects it gives null.
• FirstOrDefault(condition) gives the first element for which the condition is true, or null if there is none.
• huge?.Customer: if huge is null, the whole expression is null instead of the program stopping.
• x ?? y gives y if x is null, and x otherwise.

Your own methods also return null when they have no answer:

static string Nickname(string user)
{
    // Returns the nickname, or null if the person has none.
    return user == "maria" ? "Mimi" : null;
}

string maria = Nickname("maria");
if (maria != null)
{
    Console.WriteLine("Nickname: " + maria);
}

string georgi = Nickname("georgi");
Console.WriteLine(georgi ?? "no nickname");

Output

Nickname: Mimi
no nickname

More precisely, if you are curious

• If you forget the check:

Order huge = orders.FirstOrDefault(o => o.Total > 500);
Console.WriteLine(huge.Customer);

Error

Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at Program.Main()

• If an empty result is a mistake, write First, not FirstOrDefault. The name says honestly what will happen:

try
{
    Order huge = orders.First(o => o.Total > 500);
    Console.WriteLine(huge.Customer);
}
catch (InvalidOperationException e)
{
    Console.WriteLine("InvalidOperationException: " + e.Message);
}

Output

InvalidOperationException: Sequence contains no matching element

• int, decimal and the other value types have no null. Then FirstOrDefault gives 0, unless you give it another fallback:

List<int> ratings = new List<int> { 5, 3, 4 };
Console.WriteLine(ratings.FirstOrDefault(r => r > 5));
Console.WriteLine(ratings.FirstOrDefault(r => r > 5, -1));

Output

0
-1

• In a new project (dotnet new console) the null checks are turned on (<Nullable>enable</Nullable>). There, a type that may be null is written with ?: Order? huge. If you store the result of FirstOrDefault in an Order without ?, the compiler warns you.
• Any and All answer True or False: does any element, or every element, match the condition?

Console.WriteLine(orders.Any(o => o.Total > 100));
Console.WriteLine(orders.All(o => o.Total >= 10));

Output

True
True

Try it

Find the bug

The program counts the expensive prices and then prints them. Between the two, a new price, 99, goes into the list, but the report has to be about the prices at the moment they were counted. The count is 2, but three prices come out. Which line is the bug on?

Pick the line with the bug.

1List<int> prices = new List<int> { 12, 75, 40, 130 };2IEnumerable<int> expensive = prices.Where(p => p >= 50);3Console.WriteLine("Count: " + expensive.Count());4prices.Add(99);5Console.WriteLine("Prices: " + string.Join(", ", expensive));

Check

The idea

Grouping and joining

How many orders are there from each city, and what is the revenue?

var byCity = orders
    .GroupBy(o => o.City)
    .OrderBy(g => g.Key);
foreach (var group in byCity)
{
    Console.WriteLine($"{group.Key}: count {group.Count()}, revenue €{group.Sum(o => o.Total):F2}");
}

string customers = string.Join(", ", orders.Select(o => o.Customer).Order());
Console.WriteLine(customers);

Output

Plovdiv: count 1, revenue €120.00
Sofia: count 2, revenue €115.50
Varna: count 2, revenue €76.90
Ana, Dimitar, Elena, Georgi, Maria

GroupBy(o => o.City) splits the orders into groups by city. Each group has a Key (the city), and you go through it like a list of its orders, so Count() and Sum(...) work on it.

• OrderBy(g => g.Key) sorts the groups by city. Without it, the groups come out in the order the cities first appear: Sofia, Varna, Plovdiv.
• string.Join(", ", ...) joins the strings into one, with a comma between them.

The type of the groups is long (IOrderedEnumerable<IGrouping<string, Order>>), so it is var: the compiler works it out by itself.

More precisely, if you are curious

• ToDictionary makes a dictionary from the groups: for each city, a list of the names. The order of the keys in a Dictionary is not guaranteed, so the cities are sorted explicitly:

Dictionary<string, List<string>> namesByCity = orders
    .GroupBy(o => o.City)
    .ToDictionary(g => g.Key, g => g.Select(o => o.Customer).ToList());
foreach (string city in namesByCity.Keys.Order())
{
    Console.WriteLine(city + ": " + string.Join(", ", namesByCity[city]));
}

Output

Plovdiv: Dimitar
Sofia: Maria, Elena
Varna: Georgi, Ana

• ToDictionary(key, value) makes a dictionary with one value per key, and throws an exception if a key repeats:

try
{
    Dictionary<string, string> customerByCity = orders.ToDictionary(o => o.City, o => o.Customer);
}
catch (ArgumentException e)
{
    Console.WriteLine("ArgumentException: " + e.Message);
}

Output

ArgumentException: An item with the same key has already been added. Key: Sofia

Exercise

Reports with LINQ

Solve the tasks in your editor. In a new project, System.Linq is already included.

1. Sales by category. A class Sale with a product, a category, a quantity and a unit price (decimal), and a property Revenue (quantity times price). Group by category with GroupBy, add up each group with Sum, print the categories alphabetically and the total revenue. The sales: coffee beans, drinks, 12 at €18.50; tea, drinks, 30 at €4.20; chocolate, sweets, 40 at €2.80; biscuits, sweets, 25 at €3.10; honey, food, 8 at €12; coffee beans again, 5 at €18.50.

Output

Drinks: €440.50
Food: €96.00
Sweets: €189.50
Total: €726.00

2. Top 3 products. With the same sales, find the three products with the highest revenue. The coffee is in two sales, so first group by product and add up, then sort the groups by their sum. Hint: GroupBy(s => s.Product), then OrderByDescending(g => g.Sum(s => s.Revenue)) and Take(3): each group has a Key (the product) and its sales.

Output

Coffee beans: €314.50
Tea: €126.00
Chocolate: €112.00

3. Average grade. Read one line of grades separated by spaces. Print the average, the highest and how many are excellent (5.50 and up). Hint: Console.ReadLine().Split(' ').Select(double.Parse).

Input

4.50 5.75 6.00 3.00 5.50

Output

Average: 4.95
Highest: 6.00
Excellent: 3

4. The words in a text. Read a line of text and print the words in it with no duplicates, in alphabetical order, in lowercase, separated by commas. Turn every character that is not a letter into a space: text.Select(c => char.IsLetter(c) ? c : ' '), and new string(...) from those characters. Then split with Split(' ', StringSplitOptions.RemoveEmptyEntries): the empty parts are removed.

Input

Coffee with milk, coffee without sugar. Tea with honey!

Output

coffee, honey, milk, sugar, tea, with, without

More tasks

• The longest word. With MaxBy(w => w.Length), find the longest word in a sentence. What will the program return for an empty line?
• Average by class. Students with a name, a class and a grade: the average grade for each class with GroupBy and Average.
• Big and small orders. With Any, check whether there is an order €100 or more0, and with All, whether all of them are at least €10.

Test

Check yourself

1. What happens to the list orders after orders.Where(o => o.Total > 50).ToList()?

It is sorted by totalThe small orders are removed from itNothing: the query reads the list and makes a new result

2. Which of these starts the query?

SelectWhereCount

3. orders.FirstOrDefault(o => o.Total > 500) does not find an order. What does it return?

An empty listnullIt throws an InvalidOperationException

4. Why does the report by city have OrderBy(g => g.Key) after GroupBy?

GroupBy gives the groups in the order the cities first appear, and the report has to be alphabeticalOrderBy is fasterGroupBy does not work without OrderBy

5. IEnumerable<int> q = prices.Where(p => p >= 50); Console.WriteLine(q.Count()); prices.Add(99); Console.WriteLine(q.Count()); What happens?

InvalidOperationException: the query has already been used upThe same count twiceThe second count is one higher: the query runs again and sees 99

Check

Start the lesson over

Related lessons

• Builds on: Lists · Dictionary and HashSet · Nested classes · Lambdas and delegates
• Continues in: record, switch and patterns · Tests with xUnit · Databases · Reflection and attributes
• Tasks on this topic: Modern C#