Skip to content
wannacode

Data in files

Problem

Text, but you need numbers

The warehouse exports its products to a file. Each row is one product: name, price and stock, separated with commas. The first row says what is in each column:

products.csv
Name,Price,Stock
Bread,1.20,30
Milk,2.10,18
Coffee beans,18.90,12

You can already read the file. But to the program, the row is just text:

String line = "Bread,1.20,30";
System.out.println(line.length());
System.out.println(line.charAt(6));
Output
13
1

It does not know that 1.20 is a price and 30 is a count. To work out what the goods are worth, you have to turn each row into something with fields: a Product object.

The idea

CSV: row to object

A file like this is called CSV (comma-separated values). Excel reads it, databases read it, any program reads it. A row becomes an object in three steps: split, turning each part into the type you need, and the constructor.

It is most convenient to make this a static method in the class itself:
class Product {
    String name;
    double price;
    int stock;


    Product(String name, double price, int stock) {
        this.name = name;
        this.price = price;
        this.stock = stock;
    }


    static Product fromCsv(String line) {
        String[] f = line.split(",");
        if (f.length != 3) {
            throw new IllegalArgumentException("expected 3 columns, got " + f.length);
        }
        return new Product(f[0].strip(), Double.parseDouble(f[1].strip()), Integer.parseInt(f[2].strip()));
    }


    String toCsv() {
        return name + "," + price + "," + stock;
    }
}
Product p = Product.fromCsv("Coffee beans,18.90,12");
System.out.printf("%s: €%.2f x %d = €%.2f%n", p.name, p.price, p.stock, p.price * p.stock);
Output
Coffee beans: €18.90 x 12 = €226.80

fromCsv is the only place that knows what the row looks like. If the format changes, you change only that. The check for the number of columns throws IllegalArgumentException with a clear message, instead of the program stopping later with an index out of the array.

What fromCsv does with the row, step by step, with its body spelled out in main:

String line = "Coffee beans,18.90,12";
String[] f = line.split(",");
String name = f[0];
double price = Double.parseDouble(f[1]);
int stock = Integer.parseInt(f[2]);
Product p = new Product(name, price, stock);
System.out.println(p.name + ": €" + p.price + " x " + p.stock);
System.out.printf("€%.2f%n", p.price * p.stock);
Step by step, in words
  1. The row from the file is one string. To the program, the 18.90 in it is just four characters.
  2. split cuts the row at every comma: an array of three strings.
  3. The name stays a string.
  4. parseDouble turns the string "18.90" into the number 18.9. Now you can do math with it.
  5. parseInt makes the whole number 12 from "12".
  6. The constructor puts the three values in the fields of a new Product, and p points to it.
  7. p.price prints as 18.9: the zero at the end was only in the text.
  8. The price times the stock: now it is math with numbers.

See it

The whole file

You load all the products into an array and work out the value of the goods:
Path file = Path.of("products.csv");
try {
    Files.writeString(file, "Name,Price,Stock\nBread,1.20,30\nMilk,2.10,18\nCoffee beans,18.90,12\n");
    List<String> lines = Files.readAllLines(file);
    Product[] products = new Product[lines.size() - 1];
    int count = 0;
    boolean header = true;
    for (String line : lines) {
        if (header) {
            header = false;
            continue;
        }
        products[count] = Product.fromCsv(line);
        count++;
    }
    double value = 0;
    for (int i = 0; i < count; i++) {
        value += products[i].price * products[i].stock;
    }
    System.out.printf("%d products, stock worth €%.2f%n", count, value);
} catch (IOException e) {
    System.out.println("Cannot read " + file + ": " + e.getMessage());
}
Output
3 products, stock worth €300.60
  • The array has one element fewer than the rows, because the first row is the header.
  • header is true only on the first pass. Then the row is skipped with continue.
  • count counts how many products are in the array so far, and it is the index for the next one.

In the module "Collections" you will see a list that grows by itself, and you will not need to count with count.

The idea

Bad rows

People and other programs write the files. Sooner or later a row comes with a missing column, a word instead of a number, or an empty row:

products.csv
Name,Price,Stock
Bread,1.20,30
Milk,2.10
Coffee,seven,4


Tea,3.50,25

One error should not stop the loading of everything else. Each row is in its own try, and the message gives the row number, so that someone can fix it:

Path file = Path.of("products.csv");
try {
    Files.writeString(file, "Name,Price,Stock\nBread,1.20,30\nMilk,2.10\nCoffee,seven,4\n\nTea,3.50,25\n");
    List<String> lines = Files.readAllLines(file);
    Product[] products = new Product[lines.size()];
    int count = 0;
    int row = 0;
    for (String line : lines) {
        row++;
        if (row == 1 || line.isBlank()) {
            continue;
        }
        try {
            products[count] = Product.fromCsv(line);
            count++;
        } catch (NumberFormatException e) {
            System.out.println("Row " + row + ": bad number (" + e.getMessage() + ")");
        } catch (IllegalArgumentException e) {
            System.out.println("Row " + row + ": " + e.getMessage());
        }
    }
    System.out.println("Loaded: " + count);
} catch (IOException e) {
    System.out.println("Cannot read " + file + ": " + e.getMessage());
}
Output
Row 3: expected 3 columns, got 2
Row 4: bad number (For input string: "seven")
Loaded: 2
  • Empty rows are skipped with isBlank().
  • NumberFormatException is a subclass of IllegalArgumentException, so it comes first. Otherwise the general catch would catch it too.
  • The program says "Loaded: 2" and carries on with them.

Try it

Empty columns

In the first row the price is empty, in the second the stock:

What will the program print?

System.out.println("Coffee,,4".split(",").length);
System.out.println("Coffee,7.50,".split(",").length);
System.out.println("Coffee,7.50,".split(",", -1).length);
Answer

The idea

Objects to CSV

The other direction is a method that builds the row:

String toCsv() {
    return name + "," + price + "," + stock;
}

You write the header row, and then one row for each object:

Product[] stock = { new Product("Bread", 1.2, 30), new Product("Milk", 2.1, 18) };
Path file = Path.of("stock.csv");
try {
    try (BufferedWriter out = Files.newBufferedWriter(file)) {
        out.write("Name,Price,Stock");
        out.newLine();
        for (Product p : stock) {
            out.write(p.toCsv());
            out.newLine();
        }
    }
    System.out.print(Files.readString(file));
} catch (IOException e) {
    System.out.println("Cannot write " + file + ": " + e.getMessage());
}
Output
Name,Price,Stock
Bread,1.2,30
Milk,2.1,18

fromCsv can read this file back. Saving and reading have to use the same format.

Watch out if you save the prices with String.format. It follows the computer's language settings, and in German, Bulgarian and many other languages the decimal mark is a comma:

String de = String.format(Locale.of("de"), "%s,%.2f,%d", "Bread", 1.2, 30);
System.out.println(de + "  -> " + de.split(",").length + " columns");
String root = String.format(Locale.ROOT, "%s,%.2f,%d", "Bread", 1.2, 30);
System.out.println(root + "  -> " + root.split(",").length + " columns");
Output
Bread,1,20,30  -> 4 columns
Bread,1.20,30  -> 3 columns

1,20 becomes two columns, and the rows shift. For files that another program will read, pass Locale.ROOT (java.util.Locale). It always gives a point, on every computer.

Try it

Find the bug

The program should load the three products from products.csv, but it stops on the very first row:

Error
Exception in thread "main" java.lang.NumberFormatException: For input string: "Price"

Pick the line with the bug.

The idea

Columns by name

Another program exports the same data, but with the columns in a different order and with one more. If you write f[1] for the price, with a new format you will read the wrong column and not notice. It is safer to find the column by its name in the header row:

static int column(String[] header, String name) {
    for (int i = 0; i < header.length; i++) {
        if (header[i].strip().equalsIgnoreCase(name)) {
            return i;
        }
    }
    throw new IllegalArgumentException("No column " + name);
}


Path file = Path.of("export.csv");
try {
    Files.writeString(file, "Stock,Name,Code,Price\n30,Bread,1001,1.20\n18,Milk,1002,2.10\n");
    int nameCol = -1;
    int priceCol = -1;
    for (String line : Files.readAllLines(file)) {
        String[] f = line.split(",");
        if (nameCol == -1) {
            nameCol = column(f, "name");
            priceCol = column(f, "price");
            continue;
        }
        System.out.println(f[nameCol] + ": €" + f[priceCol]);
    }
} catch (IOException e) {
    System.out.println(e.getMessage());
}
Output
Bread: €1.20
Milk: €2.10

The first pass only finds where the columns are. After that, each row is read by those indexes. If the column is not there, column throws an exception right away, instead of reading wrong data.

More precisely, if you are curious
  • A comma in the value. "Cheddar, aged" has a comma in it. In CSV such a value goes in quotes: "Cheddar, aged",9.80,5, and a quote inside is written double: "". Then split(",") is no longer enough. Real projects use a library such as OpenCSV or Apache Commons CSV, which knows all the rules.
  • Another separator. Excel on a computer with German or Bulgarian settings often saves CSV with ;, exactly because the comma is the decimal mark.
  • Serialization. Java can write a whole object to a file with ObjectOutputStream, if the class implements Serializable. The file is then binary: a person cannot read it, and it breaks when the class changes. Text formats such as CSV and JSON are more predictable, and that is why they are used more.

Exercise

Data that comes alive

Solve the tasks in your editor. The program first creates the input file itself.

1. CSV analysis. Create people.csv with the contents below. Print the average age (integer division), who is the oldest, and how many people there are from each city, in the order the cities appear. Keep the cities in a String[] array, and the counts in an int[] of the same size.

people.csv
Name,Age,City
Ivan,34,Sofia
Maria,28,Plovdiv
Georgi,41,Sofia
Elena,25,Varna
Output
Average age: 32
Oldest: Georgi, 41
People from Sofia: 2
People from Plovdiv: 1
People from Varna: 1

2. Saving the hero. A Hero class with a name, level, health and gold. The method save(Path file) saves the hero as key=value lines, and the static method load(Path file) creates a new Hero from such a file. Create "Arin" with level 6, 100 health and 120 gold, raise his level by 1, take away 15 health, give him 220 gold, save him, print the file and load it back.

Output
name=Arin
level=7
health=85
gold=340
Loaded: Arin, level 7, health 85, gold 340

3. Duplicate text. Create text.txt with the text below. Split it into sentences at the period (and the spaces after it), and print each sentence that appears more than once, one time.

text.txt
This is text. This is also text. This is text.
Output
Duplicate text: "This is text."

More tasks

  • The most expensive product. Load products.csv from the lesson, and print the product with the highest price and how many products have stock under 20.
  • Fixed rows. Load the file with the bad rows, and write the good ones to products-clean.csv, and the bad ones, with the row number and the reason, to rejected.txt.
  • Sales report. A file sales.csv with the columns Date,Product,Count,Price. The rows are sorted by date. Print the revenue for each day with String.format in columns.

Test

Check yourself

1. What is the first row of most CSV files?
2. Why does catch (NumberFormatException e) have to come before catch (IllegalArgumentException e)?
3. What is "Coffee,7.50,".split(",").length?
4. You save prices with String.format("%.2f", price) on a computer with German settings. What happens to the CSV file?

Made by Ivo Gergov as part of Evo Tech's campaign for accessible programming education ·