Dependencies: other people’s code that yours runs

Code in:JavaC#PythonJavaScript

In short: Dependencies are other people’s code that your program runs; the lock file and the package hash guard against tampering, and a scanner finds known vulnerabilities.

Problem

The package nobody chose

The shop’s package.json lists 14 packages. node_modules has 900. Each of the 14 brings its own dependencies, and those bring theirs.

One night somebody gets into the account of the author of a small date formatting package. The package is deep in the tree: the shop uses it through the PDF invoice library, and nobody in the team knows it is there. The attacker uploads a new version with one extra line: a line that reads the process environment and sends PAYMENT_API_KEY out. In the morning the shop’s build pulls the new version, because the range ^2.1.3 allows it.

The key that you moved out of the code and into the environment in the previous lesson leaks from there. The shop got nothing wrong in its own code.

This is not made up. It happened with event-stream (2018) and with ua-parser-js (2021), and in 2024 the swap in xz utils was found shortly before it reached servers around the world.

The idea

Your process, someone else’s code

Every package the program loads runs in your process and with your rights. It reads the same environment and the same files, and talks to the same network. Some packages run code already at install time (npm’s postinstall scripts, setup.py in old Python packages).

The packages you asked for are direct dependencies. The ones they bring are transitive. The transitive ones are usually the majority, and nobody chose them.

This is the supply chain: from the package author through the registry (npm, Maven Central, NuGet, PyPI) to your build. An attacker can get in at any link:

• a hijacked account of an author: a new version with harmful code;
• typosquatting: a package named like a known one, but one letter off (reqeusts, lodahs), waiting for a typo;
• a swap on the way: a mirror, cache or proxy that hands over different bytes under the same name and version;
• an abandoned package that someone takes over and "updates" months later.

Against all of this there are two simple rules. Know exactly which versions get in. And check that the bytes are the same ones you checked.

See it

The tree and the swapped package

function verify(name, bytes, lockFile) {
  const expected = lockFile[name];
  const actual = integrity(bytes);
  if (actual !== expected) {
    console.log(`${name}: TAMPERED`);
    console.log(`  lock:    ${brief(expected)}`);
    console.log(`  fetched: ${brief(actual)}`);
    return false;
  }
  console.log(`${name}: OK`);
  return true;
}

Step by step, in words

1. The shop asks for two packages. They bring three more of their own. All five run inside the shop.
2. Somebody swaps date-fmt on the way. Without a check its code runs in the shop, with its environment and files.
3. The lock file holds the exact version and hash of every package, the deep ones too.
4. The hash of what was downloaded: SHA-512, written in Base64.
5. The same hash as in the lock file: these are the bytes that were checked. Installed.
6. date-fmt has one extra line, and the whole hash is different. It is not installed.
7. Installed: 4 of 5. The build stops, and nobody runs the swapped code.

The hash here is a relative of the hash code from the lesson on hash codes, but it is cryptographic. SHA-512 gives 64 bytes, and nobody can make another package with the same fingerprint. The hash code in collections is 4 bytes and repeats easily: it is fine for buckets in a table, but not for checking someone else’s code.

The idea

The lock file: the exact version and its fingerprint

The manifest says what you want: "pdf-invoice": "^5.0.0" means "5.0.0 or a newer 5.x". The lock file says exactly what is installed: the version of every package in the tree, the transitive ones too, and a hash of its contents. It is kept in git, and the build installs exactly that.

In JavaScript this is package-lock.json (or pnpm-lock.yaml, yarn.lock). Every entry has an integrity:

"node_modules/date-fmt": {
  "version": "2.1.3",
  "resolved": "https://registry.npmjs.org/date-fmt/-/date-fmt-2.1.3.tgz",
  "integrity": "sha512-Jeb+bpiC7IqY/wWOY2a0umsIM6ce0XcMc8Xl…"
}

CI runs npm ci, not npm install: it installs exactly the lock file and stops if it does not match package.json or if a hash does not match. npm install may change it.

The check is simple: a hash of the downloaded bytes and a comparison with the entry. Here it is as a program, with a hash in npm’s form, sha512- and Base64:

function integrity(bytes) {
  const digest = createHash("sha512").update(bytes).digest("base64");
  return "sha512-" + digest;
}

function brief(hash) {
  return hash.slice(0, 19) + "…";
}

function verify(name, bytes, lockFile) {
  const expected = lockFile[name];
  const actual = integrity(bytes);
  if (actual !== expected) {
    console.log(`${name}: TAMPERED`);
    console.log(`  lock:    ${brief(expected)}`);
    console.log(`  fetched: ${brief(actual)}`);
    return false;
  }
  console.log(`${name}: OK`);
  return true;
}

const lockFile = {
  "pdf-invoice@5.0.2": "sha512-IN0lH/YKriHbj87FMea7idgW4qdE8vgdeYTdEVVqz/GwAnMAzmU8X7TOO2TJ20CSQLaDtD9hKB7J5bw8JLos0A==",
  "font-kit@3.1.0": "sha512-19+g7wktnKQh+JGfMRNeNdrG3//S+BsOUeuBcAp8p10kuHCpM3HFgbQzKnzTQ+DRLcA2b1MmnHFfHYvjFrjm9g==",
  "date-fmt@2.1.3": "sha512-Jeb+bpiC7IqY/wWOY2a0umsIM6ce0XcMc8XlRr9udQktm8kZCNIV+qcJtad0Uip8hhhdn9P+8yE6p56qsZkZSw==",
  "pay-sdk@4.2.0": "sha512-qyQmG1zNAk4HmkwNd+yxZ0GkyOsQoLn9CGHMW2LsBWmkhwpMo7rXHYT2keUIu6n50HuYBjv411Twk69Wqc0nig==",
  "http-lite@1.8.0": "sha512-TI5SSzlpujK57oie/hoakeRXqVh5YuW22qwmU7I6moo75Gl3qXY5ZqRuEsy0sKdy6dz7uqhlJzDDliKF4tUQQA==",
};
const downloaded = [
  ["pdf-invoice@5.0.2", Buffer.from("pdf-invoice 5.0.2: renders invoices to PDF")],
  ["font-kit@3.1.0", Buffer.from("font-kit 3.1.0: loads and measures fonts")],
  ["date-fmt@2.1.3", Buffer.from("date-fmt 2.1.3: formats dates and times\n+ one line added by the attacker")],
  ["pay-sdk@4.2.0", Buffer.from("pay-sdk 4.2.0: talks to the payment provider")],
  ["http-lite@1.8.0", Buffer.from("http-lite 1.8.0: a small HTTP client")],
];
const installed = downloaded.filter(([name, bytes]) => verify(name, bytes, lockFile)).length;
console.log(`Installed: ${installed} of ${downloaded.length}`);

Output

pdf-invoice@5.0.2: OK
font-kit@3.1.0: OK
date-fmt@2.1.3: TAMPERED
  lock:    sha512-Jeb+bpiC7IqY…
  fetched: sha512-sMR370edweyw…
pay-sdk@4.2.0: OK
http-lite@1.8.0: OK
Installed: 4 of 5

The packages here are a short text, but for the hash there is no difference: they are bytes. One extra line in date-fmt and the hash is completely different, from the very first character.

Try it

Predict the output

The two texts differ only by one space at the end. What will it print?

const a = integrity(Buffer.from("total 49.90"));
const b = integrity(Buffer.from("total 49.90 "));
console.log(a.length);
console.log(b.length);
console.log(a === b);

Answer

95
95
false

95
96
false

95
95
true

Check

Try it

Fill in

Which algorithm gives the hash in the lock file?

const digest = createHash("").update(bytes).digest("base64");

Check

Try it

Find the bug

A colleague writes the check again. The registry server returns the package together with its hash, so the colleague compares with that. Which line is the bug?

Pick the line with the bug.

1function verify(pkg, lockFile) {2 const expected = pkg.integrity; // the hash the server sent along with the package3 const actual = integrity(pkg.bytes);4 return actual === expected;5}

Check

The idea

Known vulnerabilities and updating

The lock file guards against swaps, but it also freezes the old bugs. A package without harmful code can have a vulnerability: a bug that lets an attacker in. An example is Log4Shell (2021): one line in the log of a server with log4j-core before 2.17.1 was enough for the attacker to run their own code.

Vulnerabilities are published in databases (CVE, GitHub Advisory Database, OSV), and tools compare your tree with them:

• JavaScript: npm audit;
• Python: pip-audit;
• .NET: dotnet list package --vulnerable --include-transitive;
• Java: OWASP Dependency-Check (a plugin for Maven and Gradle);
• for all of them: Dependabot and Renovate open a pull request with the update.

The hygiene of updating:

• Small and often. Updating one version with tests behind it is safe. Five years at once is not. This is where tests as a safety net help.
• Critical ones at once, especially when the fix is already public.
• Read what changes before you accept a major version.
• Remove what you do not need. Every package is code you run. Six lines of your own code are sometimes cheaper than a package with 40 dependencies.
• A new dependency is a decision. Who maintains it, how many people, how many dependencies it brings, when the last version came out.

More precisely, if you are curious

Ranges. In npm ^2.1.3 allows any 2.x.y from 2.1.3 up, ~2.1.3 only 2.1.y. That is why without a lock file two builds of the same code give different trees. Libraries write ranges so they do not force everyone onto one version; applications lock the exact tree.

Install scripts. npm ci --ignore-scripts does not run postinstall and the like. Many projects do this by default in CI and allow scripts only for listed packages.

Your own registry. Companies often pull packages through their own proxy registry (Artifactory, Nexus, Azure Artifacts). It keeps a copy, lets only approved packages through and tracks what is used where.

SBOM. A Software Bill of Materials is a list of every component in the product, in the CycloneDX or SPDX format. When the next Log4Shell comes out, the answer to "are we affected?" is a search in it, not a week of digging.

Signatures. A hash says "the same bytes as when they were checked". A signature says "published by this author": npm provenance, Sigstore, signed packages in NuGet and Maven Central.

How to see the tree: npm ls --all, mvn dependency:tree, gradle dependencies, dotnet list package --include-transitive, pipdeptree.

Exercise

A small audit

Solve it in your own editor.

Installed are log-kit 2.14.1, http-lite 1.8.0, pay-sdk 4.2.0 and qr-code 2.9.0. The vulnerability database says (package, fixed in version, severity): log-kit 2.17.1 critical, http-lite 1.8.2 high, pay-sdk 3.9.0 moderate, qr-code 2.10.0 low. Print for each one whether it is vulnerable and which version to update to, and how many are vulnerable:

Output

log-kit 2.14.1: vulnerable (critical), update to 2.17.1
http-lite 1.8.0: vulnerable (high), update to 1.8.2
pay-sdk 4.2.0: fine
qr-code 2.9.0: vulnerable (low), update to 2.10.0
Vulnerable: 3

Careful with qr-code: as text, "2.9.0" comes after "2.10.0", because 9 comes after 1. Versions are compared part by part, as numbers.

More tasks

• Severity. Sort the vulnerable ones by severity: critical, high, moderate, low.
• A lock from text. Read the lock file as lines name@version hash and use it to check the downloaded packages from the lesson.
• Typosquatting. Compare the name of a new package with a list of popular ones and warn if it differs by one letter (one change, insertion or deletion).

Test

Check yourself

1. The shop depends directly on pdf-invoice. It depends on date-fmt. Whose code runs when the shop prints an invoice?

Only the shop’s and pdf-invoice’s: date-fmt is someone else’s dependencyOnly the shop’s: packages are just librariesAll three: the shop, pdf-invoice and date-fmt, all in one process and with the same rights

2. npm’s lock file says date-fmt version 2.1.3 with hash sha512-Jeb+…. The server hands over a package whose hash is sha512-sMR3…. What does npm ci do?

Stops with an integrity error: what was downloaded is not what was checkedUpdates the hash in the lock fileInstalls it, because the version is the same

3. What is typosquatting?

A package whose name is almost that of a known one (reqeusts instead of requests), waiting for someone to mistype itA package that changes its version without changing its codeA mistake in a package name in the lock file

4. npm audit says log-kit 2.14.1 has a critical vulnerability, fixed in 2.17.1. What is right?

Update to 2.17.1 (or newer), run the tests and ship it soonRemove audit from the build so it does not get in the wayWait for the next big update of everything

Check

Start the lesson over

Related lessons

• Builds on: Data, switch and destructuring · Tests as a safety net · Untrusted input · Secrets in code and in git
• Tasks on this topic: Security for developers