Code Coverage

Code coverage measures exactly one thing: which lines of production code executed while the tests ran.

It does not measure whether anything was asserted, whether the assertions were correct, or whether the behaviour anybody cares about was exercised. A suite with no assertions at all can reach 100%.

Understanding that sentence is most of what there is to know about coverage.

#Collecting it

# JavaScript / TypeScript — Jest or Vitest, V8 or Istanbul under the hood
npx jest --coverage
npx vitest run --coverage

# .NET — coverlet, producing Cobertura
dotnet test --collect:"XPlat Code Coverage"
reportgenerator -reports:"**/coverage.cobertura.xml" -targetdir:coverage -reporttypes:Html

# Java — JaCoCo
./gradlew test jacocoTestReport
mvn test jacoco:report

# Python — coverage.py via pytest-cov
pytest --cov=src --cov-report=term-missing --cov-report=xml

# Go
go test ./... -coverprofile=coverage.out && go tool cover -html=coverage.out

#Use it subtractively

The valuable use of coverage is looking at what is red, not at what the total is.

# The useful output: which lines are not covered, file by file.
pytest --cov=src --cov-report=term-missing

Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
src/pricing.py               84      0   100%
src/refunds.py               61     44    28%   23-67, 88-102
src/webhooks/stripe.py       47     47     0%

Three findings in that table, none of which is the overall percentage:

• webhooks/stripe.py at 0% is either dead code or completely untested money-handling code. Either answer is important.
• refunds.py at 28% has two large unexercised blocks. Look at them.
• pricing.py at 100% tells you nothing on its own — it might be thoroughly tested or it might have no assertions. See mutation testing.

#Gate on the patch, not the total

An absolute threshold — "the build fails below 80%" — is a textbook case of Goodhart's law. The measure becomes a target and stops being a measure, and what you get is tests written to execute lines.

it('constructs a PricingService', () => {
  new PricingService(stub);      // +40 lines covered, 0 behaviour verified
});

A patch threshold is far more useful: new and changed lines must be covered. It applies to the code someone is thinking about right now, it does not punish anyone for a legacy module, and it stops the untested area growing.

# codecov.yml
coverage:
  status:
    project:
      default:
        target: auto        # do not go down relative to the base
        threshold: 1%       # allow small noise
    patch:
      default:
        target: 80%         # new code must be covered

// Or in the runner, with per-directory thresholds that reflect reality
module.exports = {
  coverageThreshold: {
    global: { statements: 70, branches: 60 },
    './src/pricing/': { statements: 95, branches: 90 },   // the money code
    './src/ui/': { statements: 40 }                       // honest about the UI
  }
};

Different thresholds for different directories is an unfashionable and honest configuration: the pricing engine and the settings page do not deserve the same bar.

#What to exclude, and what not to

Exclude generated code, migrations, configuration and framework boilerplate — they inflate or deflate the number without informing anything.

collectCoverageFrom: [
  'src/**/*.{ts,tsx}',
  '!src/**/*.d.ts',
  '!src/**/generated/**',
  '!src/migrations/**'
]

Do not exclude a file because it is hard to test. That is the file the number was trying to tell you about.

#Reading it in a pull request

Coverage is most useful as a diff annotation: which lines this change added that nothing exercises.

# GitHub Actions
- run: npm test -- --coverage --coverageReporters=lcov
- uses: codecov/codecov-action@v4
  with:
    files: ./coverage/lcov.info
    fail_ci_if_error: true

The output that changes behaviour is the inline marker on an untested line in the diff, not the badge in the README.

#The arguments against, taken seriously

"Coverage does not measure test quality." Correct. It measures execution. That is why it is a finding tool, and why mutation testing exists to measure the thing coverage is mistaken for.

"Chasing coverage produces bad tests." Correct, when it is a target. This is an argument against absolute thresholds, not against measurement.

"100% is a waste of effort." Usually. The last 10% is generally error handling for conditions that cannot occur and glue that cannot break, and the effort is better spent on the 40%-covered module that handles money.

#The one number worth watching

Not the total. The list of files at or near zero.

That list is a direct statement of where your system is unprotected, it is short enough to read, and it is actionable. Publish it, look at it monthly, and fix the entries that handle money, permissions or customer data.