Test Strategy

Most teams have an implicit test strategy. It usually amounts to write tests for the things that are easy to write tests for, and it shows in the shape of the pyramid and in which module turns out to be uncovered when something breaks.

A strategy is the same decision, made deliberately and written down. It should fit on one page.

#Four questions

#1. What are we trying to prevent?

Not "bugs". Specifically:

• A customer being charged twice
• An order silently not reaching the warehouse
• One tenant seeing another tenant's data
• The site being down during a campaign
• A migration that cannot be rolled back

That list is short, specific to your business, and it determines everything else. An e-commerce team and a medical device team write very different lists and should have very different suites.

#2. What is each level responsible for?

Unit            every branch of business logic; no I/O; runs in seconds
Integration     every seam with something we did not write
Component       each service's HTTP contract, including authorization
Contract        that independently deployed services still agree
End-to-end      the six journeys that would end the business
Exploratory     everything we did not think of
Production      monitoring and alerting as the final layer of testing

Writing this down settles the recurring argument about where a given test belongs, and it makes gaps visible. If nothing on that list owns "authorization", nothing is testing it.

#3. What are we not going to test?

The part everyone skips, and the most useful.

NOT TESTED, DELIBERATELY
  - Internet Explorer and any browser below 1% of sessions
  - The admin CSV export (internal, two users, obvious when broken)
  - Email rendering across clients (we accept it may look wrong in Outlook)
  - Localisation beyond en-GB and fr-FR
  - Load beyond 3x current peak

Every team has a list like this. The difference is whether it was chosen or discovered at 2am. Writing it down converts an unknown risk into an accepted one, and makes it reviewable when the business changes.

#4. How will we know this is not working?

Leading indicators, so the strategy can be corrected before an incident teaches you:

• Escaped defects — bugs found in production, by area. The distribution tells you where the strategy is wrong.
• Flaky rate — above ~1% and people stop trusting red.
• Pipeline duration — over ~15 minutes and people stop waiting for it.
• Areas at zero coverage — a direct statement of what is unprotected.

#Risk-based prioritisation

Testing is a cost paid to reduce risk. Spend it where risk is highest.

The interesting rows are the cheap-failure ones. "Search relevance is not worth automated testing, we will iterate on it in production" is a legitimate and often correct strategic decision, and it is only available to a team that has decided it rather than drifted into it.

#What a strategy is not

Not a test plan per feature. Those are artefacts of a process that produces documents; strategy is the standing decision that removes the need for most of them.

Not a coverage target. See code coverage — a number without a risk model behind it produces tests written to move the number.

Not a tool choice. Tools are downstream. A team that cannot say what it is trying to prevent will not be saved by picking Playwright.

#Writing it down

One page, in the repository, next to the code:

# Test strategy

## What we are protecting
1. No customer is charged for something they did not receive.
2. No tenant can see another tenant's data.
3. Checkout works, every day, on Chrome, Safari and mobile Safari.

## Who tests what
- Developers: unit, integration and component tests for everything they write.
- The suite: six e2e journeys, tagged @critical, run on three engines.
- Everyone: exploratory testing for one hour before each release.

## Deliberately not tested
- Browsers below 1% of sessions (reviewed quarterly).
- The internal admin export.
- Email rendering outside Gmail and Apple Mail.

## How we know this is failing
- Escaped defects by area, reviewed monthly.
- Flaky rate over 1%.
- Pipeline over 15 minutes.

Reviewed: 2026-09-19. Next review: 2026-12-19.

The review date is what stops it from becoming archaeology. A strategy written for a product that has since changed is worse than none, because people still believe it.