End-to-End Testing

An end-to-end test drives the deployed system the way a person does: a real browser, a real network, a real database, every service running. It is the only level that can tell you the product actually works.

It is also the slowest, most fragile and most expensive level, which is why the discipline around what goes in it matters more than the tool you pick.

#What belongs here

Journeys, in business terms, that would be catastrophic if broken:

• Sign up, verify, sign in
• Find a product, add to cart, check out, get a confirmation
• Upload a document and see it processed
• The one admin action that, if it stopped working, means support cannot do their job

Ten to forty of these for most products. Each one crosses every layer, so each one is worth enormous confidence — and each one costs seconds of wall-clock and a permanent maintenance obligation.

#What does not belong here

• Validation messages. "Email is required" is a unit test. It is not worth a browser.
• Every permutation. One checkout journey, not one per payment method per country per currency. Those belong at the component or API level, where they run in milliseconds.
• Error paths that need a broken dependency. You cannot reliably make the real payment provider fail. That is stub territory.
• Anything already proven below. If a component test covers POST /orders returning 402 on a declined card, the end-to-end suite does not need to re-prove it through a form.

A useful filter: would a product manager recognise this as a thing a customer does? If not, it probably belongs lower.

#The shape of a good one

// TypeScript, Playwright. Data by API, auth by stored session,
// assertions by user-visible role and text.
import { test, expect } from '@playwright/test';
import { createCustomer, createProduct } from './support/api';

test('a customer can buy a product and receives a confirmation', async ({ page, request }) => {
  // Arrange through the API — fast, reliable, and not what is under test.
  const product = await createProduct(request, { name: 'Field Notes', priceCents: 1_200 });
  const customer = await createCustomer(request, { email: `buyer+${test.info().workerIndex}@example.test` });

  await page.goto(`/products/${product.slug}`);

  // Act as a user, addressed the way a user addresses things.
  await page.getByRole('button', { name: 'Add to cart' }).click();
  await page.getByRole('link', { name: 'Checkout' }).click();
  await page.getByLabel('Card number').fill('4242424242424242');
  await page.getByRole('button', { name: 'Pay £12.00' }).click();

  // Assert the outcome the customer cares about.
  await expect(page.getByRole('heading', { name: 'Order confirmed' })).toBeVisible();
  await expect(page.getByText(/we have emailed/i)).toBeVisible();

  // And the outcome the business cares about.
  const orders = await request.get(`/api/customers/${customer.id}/orders`);
  expect(await orders.json()).toMatchObject([{ status: 'paid', totalCents: 1_200 }]);
});

Four things in there are the difference between a suite that survives and one that does not:

1. Setup through the API. Creating a product through an admin UI takes fourteen seconds and breaks when the admin UI changes. It also means a failure in product creation fails the checkout test, which is a diagnosis problem.
2. Role and label selectors. getByRole('button', { name: 'Add to cart' }) survives a restyle, a class rename and a component library migration. .btn-primary:nth-child(3) survives none of them. See DOM testing.
3. Unique data per worker. The email includes the worker index, so parallel runs do not collide.
4. Web-first assertions. expect(locator).toBeVisible() retries until it is true or times out. A bare expect(await locator.isVisible()) reads once and is a race — the single most common source of flakiness in browser suites.

#Authentication, done once

Signing in through the form in every test costs three to eight seconds per test and tests the login form several hundred times. Sign in once, save the storage state, reuse it.

// playwright/auth.setup.ts — runs once, before everything else
import { test as setup } from '@playwright/test';

setup('authenticate', async ({ page }) => {
  await page.goto('/signin');
  await page.getByLabel('Email').fill(process.env.TEST_USER!);
  await page.getByLabel('Password').fill(process.env.TEST_PASSWORD!);
  await page.getByRole('button', { name: 'Sign in' }).click();
  await page.waitForURL('/dashboard');
  await page.context().storageState({ path: 'playwright/.auth/user.json' });
});

// playwright.config.ts
projects: [
  { name: 'setup', testMatch: /auth\.setup\.ts/ },
  {
    name: 'chromium',
    use: { ...devices['Desktop Chrome'], storageState: 'playwright/.auth/user.json' },
    dependencies: ['setup']
  }
]

Keep exactly one test that signs in through the form. That is the one testing the login journey; the other three hundred are not.

#Making failures diagnosable

An end-to-end test that fails in CI with a timeout and nothing else will cost you an afternoon. Configure the artefacts once and it costs five minutes:

use: {
  trace: 'on-first-retry',       // full action/network/DOM recording
  screenshot: 'only-on-failure',
  video: 'retain-on-failure'
}

See screenshots, screen recordings and the trace viewer.

#The honest position on retries

Retries hide flakiness. They are also necessary, because at this level some non-determinism is genuinely environmental and blocking every merge on it is worse than the alternative.

The workable compromise: allow one retry in CI, none locally, and treat a test that only passes on retry as a failure to investigate. Playwright reports these as "flaky" rather than "passed" specifically so you can count them. When that count goes up, something is wrong — see flaky tests.

#Where the level ends

End-to-end testing tells you the journey works. It cannot tell you why it stopped working, it cannot cover the combinatorics, and it cannot be made fast. Everything you push down into component, integration and unit tests is time and reliability you get back — which is the whole argument of the pyramid.