Accessibility Testing · End To End Tester

Accessibility testing is the rare case where the automated part is cheap, effective and improves your test suite at the same time.

It is also a case where overclaiming is common. Be clear about both halves: roughly a third of WCAG criteria are machine-checkable, and the widely quoted figure from Deque is that axe finds about 57% of real issues. For a check that runs in two seconds, that is excellent. It is not compliance.

#The automated part

typescript

// Playwright + axe — every page, every run, two seconds each.
import AxeBuilder from '@axe-core/playwright';

test.describe('accessibility', () => {
  for (const path of ['/', '/products/field-notes', '/checkout', '/account']) {
    test(`${path} has no detectable violations`, async ({ page }) => {
      await page.goto(path);

      const results = await new AxeBuilder({ page })
        .withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
        .analyze();

      expect(results.violations).toEqual([]);
    });
  }
});

typescript

// jest-axe — at the component level, where a violation is cheapest to fix.
import { axe, toHaveNoViolations } from 'jest-axe';
expect.extend(toHaveNoViolations);

it('the price tag has no accessibility violations', async () => {
  const { container } = render(<PriceTag cents={1_200} wasCents={1_500} />);

  expect(await axe(container)).toHaveNoViolations();
});

When a violation is found, the report names the rule, the element and the fix. It is one of the most actionable test failures there is.

#What it catches

Missing alternative text
Insufficient colour contrast
Form inputs with no label
Invalid or contradictory ARIA
Missing document language, duplicate ids
Heading levels skipped
Interactive elements that are not focusable
Tables without headers

All mechanical, all common, all genuinely worth fixing.

#What it cannot catch

Whether alt text is useful. alt="image" passes.
Whether focus order matches visual order.
Whether a modal traps focus, and returns it on close.
Whether an error message is announced to a screen reader.
Whether the page makes sense read aloud, in order.
Whether an animation triggers motion sickness.
Whether the reading level suits the audience.

These need a person, and specifically a person using assistive technology. That is exploratory testing and no tool substitutes for it.

#The parts you can automate that axe does not

Two things worth writing yourself, because they catch real defects:

Keyboard operability:

typescript

test('the whole checkout flow is reachable with a keyboard', async ({ page }) => {
  await page.goto('/checkout');
  await page.keyboard.press('Tab');

  const order: string[] = [];
  for (let i = 0; i < 12; i++) {
    order.push(await page.evaluate(() =>
      document.activeElement?.getAttribute('aria-label')
      ?? document.activeElement?.textContent?.trim()
      ?? document.activeElement?.tagName
      ?? ''
    ));
    await page.keyboard.press('Tab');
  }

  // Focus order matches reading order, and nothing is skipped.
  expect(order).toEqual([
    'Skip to content', 'Home', 'Card number', 'Expiry', 'CVC', 'Pay £27.95', /* … */
  ].slice(0, 12));
});

Focus management in dialogs:

typescript

test('the dialog traps focus and restores it on close', async ({ page }) => {
  const trigger = page.getByRole('button', { name: 'Edit address' });
  await trigger.click();

  const dialog = page.getByRole('dialog');
  await expect(dialog).toBeFocused();

  // Tab round the dialog: focus must never escape it.
  for (let i = 0; i < 10; i++) await page.keyboard.press('Tab');
  await expect(dialog.locator(':focus')).toHaveCount(1);

  await page.keyboard.press('Escape');
  await expect(trigger).toBeFocused();         // returned to where it came from
});

Focus restoration is the single most common accessibility defect in modern single-page applications, and it is entirely testable.

#The double benefit

This is the part worth internalising: accessible markup and maintainable tests are the same markup.

typescript

// Requires an accessible name. Survives a restyle, a class rename, a
// component library migration. Fails when accessibility regresses.
page.getByRole('button', { name: 'Add to cart' })

// Requires nothing. Breaks on the next redesign. Silent about accessibility.
page.locator('.btn.btn-primary:nth-child(2)')

A team that adopts role-and-name queries finds its accessibility improving, because a component with no accessible name is now inconvenient to test. A team that improves accessibility finds its tests getting more stable, for the same reason. See DOM testing and Testing Library.

That alignment is rare enough to be worth exploiting deliberately.

#In the pipeline

yaml

- name: Accessibility
  run: npx playwright test accessibility.spec.ts

Gate on new violations rather than all of them if you are retrofitting: a baseline of known issues, and a build that fails when the count goes up. An all-or-nothing gate on a legacy application gets switched off in a week.

typescript

const results = await new AxeBuilder({ page }).analyze();
const known = loadBaseline(path);

const introduced = results.violations.filter((v) => !known.includes(v.id));
expect(introduced).toEqual([]);

Common questions

How much of accessibility can be automated?

Roughly a third of WCAG success criteria can be checked automatically. The commonly cited figure from Deque is that axe finds about 57% of issues in practice, which is a lot for a tool that runs in two seconds — and it is not the same as compliance.

Does a clean axe scan mean the site is accessible?

No. A page can pass every automated check and be unusable with a keyboard, illogical with a screen reader, or impossible to follow because the focus order does not match the visual order. Automation catches the mechanical problems.

How does accessibility relate to test maintainability?

Directly. Querying by role and accessible name requires accessible markup, and produces tests that survive restyling. Teams that adopt getByRole find their accessibility improves as a side effect, and vice versa.

#The automated part

#What it catches

#What it cannot catch

#The parts you can automate that axe does not

#The double benefit

#In the pipeline

Common questions

Runnable samples for this page

Related topics