Skip to content
End To End Tester
Beyond Functionalpractical

Performance Testing

Latency, throughput and what breaks first — designing a performance test that answers a question, reading percentiles honestly, and putting a check in the pipeline.

2 min read · updated 19 September 2026

A performance test has to answer a question. "Is it fast?" is not one. "Can the checkout endpoint serve 200 requests a second with a 99th percentile under 400ms on the production instance size?" is.

Without that specificity you get a number, and a number with no threshold attached changes nobody's behaviour.

#Percentiles, not averages

Latency distributions have long tails. The mean hides them.

p50   82ms      half of requests
p90  140ms
p95  210ms
p99 2,400ms     ← one request in a hundred
p999 8,100ms    ← the ones people tweet about
mean 118ms      ← tells you almost nothing useful

If a page makes twenty requests, "one in a hundred is slow" means roughly one page view in five contains a slow request. The p99 is not an edge case; it is your Tuesday.

Report p50, p95 and p99. Set thresholds on p95 and p99. Never gate on a mean.

#A test with a threshold

javascript
// k6 — the most usable of the load-testing tools in 2026.
import http from 'k6/http';
import { check, sleep } from 'k6';
import { Trend } from 'k6/metrics';

const checkoutLatency = new Trend('checkout_latency', true);

export const options = {
  scenarios: {
    steady: {
      executor: 'constant-arrival-rate',   // arrival rate, not VUs — see below
      rate: 200,
      timeUnit: '1s',
      duration: '5m',
      preAllocatedVUs: 100,
      maxVUs: 500
    }
  },
  // The test fails if these are not met. This is the part that matters.
  thresholds: {
    'http_req_failed': ['rate<0.001'],
    'checkout_latency': ['p(95)<400', 'p(99)<1000']
  }
};

export default function () {
  const basket = http.post(`${__ENV.BASE_URL}/api/basket`, JSON.stringify({ sku: 'book-1' }), {
    headers: { 'Content-Type': 'application/json' }
  });
  check(basket, { 'basket created': (r) => r.status === 201 });

  const started = Date.now();
  const order = http.post(`${__ENV.BASE_URL}/api/orders`, basket.body, {
    headers: { 'Content-Type': 'application/json' }
  });
  checkoutLatency.add(Date.now() - started);

  check(order, { 'order placed': (r) => r.status === 201 });
  sleep(1);
}

Use an arrival-rate executor, not a fixed number of virtual users. With fixed VUs, each one waits for its response before sending the next, so as the system slows down the offered load falls — the test quietly stops applying pressure at exactly the moment you needed it to. This is coordinated omission, and it makes most homegrown load tests optimistic by a large factor.

#The tools

Tool Language Notes
k6 JavaScript best ergonomics; thresholds as first-class; good CI story
Gatling Scala/Java/Kotlin excellent reports, strong on complex scenarios
JMeter GUI/XML ubiquitous, enormous plugin ecosystem, painful to version-control
Locust Python write scenarios in Python; easy to extend
wrk / oha / autocannon CLI single-endpoint microbenchmarks; fine for a quick answer

#Micro-benchmarks

For a function rather than a system, use a benchmarking harness that controls for JIT warm-up, dead-code elimination and measurement overhead. A hand-rolled for loop with timestamps gets all three wrong.

csharp
// C#, BenchmarkDotNet
[MemoryDiagnoser]
public class PricingBenchmarks
{
    private readonly Order _order = OrderBuilder.WithLines(50);

    [Benchmark(Baseline = true)]
    public int Current() => Pricing.ApplyDiscount(_order, Policy).DiscountCents;

    [Benchmark]
    public int Proposed() => PricingV2.ApplyDiscount(_order, Policy).DiscountCents;
}
java
// Java, JMH
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Fork(2) @Warmup(iterations = 3) @Measurement(iterations = 5)
public class PricingBenchmark {
    @Benchmark public int current(State s) { return Pricing.applyDiscount(s.order, POLICY).discountCents(); }
}

#In the pipeline

Full load tests need a production-like environment and produce noisy results on shared CI hardware. What does belong on every pull request is a smoke performance check: enough to catch an order-of-magnitude regression.

yaml
# GitHub Actions
- name: Performance smoke
  run: |
    docker compose up -d --wait
    k6 run --vus 10 --duration 60s --quiet perf/checkout.js
  env:
    BASE_URL: http://localhost:3000
javascript
// Thresholds generous enough to survive noisy CI, tight enough to catch a
// 10x regression. The full run happens nightly against a real environment.
export const options = {
  thresholds: { 'http_req_duration': ['p(95)<2000'], 'http_req_failed': ['rate<0.01'] }
};

#Front-end performance

Server latency is half the story. The other half is what the browser does with the response.

typescript
// Playwright + Lighthouse, as a budget check rather than a score chase
import { playAudit } from 'playwright-lighthouse';

test('the product page stays within its performance budget', async ({ page }) => {
  await page.goto('/products/field-notes');

  await playAudit({
    page,
    thresholds: { performance: 85, accessibility: 95 },
    // Budgets are more actionable than the composite score
    config: { settings: { budgets: [{ resourceSizes: [{ resourceType: 'script', budget: 180 }] }] } }
  });
});

A JavaScript bundle-size budget is the single most effective front-end performance check, because bundle size is the metric that regresses quietly and monotonically.

#What to measure alongside

A latency number with no context is hard to act on. Capture, for the same window:

  • Error rate. Fast failures look like good latency.
  • Throughput actually achieved, versus offered.
  • Saturation — CPU, memory, connection pool, queue depth. This is what tells you what to fix.
  • Database time specifically. In most web systems the answer is in here — see database testing.

#The trap

The most common performance-testing mistake is testing a system that is not under realistic data conditions. A query that is 4ms against 1,000 rows and 4 seconds against 10 million is the most common production performance incident there is, and no amount of load against an empty database will find it.

Seed production-like volume — generated, never copied, for the reasons in test data management — before believing any number.

Common questions

What is the difference between performance testing and load testing?
Performance testing asks how fast the system is under a defined condition. Load testing asks what happens as the condition gets worse — more users, more data, more concurrency — and where it breaks. They use the same tools and answer different questions.
Why is the average response time misleading?
Because latency distributions are not symmetric. A system with a 100ms mean can easily have a 2-second 99th percentile, which is the experience of one request in a hundred — and with twenty requests per page view, that is most page views touching at least one slow request. Report percentiles, not means.
Should performance tests run in CI?
A small, fast one should — enough to catch an order-of-magnitude regression in a critical path. Full load tests need a production-like environment and are too slow and too noisy for every pull request; run those on a schedule or before a release.

Runnable samples for this page

last test results ↗

Working tests, not fragments — they run in CI on every push to 8exgh/endtoendtester-samples.

Was this page useful?