Beyond Functional

A functional suite answers one question: does the system do the right thing when asked nicely, once, by someone allowed to ask. Every incident you will ever have is somewhere in the gap between that sentence and reality.

#The four gaps

Load. It does the right thing once. Does it do the right thing a thousand times a second, and what breaks first when it stops? Performance testing and load testing are not the same activity and mixing them up is why so many teams have a load test that proves nothing.

Data. Database testing covers the layer everything else is built on: migrations that must be reversible, constraints that are the last line of defence, queries that are correct on ten rows and catastrophic on ten million, and the fact that most systems' hardest bugs live in a transaction boundary.

Access. Authorization testing is the single highest-value security testing most teams are not doing. Authentication gets tested constantly — everybody logs in during every end-to-end test. Authorization gets tested almost never, because the suite runs as one user who is allowed to do everything, and nothing proves that the other user cannot. Broken access control has topped the OWASP list for years for exactly this reason.

Reach. Accessibility testing catches a class of defect that functional tests are blind to by construction, and about a third of it is genuinely automatable. The other two thirds are not, and the article is honest about which is which.

#What automation can and cannot do here

Security testing is the section's honest page. Automated scanning finds known-shaped problems: dependency vulnerabilities, missing headers, injection in the obvious places, secrets in source. It does not find business-logic flaws, and the most damaging breaches are almost always business-logic flaws. Automate the known shapes, put them in the pipeline, and do not mistake a green scan for a secure system.

There is a pleasing symmetry between this section and test doubles: the more of the real system a test replaces with a fake, the less it can say about how the real system behaves under real conditions. Non-functional testing is where the fakes run out.