What testing is actually for

Ask a team what their test coverage is and you get a number. Ask which failures that number rules out and the room goes quiet.

Coverage measures how much of the code ran while the tests were running. It says nothing about whether anything was checked. A suite can execute every line of a payment calculation while asserting only that it did not throw: the number goes up and the product is no safer.

The useful question is which failures you are unwilling to ship, and what stops each one.

It is a design tool before it is a safety net

The first value of a test arrives before the feature is finished, and it has nothing to do with catching regressions.

Code that is hard to test is usually code that does too much. When a function needs a database, a clock, an HTTP client and a signed-in user before it can be called once, the test is telling you the business rule is tangled up with everything around it. Pulling the rule out so it can be tested in three lines makes it clearer to read and cheaper to change, and the test is a side effect of that.

This shows up most on rules worth arguing about. Fee calculations. Eligibility. Anything with a threshold. Getting those into a shape where a test states the rule as plainly as a specification would usually improves the specification, because it forces the boundary cases into the open. What happens exactly at the limit? Both parties tend to discover they had assumed different answers.

Where the effort goes, and where it does not

Not everything deserves the same treatment.

Money and dates get tested hard, boundaries and rounding included. They have exact right answers, they are cheap to test, and getting one wrong is a customer complaint with a paper trail. Every rule about who can see or do what gets a test asserting the negative case, because positive tests confirm a feature works while negative tests keep working as a safeguard for years. State that can arrive twice or out of order gets tested on the second arrival, since that is the one production will send.

Then a small number of end-to-end tests over the journeys the business depends on: signing in, completing a purchase, submitting an application. Small on purpose. These are the slowest and most brittle tests in any suite, and a team with fifty of them starts ignoring failures, which is worse than having none.

Some things are left alone deliberately. No test asserts that a heading contains particular words, because copy changes, the test fails, and somebody updates the test without reading it, which is a tax on editing and teaches the team that a red build means nothing. No coverage target, because aiming at a percentage produces tests written to raise a percentage and those are the ones that assert nothing. And nothing tests the framework: if a query returns rows in the order the database was asked for, that is not our behaviour to verify.

What no machine checks for you

Accessibility is the clearest case. An automated checker catches missing alt text and a contrast failure. It cannot tell you that focus order jumps from the header to the footer and back, that an error message is announced before the field it belongs to, or that a keyboard user cannot escape a dialog. Those are found by putting the mouse down and using a screen reader.

Real devices are the second. An interface that behaves in a desktop browser narrowed to 390 pixels is not the same as one tested on an actual phone, where the keyboard covers half the screen and a fixed footer lands on top of the field being typed into.

For this site there is a third, which is the same page in two languages. Turkish strings run noticeably longer than English ones, so the width that breaks a card is usually 360 pixels in Turkish rather than in English. A suite will not notice a button whose label has wrapped onto three lines. A person looking at the page will.

There is one more way a suite lies, and it is the quietest. A suite that passes against tidy fixtures and fails against production data was testing the fixtures. Real names contain apostrophes, accented characters and the Turkish dotless i. Real addresses have four lines and no postcode. Real amounts include the one exactly at the threshold and the one a penny under. Real uploads include the empty file and the photograph of a document rather than the document.

So when something reaches production that the tests missed, the first move is to write the failing test and watch it fail. Then fix the bug and watch it pass. Otherwise there is no evidence the fix addressed the actual cause, and nothing stopping the same defect coming back later under a different name.

At the end of a project the tests go with the code, along with a short note on what they cover and what they deliberately do not. A suite whose intent is undocumented gets deleted by the next team the first time it goes red at an inconvenient moment. They will be right to, because nobody could tell them what it was protecting.