Why Automated Audits Miss Most of It

You run Lighthouse, it flashes a green 100 for accessibility, and you feel done. Here's the uncomfortable truth: that score only measures the slice of accessibility a machine can check on its own, and that slice is small. A perfect automated score can sit right on top of a site that fails badly for real people.

How much a machine can actually catch

Automated tools reliably catch somewhere around a quarter to a third of the accessibility issues on a page. The rest need a person. The same is true criterion by criterion: the rules a scanner ships with touch only a minority of the WCAGWeb Content Accessibility Guidelines, the technical standard the AODA points at success criteria at all, because most of them turn on context, meaning, or whether an actual interaction works. So when a scanner says you're clean, what it has told you is that nothing a machine could catch was caught. That is not the same as passing.

You'll see a much more precise sounding number quoted around this, usually 57 percent. That figure comes from Deque, the company behind the axe testing engine, and it counts issue instances rather than criteria. It's a vendor measuring its own product, and instances and criteria are not the same unit. We'll cite it as exactly that rather than round it into an industry law, because the defensible version of the claim doesn't need the decimal place: automation handles a minority, and a person handles the rest.

What a scanner can check (the easy part)

Automated tools are genuinely good at the mechanical, yes or no checks:

• Whether images have an alt attribute at all.
• Contrast ratios it can calculate from the colours.
• Form fields with no programmatic label.
• Certain ARIA mistakes and duplicate IDs.
• Whether the page declares its language.

All real, all worth fixing. But notice that every one of these is a thing a computer can measure without understanding your content.

What it cannot check (the part that decides real usability)

The majority of accessibility lives in judgement calls a scanner can't make:

• Whether your alt text is meaningful. A tool sees that text exists, not whether it actually describes the image. Alt text of image123.jpg passes the machine and fails the person.
• Whether link text makes sense out of context, or whether read more appears fifteen times.
• Whether someone can actually complete a task by keyboard, start to finish.
• Whether a screen reader announces things in a sensible order.
• Whether error messages are understandable, not just present.
• Whether the reading level and clarity work for cognitive accessibility.
• Dynamic content that loads after the scan has already run.

Why this bites you on compliance

WCAGWeb Content Accessibility Guidelines, the technical standard the AODA points at conformance is defined by the human experience, not by a tool's score. You can sit at a shiny automated 100 and still fall short of Level AA, because the criteria that failed are the ones only a person can evaluate. If your accessibility plan is run the scanner, fix the reds, call it done, you have handled the minority and declared victory on the whole.

We use automated tools too. They're a fast, useful first pass, and there's no reason to skip them. But they're the smoke detector, not the inspection. A real audit is a person going deliberately through the part the machine can't see, which is exactly where the experience your customers actually have is decided.