LLMs Are Proving That It Is Impossible to Automate Away…

This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).

Automated accessibility tools and LLMs fall far short of replacing human specialists in web accessibility auditing. Standard scanners miss 50–70% of real barriers, and a 2025 study found that leading LLMs (GPT-4o, Gemini, Llama 3) detected only 30–38% of what traditional scanners find — meaning an LLM-only workflow may surface as little as one in ten actual accessibility issues. LLMs also hallucinate violations, citing non-existent WCAG criteria and ARIA attributes, creating a credibility problem on top of a coverage problem. Even when violations are detected, fix code is rarely compilable without manual intervention. The core argument is that true WCAG compliance requires lived disability experience and real screen reader testing (NVDA, JAWS, VoiceOver) — something no algorithm can replicate. Automated tools are a valuable first layer, not a final one.

11m read timeFrom 8thlight.com
Post cover image
Table of contents
The Gap is RealWhat Automation Does WellWhat Automation InventsWhat Automation MissesWhy the Gap Is Unfixable Without Lived ExperienceWhat a Real Specialist BringsAccessibility Initiatives in the Real World
1 Comment

Sort: