A full-stack engineer built an AI-driven E2E regression suite for Malaysia's three public open data portals using Passmark, an open-source Playwright library where tests are written in plain English. The key insight is range-bounded assertions that verify whether dashboard numbers are semantically correct (e.g., population between 20M–40M), catching data-pipeline bugs that selector-based tests miss entirely. The post covers cross-field math assertions (e.g., male + female = total population), a debugging loop driven by Passmark's natural-language failure messages, and honest caveats including a 14% cache-hit rate, OpenRouter dependency, and two-model voting blind spots. Over three runs, pass rate improved from 31% to 92% without writing a single selector.
Table of contents
Table of ContentsWhy Malaysia's Open Data Portals?What Is Passmark?The Hero Spec: Range-Bounded AssertionsGoing Further: Cross-Field MathWhat I Found Across Three RunsWhat It Cost, and Why Cache Rate Is Cost RateThe Pattern Worth StealingHonest VerdictResourcesSort: