<p><strong>TLDR:</strong> This week’s signal is mostly tooling. A new GitHub Action called Proof Agent separates worker and verifier agents to catch security issues in AI-generated PRs before they merge. CopilotKit dropped aimock, a single-port mock server that covers LLMs, MCP, vector DBs, and agent-to-agent protocols in one package. Netflix shared how they scaled automated dependency updates across 3,000+ apps, and the conversation around testing AI agents in production rather than in isolation kept coming up from multiple directions.</p>
<hr>
<h2>Proof Agent: adversarial verification for AI-generated PRs</h2>
<p>Proof Agent is an open-source GitHub Action that runs a separate verifier agent against every PR diff, independent of whatever agent wrote the code. It checks for SQL injection, hardcoded secrets, XSS, CSRF, and logical correctness. Verification triggers automatically when 3+ files change or sensitive files like Dockerfiles or auth configs are touched. FAIL verdicts block merges and kick off up to three auto-fix retries — PARTIAL verdicts pass but flag issues for human review.</p>
<h2>aimock: one mock server for the entire AI stack</h2>
<p>CopilotKit released aimock, a zero-dependency npm package that mocks LLM APIs (OpenAI, Claude, Gemini, Bedrock, and more), MCP tools, A2A protocols, vector databases, and search APIs — all on a single port with one config file. The standout feature is Drift Detection: a daily three-way comparison between SDK types, real API responses, and mock output to catch silent divergence before it bites you in CI. Record-and-replay and chaos testing (injecting 500s, malformed JSON, mid-stream TCP disconnects) are also included. AG-UI reportedly uses it in production for E2E testing across providers.</p>
<h2>Netflix on scaling automated dependency updates</h2>
<p>Netflix engineers shared how they manage automated dependency updates across 3,000+ applications and 5,000+ repositories. The core pieces are a custom Gradle wrapper with resolution rules and dependency locking, an internal tool called Astrid that tracks which library versions are actually running in production, and a campaign tool for pushing nightly automated PRs at scale. They also built testing scorecards, predictive test selection, and flaky test detection into the pipeline. The honest part of the talk: tooling alone didn’t move the needle — developer confidence and cultural buy-in were what pushed daily deployment rates from under 50% to over 60%.</p>
<h2>Testing AI agents: production keeps winning the argument</h2>
<p>Two separate talks this week landed on the same conclusion from different angles. One argued that traditional test suites fail for agents because outputs are non-deterministic, and that collecting observability traces from real interactions is the only reliable evaluation loop. The other noted that Anthropic’s own research shows models behave differently when they detect they’re being tested — which makes production evaluation not just useful but necessary. The adversarial QA angle also got coverage, with a pattern of deploying “Source Agents” with distinct personas (novice, auditor, malicious actor) to stress-test target agents simultaneously.</p>
<hr>
<h2>Also notable</h2>
<ul>
<li><strong>Cypress AI Skills:</strong> Cypress released instruction sets for AI coding assistants (cypress-author and cypress-explain) that guide tools like Cursor and Copilot to write tests using stable selectors and project-specific patterns rather than generic defaults. Installable via a single npx command.</li>
<li><strong>Playwright token efficiency:</strong> A workshop compared Playwright MCP (~475K tokens), Playwright CLI (~247K), and CLI with skills (~145K) for AI-assisted test generation. Markdown-based “skills” files cut token usage nearly in half versus raw MCP.</li>
<li><strong>Evil Martians on 30k-spec Rails suite:</strong> Whop’s RSpec CI time dropped from 4m30s to under 2 minutes. Culprits included a RuboCop memory leak via a global RSpec hook, Argon2id running at production settings in tests, and SimpleCov overhead fixed by switching to oneshot_line mode.</li>
<li><strong>Self-healing tests:</strong> A post argued self-healing locator tools are a band-aid that lets teams avoid fixing the real problem — developers not communicating UI changes to whoever owns the tests.</li>
<li><strong>data-testid as accessibility smell:</strong> React Status covered the argument that role-based selectors are preferable to test IDs, since test IDs add no semantic value and can mask missing ARIA roles.</li>
<li><strong>zunit:</strong> A zero-dependency, single-file test runner for Java 25 that runs *Test.java files directly via <code>java --source 25</code> with no compilation step, no JUnit, and no build tool. Each test gets its own JVM.</li>
<li><strong>Floci AWS emulator:</strong> A new LocalStack alternative built with Quarkus and compiled to a native binary via GraalVM. Claims ~24ms startup, ~13 MiB idle memory, supports 25+ AWS services, and has passed 400+ SDK compatibility tests.</li>
<li><strong>Rails integration test helpers:</strong> Rails added explicit <code>query:</code> and <code>body:</code> keyword arguments to integration test helpers to resolve ambiguity with GET+JSON requests in API-only apps.</li>
<li><strong>Contract testing ROI numbers:</strong> A post cited schema drift incidents averaging $35,000 each and poor data quality costing $12.9–15M annually per organization, with automated contract testing showing 200% ROI within 12 months.</li>
<li><strong>Co-located tests:</strong> A practical post made the case for putting test files next to source files rather than in a centralized folder — shorter import paths, automatic movement during refactors, and visible coverage gaps without a coverage tool.</li>
</ul>


Testing strategies, CI/CD pipelines, code quality, and test automation.

Testing Digest

A curated roundup of developer tooling news focused on AI agent testing and verification. Proof Agent is a GitHub Action that runs an adversarial verifier agent against AI-generated PRs to catch security issues before merge. CopilotKit released aimock, a zero-dependency mock server covering LLMs, MCP, vector DBs, and A2A protocols with drift detection and chaos testing. Netflix shared how they scaled automated dependency updates across 3,000+ apps using Gradle tooling, an internal tracker called Astrid, and cultural buy-in. Multiple talks converged on the conclusion that production evaluation is necessary for AI agents due to non-determinism and model behavior changes under test conditions. Additional items cover Cypress AI instruction sets, Playwright token efficiency comparisons, a Rails RSpec CI optimization case study, self-healing test criticism, data-testid accessibility concerns, a zero-dependency Java 25 test runner, a LocalStack alternative, and contract testing ROI data.

Proof Agent ships adversarial PR verification, aimock covers the full AI stack