Write the step
“Type {{TEST_USER}} into the 'Email address' field.” Name what is on the screen, not what is in the HTML. Assertions are claims, not commands.
An AI agent reads a screenshot and decides what to click. The first run records ordinary Playwright locators — every run after that replays them with no model involved.
Bring your own OpenAI-compatible vision endpoint. Chromium. Tests commit to git as JSON.
Launching Chromium 1440x896 (Replay mode, headless) ▶ Successful User Login — /testbed/login.html (signed out) PASS Step 1 Clicked via testid locator · Typed "qa_tester@shopsphere.com" PASS Step 2 Clicked via testid locator · Typed "••••••" PASS Step 3 Clicked via testid locator PASS Step 4 Verified via probe: "My Account Dashboard" is visible === PASSED — 1/1 scenarios in 2.8s using 0 model calls ===
That split is the whole design. You pay a vision model once, to work out what to click. You never pay it again to click the same thing.
“Type {{TEST_USER}} into the 'Email address' field.” Name what is on the screen, not what is in the HTML. Assertions are claims, not commands.
The agent looks at a screenshot and acts. From the element it actually hit, the run records a chain of ordinary Playwright locators. 30s–2min per step, once.
--run replay re-runs the recorded locators and exits non-zero on
failure. No model, no API key, no flake budget. Roughly 0.2s per step.
The agent is shown pixels. It cannot read your markup, so it cannot pass a test by finding a hidden node — and no selector is ever written by hand.
| Sees | Does not see | |
|---|---|---|
| The agentperforms steps | Last 3 screenshots, current URL, one line describing its own previous action | The DOM, the accessibility tree, your selectors, the page source |
| The judgeverifies assertions | One screenshot, the current URL | Everything else — including the agent's action history |
A separate judge decides pass or fail, and it must fail unless it can quote text that is on screen. So “visible” means visible: an error message you have to scroll to reach does not count as shown.
Five strategies recorded per element — data-testid → id
→ ARIA role+name → text → CSS path. Rename a testid and the ARIA fallback
usually catches it for zero model calls. Healed steps are flagged, never silent.
One JSON file per page under tests/, recordings included. A weakened
assertion shows up in a diff and can be reviewed. Secret values never commit —
they resolve per machine from env vars or a gitignored file.
Signed out, the run's one shared sign-in, or an isolated login of its own. The third exists so a sign-out or password-change test cannot invalidate the session every other test is using.
Give a test its own preconditions, or point it at another test to run first so a login flow is fixed in one place. References are transitive, deduplicated, and reject cycles.
1–8 browsers pulling from one queue. Replay scales close to linearly. Discover only reaches 1.3–1.6× — every worker queues on the same model server, and the UI says so rather than promising N×.
Per-step screenshots, the agent's reasoning, the locator it used. Optional Playwright video and trace per test. Self-contained HTML and JSON exports that keep working with the app switched off.
Median of 1–5 Lighthouse iterations across three device profiles, a request waterfall and third-party weight breakdown, plus p75 field data from real Chrome users over the trailing 28 days.
--run replay as a build gate, --selftest to check the
model wiring, Docker image with Lighthouse baked in. A cancelled or partial run
reports incomplete — never a green build.
The tool refuses to report a false green. Same standard applies to this page.
A bundled offline testbed ships with it, so there is something to run before you wire up your own app.
Launch platform