AI web testing for QA teams: write the test in English, let an agent run it

You write a test the way you'd describe it to a teammate: "open the homepage, click 'learn more', verify the url contains /about". assertly hands that to an AI agent, which translates the intent into concrete browser steps and runs them in a sandboxed headless browser — Playwright under the hood — then reports back pass or fail, the execution log of what it did at each step, and a screenshot captured at the point of failure so you can see exactly what the page looked like when it broke.

The difference from a recorded or hand-coded script is where the brittleness goes. A static script stores "click the element at button.cta-primary > span". The agent stores "click 'learn more'" and resolves that against the actual page each run. So when the markup shifts under it — a renamed class, a reordered DOM, a new wrapper — the agent still finds the right thing, and your test stays green because the user-facing behaviour didn't change. That's the whole wedge: the test describes intent, not implementation.

The clearest win is regression coverage on the flows a team re-checks constantly: signup, login, checkout, search, the contact form. Those are exactly the flows that get touched by every redesign, so they're exactly the static tests that break most. Describing them in English once, then re-running on demand, turns a maintenance liability into a few saved tests you can fire whenever a change lands.

The second win is reach. Plenty of teams have flows that nobody ever automated because writing the Playwright was more work than it was worth — an internal admin screen, a rarely-touched settings page, a marketing landing route. Plain-English authoring lowers that bar enough that a product manager or designer can write the test themselves, no framework, no selectors. assertly keeps a short run history per saved test, so a flow you wrote in a slow week is still there, one click from a fresh run, when you need to confirm a release didn't disturb it.

AI web testing is not a silver bullet, and pretending otherwise is how teams get burned. assertly is web-only for the MVP: it drives a browser, so it doesn't test native mobile apps or raw API endpoints. It has no CI/CD integration yet — you run tests from the app, not from a pipeline gate (that's on the roadmap, not in the box today). And it is not a record-and-playback tool; you author intent in words rather than clicking through a recorder.

The most important caveat is judgement. An agent that resolves intent flexibly is a feature for surviving cosmetic changes and a risk for masking real ones — if a flow genuinely changes, you want the test to fail, not to cleverly route around the problem. So treat generated steps the way you'd treat any generated code: review them for your critical paths, watch the failure screenshots, and keep a human in the loop where a missed regression would actually hurt. Used that way, the agent absorbs the maintenance grind while you keep the calls that matter.