Watch an AI agent run a regression test sometime. Really watch it.
It lands on a login page. It reads the entire page structure: every element, every attribute, every nested div someone added in 2019 that nobody has dared to remove since. Then it reasons. It weighs its options. It explains to itself, at some length, that the button labeled "Sign in" is probably the one that signs you in. And then, finally, it clicks.
Now multiply that by every step, in every test, in every regression run, every night.
An experienced tester doing the same thing doesn't deliberate. Their hand is on the button before they've consciously read the label. That's not carelessness. They've done it a thousand times, and their brain has correctly filed it under "doesn't need thinking."
That distinction, between what needs thinking and what doesn't, is where I think the next real progress in AI testing will come from. It's also why September's launch of Jev, a model built for exactly that kind of fast decision, has the testing community paying attention.
The short answer
Where Jev fits in AI software testing, at a glance:
Jev isn't a smarter model. It's a different kind of model, for a different kind of decision.
Daniel Kahneman gave us the vocabulary in Thinking, Fast and Slow. System 1 is fast, automatic and intuitive: recognizing a face, reading a stop sign, clicking "Sign in." System 2 is slow, deliberate and effortful: working through a tax return, or noticing that an order total is off by exactly the discount that should have applied.
Good testers live in both. Most of a regression run is System 1: navigate, fill, click, confirm, move on. System 2 kicks in when something is off. The page loaded but the balance didn't refresh. The confirmation appeared but the email never arrived. The flow worked, but it felt slower than yesterday. Those are the moments that matter. They're the whole reason a human is in the loop.
Most AI testing agents today don't work that way. They run everything through System 2. Every click becomes a reasoning exercise. Every choice gets the full, wordy treatment from a large language model that was built to write essays and is being asked to pick option B. If agents that run tests on their own are new to you, our guide to agentic QA covers the basics.
It works. It's also slow, and at regression scale it gets expensive fast. There's a subtler cost too: when an agent deliberates about everything with the same intensity, nothing stands out. The "that's weird" moment gets the same attention as the login button.
On September 15, TypeSafe AI released Jev, which they describe as the first "System One model." It doesn't generate text at all. Instead of writing an answer, it picks one from a list you define in advance, and tells you how confident it is.
In testing terms, you hand it two things. The first is a snapshot of where the test is right now: the page the agent is on, the elements on the page that can be clicked or filled in, and the step it's trying to complete. The second is the list of moves it's allowed to make on that page.
Say the step is "Submit the payment." The agent is on a checkout page with five clickable elements: "Pay now," "Save for later," "Edit billing address," "Cancel" and a help icon. You ask Jev which one completes the step, and it comes back with something like "Pay now, 97% confident" in well under a second. No paragraph of reasoning, no essay on how payment flows work. Just the choice and how sure it is. The same pattern works for yes/no questions ("Did the order confirmation appear?") and for scores.
TypeSafe reports it at 40 to 200 times faster than frontier LLMs on comparable tasks, at a fraction of the cost.
Two caveats are worth stating plainly. Those figures are TypeSafe's own benchmarks, and they've said themselves that real-world results will probably come in lower. And a model like this can only choose among the options it's given. It can't generate a new one.
That limitation is exactly the point, though. Jev isn't a smarter model. It's a different kind of model, for a different kind of decision. For a plain-English walkthrough of how it works, see What Is Jev? A Plain-English Guide for Testers.
Look closely at what a testing agent does from one step to the next, and a surprising share of it is choosing from a known set. Here's how the decisions sort.
| Test decision | Kind of thinking | What it needs |
|---|---|---|
| Which of these elements is "the submit button"? | System 1 | A fast, calibrated choice |
| Which tool should I use for this step? | System 1 | A fast choice, and a clear signal when confidence is low |
| Did the page reach the expected state, yes or no? | System 1 | A fast, calibrated answer |
| Is this failure the same one we saw yesterday? | System 1 | A fast, calibrated answer |
| Understanding what a test case is actually trying to prove | System 2 | Reasoning |
| Finding a new path when the application has changed and the old one no longer works | System 2 | Reasoning, because a fast model can't generate a new option |
| Deciding whether the software is right | System 2 | Not "did the step complete," but "did the business outcome actually happen?" |
None of the first four decisions need an essay. They need a fast, calibrated choice, and a clear signal when confidence is low. For more on where Jev helps QA teams and where it's the wrong tool, see our Jev for QA post.
The other three still need System 2. Understanding what a test case is actually trying to prove, and finding a new path when the application has changed and the old one no longer works, are reasoning problems. The last one is different in kind: deciding whether the software is right.
That last one deserves emphasis. The call on whether software is fit to release is the most consequential decision in testing, and it's the one you should never rush.
So the question isn't "which model is best for testing?" It's "which kind of thinking does each decision need?"
Get that wrong in either direction and it costs you. Pay System 2 prices for System 1 decisions, and your regression suite becomes a very expensive way to click buttons. Let System 1 make System 2 calls, and you get fast, confident answers to questions the model never understood. Anyone who has spent time in QA knows exactly what kind of day that turns into.
If you're evaluating AI testing tools, or building agents in-house, these are worth putting on the table:
Testers have always known that not every step deserves the same attention. It's taken a while for the tools to catch up. The interesting part is just getting started.