What Jev Means for Software Testing: Your AI Agent Is Thinking Too Hard

Watch an AI agent run a regression test sometime. Really watch it.

It lands on a login page. It reads the entire page structure: every element, every attribute, every nested div someone added in 2019 that nobody has dared to remove since. Then it reasons. It weighs its options. It explains to itself, at some length, that the button labeled "Sign in" is probably the one that signs you in. And then, finally, it clicks.

Now multiply that by every step, in every test, in every regression run, every night.

An experienced tester doing the same thing doesn't deliberate. Their hand is on the button before they've consciously read the label. That's not carelessness. They've done it a thousand times, and their brain has correctly filed it under "doesn't need thinking."

That distinction, between what needs thinking and what doesn't, is where I think the next real progress in AI testing will come from. It's also why September's launch of Jev, a model built for exactly that kind of fast decision, has the testing community paying attention.

Two Kinds of Thinking

Daniel Kahneman gave us the vocabulary in Thinking, Fast and Slow. System 1 is fast, automatic and intuitive: recognizing a face, reading a stop sign, clicking "Sign in." System 2 is slow, deliberate and effortful: working through a tax return, or noticing that an order total is off by exactly the discount that should have applied.

Good testers live in both. Most of a regression run is System 1: navigate, fill, click, confirm, move on. System 2 kicks in when something is off. The page loaded but the balance didn't refresh. The confirmation appeared but the email never arrived. The flow worked, but it felt slower than yesterday. Those are the moments that matter. They're the whole reason a human is in the loop.

Most AI testing agents today don't work that way. They run everything through System 2. Every click becomes a reasoning exercise. Every choice gets the full, wordy treatment from a large language model that was built to write essays and is being asked to pick option B. If agents that run tests on their own are new to you, our guide to agentic QA covers the basics.

It works. It's also slow, and at regression scale it gets expensive fast. There's a subtler cost too: when an agent deliberates about everything with the same intensity, nothing stands out. The "that's weird" moment gets the same attention as the login button.

What Jev Is, and What It Isn't

On September 15, TypeSafe AI released Jev, which they describe as the first "System One model." It doesn't generate text at all. Instead of writing an answer, it picks one from a list you define in advance, and tells you how confident it is.

In testing terms, you hand it two things. The first is a snapshot of where the test is right now: the page the agent is on, the elements on the page that can be clicked or filled in, and the step it's trying to complete. The second is the list of moves it's allowed to make on that page.

Say the step is "Submit the payment." The agent is on a checkout page with five clickable elements: "Pay now," "Save for later," "Edit billing address," "Cancel" and a help icon. You ask Jev which one completes the step, and it comes back with something like "Pay now, 97% confident" in well under a second. No paragraph of reasoning, no essay on how payment flows work. Just the choice and how sure it is. The same pattern works for yes/no questions ("Did the order confirmation appear?") and for scores.

TypeSafe reports it at 40 to 200 times faster than frontier LLMs on comparable tasks, at a fraction of the cost.

Two caveats are worth stating plainly. Those figures are TypeSafe's own benchmarks, and they've said themselves that real-world results will probably come in lower. And a model like this can only choose among the options it's given. It can't generate a new one.

That limitation is exactly the point, though. Jev isn't a smarter model. It's a different kind of model, for a different kind of decision. For a plain-English walkthrough of how it works, see What Is Jev? A Plain-English Guide for Testers.

Where Jev Fits in a Testing Agent

Look closely at what a testing agent does from one step to the next, and a surprising share of it is choosing from a known set. Here's how the decisions sort.

None of the first four decisions need an essay. They need a fast, calibrated choice, and a clear signal when confidence is low. For more on where Jev helps QA teams and where it's the wrong tool, see our Jev for QA post.

The other three still need System 2. Understanding what a test case is actually trying to prove, and finding a new path when the application has changed and the old one no longer works, are reasoning problems. The last one is different in kind: deciding whether the software is right.

That last one deserves emphasis. The call on whether software is fit to release is the most consequential decision in testing, and it's the one you should never rush.

The Real Design Question

So the question isn't "which model is best for testing?" It's "which kind of thinking does each decision need?"

Get that wrong in either direction and it costs you. Pay System 2 prices for System 1 decisions, and your regression suite becomes a very expensive way to click buttons. Let System 1 make System 2 calls, and you get fast, confident answers to questions the model never understood. Anyone who has spent time in QA knows exactly what kind of day that turns into.

Questions Worth Asking Now

If you're evaluating AI testing tools, or building agents in-house, these are worth putting on the table:

1. Which decisions in your agent's loop actually need reasoning? Can anyone show you?
2. What does one regression run cost in model calls today, and what happens to that number when your suite is ten times bigger?
3. When the fast path isn't sure, what happens? Does the system know when to slow down?
4. Where does the final pass/fail judgment live, and is it independent of whatever did the clicking? Our guide to AI testing best practices is a good place to start.

Testers have always known that not every step deserves the same attention. It's taken a while for the tools to catch up. The interesting part is just getting started.