Last updated: September 30, 2026. Jev is in early access and changing quickly. We review this page monthly.
Jev is a new kind of AI model built to make fast, structured decisions rather than write text. For QA teams, that makes it useful for one specific job: the small judgment calls inside a test loop, like "which element should the agent click next?" or "is this failure flaky or a real regression?" It is not a replacement for assertions, and its headline numbers have not been independently verified yet.
This guide covers what Jev is, where it helps testers, where it doesn't, and how to think about it next to the tests you already run. For the bigger argument about matching the model to the decision, read Alex Martins on what Jev means for software testing.
THE SHORT ANSWER
Jev is a "System One" model from TypeSafe AI, released in early access on September 15, 2026. You send it a state (text or JSON) and one or more typed questions. It returns a structured answer with probabilities, not a paragraph.
It answers three kinds of question:
Because the answer must be one of the options you defined, Jev can't return malformed output your code fails to parse [1]. That is a different guarantee from being right, which we come back to below.
If you want the full explainer, read What is Jev? A plain-English guide for testers.
Large language models are good at the parts of testing that need reasoning: drafting test cases from requirements, writing scripts, explaining a failure. They're built to generate text, though, so even a one-word decision comes back as generated text your code has to parse. In a loop that makes dozens of small decisions, that time and cost add up.
Jev is built for those small decisions only. TypeSafe reports end-to-end response times of 70 to 500 milliseconds. Early independent demos are in that range:
Cost is the other half. Jev is priced at $0.042 per million input tokens, and output is not billed. For a team running AI checks on every build, that changes what is affordable to run.
These are the patterns that have public, working examples today. Each one is a bounded decision with a short list of possible answers.
A red build with 40 failures is mostly noise. The open-source jev-ci-triage project sorts each failure into regression, flaky, environment, test data, or unknown. Deterministic rules decide what they can first (for example, a test that failed then passed in the same run is flaky). Jev only sees the ambiguous cases, and anything below 0.75 confidence goes to a human.
If you run Katalon Studio Enterprise, part of this already exists. AI failure analysis reads the logs of failed tests, explains each failure in plain language, and tags it with one of eight categories, including Application Under Test (AUT) Issue, Test Script Issue, Test Data Issue, and Environment Issue. When confidence is low, it falls back to "Others." A Jev step is an option if you want your own rules-first classifier in the pipeline, or a different set of categories.
In both Arbon's web demo and Callstack's mobile demo, Jev read a structured snapshot of the screen and picked the next action from a list.
Some checks are about meaning. Does this error message tell the user how to fix the problem? Is this confirmation text consistent with the order? A Noul question can score that, and you decide what probability counts as a pass.
Classifying incoming defects by component or severity is close to TypeSafe's own headline use case of ticket routing. The Score primitive's documentation example is literally a bug severity scale, from "cosmetic" to "blocking issue; no workaround exists."
LangChain uses Jev to classify risky tool calls and block them before they run. If you let AI agents act on a test environment, the same pattern can stop an agent from deleting data or leaving the environment it should be in.
The pattern across all five: Jev makes a small call, and your code decides what to do with it.
TypeSafe publishes a list of Jev's known weak spots, and several hit common test checks directly:
It also doesn't write tests, summarize runs, or chat. That's still LLM work, and in Katalon Studio it's what the Katalon AI Assistant (Kai) is for.
Jev launched on September 15, 2026, and is still in early access. Here is what a careful test lead should weigh.
The benchmarks are the vendor's. TypeSafe ran its own evaluations, acknowledges possible design bias, and says of its headline speed and cost figures: "we expect that these are on the higher end of real world gains." As of late September 2026, there is no independent benchmark of Jev's decision quality. The first independent comparisons are promising: Lightfield found Jev matched or beat three Claude models on three 100-item text benchmarks without reasoning enabled. Those are general text tasks, though, not QA tasks.
"Can't hallucinate" is narrower than it sounds. A constrained answer can still be the wrong option. As one early review put it, the schema "does nothing to stop the model from picking the wrong option." Arbon's demo shows what that looks like in practice: one normal checkout scored 0.49 for expected behavior and 0.47 for a bug, a near coin flip, and an earlier run falsely flagged a normal add-to-bag.
Answers can change under you. The jev-latest alias moves when a new release ships, "so the answers behind it can change without a change on your side" [5]. For regression testing, that's a silent variable. Pin a specific version.
Page text can attack the model. Jev is susceptible to injected instructions and misleading framing [9]. In testing, the state you send is often page content or log output you don't fully control.
Data and availability. TypeSafe says Jev is not trained on customer requests, and offers zero data retention for enterprise customers. It is a hosted service with published rate limits (1,200 requests per minute) [5]. Decide in advance what your pipeline does when it's unavailable. The safe default is "unknown," never "pass" [6].
The public QA experiments with Jev land in the same place [3][6]: rules and assertions first, Jev only for the calls they can't make. Here's how that maps to a stack you may already run:
| Layer | What it does | Examples | In a Katalon stack |
|---|---|---|---|
| Deterministic tests and assertions | Decide anything with a precise expected result | Exact values, status codes, totals, dates, element state | Katalon Studio test cases, run locally or with Katalon Runtime Engine (KRE) in CI |
| Jev (optional) | Make bounded judgment calls, fast and cheap | Next action, failure category, "did this step work?", severity | A pipeline step or custom keyword you add |
| LLMs | Generate and explain | Draft test cases, write scripts, explain failures | Katalon AI Assistant (Kai), AI agents that draft test cases from requirements |
| Record | Keep evidence of what was tested and approved | Traceability, approvals, audit trail | Katalon TestOps and Katalon True Platform |
Arbon's own recommendation after his demo is the right rule of thumb: "use deterministic assertions wherever the expected result is already precise." Jev earns its place in the gaps between those assertions, where a rule can't decide and a full LLM call is more than the decision needs.
The last layer is the one teams skip. Jev returns an answer and probabilities, not its reasoning [10]. If an AI call labels a failure as flaky and nobody checks, you've lost track of why a test stopped counting. Whatever makes the call, keep a record of what the AI decided, what a person approved, and what changed. That's the problem Katalon True Platform is built around: one system of record for manual, automated, and AI-generated testing, with traceability and approvals. More on that in test management in the AI-driven era.
Nothing in your existing suites needs to change to try Jev. Katalon doesn't have a native Jev integration, and for the patterns that work today you don't need one: Jev sits beside your tests, not inside the test engine.
Your base layer stays the same. Katalon Studio supports deterministic execution for regulated environments and is usable with little to no AI, and KRE runs your suites from the command line in any CI system, inside your own infrastructure. That's the layer to keep as your source of truth, whatever AI you add on top.
Two practical places to try Jev:
Kai and Jev do different jobs. Kai helps you write and understand tests, using the model provider you choose: Katalon's own AI service, OpenAI, Azure OpenAI, Gemini, AWS Bedrock, or an OpenAI-compatible provider. Jev doesn't generate text, so it isn't a Kai provider and isn't meant to be.
Jev is fast, cheap, and structured, and that suits the small decisions inside AI-driven testing. It is also new, vendor-benchmarked, and capable of confident mistakes. Keep your deterministic suite as the source of truth. Then start small: pick one decision, put rules in front of it, pin the model version, and compare Jev's answers against a set of past failures your team has already labeled. If it doesn't beat your rules on that set, you've lost a week, not a release.