AI-generated code is failing in production at teams that feel confident about it. The cheapest place to catch the problem isn't the pull request or the release gate. It's the spec, before anything gets generated.
THE SHORT ANSWER
Most teams check AI-generated code after it's written: at review, at commit, or before release. But when an AI builds from a spec, a wrong assumption in that spec flows straight into the code and into any tests the AI writes for it. Everything passes, and the software is still wrong. AI code validation should start earlier, with a person confirming the spec reflects what the business actually intends, and with acceptance tests written from that spec independently of the AI that builds it.
Here's a story I've heard, in one form or another, from more QA leaders this year than I can count.
A ticket says: "Users can export their invoices." An AI coding assistant picks it up. It builds a clean CSV export, writes unit tests for it, and every test passes. Code review looks fine. The release goes out on time.
Two weeks later, finance asks where the PDF export is. The one with the tax breakdown their auditors need. That's what "export" meant to everyone who wrote the ticket. It just never made it into the ticket.
Nothing in that pipeline failed. The code did exactly what the spec said. The tests confirmed exactly what the code did. Every check was green, because every check was measuring the code against the same incomplete spec.
That's a validation failure, and it happened before a single line was generated.
SmartBear just published its 2026 State of Software Quality and Testing study, based on almost 1,500 technology practitioners and senior leaders, mostly in the US and UK. It's worth reading in full. A few findings stood out to me.
46% of teams have shipped AI-generated code that failed once it reached production. And 69% of those same teams say they have a lot of or complete confidence that their AI-generated code behaves as intended.
Read that twice. Most of the teams that have watched AI code fail in production still trust it.
The confidence isn't evenly spread, either. 73% of leaders report a lot of or complete confidence in AI-generated code, compared with 52% of practitioners. The further you sit from the code, the better it looks.
Then there's where the checking happens. According to SD Times' coverage of the report:
| Stage | What teams report |
|---|---|
| Before generation | 46% check that the spec reflects intent before AI generates from it |
| Before commit | 49% independently review more than 60% of AI-generated code |
| Before release | 60% independently test more than 60% of AI-written code |
These three questions measure slightly different things, so I wouldn't stack them as a funnel. But the direction is hard to miss. The closer we get to release, the more checking we do. The earliest point, the one where a bad assumption costs the least to fix, gets the least attention.
That's backwards. And I don't think it's because QA teams don't know better. I think it's because nobody's job description covers it.
For most of my career, an ambiguous requirement wasn't a disaster. It was a conversation. A developer would read "users can export invoices," frown, and ask at standup: "Export to what? For whom?" Someone would answer. The gap got filled with judgment, and usually nobody wrote that down.
AI doesn't frown. It doesn't ask at standup. It reads the ticket, fills the gap with the most plausible guess, and builds that guess quickly and confidently. Then, if you let it, it writes tests that confirm its own guess.
That last part matters. Only about a quarter of SmartBear's respondents think it's always appropriate for the AI that wrote the code to also test it. I'm with the other three quarters. When the same system writes the code and the tests from the same spec, you don't have two checks. You have one assumption, recorded twice.
So when an AI builds from a spec, the spec stops being a starting point for a conversation. It becomes the program. Whatever's missing from it will be missing from the software, and from every test generated alongside it.
Think about who touches a ticket on its way to production.
Product writes it. A developer, or now an agent, turns it into code. QA usually arrives after that, with review, automation, and regression. Leaders see the outcome in a dashboard that summarizes all of it.
Now ask: whose job is it to confirm the ticket is fit to generate from? Product thinks it's done when the ticket is written. Engineering assumes it's right because product wrote it. QA isn't in the room yet.
That seam between intent and generation used to be covered by human judgment that nobody planned for. AI has removed that judgment and made everything after it faster. So the gap hasn't gotten bigger. It's just gotten more expensive.
It also explains the confidence gap. Leaders aren't wrong about what they see. They're looking at green checks and on-time releases. Practitioners are closer to the evidence, and they can see the checks were measuring the wrong thing.
Here's the good news. Test Leads have been doing this for decades. Requirements-based testing, reviewing a requirement for testability, writing acceptance criteria before development, tracing tests back to what the business asked for: that's core QA work. We've just tended to do it alongside development, not in front of it.
With AI in the loop, that work needs to move to the front of the line. Concretely, these are the five checks I'd want on any spec before an AI generates from it.
If the only full statement of what you want is the prompt someone typed into a coding assistant, the AI is both the interpreter and the judge. Intent should live in a requirement a person owns, that's versioned, and that someone other than the AI can read.
This is the simplest test of a spec I know. If you can't say what passing looks like before the code exists, the AI can't either. It'll just decide for you. Writing acceptance tests from the requirement, independently of whatever generates the code, gives you something to check against that doesn't share the AI's assumptions.
Specs are good at describing the happy path. They're bad at saying what must never happen: never expose another customer's invoice, never drop the tax line, never change the format finance's systems import. People used to supply those from experience. AI won't unless you tell it.
Not "the team." A person. That might be the product owner, the Test Lead, or both. What matters is that someone with context has read the spec and said, "Yes, this is what we mean." That's the check only 46% of teams are doing today.
When something fails in production, the first question is always "what were we trying to build?" If the requirement, the tests written from it, the runs, the approvals, and the release are connected, you can answer that in minutes. If they're spread across a ticket, a chat thread, and someone's memory, you're guessing. And SmartBear found only 50% of teams can fully track AI output for audits.
None of this slows delivery down much. A spec check takes minutes. Finding the invoice problem in production took two weeks and an unhappy finance team.
Moving validation earlier doesn't replace code review or release testing. It makes them worth more.
When a reviewer at commit, or a tester before release, has acceptance tests that came from a confirmed spec, they're checking the code against intent, not against itself. The late checks finally have something independent to measure against.
And the confidence gap starts to close for a simple reason: leaders and practitioners are looking at the same record. A leader's confidence can rest on "here's the requirement, here's who confirmed it, here are the tests and results, here's who approved the release" instead of a summary of a summary.
That's the difference between confidence and evidence. Confidence is how you feel about the release. Evidence is what you can show someone who wasn't in the room.
This is why we're building Katalon True Platform as the system of record for quality: one place connecting requirements, test cases, execution, evidence, defects, approvals, and releases, whether the work was done by a person or an AI agent.
In practice, that means AI agents can draft test cases directly from requirements in Jira or Azure DevOps, and the platform can flag when a requirement needs clarification before generating. Every generated test case stays linked to its requirement, and a person reviews it before it's saved. The AI does the drafting. People decide what counts.
I've written before about what QA teams should expect from a test management platform in the AI era. If you want to see how this would work for your team, talk to our team. And if you want the bigger picture on why validation has to stay independent of the AI doing the work, I wrote about that in The agent passed its own test. That's the problem.
If the answers are "nobody," "alongside," "no," and "no," you're in good company. Over half the industry is in the same spot. But it's a gap QA is better placed than anyone to close. We just have to start earlier.