"Agentic" is the word of the year in software marketing. Every product that sends an automated email is agentic now. Every tool with a chatbot interface is powered by AI agents. The term has been stretched to the point where it tells you almost nothing about what a product actually does.
So when Katalon says True Platform has six AI agents that take a Jira ticket through to a filed bug report, it is worth asking what is actually happening under the hood, and how much of the "agentic" label holds up per agent. This article is the honest assessment: what each agent is genuinely doing, where the label is more generous than the reality, and what the system actually needs from your team to work. For the full walkthrough of each agent's setup and workflow, see the full breakdown here.
THE SHORT ANSWER
What makes the system valuable is less the label "agentic" and more the fact that the workflow keeps the same context from one step to the next.
Five of the six agents support an end-to-end workflow from requirement refinement through bug filing. In practice, teams can use them together or separately, with shared context reducing manual re-entry between steps. Human review remains part of the workflow at key checkpoints. The sixth agent, the Report and Insight Generator, sits across the whole pipeline rather than inside it. It reads what the other five agents produced and answers the question teams ask at the end of every sprint: are we ready to ship?
| The Six-Agent Pipeline | ||
|---|---|---|
| Input | Output | |
| 1. Requirement Analyzer | Jira or ADO requirement context, plus uploaded supporting files where needed | Gap analysis + clarifying questions |
| 2. Test Generation Agent | Reviewed requirement + context | Full test cases with steps (highlighted in purple, pending approval) |
| 3. Autonomous Test Runner | High-confidence manual test run | Pass/fail + screenshots + video + AI log |
| 4. Root Cause Analyzer | Failed test result | Bug / test script issue / flaky classification |
| 5. Bug Reporter | Confirmed bug classification | Filed Jira or ADO ticket with execution logs and screenshots |
| 6. Report & Insight Generator | All agent outputs across the pipeline | Release-readiness summary and risk-based recommendation |
The order of the first five is not incidental. Each agent's output is designed to feed the next one. Strip one agent out and the chain weakens.
For the setup steps, screenshots, and best practices for each agent, see the full breakdown here. What follows is the assessment: is each agent genuinely agentic, or fast automation wearing the label?
1. Requirement Analyzer: Reads the ticket or an uploaded PDF, DOCX, spreadsheet, or image before a single test is written and surfaces missing acceptance criteria, contradictory conditions, and untestable requirements with ticket-specific questions. This is real analytical work. Its main limitation is that it works best when requirement context is provided directly in the ticket or uploaded into the workflow, rather than assumed to be available through external linked documents.
2. Test Generation Agent: Turns a reviewed requirement into a minimum-viable set of test cases with steps included, flagged purple until you approve them. Fast and consistent, but generating test cases from a spec is something LLMs have done well for a while - the real value isn't the generation, it's that the output lands pre-linked to the requirement in your test management system with traceability built in.
3. Autonomous Test Runner: Reads natural-language test steps and executes the corresponding browser actions directly, gated by an AI Confidence rating - a suite only runs with AI once every case in it is rated High. This is the most genuinely agentic piece of the six: it's interpreting instructions and acting on them, not replaying a script, and the confidence gate has real teeth rather than being a soft suggestion.
4. Root Cause Analyzer: Classifies a failed result as an application bug, a test script issue, or a flaky test, with a plain-language explanation. Practically useful rather than technically flashy - the classification isn't always right, but it's right often enough to be the first thing worth checking. Important scope note: failure analysis is unavailable for manual tests. The Root Cause Analyzer covers automated test failures only.
5. Bug Reporter: Drafts a bug ticket from execution context and screenshots and files it directly to Jira or ADO. Its current value is speed and traceability: it creates a strong starting ticket, while richer fields and evidence are still evolving.
6. Report & Insight Generator. Synthesizes everything the other five agents produced into a plain-language release readiness answer, with a release-readiness verdict based on the thresholds and quality signals you configure. The most strategically valuable agent and the most dependent on the others - its signal is only as good as the classifications and coverage data feeding it, and it knows nothing about support tickets or in-flight fixes outside the test suite.
The through-line across all six: the value isn't any single agent, it's the shared context. The Test Generation Agent's output is already what the Runner needs to execute it. The Runner's log is already what the Root Cause Analyzer reads. The Root Cause Analyzer's classification is already what the Bug Reporter files. Individual AI features are easy to build. An agent that picks up exactly where the last one left off, without you moving data between tools, is the actual thing worth paying attention to here.
Katalon's own guide to agentic QA describes the category as a four-phase loop: Analyze and Plan, Generate and Create, Execute and Observe, and Adapt and Heal. It's worth checking the six-agent pipeline against that same framework, since it's the definition Katalon is using to make its own case.
The first three phases map cleanly. Requirement Analyzer and Test Generation Agent cover Analyze/Plan and Generate/Create. Autonomous Test Runner and Root Cause Analyzer cover Execute/Observe, including the triage step the framework calls out specifically - is this a real bug, a maintenance issue, or a flake?
Phase four turns out to be a real capability - just not where the six-agent pipeline lives. Self-healing exists as a Katalon Studio capability, not a True Platform agent.
It's also a fairly mature version of the feature where it does live. Studio tries a prioritized list of alternative locators first (XPath, Attributes, CSS, Image, Smart Locator), and only if all of those fail does an LLM step in to read the page and propose a new one. Every suggestion surfaces for human approval before it becomes the new default - it doesn't rewrite tests silently.
The catch for this pipeline specifically: self-healing is locator-based and tied to scripted automation in Studio, not to the Autonomous Test Runner's natural-language manual execution in True Platform.
The agents are only as good as what they read. A vague Jira ticket produces vague test cases. Vague test cases produce Medium or Low confidence ratings. Those ratings stop the autonomous runner before it starts, since a suite only runs with AI once every case in it is rated High. And if the runner does not run, the Root Cause Analyzer has nothing to analyze, the Bug Reporter has nothing to file, and the Report and Insight Generator has no data to synthesize.
This is not a flaw in the system. It is the system doing exactly what it should. The agents create pressure to write better requirements and cleaner test steps, which improves your test suite independently of whether the autonomous runner ever gets used.
Teams that go through this workflow once tend to write tickets differently afterward - not because someone mandated a new template, but because the Requirement Analyzer asked the same question three times in a row and eventually someone put the answer in the ticket description.
Current limitations are important to understand upfront. Requirement analysis works best when relevant context is provided directly, rather than assumed to be accessible through linked external systems. Root Cause Analyzer currently applies to automated test failures, not manual test runs. Bug Reporter focuses on core execution context and screenshots today, with richer metadata and evidence continuing to mature. And Autonomous Test Runner depends on sufficiently clear, high-confidence test steps before AI execution can proceed.
None of these make the pipeline less useful. They do make "fully autonomous, end to end" the wrong way to describe it today. "Autonomous within defined boundaries, with human review at key checkpoints" is the more accurate pitch - which, per the platform's own governance framing, is also the point.
For the Test Generation Agent and Bug Reporter, less than the marketing implies - both do things language models have handled well for a while, and their value is the integration, not the underlying AI task. For the Autonomous Test Runner, yes: interpreting a natural-language step and deciding what action it maps to on the current page is a different mechanism than replaying a recorded script.
Every agent in this pipeline produces a draft or a proposal, not a final action - test cases wait for approval, bug tickets wait for review, and the Runner asks for user input when it cannot confidently complete a step, rather than guessing and continuing silently. The failure mode built into the design is "ask a human," not "proceed anyway."
No, and not because of a marketing line - because several of the judgment calls in this pipeline are explicitly left to humans: which edge cases matter for this sprint, whether a generated test case reflects actual product risk, and whether a release-readiness recommendation accounts for context outside the test suite. The agents remove mechanical work, not the decisions.
No. The docs explicitly state that failure analysis is unavailable for manual tests. If your team runs primarily manual tests through the Autonomous Test Runner and plans to rely on Root Cause Analyzer for triage, that gap is a real planning consideration today, not a question to follow up on with your Katalon contact.
For a deeper look at what changes when an organization actually commits to agentic testing - not just the tooling, but the team structure and the definition of QA work itself - Huy Tieu's interview on the evolution of agentic QA is worth reading alongside this one. And if you are still working out whether agentic QA is the right frame for where your team is headed, that is the place to start.
📚 Read more: Building a Playwright AI Agent: How Claude Code Drives Katalon True Platform End to End