Trust is our foundation

Katalon TrueTest is designed to help quality assurance (QA) teams, developers, and product managers accelerate test creation and improve test coverage. By observing real user interactions in a production or pre-production environment, TrueTest automatically generates robust, maintainable automated tests. This is intended to:

• Generate automated tests: Automatically create test scripts and objects for Katalon Studio based on captured user journeys.
• Identify coverage gaps: Highlight business-critical user flows that are not currently covered by existing test suites.
• Accelerate test maintenance: Simplify the process of updating tests when the application UI changes.

TrueTest uses a combination of a data-capturing agent and a sophisticated AI processing backend.

• Capture: The TrueTest agent (installed as a browser extension or code snippet) observes user interactions within your web application. It captures the underlying structure (DOM) and events of each user session.
• Anonymize & filter: At the source, the agent automatically filters and anonymizes sensitive data to protect user privacy before any data is sent for processing.
• Process: The anonymized structural data is sent to the Katalon Platform, where it is used to construct a prompt. This prompt is then sent to a secure instance of a third-party Large Language Model (LLM).
• Generate: The LLM processes the prompt and generates a series of test steps and corresponding test objects. This output is returned to the Katalon Platform and made available in your TestOps environment as a ready-to-use automated test case.

The following diagram illustrates our multi-layered approach to data protection during the AI processing lifecycle. Your sensitive data is explicitly filtered and anonymized before any interaction with third-party models.

• Zero data retention: We have a contractual, zero-retention and no-log agreement with our AI sub-processor (Microsoft Azure OpenAI). Customer data submitted for processing is never stored, logged, or used to train or improve third-party AI models. The processing is ephemeral and stateless.
• Proactive data filtering: The TrueTest agent is designed to automatically detect and filter common sensitive data types at the source using regular expressions (regex). This includes Social Security Numbers, emails, IP addresses, phone numbers, credit card information, and passwords.
• Customer-defined exclusions: You have the control to manually define specific UI elements to be excluded from data capture by adding a simple CSS class to your application's code, ensuring sensitive or proprietary information is never captured.
• Bring Your Own Key (BYOK): For customers with stringent data sovereignty requirements, TrueTest supports a BYOK model, allowing you to route all AI processing through your organization's own private instance of Microsoft Azure OpenAI.
• Complete customer control: AI features are optional. Your organization's administrators have full, granular control to enable or disable AI services at any time.
• Data ownership: You retain ownership of your data. Per our terms of service, any inputs provided to our AI products and any outputs generated by them are considered "Your Data."

• Like all AI systems, TrueTest is a tool to augment human expertise, not replace it.
• Contextual understanding: The AI may not always grasp the full business context or intent behind a complex user flow. Tests generated by TrueTest should always be reviewed by a human QA professional to ensure they align with business requirements.
• Complex dynamic elements: While robust, the AI may occasionally struggle with highly complex, non-standard, or dynamically generated UI elements. Some manual adjustment in Katalon Studio may be required in these edge cases.
• Hallucinations: In rare cases, the AI may generate a test step that is not logical or does not correspond to a real user action. We mitigate this through prompt engineering and by ensuring generated tests are easily editable in Katalon Studio.