Katalon AI Assistant

This document explains how Katalon AI Assistant works in Katalon True Platform to help QA teams move through planning, execution, and analysis in one place.

This page focuses on capabilities available in Katalon True Platform. To learn about AI capabilities in Katalon Studio, see Katalon AI Assistant Overview.

You can also use Katalon AI Assistant in Katalon Studio to write automated test cases, sync those test artifacts to Katalon True Platform, and then create automated executions from the synced repository.

Overview

Katalon AI Assistant is your virtual testing assistant in Katalon True Platform. Through a single conversational UI, it routes your request across multiple agents and uses platform skills and tools to complete tasks such as:

• Refining requirements
• Generating tests from requirements
• Executing manual tests
• Analyzing failures
• Filing bugs
• Providing project insights

With Katalon AI Assistant, QA teams can move between planning, execution, and analysis workflows without switching tools, using automation to spare human effort for where it truly matters.

Get started

Click the Katalon AI Assistant tab, to open the assistant in full-screen mode. Quick chat options are available to get you started quickly.

Available group options:

• Manual Testing
• Automation Testing
• Quality & Release

Click on any option and a list of key suggested prompts along with requirements appears (e.g. Generate test cases for this requirement > Needs ALM).

Alternatively, click the top left corner button to open the assistant as a side panel. The assistant stays active and is aware of the current page you're on, allowing constant context awareness as you move through tasks.

End-to-end workflow

In one conversation, you can move from refining requirements, to generating tests, executing runs, analyzing failures, filing bugs, and reviewing project health.

To allow full control of the workflow, True Platform extends individual capabilities to dedicated workflows with convenient access points and additional fine-tuning options. Check out our specialized workflows section below for more details.

Specialized workflows

Below are common workflows for using Katalon AI Assistant effectively.

Refine requirements and generate test cases

You can upload a requirement, and work with Katalon AI to refine it, removing ambiguities or add more context. Katalon AI can turn a clear requirement into clear test cases, with a higher chance of it being automated later on (see execute and monitor manual tests workflow below for details).

With Jira or ADO integration, you can also view the requirement in Plans module, and refine the requirement/create tests from it:

1. Click the Release or sprint that contains the requirement ticket. The requirement-test workbench appears.
2. Read Katalon AI analysis of the requirement. Answer the questions, and provide more context with Add more context button. You can specify test data or AUT URL here.
3. Once you're done with clarifying, click Generate Test Cases. Wait while your test cases are being generated.
4. Review the test cases, discard/save, or edit if you'd like. Clicking a test case opens a test case detail drawer that allows two editing options: manual editing each step, and AI-assisted regenerating the steps.
5. Once you're happy with the test cases, click Save All.
6. Result: The test cases are generated, and linked to the requirement.

Best practices

• Provide AUT URL in the requirement.
• Specify test data, if any, in the requirement or when noting context.
• Use either a test requirement or a feature requirement.
• Supported file types include:
  • Documents: .pdf, .doc, .docx
  • Text files: .txt, .md
  • Spreadsheets: .xls, .xlsx, .csv
  • Images: .png, .jpg, .jpeg, .gif

Generate automated test case scripts in Katalon Studio

You can use Katalon AI Assistant in Katalon Studio to write automated test case scripts from prompts, project context, and available test objects. For more details about Studio-side Katalon AI capabilities, check out Katalon AI Assistant Overview.

Execute and monitor manual tests

In addition to creating/scheduling runs from automated test scripts, Katalon AI Assistant can now execute and help you monitor a manual test run so you don't need to stay on the page. During the run, the assistant will ask for your input (if any) via snack bar notifications; otherwise, it will take care of the execution to completion and notify you then.

1. Go to Executions.

2. Click Create > Create Manual Test Run.

3. Select the test suites and provide additional details such as name, sprint or release, and environment.

4. Click Create Test Run.

5. On the test run details page, open the Tests tab and review the AI-confidence status of each test case.

   When you create a manual test run, the test run panel displays test cases in the selected test suite. Each test case has an AI-confidence status with a confidence level of High, Medium, or Low. A suite can be executed with AI only when all test cases have a High status.

6. During the execution, if Katalon AI fails to perform the step, it will ask for your input. Provide a solution to allow the assistant to continue with the test run.

7. If some test cases are Medium or Low, edit the steps to make sure they are as clear as possible. Once all are High in the AI-confidence column, click Run with AI.

8. When the execution finishes, review the status of each test case and the result of each test step.

9. Check the execution carefully. After you confirm everything is correct, click End Run to finish the test run and make the results available for viewing.

10. Review the video, screenshots, and AI log to inspect the execution context and validate each test result.

11. Click End test run to end the test.

Alternatively, you can also prompt the assistant from the chat window:

1. Provide test case IDs, or navigate to the requirement with linked test cases, so the assistant has context.
2. Prompt Create a manual test run from these test cases: [test case ID...].
3. Prompt Is the AI-confidence level high for these test cases?. If the level is medium or low, work with the assistant to improve.
4. Prompt Run with AI to make the assistant run and monitor the test for you. Activate only if the confidence level of all test cases are high.

best practices

• Ask the assistant if the requirement is clear, and iterate many times before creating test cases.
• Always specify the AUT URL in the test step.
• End the run before viewing the final results.

Triage failures quickly

When a test fails, Katalon AI Assistant can analyze the result and provide a concise summary of the likely root cause together with suggested resolution strategies.

You can trigger failure analysis by opening a failed result and click Analyze (see Reports and Analytics Features for more details).

note

• This feature is unavailable for manual tests.
• For automated tests, you can ask Katalon AI to determine if this failure indicates a bug, a test script issue, or a flaky test. For detailed workflows, see Investigate Flaky Tests and Investigate Failures

File bug tickets

If the failure truly suggests a bug in the application, you can prompt Katalon AI Assistant to create a bug ticket in Jira or ADO.

Katalon AI Assistant can draft the ticket from the failed result context, attach screenshots as evidence (fields like videos, reporter, assignee, priority/severity indicators... will be supported in later versions), and link the ticket to the current failure.

1. In Executions, click on a failed test run.
2. Select Tests tab, and click on the failed test case in the run that you want to file a bug ticket. If it's a manual test, make sure that you have ended the test, so its test report becomes available for filing.
3. (Optional) Prompt to ask the assistant if the failure could also be a test case issue or a flaky test. This helps prevent creating false bug tickets.
4. Prompt the assistant to file a bug ticket. It's helpful to specify these details:

• Title, description: If not given, they will be auto-generated from the execution logs.
• Execution Logs: Useful for automated test cases. For manual tets, the execution log is in plain language.
• Images: Ask the assistant to attach images of the execution. This feature will become available for manual test cases in later versions.
• Videos: Will become available in later versions.
• Priority/Severity and custom fields: Will become available in later versions.

best practices

• File 1 bug ticket for 1 test case result at a time.
• Make a ticket structure sample file (in .txt, .md...) that your team often use, then attach to the chat. The assistant will follow the ticket structure to produce a consistent outcome every time.

Analyze project health and readiness

With access to your project's tests, runs, results, and related entities, Katalon AI Assistant can summarize project health and release readiness in natural language.

You can trigger this analysis in two ways:

• Click Insight inside a report or dashboard. See Reports and Analytic - Insights for more details.
• Open the Ask Katalon AI Assistant chat and ask a project-level question directly.

After answering your initial question, Katalon AI Assistant can also suggest follow-up questions so you can continue exploring the same topic in one conversation.

best practices

• Provide the following details for accurate analysis:
  • Data scope (time period, sprint, release, or current dashboard/report)
  • The kind of analysis you want (e.g. readiness, flaky tests, failures, or trends).
• To uncover more targeted insights, open a specific dashboard or report first, then ask a follow-up question about trends or anomalies in that view. For example, in a Test Results Analysis Report, you can ask: Why did failures spike in the second week?

Common analysis topics

Overall analysis	Test patterns	Defects	Coverages
Release readiness Project's health	Flaky test causes Test case health Automation progress Run and result trends Failure trends Test publishing trends	Defects trends Defects status	Configuration coverage Traceability Requirement coverage

Interact with testing entities

Katalon AI Assistant integrates with the Katalon True Platform MCP Server and related tools, allowing you to create, update, move, duplicate, delete, and search for Katalon True Platform entities in the same chat window.

To learn more, see Katalon True Platform MCP Server.