Katalon TestOps Terminology
Release
Represents a version or a milestone of a software product, which consists of one or more Test Cases.
Usage
Manages your test cases and testing progress by milestone. With Katalon TestOps Release, you can view the overall performance and readiness of the software product to be deployed with confidence.
Releases can be managed in Test Planning section.
Example
Test Run
Consists of one or more test case, depending on whether it is conducted parallelly, to be scheduled or executed.
Usage
Gives you the related information of running Test Cases. A Test Run will keep track of the test result, when the test was ran, Test Run status, duration, assignee, etc.
Test Runs can be planned in Test Planning section.
After being executed, you can view their results in Reports & Analytics section.
Examples
Trigger
By when the Test Run should be executed. (A Trigger can be an Event (e.g. from Github or Jira), or a time interval).
Usage
Katalon TestOps provides you an ability to determine when the Test Run is executed so that you can actively plan your testing procedure.
To create a Trigger, go to Test Planning > Test Run Types > Select the one you want to create a Trigger for > Click "Create Trigger" button.
Example
Test Environment
In what machines the Test Run should be executed. If a Test Environment is a local machine, it has to be controlled by an Agent.
Usage
Used to execute test cases along with Script Repository after being configured. Katalon TestOps allows you to create a Test Environment with the following options:
- Your local machine
- Kubernetes environment
- CircleCI environment
Test Environments can be managed in Configurations section.
Example
Script Repository
Stores the actual test automation scripts to be executed.
Usage
Gives you a space to manage all test scripts and decide which one to be executed along with the given Test Environment. In Katalon TestOps, Script Repository can be a Git repository or a ZIP file.
Script Repository can be managed in Configurations section.
Example
Test Run Type
A collection of Test Runs having the same configurations.
Usage
Gives you an ability to view the list of Test Run with similar configurations (Test Environment, Trigger, Script Repository) so that you can better manage your Test Runs.
You can find Test Run Types in Test Planning section.
Example
Test Session
Being created once a Test Run is executed. Each Test Run can be executed as one or many Test Sessions depending on whether it is conducted parellely.
Usage
Gives you information after executing a subset of the overall Test Run. In Katalon TestOps, each Test Session contains:
- One or more Test Cases
- Test Environment and test scripts in Script Repository used for the execution of each Test Case
To view Test Session, in Reports & Analytics section, go to Test Runs > select a Test Run you want to view > Test Session tab.
Example
Test Result
A result of a Test Case after being executed.
Usage
Gives you detailed results after a Test Case is executed from Katalon Studio (or other apps). Usually, Test Results are organized into one or more Test Suites and can link to one or more Defects to track bugs and failures.
To view Test Result, in Reports & Analytics, go to Test Runs > select one Test Run to view details. Here you can view all details of the Test Case after being executed.
Example
Defect
Represents a Jira Issue linked to a (failed) Test Run in Katalon TestOps.
Usage
Gives you an oveview of performance of Test Runs by Jira issue so that you can have full control and awareness of any problem for better test maintenance.
You can find Defects in Test Management section.
Learn how to link Test Runs to Jira Defects here.
Example
Assertion
A statement that a predicate (Boolean-valued function, a true–false expression) is expected to always be true at that point in the code. Assertion is the validation step that determines whether the automated Test Case succeeded or not.
Usage
Katalon TestOps provides you a specific view of Assertions in each Test Case. This used to evaluate the quality of Test Case and ensure that your application is working correctly by checking whether a condition is true, i.e., whether labels, data, API responses, etc., are rendered correctly.
Assertion statistics of each Test Case are displayed in the list of Test Cases from Test Management.
To view details of Assertions, in Reports & Analytics, go to Test Runs > select a Test Run to view details. Here you can view the list of Assertions with status, errors and logs.
Example
Requirement
Represents a Jira Issue linked to a Test Case in Katalon TestOps.
Usage
Gives you an oveview of performance of Test Cases by Jira issue so that you can oversee the Test Case quality for better test performance.
Requirements can be managed in Test Management section.
Example
Test Case
Has multiple Test Results, usually when the Test Case is a data-driven one. A Test Case can link to one or more Requirements.
Usage
Ensures if different features within an application are working as expected. Katalon TestOps provides you detailed information of Test Cases that helps your team validate if the software is free of defects and if it is working as per your expectations. You can also link Test Cases to Jira Requirements to ensure good test coverage.
You can find Test Cases in Test Management section.
Example
Test Suite
A collection of Test Cases.
Usage
Organizes Test Cases for better management and maintenance.
You can find Test Suites in Test Management section.
Example
