Smart Mobile Recorder

This document explains the Smart Mobile Recorder feature in Katalon Studio, how beneficial it is to your workflow, and how to use it.

Starting from version 11.0.0, Smart Mobile Recorder is available in Katalon Studio as a Beta feature.

Overview

The Smart Mobile Recorder in Katalon Studio provides an upgraded experience for recording mobile app interactions. It allows you to capture user live actions, and generate test steps automatically.

Katalon Studio's Smart Mobile Recorder modernizes the existing Mobile Recorder experience while introducing a new Interactive Mode that enables live actions capturing directly from the live device preview. With Interactive Mode, you can review test steps and see the app UI all in one place, making test case creation more intuitive and reducing the manual back-and-forth between recording and debugging.

requirement

• A configured mobile device. See instructions at Mobile Recorder - Device Type.

Key Benefits

• Enhanced usability with a modern UI — Supports dynamic resizing, and object outlining on hover.
• Supports interactive mode with real-time test step capturing, for both Android and iOS.
• Heartbeat mechanism prevents Appium session timeouts during long recordings.
• Pause and Resume capability to control recording flow.
• Auto refresh mechanism to auto-update the device view and object tree every 60 seconds, to capture the latest app state (new screens, updated elements, or layout changes) alongside manual reload option.

Enabling Smart Mobile Recorder (Beta)

To switch to Smart Mobile Recorder:

1. Go to Preferences → Beta Features.
2. Tick the Smart Mobile Recorder (Beta) checkbox.
3. Click Apply and Close.

Once enabled, launching Mobile Recorder opens the Smart Mobile Recorder UI instead of the usual Mobile Recorder. You can toggle off this mode to opt-out anytime.

Get started with Smart Mobile Recorder

Follow these steps:

1. Open Mobile Recorder from the main toolbar.
2. Select a record mode: Local Device, Remote Device, or Custom Desired Capabilities. Cloud Provider will be supported in future phases.
3. Pick your app source (upload an Application file or specify the Application ID). Application ID is the package name for Android apps, or the bundle identifier for iOS apps.
4. Click Start Record.
5. By default, you enter Manual Mode - this mode functions like the legacy Mobile Recorder, with a modern, revamped UI. Turn on Interactive Mode to record actions directly from the live device view.
   • Your interactions will appear in the Recorded Steps panel in real time.
   • Each interaction automatically creates a new Test Object, or update existing one in the current recording session.
6. Throughout the session, you can:
   • Enable auto-refresh to update the device view every 60s.
   • Verify and highlight objects to verify objects via different locators.
   • Capture elements via screenshots, for image locator (see more details at Mobile object locator strategies - Image).
   • Add/Edit/Remove test steps.
7. When done, click Save to save the test script including test steps, test objects and variables (if any).

Smart Mobile Recorder - Feature details

Interactive mode

Interactive Mode lets you capture actions directly on the live mobile preview. It’s fast and visual, ideal for exploratory testing phase.

Supported actions are:

• Tap → Mobile.tap()
• SetText → Mobile.setText()
• Swipe → Mobile.swipe()

We are working to support more actions in this mode.

Key behaviors

• Hovering over elements highlights them. Selecting objects in the hierarchy also highlights them.
• Your interactions with the AUT are instantly captured and listed in Recorded Steps. Test Objects are automatically generated.
• Pause the session if you wish to interact with the AUT without recording your interactions. Resume the session to continue recording.

Global actions

Global actions are available on the left panel in both modes:

Action	Description
Hide Keyboard	Hides the on-screen keyboard if it is blocking elements.
Press	Provides quick-access buttons to perform Press Home, Press Back (Android only), or Press Key Code on the device.
Take Screenshot	Captures the current device screen. Screenshots can be used for future reference or for creating image-based locators (see more details at Mobile object locator strategies - Image).
Switch Context	Switch between Native, WebView, or other available contexts when testing hybrid apps.
Rotate Screen	Rotates the device orientation (portrait/landscape) and updates the device preview accordingly.

Device View vs Screen Objects panel

Smart Mobile Recorder provides two synchronized views of your application under test (AUT): the Device View and the Screen Objects panel.

The Device View shows a live snapshot of your mobile app exactly as it appears on the device. Use this view to:

• Interact with the AUT in interactive mode.
• Confirm that UI updates or transitions appear correctly.
• Validate that the displayed content is what the test step is interacting with.

You can click Detected Objects to view the Screen Objects panel. This panel displays the underlying view hierarchy of the current UI, showing every UI element and its parent–child relationships. Use this view to:

• Verify that an element exists in the expected place in the structure.
• Verify overlapping/repeated elements.
• Inspect parent containers and grouping logic.
• Determine whether multiple similar-looking elements are actually different nodes in the hierarchy.
• Understand how deeply nested components are built behind the scenes.

Both views work together to provide a full understanding of your app's UI, ensuring both visual and structural accuracy.

Heartbeat Mechanism

The heartbeat runs silently while Smart Mobile Recorder is active, sending periodic commands to keep the Appium session alive.

Benefits

• Prevents session timeouts during idle periods.
• Reduces session drops on cloud providers such as TestCloud and BrowserStack.
• Improves stability for long recordings or debugging sessions.

The heartbeat stops automatically when you close the recorder.

Refresh Mechanism

You can manually refresh with the Refresh button, to update the latest device view and object tree. This is helpful when:

• The device UI changes but does not automatically appear in the preview.
• You want to confirm the current state of the AUT before performing new actions.

Optionally, you can turn on Auto-refresh to automatically update the device view every 60 seconds. Auto-refresh only fetches the UI preview, without performing any actions on the device.

Manage Test Steps, Objects, and Variables while you record

While you record, you can switch between Recorded Steps, Captured Objects, and Variables panels.

Recorded Steps panel

All recorded steps automatically appear in the Recorded Steps panel. In this panel, you can:

• Edit, remove, or arrange recorded steps by dragging.
• Manually add new steps.

Adding and editing a step involves editing these fields: Test Object, Input, Output, and Description.

Captured Objects panel

The Captured Objects panel lists all mobile UI elements detected during recording. In this panel, you can:

• Switch between locator types and Highlight & Verify objects on the device to confirm accuracy. This version of KS supports viewing the locators only - editing locators during recording will become available in later versions.
• Select a default locator with the Default button. Default locator will be marked with the bookmark icon.
• Generate an image locator by selecting Image and take a screenshot with the Take Screenshot button.

Variables panel

Manage temporary variables (such as credentials or config values) directly within your recording session.

You can:

• Add, edit, or delete variables during the session.
• Define attributes like type, default value, and description.
• Mask sensitive fields to hide passwords or tokens.
• Link variables to existing project data tables for data-driven testing.

Supported variable types:

• Basic types: string, number, boolean, null
• Global variables:
• Test Object (directory)
• Test data
• Test data value
• Property
• List
• Map