Skip to content

The test agent

Write what should happen. The agent does it on the phone.

You describe the test in a plain sentence. The agent opens your app on an Android phone, looks at the screen, decides what to do next on its own, and tells you whether the result was right.

AI Instructions

AI

PROMPT

Add the running shoes and the water bottle to the cart. Type SAVE10 into the coupon field and apply it. The subtotal and the total should both drop by 10.

FLOW PORTS

Success
Error

One sentence

What a sentence turns into on the device

You write the goal. The agent chooses the actions, and every action it took is in the report afterwards with the time it needed.

An AI Instructions box on the TestSafe scenario canvas. It says: Add the running shoes and the water bottle to the cart. Type SAVE10 into the coupon field and apply it. The subtotal and the total should both drop by 10. Below it are the Success and Error outputs.

Every box has two exits, Success and Error, so you decide what the run does when a step does not go the way you wrote it.

What the agent did

  1. Start appcom.shopgo.android1.2 s
  2. Find and tap“Add to cart”, on two items1.4 s
  3. Find and write textcoupon: SAVE100.9 s
  4. Wait for the screenthe cart recalculated0.6 s
  5. Read element texttotal: ₺998.000.4 s
  6. AI Verificationexpected ₺988.00, the coupon never reached the total0.7 s

The boxes

You build a scenario from boxes you connect. They do not all do the same job.

A scenario is built from boxes you connect. These are the ones that think for themselves.

  • Whole run

    Autonomous device agent

    You write what should be tested and it builds the rest of the run itself.

  • Instruction

    AI Instructions

    Carries out the goal in the prompt on the device by itself.

  • Check

    AI Verification

    Decides whether what you expected actually happened.

  • Read

    AI Extraction

    Reads a value off the screen and hands it to the next step.

  • Branch

    AI If

    Splits the flow in two based on what it sees.

  • Repeat

    AI Loop

    Keeps doing the same work for as long as a condition holds.

Autonomous Device Agent

The strongest box: it builds the whole test for you

Put one Autonomous Device Agent box in the scenario and write what to test in a single sentence. The agent does the rest. It adds the other boxes itself, connects them, changes or deletes them when needed, and runs the test.

One sentence

Search for Classic Hoodie and add it to the cart in size XL. The app closing is a bug.

  1. 01

    Looks at the screen

    Reads which screen the app is on right now.

  2. 02

    Plans the test

    Writes down up front, as criteria, what has to happen for the test to pass.

  3. 03

    Builds the boxes

    Adds a box for each step and connects them in order.

  4. 04

    Runs and fixes

    Runs the chain. If a step fails, it changes or deletes the box and tries again.

The Autonomous Device Agent step in a TestSafe report. The task is a single sentence. Above it are the six boxes the agent built itself, below it each criterion with its result and its moment in the video.

Pinned AI Instructions

Pinned AI Instructions learn the job once, then run the same steps again

Put a Pinned AI Instructions box in the scenario instead of AI Instructions, and the agent uses the model only on the first run. It saves the steps it took under the box. Later runs play those steps back directly, so runs are faster and use far fewer AI tokens. If a step fails, the model steps back in.

  1. 1First run

    It learns

    The agent does the job with the model and writes the steps it took under the Pinned AI Instructions box.

  2. 2After that

    It replays

    The saved steps run again with no model in the loop. Same path, a fraction of the time.

  3. 3If a step runs into a problem

    The model wakes up

    Only for that step. It repairs it and the run carries on.

Boxes that work the same way

  • Pinned AI Instructions
  • Pinned Validation
  • Pinned AI Extraction
  • Pinned Definition Agent

Pinned Validation works the same way. When a check fails, the model decides whether the expectation really broke or the check simply went stale.

Effort

You choose how deeply the agent thinks

A sign-in screen does not need what a crowded game level needs. Leave it on automatic and the agent sets it per scenario.

  • Automaticset per scenario
  • Simple0.5x
  • Medium1x
  • Hard2x
  • Very hard5x

When it has to be exact

Ninety-two ready steps sit in the same flow

Some things you do not want decided for you. Those you place by hand, next to the AI boxes, in the same scenario.

92 ready steps

Click a group to see its boxes as they look in the product.

System Settings25

  • Phone Call

    Rings the device, answers the ringing call or ends it.

  • Control Camera Playback

    Pauses, resumes, restarts or stops the media playing on a camera.

  • Route Playback

    Starts, pauses, resumes or seeks the route that was loaded.

  • Read Camera Frame

    Takes the picture the camera is producing right now as a frame and changes nothing.

  • Read Device Cameras

    Lists the device's cameras, what they are doing and the media files they can play.

  • Read Network Conditions

    Reads which connection the device is using and the speed limits in force.

  • Receive SMS

    Makes the device receive an SMS as if it came from the network.

  • Refresh Sensor List

    Makes the device and the running apps read the sensor list again.

  • Reinsert SIM

    Takes the SIM of a slot out and puts it back so the identity is read again.

  • Set Device Battery

    Sets the device battery level, charging state, health and temperature.

  • Set Device Bluetooth

    Turns Bluetooth on or off and sets the nearby devices and pairing behaviour.

  • Set Device Camera

    Plays an uploaded image or video on one of the device's cameras as if it were the live view.

  • Set Device Mobile Network

    Sets the reception level, network generation, roaming and SIM card presence of a slot.

  • Set Device Display

    Sets the device text size, screen density and resolution table.

  • Set Device Language

    Sets the device system language and time zone.

  • Set Device Location

    Moves the device GPS position to the given coordinates and optionally makes it travel.

  • Set GPS Reception

    Sets how well the device receives GPS and whether location is allowed at all.

  • Set Device Mobile Data

    Turns the device mobile data connection on or off.

  • Set Network Conditions

    Limits the device's real internet speed, latency and packet loss.

  • Set Device Sensors

    Sets how the device is held, what the user is doing with it and the screen rotation setting.

  • Set SIM Identity

    Sets what is written on the SIM card of a slot and re-inserts the card.

  • Set SIM Operator

    Sets which mobile operator the SIM of a slot belongs to.

  • Set Device Wi-Fi

    Sets the Wi-Fi radio, reception level and which network the device is joined to.

  • Teleport Device

    Moves the device to the given coordinates at once without saving it into the environment.

  • Play Gesture

    Plays a one time physical gesture, such as a shake or raising the device to the ear.

Traffic capture puts your app's HTTPS traffic in front of you, alongside the rest of the test.

Workflows

Group scenarios and run them in order with one click

The flows you check before a release go into one workflow. Sign-in, cart, checkout, profile. They run one after another on the environment you pick. The list shows each workflow's last ten runs, its success rate and how many tests broke in the last run.

NameLast 10 runsSuccessWhen it runsLast run
  • ShopGo pre-release90% last 10ManualPassed2 hours ago · 6/6 passed
  • RideGo sign-in and ride50% last 6ManualFailedyesterday · 2 tests failed
  • Minesweeper level run100% last 3ManualCancelled3 days ago · 2/3 passed

You start a workflow with one click. It can also run on its own on the days and hours you pick, or when a new version of your app is uploaded. See the integrations

Let's write your first scenario together.

Tell us about your app and we'll watch a live run with you.