The most common reason people stop trusting test results is that the report says nothing on its own. There is a red line on a screen: “checkout_flow failed”. Then what? Someone runs that test on their own machine, it passes, “that test is flaky anyway”, and the line turns into noise nobody reads.
The problem isn’t the test. It’s what the report hands you.
A report has to answer one question
Whoever opens the report has a single question: is this real?
To answer it in ten seconds, the report needs:
- What was expected. Not “failed”, but “tapping a cancelled trip must not crash the app”.
- What happened. In one plain sentence, without jargon.
- Evidence. The exact moment in the video, the error on screen, the device’s own record.
- Where it happened. Which screen, which step, which part of the app.
Without those, a report is a claim. With them, it’s proof.
What is in a TestSafe report
When a test ends, the report sits on one page:
Summary. The result, the duration, how many steps ran, how many warnings came up. Under it, a result report written in plain words: what was tried, what happened.
Steps. How long each step took and what the agent did, one action at a time: “Start app”, “Write text: phone number”, “Tap: CONTINUE”. If a step failed, you can see which one.
Video. A screen recording of the whole test. Every criterion in the report points at a range of seconds, and opens there.
Crash record. If the app closed, the record Android produced goes into the report: the name of the error, which screen of the app it came from, and the full stack trace. From the same place you can jump into the device logs.
Performance. CPU, memory, frame rate and network use are charted for the whole test. They are recorded even if you didn’t ask, because when someone says “the app got slow” you need somewhere to look back at.
Info. Which device, which version, which environment, and what started the test.
Why the criteria table earns its place
The most used part of the report turned out to be the criteria table. Each row is an expectation, with its result and the range of seconds in the video. It does two jobs at once:
- It explains what the test checked to someone who didn’t write it.
- It makes “is this real?” clickable.
When a criterion warns, the conversation isn’t “is the test broken” but “look at 2:03 in the video”.
Where the report should arrive
The last property of a good report: you shouldn’t have to go looking for it. It should land where the team already works. Ours arrives as a comment on the GitHub issue, a notification in Slack, and an issue in Jira.
Every report you put on a separate dashboard becomes, after a while, a tab nobody opens.
Screenshots of a report are on the Quick Start page.