Skip to content
Get Free Trial

Reports

Every test run creates a report. Learn how to open reports, how they are organised, and what each status means.

When a test fails, you need to know which step failed, what the page looked like and whether it failed before, without running it again. Every test run in Sahi Pro creates a report that answers these questions. Sahi Pro saves the report as the test runs, so you can watch a long suite run while it’s still going.

Open Reports in any of these ways:

  • In the No-Code Test Editor, click Reports in the left bar.
  • In the Controller, click Reports in the bottom bar.
  • In a browser, go to http://localhost:9999/logs.

Reports opens on the reports list, with the latest test runs first.

Reports have three levels. Each one opens the next:

Level What it shows
Reports list Every test run, newest first, with its status and a summary of results. Select runs here to compare, consolidate, download, sync or delete them.
Suite report One run of a suite: its details, charts of the results, the time each scenario file took, and every scenario file and test case in it. Its other tabs hold the Suite Analysis and the JS Code Coverage report.
Script report One scenario file or script: each test case, keyword and step, with the error message, the element and the screenshot for each step that failed.

Running a single scenario file creates a suite report too, with that one file in it.

Each scenario file or script ends a run with one status, shown as a coloured pill, so you can see at a glance what needs attention. A suite takes the worst status of the files in it.

Status Colour What happened
Success (SUCCESS) Green Every step and test case passed.
Failure (FAILURE) Red At least one step or test case failed, usually an assertion, and the run wasn’t aborted or stopped by an error.
Error (ERROR) Red At least one step or test case hit an error, such as an element that wasn’t found or a syntax error, and the run wasn’t aborted.
Aborted (ABORTED) Orange The browser stopped responding for too long, 90 seconds by default.
User Aborted (USER_ABORTED) Orange Someone stopped the run by hand, for example with Tools › Stop All in the Editor, or Stop in the Controller while a test runs.
Aborted Timeout (ABORTED_TIMEOUT) Orange The scenario file ran longer than the time it was given. Set a time limit in the suite file’s timeout_minutes column (see Test Suites) or for the whole suite in the run’s Advanced Settings.
Skipped (SKIPPED) Grey It wasn’t run, because a scenario file it depends on didn’t pass. See depends in Test Suites.
Not Supported (NOT_SUPPORTED) Light grey It wasn’t run, because something it needs, such as a browser or device, wasn’t available.
Internal Aborted Retry (INTERNAL_ABORTED_RETRY) Orange In a distributed run, the master lost contact with the node running it, and drun.auto_rerun_scripts_if_internal_aborted is true, so it runs again.
Internal Aborted (INTERNAL_ABORTED) Orange In a distributed run, the master lost contact with the node running it, and drun.auto_rerun_scripts_if_internal_aborted is false.
Running (RUNNING) Blue It’s still running.

The charts in a suite report say Passed and Failed for Success and Failure. They’re the same statuses.

A final status doesn’t say what happened along the way. A scenario file can pass after Sahi Pro healed one of its steps, or end as a Failure after an earlier error. The intermediate status records the most serious thing that happened during the run:

Intermediate status What happened
Healed At least one step or test case was healed: an element wasn’t found, so Sahi Pro used a matching one instead.
Error At least one step or test case hit an error, including syntax errors.
Failure At least one step or test case failed, such as an assertion failure.
NA Nothing was healed, and nothing failed or hit an error.

You’ll find it in the Intermediate Statuses column of a suite report and in the Script Summary.

Sahi Pro saves every report in its own database, as the test runs. You can also:

  • keep reports in another database, such as MySQL, MSSQL, Oracle or PostgreSQL
  • sync reports to a central database, to see the runs from several machines in one place
  • save reports as files, such as HTML, JUnit or Excel, when a run ends (see Customize Reports)

To change what reports look like, or to delete old reports, see Customize Reports.