Skip to content
Get Free Trial

Script Report

Read a script report: each test case, keyword and step, the error and screenshot for a failed step, healed steps, data sets and retries.

A script report shows one scenario file or script from a run, down to each step. It’s where you find out why a test failed: which step, what Sahi Pro expected, what it found, and what the page looked like.

Open it from a suite report: click the scenario file in the navigator or in the list of scenario files. It opens in a tab of its own.

The script report for book_orders.s.csv in the suite report: the header with Failure, 1 test case passed, 1 failed and 0 errors, Chrome, the time taken, the start time and localhost:9999; Filters, Script Summary, Auto Refresh and Export; the test case Order Books at Cart Prices with the data chip nightly_orders.csv - cart_totals and 3 sets executed, a table of data sets with their status, and the keywords of each set; the failed keyword Check Cart Total is open in the steps panel on the right; numbered badges mark each part
  1. Header: the status, how many test cases passed, failed and hit an error, and the browser, time taken, start time and node.
  2. Filters for what the report shows (see Choose What to Show).
  3. Script Summary, Auto Refresh and Export.
  4. Test data: the data file and group a data-driven test case used, and how many sets ran.
  5. Data sets: each set’s status and values.
  6. Steps panel: the steps of the keyword you clicked.
  7. A failed keyword, in red.

The report lists the scenario file the way it’s built:

  • INCLUDES/GLOBAL comes first, with the keyword files the scenario file loads, such as store.sah.
  • Then each test case, with its description and a status icon.
  • Under each test case, its keywords, with the values passed to them in grey on the right, such as test, secret.

Click a test case’s arrow to open or close it.

Click a keyword to see its steps in the panel on the right. Each step reads as a sentence, such as “Enter value 2 in q textbox”, with the time it took. A keyword’s parameters, such as $total, show above its steps.

A failed or errored keyword is red, and so is its test case. Click the keyword, then click the red step in the panel to open its details:

The steps panel for the keyword Check Cart Total with the step Assert that value of total textbox is 800 open: Failed at 02:09:42.858 pm, STEP 1; the Failure Message [Assertion Failed] Expected:800 Actual:"850" at /sahi_training/store.sah&n=29 CheckCartTotal at /sahi_training/book_orders.s.csv&n=7; the Element _getValue(_textbox("total")); and the actions Lock the window, Bring window into focus and Take page screen shot
  • Failed or Error, the time and the step number.
  • Failure Message or Error Message: what went wrong, such as an assertion that expected 800 and found 850, or an element that wasn’t found. Under it, the line in the keyword file and in the scenario file where it happened, such as store.sah&n=29 for line 29 of store.sah.
  • Element: the element the step acted on, such as _getValue(_textbox("total")).
  • Screenshot: when a step fails or hits an error, Sahi Pro locks the browser window, brings it into focus and takes a screenshot, so you can see what the page looked like.

An error looks the same, with Error Message in place of Failure Message:

The steps panel for the keyword Open Forgot Password Page: step 1 Navigate to https://sahitest.com/demo/training/login.htm passed; step 2 Click on Forgot Password? link is open, with Error at 02:10:22.730 pm, STEP 2, the Error Message Error: _link("Forgot Password?") was not found on the browser, the lines in store.sah and login_errors.s.csv, the Element _link("Forgot Password?"), and the actions Lock the window, Bring window into focus, Take page screen shot (blurred here) and Unlock the window

When an element isn’t found, Autoheal can find a matching one, so the test carries on instead of failing. This happens when the application changes, for example when a button’s label changes from “Add” to “Add to Cart”. A healed step passes, but you need to know about it, because the test should be updated.

Healed steps are marked Healed on the test case, the keyword and the step, and the header counts them, such as 1 healed. Open the step to see what changed:

The steps panel for the keyword Add Books To Order with 1 healed; step 4 Click on Add button is open, marked Passed and Healed, with Result Step completed successfully, Element _button("Add"), and under Auto Heal: Original element _button("Add"), Healed element _button("form-btn"), Executed step _click(_button("form-btn")), and Recommendation Reason might be page update. Please update script with new value, with a link to /sahi_training/store.sah&n=22
  • Original element: the element in your test, which wasn’t found.
  • Healed element: the element Sahi Pro used instead.
  • Executed step: the step as it actually ran.
  • Recommendation: what to do, with a link to the line in the keyword file to update.

A test case that runs with a test data file runs once for each row. The report shows the data file and group under the test case, such as nightly_orders.csv - cart_totals, and how many sets ran, such as 3 sets executed.

Each set has its own row in a table, with its status and the values it used, one column for each column of the data file:

The test case Order Books at Cart Prices with the data chip nightly_orders.csv - cart_totals and 3 sets executed, above a table with the columns Status, Username, Password, javaQty, rubyQty, pythonQty and total: Success with test, secret, 2, 3, 1 and 1550; Failure with test, secret, 1, 1, 1 and 800; Success with test, secret, 0, 2, 2 and 1100

So when one set out of many fails, you can see straight away which values it used, such as 1, 1, 1 and 800. Click the arrow at the start of a row to see the keywords that ran with those values, or the arrows above the table to open or close every set.

When a run is set to retry failed scenario files (Retry Count in the run’s Advanced Settings), the report keeps every attempt. The header shows which one you’re looking at. Click it to switch:

The retry menu open in the script report header: 1 Retry No. 1 (final), Failure, 00:30, 02:17:11 pm; and Actual execution, Failure Retry, 00:30, 02:16:36 pm
  • Actual execution is the first attempt.
  • Retry No. 1, Retry No. 2 and so on are the retries. The last one is marked (final), and its status is the file’s final status.

Click Filters to choose what the report shows. The number on the button is how many are on:

The Filters menu open with Show Failed unticked, and Show Screenshots and No Code View ticked
  • Show Failed shows only the test cases that failed or hit an error. Turn it on in a long report to go straight to the problems.
  • Show Screenshots shows the screenshots in each step’s details.
  • No Code View shows the steps as sentences, such as “Enter value 2 in q textbox”. Turn it off to see each step as Sahi Script, such as _setValue(_textbox("q"), "2").

Click Script Summary for an overview of the file:

The Script Summary for book_orders.s.csv, 2 test cases, 1 failed: Script Info with Browser Version Chrome 157.0.8103.79 and the User Agent, and Test Cases with Order Books at Cart Prices (Failure, intermediate status FAILURE) and Place Bulk Orders (Success), with their tags, start and end times
  • Script Info: the browser version and user agent the file ran in.
  • Test Cases: each test case with its tags, final status, intermediate status, and start and end time.
  • Artifacts: the files the run saved, such as screenshots, when there are any.
  • Turn on Auto Refresh to follow a scenario file while it runs. The report reloads every 2 seconds until the file ends.
  • Export has Export to PDF, Export to Excel and Download all Artifacts, the same as in the suite report.