Suite Analysis

abstract Suite Analysis is designed to help quick fix of scripts causing most failures with a consolidated failure report as well quick update of steps which were healed with a consolidated healed report. Historical results, for each scripts which are having at least one failure history, are also provided. This feature is available since Sahi Pro V8.0.0.

When suites fail, a good amount of time may be spent analyzing the test results to find the core problem. We have enhanced our Suite analysis to analyze the reports and pinpoint the source of failures. When you know that the 23 failing scripts are all failing due to just 3 specific steps, it becomes very easy and fast to fix and rerun.

Suite Analysis Report

Click on Suite Analysis link on Suite Report and Script Report pages.

Step gives the first step which has failed. Click this to see the failed step in the script.
Number of Steps Failed link gives the total number of steps failed in the whole suite. Click this link to see all failed step messages.
Tree structure shows all the scripts and functions called in between the failed step and scenarios/ scripts got failed because of the step.
Affected Script Reports section contains the scripts which are failed. Clicking on the scripts takes you to the corresponding script report.

In the image above, it is seen that _assertEqual("1152", _textbox("total").value) was called from addBooks function, which in turn was called by 3 different scripts. If this one line is fixed, all the 3 scripts will start working.

info Note: If you are upgrading from versions below Sahi Pro V8.0.0 and want to generate report for old data as well, do the following:

Navigate to http://localhost:9999/_s_/dyn/pro/LoadOriginalSuitePath_loadData. (Use appropriate Sahi host and port.)
Wait for SUCCESS response

Failed Scripts History

The history of past run results for the failed scripts along with their pass percentage are showcased in this table. It helps to keep track of failed scripts.
The history tables are categorized into 2 as:

Within the same Suite: Script executed from the same suite is considered in this section.
Within Sahi: This section covers the history of script whenever it is executed irrespective of the suite from which it is executed.

info Latest Status column gives, the status of each script execution shown as corresponding status batches.
The sequence of status shown starts with the most recent execution.

Success Scripts With Failure History

Scripts which are passed in the current suite but having atleast one failure history are displayed here.

If script is retried multiple times by setting Retry Count while playback, final status of the script followed by corresponding status batches.
Eg: If a script is retried 1 time after it fails in the first run, and if it is failed in the second run as well, it is denoted in the Failed Scripts History table as two FAILURE status batch.

infoNote: Scripts which are in ABORTED and ERROR status is considered as FAILURE.

Maximum number of report status shown in the table can be configured by setting below property in sahi_pro/userdata/config/userdata.properties

sahi.count.script_status_to_be_shown=10

infoNote: Default value is 10. Can change it to any desired value