abstract When a suite or script is executed, detailed reports of suite, script, steps and errors are automatically created by Sahi and stored in a database. Reports can be viewed as a web page, or downloaded as offline files in various formats. Reports can also be synced to a central database or between Sahi installations. The appearance of reports can be modified via XSL style sheets. Various other parameters of reports can be also be configured.
|HEALED||Scripts which has at least one step or test case healed.|
|ERROR||Scripts which has at least one step or test case errored. Syntax error also come under this.|
|FAILURE||Scripts which has at least one step or test case failed. Assertion failure comes under this.|
|NA||Scripts which has no step healed or errored or failed.|
|SUCCESS||Scripts which has passed all the steps or test cases.||GREEN|
|ABORTED||When browser hangs or goes unresponsive for certain period of time. Default is 90 second.||BROWN|
|USER_ABORTED||Occurs when script execution has stopped manually by user through ||BROWN|
|ABORTED_TIMEOUT||Scripts which abort due to provided time elapsed. To enable timeout feature refer here.||BROWN|
|ERROR||Scripts which has at least one step or test case errored and not ABORTED or USER_ABORTED or ABORTED_TIMEOUT. Syntax error also come under this.||RED|
|FAILURE||Scripts which has at least one step or test case failed and not ABORTED or USER_ABORTED or ABORTED_TIMEOUT or ERROR. Assertion failure comes under this.||RED|
|SKIPPED||Scripts which are skipped from execution due to depended scripts failure. Refer Data driver suite section.||GREY|
|NOT_SUPPORTED||Scripts which are not executed due to unavailability of required resources.||DARK GREY|
|INTERENAL_ABORTED_RETRY||Scripts not able to decide status due to communication lost between Sahi Master and node while the execution is in progress when ||BROWN|
|INTERENAL_ABORTED||Scripts not able to decide status due to communication lost between Sahi Master and node while the execution is in progress when ||BROWN|
|RUNNING||Scripts which are still in execution state.||ORANGE|
- To view the script execution logs, click on "Logs" link from either Sahi Editor or Sahi Controller or navigate to http://localhost:9999/logs
This shows the Suites List page.
- Consolidate Report: Click on Consolidate Report to get a consolidated report of Scripts and Testcases of selected Suites.
- Click on Suite or Script to view the Suite Report page.
Suite reports are displayed as a nested tree structure of suites and scripts.
It shows the exact nested level for scripts the way it is written in suite / dd.csv file so that user can get the idea of which script belongs to which suite / dd.csv file.
Offline reports will also look as nested tree structure.
- Click on any individual script to view the Script Report page.
- The steps which failed during script execution will be in shown in red color, with an error message, stating the reason for the step failure.
- Sahi takes screenshots on step failures and errors. Screenshots give visual information on the state of the application during failure.
onScriptErrorDefaultto view the screenshot.
- Clicking on a line in the logs will drill down to exact line in script.
- To edit the script, click on
Edit Script. Sahi Editor will be opened.
The reports can be downloaded with varying formats including email, html, xml, excel, junit.
One can also download multiple script reports inside a suite, if there exists any script inside that suite.
JS Code Coverage Reportlink on Suite and Script Reports pages, shows code coverage reports. Refer: JS Code Coverage Report section for details.
Suite Analysislink on Suite and Script Reports pages, shows script failure reports and healed reports. Refer: Suite Analysis section for details.
- All report data is stored in the database by default.
- At the end of a suite, Sahi Pro reads the data from the database and converts them to XML
- The XML is then converted into desired reports using XSL transformations
info By default, Sahi Pro ships with XSLs for html, xml, junit, excel and email formats. These are stored in
sahi_pro/config/reports/folder under folders named
html, xml, junit, excel and emailreport types. The words
xmlare not special keywords. These can be defined by the user. When Sahi sees the logType as
html, it will look for a folder
This folder contains various XSL files for transforming data as desired, and a
A brief description of various XSL files:
|script_report.xsl||Output for individual scripts|
|suite_report.xsl||Output for a single suite|
|suites_list.xsl||Creates the default listing of all suites (This may only be needed by the default online reporting)|
|testcase_report.xsl||Creates the output for individual scripts but also shows summary of testcases. (Testcases are defined in Scenario files.)|
|reconciled_report.xsl||Output for different view of data. For example, to show stepwise time comparisons in a load test|
The names of the output files are defined in
Variables available are
script_file.name=<reportId>.html suite_file.name=index.html images.copy=true images.dir=images
<reportId>and will be replaced correctly in the file names.
images.copytells Sahi whether screenshots images should be copied and images.dir specifies the copy destination folder.
config/reports/. They can be overridden in
Sahi will first look for any report xsl in
userdata/config/reports/. If it is not found, it will default to
For example, xsls for
htmlreports are in
To modify an existing
xsl, copy the
userdata/config/reportswith the same subfolder name.
Eg. If you want to modify
config/reports/html/suites_list.xsl, copy it along with
report.propertiessuch that, both of the files
report.propertiesshould be present in the folder
userdata/config/reports/html/. Now modify this copied
xslfile as needed.
Edit the xsl files as desired. You can test your xsl at runtime by navigating to any existing logs and adding
&type=mytypeat the end of the log url.
For example, click on the "Logs" link on the dashboard and drill down to any particular script report.
You may see a URL like this on the browser address bar:
&type=mytypeto the end such that it looks like
This will show you your modified output according to the xsl in
When ready with the correct XSL, open testrunner.bat or drun.bat and pass
-logsInfo=mytype. You will now see the logs created according to your xsls.
- Take a backup of
- Search for
and uncomment it.
<!-- <xsl:sort select="SCRIPTNAME" /> -->
- Take a backup of
Custom field headers
and change it to
<!-- Custom Field header--> <!--<td id="RowNo">Custom Field</td>-->
<!-- Custom Field headers--> <td id="CustField1">Custom Field</td> <td id="CustField2">Another Custom Field</td>
Custom field values
<!--Retrieve custom field value--> <!-- <td> <xsl:if test="$suiteInfo != 'null'"> <pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'customField')" /></pre> </xsl:if> </td> -->infoNOTE: There will be two occurences for Retrieve custom field value. Make sure you replace at both occurences.and change it to
<!--Retrieve custom field values--> <td> <xsl:if test="$suiteInfo != 'null'"> <xsl:value-of select="util:fetchInfo($suiteInfo, 'customField')" /> </xsl:if> </td> <td> <xsl:if test="$suiteInfo != 'null'"> <xsl:value-of select="util:fetchInfo($suiteInfo, 'anotherCustomField')" /> </xsl:if> </td>info NOTE: Add as many entries as there are custom fields.
- Custom field headers search for
- Click the
Configurelink on Sahi dashbord and go to
- Then select
sahi propertiesand search for sahi.pass_threshold_percentage.
- The default value is set to 100, update the value as required.
# Sets the type of offline logs created when script is run from Controller. Set to blank to disable offline logs controller.offline_logs.type=html
- Close Sahi(if running).
- Open a command prompt and cd to
- Run delete_reports.bat/delete_reports.sh to see the syntax
D:\SahiPro\userdata\bin>delete_reports.bat -- Usage: delete_reports.bat "-duration=<duration_in_number_of_days>" Example: To delete reports older than 30 days: delete_reports.bat "-duration=30" The duration_in_number_of_days is greater than or equal to 0. delete_reports.bat "-time_taken<=<hh:mm:ss>" Example: To delete script reports which took time less than or equal to 3 minutes: delete_reports.bat "-time_taken<=00:03:00" delete_reports.bat "-time_taken>=<hh:mm:ss>" Example: To delete script reports which took time greater than or equal to 3 minutes: delete_reports.bat "-time_taken>=00:03:00" -- D:\SahiPro\userdata\bin>
Parameter IS_LARGE_DB and related parameters DELETE_FILES, BATCH_DURATION with it, can be modified by editing
Set DELETE_FILES to false to avoid deleting files related to reports which are supposed to be deleted. Default value is true.
Set BATCH_DURATION to no of days which will decide individual batch duration days. Default value is 10.
- Set DELETE_FILES to false to avoid deleting files related to reports which are supposed to be deleted. Default value is true.