Reports

Sahi Intermediate Statuses

Intermediate Status	Description
HEALED	Scripts which has at least one step or test case healed.
ERROR	Scripts which has at least one step or test case errored. Syntax error also come under this.
FAILURE	Scripts which has at least one step or test case failed. Assertion failure comes under this.
NA	Scripts which has no step healed or errored or failed.

Sahi Statuses

Status	Description	Color
SUCCESS	Scripts which has passed all the steps or test cases.	GREEN
ABORTED	When browser hangs or goes unresponsive for certain period of time. Default is 90 second.	BROWN
USER_ABORTED	Occurs when script execution has stopped manually by user through Stop All button in Editor or Stop button in Controller.	BROWN
ABORTED_TIMEOUT	Scripts which abort due to provided time elapsed. To enable timeout feature refer here.	BROWN
ERROR	Scripts which has at least one step or test case errored and not ABORTED or USER_ABORTED or ABORTED_TIMEOUT. Syntax error also come under this.	RED
FAILURE	Scripts which has at least one step or test case failed and not ABORTED or USER_ABORTED or ABORTED_TIMEOUT or ERROR. Assertion failure comes under this.	RED
SKIPPED	Scripts which are skipped from execution due to depended scripts failure. Refer Data driver suite section.	GREY
NOT_SUPPORTED	Scripts which are not executed due to unavailability of required resources.	DARK GREY
INTERENAL_ABORTED_RETRY	Scripts not able to decide status due to communication lost between Sahi Master and node while the execution is in progress when drun.auto_rerun_scripts_if_internal_aborted property value is true.	BROWN
INTERENAL_ABORTED	Scripts not able to decide status due to communication lost between Sahi Master and node while the execution is in progress when drun.auto_rerun_scripts_if_internal_aborted property value is false.	BROWN
RUNNING	Scripts which are still in execution state.	ORANGE

Report Screens and Navigation

• To view the script execution logs, click on "Logs" link from either Sahi Editor or Sahi Controller or navigate to http://localhost:9999/logs
  This shows the Suites List page.
• Consolidate Report: Click on Consolidate to get a consolidated report of Scripts and Testcases of selected Suites.
• Click on Suite or Script to view the Suite Report page. Suite reports are displayed as a nested tree structure of suites and scripts. It shows the exact nested level for scripts the way it is written in suite / dd.csv file so that user can get the idea of which script belongs to which suite / dd.csv file. Offline reports will also look as nested tree structure.
  
  
• Click on any individual script to view the Script Report page.
• The steps which failed during script execution will be in shown in red color, with an error message, stating the reason for the step failure.
• Sahi takes screenshots on step failures and errors. Screenshots give visual information on the state of the application during failure. Click on onScriptErrorDefault to view the screenshot.
• Clicking on a line in the logs will drill down to exact line in script.
• To edit the script, click on Edit Script. Sahi Editor will be opened.

Downloading reports

Multiple suite reports can be downloaded directly from the logs page.

1. Select the suites/scripts by checking the checkBox.
2. Click on Download button, which will open Download As dialog.
3. Select the report type format available by clicking the radio button besides it.
4. Click on download button to download the selected format.

One can also download multiple script reports inside a suite, if there exists any script inside that suite.

Auto Refresh Logs

Check the auto-refresh checkbox to automatically refresh the page to show current status of suite. The logs page will refresh every 2 seconds till the excecution completes.

Convert Logs into English

Checking the "Convert To English" check box will translate the executed script steps into corresponding English phrases.

Javascript Code Coverage Reports

Clicking on JS Code Coverage Report link on Suite and Script Reports pages, shows code coverage reports. Refer: JS Code Coverage Report section for details.

Suite Analysis

Clicking on Suite Analysis link on Suite and Script Reports pages, shows script failure reports and healed reports. Refer: Suite Analysis section for details.

Reports - Architecture and Flow

When a script is executed, steps are automatically logged by Sahi.

1. All report data is stored in the database by default.
2. At the end of a suite, Sahi Pro reads the data from the database and converts them to XML
3. The XML is then converted into desired reports using XSL transformations
   info By default, Sahi Pro ships with XSLs for html, xml, junit, excel and email formats. These are stored in sahi_pro/config/reports/ folder under folders named html,xml etc.

Understanding reports and report types

Sahi Pro reports can be generated and downloaded in various report types. By default, Sahi Pro ships with html, xml, junit, excel and email report types. The words html and xml are not special keywords. These can be defined by the user. When Sahi sees the logType as html, it will look for a folder userdata/config/reports/html or in config/reports/html.

This folder contains various XSL files for transforming data as desired, and a report.properties file.

A brief description of various XSL files:

script_report.xsl	Output for individual scripts
suite_report.xsl	Output for a single suite
suites_list.xsl	Creates the default listing of all suites (This may only be needed by the default online reporting)
testcase_report.xsl	Creates the output for individual scripts but also shows summary of testcases. (Testcases are defined in Scenario files.)
reconciled_report.xsl	Output for different view of data. For example, to show stepwise time comparisons in a load test

The names of the output files are defined in report.properties Eg.

script_file.name=<reportId>.html
suite_file.name=index.html
images.copy=true
images.dir=images

Variables available are <scriptName> and <reportId> and will be replaced correctly in the file names.
images.copy tells Sahi whether screenshots images should be copied and images.dir specifies the copy destination folder.

Modifying an existing reporting type

Reports are generated using XSLs. By default, XSL files are present in subfolders of config/reports/. They can be overridden in userdata/config/reports/ folder

Sahi will first look for any report xsl in userdata/config/reports/. If it is not found, it will default to config/reports/

For example, xsls for html reports are in config/reports/html folder.

To modify an existing xsl, copy the xsl along with report.properties into userdata/config/reports with the same subfolder name.

Eg. If you want to modify config/reports/html/suites_list.xsl, copy it along with report.properties such that, both of the files suites_list.xsl and report.properties should be present in the folder userdata/config/reports/html/. Now modify this copied xsl file as needed.

Adding a new custom reporting type

To create a new log type copy config/reports/html to userdata/config/reports/mytype

Edit the xsl files as desired. You can test your xsl at runtime by navigating to any existing logs and adding &type=mytype at the end of the log url.

For example, click on the "Logs" link on the dashboard and drill down to any particular script report.

You may see a URL like this on the browser address bar:

http://localhost:9999/_s_/dyn/pro/DBReports_scriptReport?id=clicksTest__03Nov2011__09_17_36

Add &type=mytype to the end such that it looks like

http://localhost:9999/_s_/dyn/pro/DBReports_scriptReport?id=clicksTest__03Nov2011__09_17_36&type=mytype

This will show you your modified output according to the xsl in mytype folder.

When ready with the correct XSL, open testrunner.bat or drun.bat and pass -logsInfo=mytype. You will now see the logs created according to your xsls.

Report Customizations

Sort suite report

By default the suite report will be sorted by script STARTTIME, but if user wants to sort the suite report by SCRIPTNAME, follow the below steps.

1. Take a backup of config/reports/html/suite_report.xsl.
2. Open config/reports/html/suite_report.xsl
3. Search for

   <!-- <xsl:sort select="SCRIPTNAME" /> -->

   and uncomment it.

Add custom fields in Suite reports

From Sahi Pro v6.3.0, user can add custom fields in Suite report as well.

For users using Sahi Pro v10.0.0 and above, follow the steps given below to add custom fields to your Suite report.
Suppose you need two custom fields: customField and anotherCustomField

1. Take a backup of config/reports/html/suite_report.xsl.
2. Open config/reports/html/suite_report.xsl
   1. Custom field headers search for

      <!-- Custom Field header-->
      <!--<td id="RowNo">Custom Field</td>-->

      and

      <!-- Uncomment in case of using Custom Field header -->
      <!-- <th id="RowNoAcc"></th> -->

      and change these to

      <!-- Custom Field headers-->
      <td id="CustField1">Custom Field</td>
      <td id="CustField2">Another Custom Field</td>

      and

      <th id="RowNoAcc"></th>
      <th id="RowNoAcc"></th>

   2. Custom field values search for

      <!--Retrieve custom field value-->
      <!-- <td class='custom-row'>
          <xsl:if test="$suiteInfo != 'null'">
              <pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'customField')" /></pre>
          </xsl:if>
      </td> -->

      and change it to

      <!--Retrieve custom field values-->
      <td class='custom-row'>
          <xsl:if test="$suiteInfo != 'null'">
              <pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'customField')" /></pre>
          </xsl:if>
      </td>
      <td class='custom-row'>
          <xsl:if test="$suiteInfo != 'null'">
              <pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'anotherCustomField')" /></pre>
          </xsl:if>
      </td>

      info NOTE: Add as many entries as there are custom fields.

Set pass threshold percentage for Suite or Data Driven Suite

From Sahi Pro v7.5.0, user can set pass threshold percentage for Suite or Data Driven Suite. Suppose the user need to see log result as success for more than 75% in the scripts of a suite or dd.csv, then the user can set pass threshold percentage to 75%.

1. Click the Configure link on Sahi dashbord and go to userdata properties.
2. Then select sahi properties and search for sahi.pass_threshold_percentage.
3. The default value is set to 100, update the value as required.

Now in log result user will see reports as success, if scrips pass percentage is more than required value. The user can see the details of success and failure of all the scripts in suite report.

Offline logs for Controller Playback

By default offline logs are not created for playback via Controller. If needed, offline logs for playback via Controller can be specified in userdata.properties by setting controller.offline_logs.type property

# Sets the type of offline logs created when script is run from Controller. Set to blank to disable offline logs
controller.offline_logs.type=html

Deleting DB reports

If you want to delete the unwanted logs and reduce the size of database, follow the below procedure:

1. Close Sahi(if running).
2. Open a command prompt and cd to sahi_installation_folder/userdata/bin
3. Run delete_reports.bat/delete_reports.sh to see the syntax
   D:\SahiPro\userdata\bin>delete_reports.bat -- Usage: delete_reports.bat "-duration=<duration_in_number_of_days>" Example: To delete reports older than 30 days: delete_reports.bat "-duration=30" The duration_in_number_of_days is greater than or equal to 0. delete_reports.bat "-time_taken<=<hh:mm:ss>" Example: To delete script reports which took time less than or equal to 3 minutes: delete_reports.bat "-time_taken<=00:03:00" delete_reports.bat "-time_taken>=<hh:mm:ss>" Example: To delete script reports which took time greater than or equal to 3 minutes: delete_reports.bat "-time_taken>=00:03:00" -- D:\SahiPro\userdata\bin>
4. Parameter IS_LARGE_DB and related parameters DELETE_FILES, BATCH_DURATION with it, can be modified by editing delete_reports.bat/delete_reports.sh.
   1. Set IS_LARGE_DB to true for deleting reports when database size is large to delete reports in batches of BATCH_DURATION days. Default value is false.

      SET IS_LARGE_DB=true

      Parameters DELETE_FILES, BATCH_DURATION will come into picture only when IS_LARGE_DB set to true.
   2. Set DELETE_FILES to false to avoid deleting files related to reports which are supposed to be deleted. Default value is true.

      SET DELETE_FILES=false

   3. Set BATCH_DURATION to no of days which will decide individual batch duration days. Default value is 10.

      SET BATCH_DURATION=24