Skip to content
Get Free Trial

Customize Reports

How Sahi Pro builds reports, and how to change a report type, add your own, add custom fields, set a pass threshold, keep offline reports and delete old reports.

Reports work as they are, but teams often need more: a company layout, a field such as the build number in the suite report, or a format another tool can read. This page explains how reports are made, and how to change them.

  1. As a test runs, Sahi Pro saves each step in its database.
  2. When a suite ends, Sahi Pro reads the data from the database and turns it into XML.
  3. The XML is turned into the report you see, or into report files, with XSL transformations.

Sahi Pro ships with XSL files for the html, xml, junit, excel, email and pdf report types. They’re in config/reports/, in a directory for each type, such as config/reports/html.

A report type is a directory of XSL files and a report.properties file. The names html and xml aren’t special keywords: a report type is simply the name of its directory. When a report of type html is needed, Sahi Pro looks for userdata/config/reports/html first, then for config/reports/html.

The main XSL files in a report type:

File Output
suites_list.xsl The reports list. Only needed for the reports you view in Sahi Pro.
suite_report.xsl A suite report.
testcase_report.xsl A suite report with a summary of test cases.
script_report.xsl A script report.
failure_report.xsl The failure and healed reports in Suite Analysis.
js_code_coverage_report.xsl The JS Code Coverage report.
reconciled_report.xsl Another view of the data, such as the time of each step across a load test.

report.properties sets the names of the report files and what’s copied with them. For example, in config/reports/html/report.properties:

script_file.name=<reportId>.html
suite_file.name=index.html
testcase_summary_file.name=testcase_summary.html
images.copy=true
images.dir=images
files.copy=all
  • <scriptName> and <reportId> in a file name are replaced with the real values.
  • images.copy sets whether screenshots are copied with the report files, and images.dir the directory they’re copied to.
  • files.copy sets which files are saved: summary for the suite report and test case summary only, failed_scripts to add the reports of failed scripts, and all for every report.

To change a report type without touching the files Sahi Pro ships with, copy them to userdata/config/reports/ and change the copies. Sahi Pro uses a file in userdata/config/reports/ when there is one, and the one in config/reports/ otherwise.

For example, to change config/reports/html/suites_list.xsl:

  1. Copy suites_list.xsl and report.properties from config/reports/html/ to userdata/config/reports/html/. Both files must be there.
  2. Edit the copied suites_list.xsl.

  1. Copy config/reports/html to userdata/config/reports/mytype.

  2. Edit the XSL files as you need.

  3. Try your XSL on a report you already have: open the report, and add &type=mytype to the end of its address. For example, change

    http://localhost:9999/_s_/dyn/pro/DBReports_scriptReport?id=book_orders_chrome__243e73c20d63804cff09d1807d124f0d5cde

    to

    http://localhost:9999/_s_/dyn/pro/DBReports_scriptReport?id=book_orders_chrome__243e73c20d63804cff09d1807d124f0d5cde&type=mytype

    The report then uses the XSL files in mytype.

  4. When the report looks right, save reports in your type at the end of each run. In testrunner.bat, testrunner.sh, drun.bat or drun.sh in userdata/bin, set LOGS_INFO, which the file passes to -logsInfo:

    Terminal window
    export LOGS_INFO=mytype

    In the .bat files, write set LOGS_INFO=mytype. To also choose where the files are saved, add a path after a colon, such as mytype:D:/reports. Separate several types with commas, such as html,mytype.

A suite report lists its scenario files by start time. To list them by name instead:

  1. Back up config/reports/html/suite_report.xsl.

  2. Open config/reports/html/suite_report.xsl, find

    <!-- <xsl:sort select="SCRIPTNAME" /> -->

    and remove the comment marks, so it reads <xsl:sort select="SCRIPTNAME" />.

You can show your own values in the suite report, such as a build number or the environment a suite ran in. For example, to add two fields called customField and anotherCustomField:

  1. Back up config/reports/html/suite_report.xsl.

  2. Open config/reports/html/suite_report.xsl.

  3. Add the headings. Find

    <!-- Custom Field header-->
    <!-- <th id="RowNo">Custom Field</th> -->

    and

    <!-- Uncomment in case of using Custom Field header -->
    <!-- <th id="RowNoAcc"></th> -->

    and change them to

    <!-- Custom Field headers-->
    <th id="CustField1">Custom Field</th>
    <th id="CustField2">Another Custom Field</th>

    and

    <th id="RowNoAcc"></th>
    <th id="RowNoAcc"></th>
  4. Add the values. Find

    <!--Retrieve custom field value-->
    <!-- <td class='custom-row'>
    <xsl:if test="$suiteInfo != 'null'">
    <pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'customField')" /></pre>
    </xsl:if>
    </td> -->

    and change it to

    <!--Retrieve custom field values-->
    <td class='custom-row'>
    <xsl:if test="$suiteInfo != 'null'">
    <pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'customField')" /></pre>
    </xsl:if>
    </td>
    <td class='custom-row'>
    <xsl:if test="$suiteInfo != 'null'">
    <pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'anotherCustomField')" /></pre>
    </xsl:if>
    </td>

Add one heading and one value for each custom field. To pass the values when a suite runs, see How to show custom fields in reports.

By default, a suite is a Success only if every scenario file in it passes. To call a suite a Success when most of its files pass, such as 75% of them, set a pass threshold:

  1. Open userdata/config/userdata.properties.

  2. Add sahi.pass_threshold_percentage with the percentage you want. The default is 100:

    sahi.pass_threshold_percentage=75

The suite report then shows the suite as a Success when at least that share of its scenario files pass. Each file still shows its own result in the suite report.

Save Report Files for Runs from the Controller

Section titled “Save Report Files for Runs from the Controller”

Runs from the Controller are saved in the database, but no report files are made for them. To also save report files, set controller.offline_logs.type in userdata/config/userdata.properties to a report type:

# Sets the type of offline reports created when a test is run from the Controller. Leave it blank for none.
controller.offline_logs.type=html

To delete a few reports, select them in the reports list and click Delete. To clear out many old reports and keep the database small, use delete_reports:

  1. Stop Sahi Pro if it’s running.

  2. Open a command prompt or terminal and go to userdata/bin in the Sahi Pro installation directory.

  3. Run delete_reports.bat (Windows) or ./delete_reports.sh (macOS and Linux) with one of these:

    To delete Run
    Reports older than 30 days delete_reports.bat "-duration=30"
    Script reports that took 3 minutes or less delete_reports.bat "-time_taken<=00:03:00"
    Script reports that took 3 minutes or more delete_reports.bat "-time_taken>=00:03:00"

    The number of days must be 0 or more. Run it without anything after it to see these options:

    D:\SahiPro\userdata\bin>delete_reports.bat
    --
    Usage:
    delete_reports.bat "-duration=<duration_in_number_of_days>"
    Example:
    To delete reports older than 30 days: delete_reports.bat "-duration=30"
    The duration_in_number_of_days is greater than or equal to 0.
    delete_reports.bat "-time_taken<=<hh:mm:ss>"
    Example:
    To delete script reports which took time less than or equal to 3 minutes: delete_reports.bat "-time_taken<=00:03:00"
    delete_reports.bat "-time_taken>=<hh:mm:ss>"
    Example:
    To delete script reports which took time greater than or equal to 3 minutes: delete_reports.bat "-time_taken>=00:03:00"
    --

delete_reports.bat and delete_reports.sh have three settings you can change by editing the file:

Setting What it does Default
IS_LARGE_DB Deletes reports in batches of BATCH_DURATION days, which works better for a large database. DELETE_FILES and BATCH_DURATION only apply when this is true. true
DELETE_FILES Also deletes the files that belong to the deleted reports, such as screenshots. Set it to false to keep them. true
BATCH_DURATION The number of days in each batch. 10

For example, in delete_reports.bat:

Terminal window
SET IS_LARGE_DB=true
SET DELETE_FILES=false
SET BATCH_DURATION=24