Customize Reports
How Sahi Pro builds reports, and how to change a report type, add your own, add custom fields, set a pass threshold, keep offline reports and delete old reports.
Reports work as they are, but teams often need more: a company layout, a field such as the build number in the suite report, or a format another tool can read. This page explains how reports are made, and how to change them.
How Reports Are Made
Section titled “How Reports Are Made”- As a test runs, Sahi Pro saves each step in its database.
- When a suite ends, Sahi Pro reads the data from the database and turns it into XML.
- The XML is turned into the report you see, or into report files, with XSL transformations.
Sahi Pro ships with XSL files for the html, xml, junit, excel, email and pdf report types. They’re in config/reports/, in a directory for each type, such as config/reports/html.
Report Types
Section titled “Report Types”A report type is a directory of XSL files and a report.properties file. The names html and xml aren’t special keywords: a report type is simply the name of its directory. When a report of type html is needed, Sahi Pro looks for userdata/config/reports/html first, then for config/reports/html.
The main XSL files in a report type:
| File | Output |
|---|---|
suites_list.xsl |
The reports list. Only needed for the reports you view in Sahi Pro. |
suite_report.xsl |
A suite report. |
testcase_report.xsl |
A suite report with a summary of test cases. |
script_report.xsl |
A script report. |
failure_report.xsl |
The failure and healed reports in Suite Analysis. |
js_code_coverage_report.xsl |
The JS Code Coverage report. |
reconciled_report.xsl |
Another view of the data, such as the time of each step across a load test. |
report.properties sets the names of the report files and what’s copied with them. For example, in config/reports/html/report.properties:
script_file.name=<reportId>.htmlsuite_file.name=index.htmltestcase_summary_file.name=testcase_summary.htmlimages.copy=trueimages.dir=imagesfiles.copy=all<scriptName>and<reportId>in a file name are replaced with the real values.images.copysets whether screenshots are copied with the report files, andimages.dirthe directory they’re copied to.files.copysets which files are saved:summaryfor the suite report and test case summary only,failed_scriptsto add the reports of failed scripts, andallfor every report.
Change a Report Type
Section titled “Change a Report Type”To change a report type without touching the files Sahi Pro ships with, copy them to userdata/config/reports/ and change the copies. Sahi Pro uses a file in userdata/config/reports/ when there is one, and the one in config/reports/ otherwise.
For example, to change config/reports/html/suites_list.xsl:
- Copy
suites_list.xslandreport.propertiesfromconfig/reports/html/touserdata/config/reports/html/. Both files must be there. - Edit the copied
suites_list.xsl.
Add a Report Type
Section titled “Add a Report Type”-
Copy
config/reports/htmltouserdata/config/reports/mytype. -
Edit the XSL files as you need.
-
Try your XSL on a report you already have: open the report, and add
&type=mytypeto the end of its address. For example, changehttp://localhost:9999/_s_/dyn/pro/DBReports_scriptReport?id=book_orders_chrome__243e73c20d63804cff09d1807d124f0d5cdeto
http://localhost:9999/_s_/dyn/pro/DBReports_scriptReport?id=book_orders_chrome__243e73c20d63804cff09d1807d124f0d5cde&type=mytypeThe report then uses the XSL files in
mytype. -
When the report looks right, save reports in your type at the end of each run. In
testrunner.bat,testrunner.sh,drun.batordrun.shinuserdata/bin, setLOGS_INFO, which the file passes to-logsInfo:Terminal window export LOGS_INFO=mytypeIn the
.batfiles, writeset LOGS_INFO=mytype. To also choose where the files are saved, add a path after a colon, such asmytype:D:/reports. Separate several types with commas, such ashtml,mytype.
Sort the Suite Report by Name
Section titled “Sort the Suite Report by Name”A suite report lists its scenario files by start time. To list them by name instead:
-
Back up
config/reports/html/suite_report.xsl. -
Open
config/reports/html/suite_report.xsl, find<!-- <xsl:sort select="SCRIPTNAME" /> -->and remove the comment marks, so it reads
<xsl:sort select="SCRIPTNAME" />.
Add Custom Fields to the Suite Report
Section titled “Add Custom Fields to the Suite Report”You can show your own values in the suite report, such as a build number or the environment a suite ran in. For example, to add two fields called customField and anotherCustomField:
-
Back up
config/reports/html/suite_report.xsl. -
Open
config/reports/html/suite_report.xsl. -
Add the headings. Find
<!-- Custom Field header--><!-- <th id="RowNo">Custom Field</th> -->and
<!-- Uncomment in case of using Custom Field header --><!-- <th id="RowNoAcc"></th> -->and change them to
<!-- Custom Field headers--><th id="CustField1">Custom Field</th><th id="CustField2">Another Custom Field</th>and
<th id="RowNoAcc"></th><th id="RowNoAcc"></th> -
Add the values. Find
<!--Retrieve custom field value--><!-- <td class='custom-row'><xsl:if test="$suiteInfo != 'null'"><pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'customField')" /></pre></xsl:if></td> -->and change it to
<!--Retrieve custom field values--><td class='custom-row'><xsl:if test="$suiteInfo != 'null'"><pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'customField')" /></pre></xsl:if></td><td class='custom-row'><xsl:if test="$suiteInfo != 'null'"><pre><xsl:value-of select="util:fetchInfo($suiteInfo, 'anotherCustomField')" /></pre></xsl:if></td>
Add one heading and one value for each custom field. To pass the values when a suite runs, see How to show custom fields in reports.
Set a Pass Threshold
Section titled “Set a Pass Threshold”By default, a suite is a Success only if every scenario file in it passes. To call a suite a Success when most of its files pass, such as 75% of them, set a pass threshold:
-
Open
userdata/config/userdata.properties. -
Add
sahi.pass_threshold_percentagewith the percentage you want. The default is 100:sahi.pass_threshold_percentage=75
The suite report then shows the suite as a Success when at least that share of its scenario files pass. Each file still shows its own result in the suite report.
Save Report Files for Runs from the Controller
Section titled “Save Report Files for Runs from the Controller”Runs from the Controller are saved in the database, but no report files are made for them. To also save report files, set controller.offline_logs.type in userdata/config/userdata.properties to a report type:
# Sets the type of offline reports created when a test is run from the Controller. Leave it blank for none.controller.offline_logs.type=html
Delete Old Reports
Section titled “Delete Old Reports”To delete a few reports, select them in the reports list and click Delete. To clear out many old reports and keep the database small, use delete_reports:
-
Stop Sahi Pro if it’s running.
-
Open a command prompt or terminal and go to
userdata/binin the Sahi Pro installation directory. -
Run
delete_reports.bat(Windows) or./delete_reports.sh(macOS and Linux) with one of these:To delete Run Reports older than 30 days delete_reports.bat "-duration=30"Script reports that took 3 minutes or less delete_reports.bat "-time_taken<=00:03:00"Script reports that took 3 minutes or more delete_reports.bat "-time_taken>=00:03:00"The number of days must be 0 or more. Run it without anything after it to see these options:
D:\SahiPro\userdata\bin>delete_reports.bat--Usage:delete_reports.bat "-duration=<duration_in_number_of_days>"Example:To delete reports older than 30 days: delete_reports.bat "-duration=30"The duration_in_number_of_days is greater than or equal to 0.delete_reports.bat "-time_taken<=<hh:mm:ss>"Example:To delete script reports which took time less than or equal to 3 minutes: delete_reports.bat "-time_taken<=00:03:00"delete_reports.bat "-time_taken>=<hh:mm:ss>"Example:To delete script reports which took time greater than or equal to 3 minutes: delete_reports.bat "-time_taken>=00:03:00"--
delete_reports.bat and delete_reports.sh have three settings you can change by editing the file:
| Setting | What it does | Default |
|---|---|---|
IS_LARGE_DB |
Deletes reports in batches of BATCH_DURATION days, which works better for a large database. DELETE_FILES and BATCH_DURATION only apply when this is true. |
true |
DELETE_FILES |
Also deletes the files that belong to the deleted reports, such as screenshots. Set it to false to keep them. |
true |
BATCH_DURATION |
The number of days in each batch. | 10 |
For example, in delete_reports.bat:
SET IS_LARGE_DB=trueSET DELETE_FILES=falseSET BATCH_DURATION=24
