Best Practices

It is trivial to write automation scripts with Sahi. However once your scripts grow in size, you will realize that maintaining a large suite is hardly trivial.

We recommend the following for maintainable test automation.

Basic Recommendations

• Use Business Driven Test Automation as much as possible. Thinking about the business first leads to much cleaner and maintainable test automation suites.
  • Use the Scenario Editor to create new Scenario file and add testcases. Ensure that keywords express business intent. Don't call any Sahi APIs directly from scenario files.
  • Use Run/Record to implement Keywords.
  • While recording, use an Accessor Repository to store accessors in an .ar.csv file. (AR can be extracted later also, but once extracted, continue to use that AR file during recording.) Use one AR file per module. Same entries should not be in multiple AR files. The intent is to have a single place of modification in case an element identifier changes.
  • Follow a naming convention with filenames, keywords, parameter names etc. We recommend PascalCase (Uppercase first letters words without space) for almost everything. For Key Word and Parameter names in Scenario files we recommend Title Case (Uppercase first letters words with space).
  • Group functions in Library files according to module. If needed, create sub folders for different modules of your application. Have Lib folders inside them with library functions. Have a common folder too, which holds scripts with common functionality
  • Use the newer mode of Data Drive as described in BDTA. This is a neater and more readable implementation.
  • To use better accessor for a particular element, put a break point and run till the necessary step. Then on Controller, CTRL Hover to identify the element in a better way (using Relation APIs etc.). Then store the AR from Controller itself.
• If using the older approach
  • Use the recorder to create snippets of automation code
  • Record with "Accessor Repository" turned on. Have a single Accessor Repostitory file for the entire code-base if possible
  • Use the Sahi Script Editor to extract functions out of the recorded steps
  • Store such functions into library files. (these are normal .sah files. You may store them in a folder like lib. Eg. lib/admin.sah)
  • If needed, create sub folders for different modules of your application. Have lib folders inside them with library functions. Have a common folder too, which holds scripts with common functionality
  • Create Scenario Files (.s.csv files) and use the above created functions as Key Words. The Editor auto-completes for function names, parameters etc. of all files in the Script folder. Scenario files can easily be used by non-technical testers. Scenario files can also be tagged and their reports are rich.
    infoScenario files should ideally not contain any Sahi APIs. If you choose to move from Sahi to another framework, the Scenario files should be reusable since they are just csv files which specify testcases as a sequence of business functionality. Keep all Sahi code inside library functions if possible.
• Club together scenario files in a data driven suite (.dd.csv file). Add tags for module, priority, version etc. as needed
• Execute .dd.csv files from the Editor or ant target. The ant target can be created from Editor.
• Run with Autoheal enabled if the application under test is evolving. This will catch and rectify element identification problems and reduce failures.
• Use Suite Analysis to pinpoint errors in the suite playback.

Advanced Team Recommendations

• Have a single Accessor Repository file which is used by all scripts.
• Create a few simple smoke tests which pass through elements of all pages and assert for their existence.
• Make sure these smoke tests run with the developers' Continuous Integration build
• If these scripts fail, the developers need to modify the Accessor Repository file and make the build pass. This ensures that the Accessor Repository reflects any changes that the developer has made to the web application. It also ensures that QA scripts fail only for Scenario and flow changes and not for element changes.
• Write scripts aimed at Parallel execution. Testers many a times claim that this cannot be done for THEIR case. The claim is almost never true.
• Run your QA builds as frequently as your processes allow (Aim for once a day at least). Scripts which have not run for some time, invariably become unmaintainable.
• If you believe you need to build a framework to make things work, talk to our support team. You can save significant time and money by adapting to what is already available. Sahi Pro has been evolving with various customer requirements and we already have most things that you may need. We are also constantly tweaking and innovating on the product, so if you need a feature which does not exist, we may easily build it for you.
• Check in scripts into some Version Control System (VCS) (eg. git or svn). Instead of working on scripts in a common shared folder, work on individual versions of scripts and use the VCS features like versioning, diff, rollback, merge etc. to your advantage. Data files should also be checked in along with scripts.
• Use file types which allow easy "diff"ing between versions. When a script fails today which passed yesterday, you may pinpoint the error easily by performing a diff with the older file. However diff may only be possible on text based files. So we recommend using .csv format files instead of Excel files.
• Hire good, thinking testers rather than "automation engineers".

Language Recommendation

We recommend using the Javascript (Sahi) version for automation. Since Sahi controls the execution of such scripts, it is easy to do a lot of operations like automatic logging, code folding in reports, distributed playback handling etc. internally. Javascript as used in Sahi does not need as much programming expertise as Java. If you are using the Java driver, we recommend you to use it with Sahi's Scenario files. It will give you the same mentioned benefits as a Sahi script.

If you are using the Java driver WITHOUT Sahi's framework, please read below:

The Java driver is bare bones libraries to launch a browser and perform actions on it. If you use it you will have to do everything else that is needed as a framework. Eg. logging, reporting, storage of reports, building UI for viewing historic reports, building suites, running them in parallel, running them across machines as a distributed playback and collecting reports, provide an Excel like interface, allow auto discovery of functions, keep code and testcases separate, maintain object repositories or page object patterns etc.

Almost 70-80% of your effort in automation will go towards building all these peripheral components. This also means specialized skills are needed in your automation engineers. Sahi Pro already has all these features and more in its Javascript version. We have been evolving with our customer requirements and most of what you need would mostly already be there. It even has a good editor now which will report errors as you work on the file, suggest functions and function parameters, report if a function is misspelt or missing and a lot of other features. Mostly people look to java to build a framework and to allow use of good editors. Both of these have been addressed since Sahi Pro 6.1.0. The JS version can also call and reuse any Java code or libraries, so you can have the simplicity of JS but also use the power of Java when needed.