Mobile Automation integration with AWS Device Farm

abstract This section helps you for integration Sahi with AWS Device Farm.

Pre-requisites

  1. Sahi Pro Mobile Add-on and AWS Command Line Interface (CLI) version 2 should be installed with in the AWS system.
    • Please refer here to quickly setup AWS CLI.
    • Please make sure that Default region name is us-west-2.
    • Please make sure that Default output format is json.
  2. cURL should be installed on the system.
    • To verify, run curl in the command line.

Android Integration

  1. Goto: AWS Device Farm
  2. Create a new project under Mobile device testing projects.
  3. Click on Project settings.
  4. OPTIONAL: Set the maximum Execution Timeout according to the script execution timeout.
  5. Click on Device Pools.
  6. Click on Create a new device pool. Give a name to device pool and add one device in it. Click on Save device pool → Done.
    infoNote: Add only ONE device in each device pool. Each device pool is considered as a Single Device for Sahi Pro Mobile Add-on.
  7. Click on Uploads.
  8. Click on Upload a file and upload 3 APKs.
    1. Test Application: Select Type as Android App.
      infoUploading Sahi test app from <SahiPro>/userdata/android/testapps/AndroidApp.apk
    2. Sahi Instrumentation App: Select Type as Android App and upload APK from <SahiPro>/android/lib/app-debug.apk.
    3. Sahi Instrumentation App: Select Type as Instrumentation Test Package and upload APK from <SahiPro>/android/lib/app-debug-androidTest.apk.
  9. Configure <SahiPro>/userdata/config/aws_devicefarm.properties file.
    1. Click on the All Settings button on the Dashboard. It will open up all Settings dialog.
    2. Click on the Sahi Pro Settings button. It will open up Configure Sahi Pro page.
    3. Select the AWS DeviceFarm Configuration link on the left panel of Configure Sahi Pro page. This will open page similar to image given below.

    • Using Device Farm Configuration UI
      1. Select Type:Android and click on the Fetch Device Farm Projects button to fetch all created device farm projects.
      2. Select Device Farm Project: Sahi Pro - Android
        Select Test Application: AndroidApp.apk
        Select Sahi App: app-debug.apk
        Select Sahi App Test: app-debug-androidTest.apk

        Click on Update aws_devicefarm.properties button. This will update the aws_devicefarm.properties file.
      3. Configure android_devices.xml

        Select Device Pool: Google Pixel 8 Pro
        Set Device Capacity: 5 (default is 1).
        Click on Generate button to generate the device xml. Rename the name tag value to AWS-Google-Pixel and click on Update in android_devices.xml button. This will update the android_devices.xml file.
        infoRepeat this step, if you wish to configure multiple device pools.
        infoYou can also select more than one device pool at a time to generate android device xml.
    • Using Command Line
      1. Click on Show in Text view button.
      2. Set android.aws.devicefarm.project.arn property.
        To get the ARN of the project, execute aws devicefarm list-projects in Command Prompt/Terminal. 
        {
	"projects": [
		{
			"arn": "arn:aws:devicefarm:us-west-2:332167198415:project:f8fa4d62-dec9-4be3-8bbf-f37be0249897",
			"name": "Sahi Pro - Android",
			"created": "2020-04-17T02:53:39.857000+00:00"
		}
	]
}
      3. Set android.aws.devicefarm.main_app.arn property.
        To get the ARN of the main test app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
        infoReplace <<project_arn>> with the above project ARN
        {
	"uploads": [
		....
		{
			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:f8fa4d62-dec9-4be3-8bbf-f37be0249897/7fc96cdc-6ccc-404a-8ba7-ac35957ae751",
			"name": "AndroidApp.apk",
			"created": "2020-04-17T03:36:52.807000+00:00",
			"type": "ANDROID_APP",
			"status": "SUCCEEDED",
			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/7fc96cdc-6ccc-404a-8ba7-ac35957ae751/AndroidApp.apk?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200417T050507Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74ENYFBITRQ%2F20200417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=dfd1f87ddc29c4c68e56d7d7ddd636cba2810b8d9472e1682760209ee7e4774b",
			"metadata": "{\"device_admin\":false,\"activity_name\":\"com.sahipro.SahiTestApp.MainActivity\",\"version_name\":\"1.0\",\"screens\":[\"small\",\"normal\",\"large\",\"xlarge\"],\"error_type\":null,\"sdk_version\":\"18\",\"package_name\":\"com.sahipro.SahiTestApp\",\"version_code\":\"1\",\"native_code\":[],\"target_sdk_version\":\"26\"}",
			"contentType": "application/octet-stream",
			"category": "PRIVATE"
		}
	]
}
      4. Set android.aws.devicefarm.sahi_app.arn property.
        To get the ARN of the sahi instrumentation app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
        infoReplace <<project_arn>> with the above project ARN
        {
	"uploads": [
		....
		{
			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:f8fa4d62-dec9-4be3-8bbf-f37be0249897/348c5af7-f13d-417d-aa16-805f4dc6c7e5",
			"name": "app-debug.apk",
			"created": "2020-04-17T03:53:49.858000+00:00",
			"type": "ANDROID_APP",
			"status": "SUCCEEDED",
			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/348c5af7-f13d-417d-aa16-805f4dc6c7e5/app-debug.apk?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200417T050507Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74ENYFBITRQ%2F20200417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=0ccd167ea46837fb74fe54320825fc523a4a4f5243775d87ede5d8744286c24a",
			"metadata": "{\"device_admin\":false,\"activity_name\":\"\",\"version_name\":\"9.0.0\",\"screens\":[\"small\",\"normal\",\"large\",\"xlarge\"],\"error_type\":null,\"sdk_version\":\"18\",\"package_name\":\"in.co.sahi.android\",\"version_code\":\"900\",\"native_code\":[],\"target_sdk_version\":\"24\"}",
			"contentType": "application/octet-stream",
			"category": "PRIVATE"
        }
	]
}
      5. Set android.aws.devicefarm.sahi_app_test.arn property.
        To get the ARN of the sahi instrumentaion test app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
        infoReplace <<project_arn>> with the above project ARN
        {
	"uploads": [
		....
		{
			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:f8fa4d62-dec9-4be3-8bbf-f37be0249897/6142cb93-671a-437e-abd3-6f2a4e5c6948",
			"name": "app-debug-androidTest.apk",
			"created": "2020-04-17T03:57:51.722000+00:00",
			"type": "INSTRUMENTATION_TEST_PACKAGE",
			"status": "SUCCEEDED",
			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/6142cb93-671a-437e-abd3-6f2a4e5c6948/app-debug-androidTest.apk?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200417T050507Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74ENYFBITRQ%2F20200417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=805d11d2d673996750ed3aaa9156e0559cb8320765afde97268435ed3663187a",
			"metadata": "{\"test_package_name\":null,\"instrumentation_runner\":\"android.support.test.runner.AndroidJUnitRunner\",\"error_type\":null,\"package_name\":\"in.co.sahi.android.test\"}",
			"contentType": "application/octet-stream",
			"category": "PRIVATE"
		}
	]
}
      6. Configure android_devices.xml

        • Get AWS Device Pool ARN by executing aws devicefarm list-device-pools --arn <<arn_of_project>> in Command Prompt/Terminal.
          infoReplace <<project_arn>> with the above project ARN
          {
	"devicePools": [
		....
		{
			"arn": "arn:aws:devicefarm:us-west-2:332167198415:devicepool:f8fa4d62-dec9-4be3-8bbf-f37be0249897/9c614bb4-b950-48b4-b85e-2d54a4b677e0",
			"name": "Google Pixel 2",
			"type": "PRIVATE",
			"rules": [
				{
					"attribute": "ARN",
					"operator": "IN",
					"value": "[\"arn:aws:devicefarm:us-west-2::device:5F20BBED05F74D6288D51236B0FB9895\"]"
				}
			]
		}
	]
}
        • Configure device pool arn in android_devices.xml. 
          <!-- Google pixel device with 5 capacity. -->
<androidDevice>
	<name>AWS-Google-Pixel</name>
	<displayName>Google Pixel 2</displayName>
	<deviceId aws="true" capacity="5">arn:aws:devicefarm:us-west-2:332167198415:devicepool:f8fa4d62-dec9-4be3-8bbf-f37be0249897/9c614bb4-b950-48b4-b85e-2d54a4b677e0</deviceId>
</androidDevice>
  10. Open Sahi Pro Editor and execute sahitests/android/testSampleApp.sah script.
    • Click on Playback.
    • Select Android as Start Mode.
    • Select AWS-Google-Pixel as Android Device.
  11. Open Sahi Pro Script Report for the above executed script.

iOS Integration

  1. Goto: AWS Device Farm
  2. Create a new project under Mobile device testing projects.
  3. Click on Project settings.
  4. OPTIONAL: Set the maximum Execution Timeout according to the script execution timeout.
  5. Click on Device Pools.
  6. Click on Create a new device pool. Give a name to device pool and add one device in it. Click on Save device pool → Done.
    infoNote: Add only ONE device in each device pool. Each device pool is considered as a Single Device for Sahi Pro Mobile Add-on.
  7. Create IPA files for Uploads in next step.
    1. Generate IPA file for your application which needs to test.
      • Open Finder → Desktop, create a new folder and name it Payload.
        • warningFolder name should be Payload must required and no other files should be present in the Payload folder.
      • Copy the test-appication.app file, and paste it in the Payload folder.
      • Compress the Payload folder into .zip file.
      • Change the file name and extension of Payload.zip to test-application.ipa.
    2. Generate IPA file for SahiUITestAutomator-Runner.app.
      • Open Finder → Desktop, create a new folder and name it Payload.
        • warningFolder name should be Payload must required and no other files should be present in the Payload folder.
      • Copy the SahiUITestAutomator-Runner.app file from <SahiPro>/iOS/build_output/DebugLocation/Device/<xcode_version>/Debug-iphoneos location, and paste it in the Payload folder.
        infoIf SahiUITestAutomator-Runner.app is not present then follow steps mentioned in Testing on a Device document.
        The iOS Version of device should be same as iOS version of device which we selected at the time of Device pool creation.
      • Compress the Payload folder into .zip file.
      • Change the file name and extension of Payload.zip to SahiUITestAutomator-Runner.ipa.
  8. Click on Uploads.
  9. Click on Upload a file and upload 2 IPAs.
    1. Test Application: Select Type as iOS App.
      infoUploading test app test-appication.app which is generated in Step - 7a
    2. Sahi SahiUITestAutomator-Runner App: Select Type as XCTest UI Test Package and upload SahiUITestAutomator-Runner.app which is generated in Step - 7b.
  10. Configure <SahiPro>/userdata/config/aws_devicefarm.properties file.
    1. Click on the All Settings button on the Dashboard. It will open up all Settings dialog.
    2. Click on the Sahi Pro Settings button. It will open up Configure Sahi Pro page.
    3. Select the AWS DeviceFarm Configuration link on the left panel of Configure Sahi Pro page. This will open page similar to image given below.

    • Using Device Farm Configuration UI
      1. Select Type:iOS and click on the Fetch Device Farm Projects button to fetch all created device farm projects.
      2. Select Device Farm Project: Sahi Pro - iOS
        Select Test Application: SahiTestApp.ipa
        Select Sahi App Test: SahiUITestAutomator-Runner.ipa

        Click on Update aws_devicefarm.properties button. This will update the aws_devicefarm.properties file.
      3. Configure ios_devices.xml

        Select Device Pool: Apple iPhone 14
        Set Device Capacity: 5 (default is 1).
        Click on Generate button to generate the device xml. Rename the name tag value to AWS-iPhone and click on Update in ios_devices.xml button. This will update the ios_devices.xml file.
        infoRepeat this step, if you wish to configure multiple device pools.
        infoYou can also select more than one device pool at a time to generate iOS device xml.
    • Using Command Line
      1. Click on Show in Text view button.
      2. Set ios.aws.devicefarm.project.arn property.
        To get the ARN of the project, execute aws devicefarm list-projects in Command Prompt/Terminal. 
        {
	"projects": [
		{
			"arn": "arn:aws:devicefarm:us-west-2:3325467198415:project:16csd039-a132-4683-v3d7-cswsdadf7c093",
			"name": "Sahi Pro - iOS",
			"defaultJobTimeoutMinutes": 10,
			"created": "2019-06-18T11:46:52.589000+00:00"
		}
	]
}
      3. Set ios.aws.devicefarm.main_app.arn property.
        To get the ARN of the main test app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
        infoReplace <<project_arn>> with the above project ARN
        {
	"uploads": [
		....
		{
			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:16csd039-a132-4683-v3d7-cswsdadf7c093/c1bcddb2-be82-49fd-d632-eac5c31e4f77",
			"name": "SahiTestApp.ipa",
			"created": "2020-03-03T15:08:56.950000+00:00",
			"type": "IOS_APP",
			"status": "SUCCEEDED",
			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Aupload:16csd039-a132-4683-v3d7-cswsdadf7c093/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3A16ccd039-a132-4683-a3d7-caedadf7c093/c1bcddb2-be82-49fd-d632-eac5c31e4f77/your-app.ipa?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200303T151908Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74ENYHBBSFBITRQ%2F20200303%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=f9a3c56126c60c3f5ad2e7f5fc70fb85153a2f31cf530d3e4c71d858eea42cd9",
			"metadata": "{\"activity_name\":\"\",\"minimum_arm\":7,\"error_type\":null,\"package_name\":\"com.sahipro.ios.test.SahiTestApp\",\"sdk_version\":930,\"files\":{},\"warning_type\":null,\"supported_os\":\"9.3\",\"executable\":\"SahiTestApp\",\"platform\":[\"iPhoneOS\"],\"form_factor\":[1,2]}",
			"contentType": "application/octet-stream",
			"category": "PRIVATE"
		}
	]
}
      4. Set ios.aws.devicefarm.sahi_app_test.arn property.
        To get the ARN of the sahi instrumentation app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
        infoReplace <<project_arn>> with the above project ARN
        {
	"uploads": [
		....
		{
			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:upload:16csd039-a132-4683-v3d7-cswsdadf7c093/0d63fefc-2cc5-4acd-8fe4-88395512e2cd",
			"name": "SahiUITestAutomator-Runner.ipa",
			"created": "2020-03-03T15:16:47.256000+00:00",
			"type": "XCTEST_UI_TEST_PACKAGE",
			"status": "SUCCEEDED",
			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Aupload:16csd039-a132-4683-v3d7-cswsdadf7c093/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3Ac1bcddb2-be82-49fd-d632-eac5c31e4f77/0d63fefc-2cc5-4acd-8fe4-88395512e2cd/SahiUITestAutomator-Runner.ipa?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200303T151908Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74EKMNSNYFBITRQ%2F20200303%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=c3d4e877361d5f76e11adacdbec188e30c5bef311661aadc28c7937111d91eb3",
			"metadata": "{\"test_package_name\":\"com.sahipro.ios.UITestAutomator_ff5e470f0e28c043d20b48006a0f14e90ca3\",\"activity_name\":\"\",\"warning_type\":null,\"supported_os\":\"8.0\",\"executable\":\"SahiUITestAutomator-Runner\",\"platform\":[\"iPhoneOS\"],\"minimum_arm\":7,\"error_type\":null,\"package_name\":\"com.sahipro.ios.UITestAutomator_ff5e470f0e28c043d20b48006a0f14e90ca3.xctrunner\",\"sdk_version\":800,\"files\":{},\"test_executable\":\"SahiUITestAutomator\",\"form_factor\":[1,2]}",
			"contentType": "application/octet-stream",
			"category": "PRIVATE"
        }
	]
}
      5. Restart Sahi.
      6. Configure ios_devices.xml

        • Get AWS Device Pool ARN by executing aws devicefarm list-device-pools --arn <<arn_of_project>> in Command Prompt/Terminal.
          infoReplace <<project_arn>> with the above project ARN
          {
	"devicePools": [
		....
		{
			{
			    "arn": "arn:aws:devicefarm:us-west-2:332167558415:devicepool:b83ss0c-84a8-4a4d-b9de-5fddsw3f64/9789465c-c2e2-4064-a043-83b977293c98",
			    "name": "iPhone 11",
			    "type": "PRIVATE",
			    "rules": [
			        {
			            "attribute": "ARN",
			            "operator": "IN",
			            "value": "[\"arn:aws:devicefarm:us-west-2::device:A9AD8EC023394AC2BFC5148593BD6883\"]"
			        }
			    ]
			}
		}
	]
}
        • Configure device pool arn in ios_devices.xml. 
          <!-- iPhone 11 device with 5 capacity. -->
<iOSDevice>
	<name>AWS-iPhone</name>
	<displayName>iPhone 11</displayName>
	<deviceId aws="true" capacity="5">arn:aws:devicefarm:us-west-2:332167558415:devicepool:b83ss0c-84a8-4a4d-b9de-5fddsw3f64/9789465c-c2e2-4064-a043-83b977293c98</deviceId>
	<type>device</type>
</iOSDevice>
  11. Open Sahi Pro Editor and execute testSampleApp.sah script.
    • Click on Playback.
    • Select iOS as Start Mode.
    • Select AWS-iPhone as iOS Device.
  12. Open Sahi Pro Script Report for the above executed script.