Mobile Automation integration with AWS Device Farm

abstract This section helps you for integration Sahi with AWS Device Farm.

Pre-requisites

  1. Sahi Pro Mobile Add-on and AWS Command Line Interface (CLI) version 2 should be installed with in the AWS system.
    • Please refer here to quickly setup AWS CLI.
    • Please make sure that Default region name is us-west-2.
    • Please make sure that Default output format is json.
  2. cURL should be installed on the system.
    • To verify, run curl in the command line.

Android Integration

  1. Goto: AWS Device Farm
  2. Create a new project under Mobile device testing projects.
  3. Click on Project settings.
  4. OPTIONAL: Set the maximum Execution Timeout according to the script execution timeout.
  5. Click on Device Pools.
  6. Click on Create a new device pool. Give a name to device pool and add one device in it. Click on Save device poolDone.
    infoNote: Add only ONE device in each device pool. Each device pool is considered as a Single Device for Sahi Pro Mobile Add-on.
  7. Click on Uploads.
  8. Click on Upload a file and upload 3 APKs.
    1. Test Application: Select Type as Android App.
      infoUploading Sahi test app from <SahiPro>/userdata/android/testapps/AndroidApp.apk
    2. Sahi Instrumentation App: Select Type as Android App and upload APK from <SahiPro>/android/lib/app-debug.apk.
    3. Sahi Instrumentation App: Select Type as Instrumentation Test Package and upload APK from <SahiPro>/android/lib/app-debug-androidTest.apk.
  9. Configure <SahiPro>/userdata/config/aws_devicefarm.properties file. Set properties:
    • android.aws.devicefarm.project.arn.
      To get the ARN of the project, execute aws devicefarm list-projects in Command Prompt/Terminal.
      {
      	"projects": [
      		{
      			"arn": "arn:aws:devicefarm:us-west-2:332167198415:project:f8fa4d62-dec9-4be3-8bbf-f37be0249897",
      			"name": "Sahi Pro - Android",
      			"created": "2020-04-17T02:53:39.857000+00:00"
      		}
      	]
      }
    • android.aws.devicefarm.main_app.arn.
      To get the ARN of the main test app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
      infoReplace <<project_arn>> with the above project ARN
      {
      	"uploads": [
      		....
      		{
      			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:f8fa4d62-dec9-4be3-8bbf-f37be0249897/7fc96cdc-6ccc-404a-8ba7-ac35957ae751",
      			"name": "AndroidApp.apk",
      			"created": "2020-04-17T03:36:52.807000+00:00",
      			"type": "ANDROID_APP",
      			"status": "SUCCEEDED",
      			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/7fc96cdc-6ccc-404a-8ba7-ac35957ae751/AndroidApp.apk?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200417T050507Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74ENYFBITRQ%2F20200417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=dfd1f87ddc29c4c68e56d7d7ddd636cba2810b8d9472e1682760209ee7e4774b",
      			"metadata": "{\"device_admin\":false,\"activity_name\":\"com.sahipro.SahiTestApp.MainActivity\",\"version_name\":\"1.0\",\"screens\":[\"small\",\"normal\",\"large\",\"xlarge\"],\"error_type\":null,\"sdk_version\":\"18\",\"package_name\":\"com.sahipro.SahiTestApp\",\"version_code\":\"1\",\"native_code\":[],\"target_sdk_version\":\"26\"}",
      			"contentType": "application/octet-stream",
      			"category": "PRIVATE"
      		}
      	]
      }
    • android.aws.devicefarm.sahi_app.arn.
      To get the ARN of the sahi instrumentation app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
      infoReplace <<project_arn>> with the above project ARN
      {
      	"uploads": [
      		....
      		{
      			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:f8fa4d62-dec9-4be3-8bbf-f37be0249897/348c5af7-f13d-417d-aa16-805f4dc6c7e5",
      			"name": "app-debug.apk",
      			"created": "2020-04-17T03:53:49.858000+00:00",
      			"type": "ANDROID_APP",
      			"status": "SUCCEEDED",
      			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/348c5af7-f13d-417d-aa16-805f4dc6c7e5/app-debug.apk?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200417T050507Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74ENYFBITRQ%2F20200417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=0ccd167ea46837fb74fe54320825fc523a4a4f5243775d87ede5d8744286c24a",
      			"metadata": "{\"device_admin\":false,\"activity_name\":\"\",\"version_name\":\"9.0.0\",\"screens\":[\"small\",\"normal\",\"large\",\"xlarge\"],\"error_type\":null,\"sdk_version\":\"18\",\"package_name\":\"in.co.sahi.android\",\"version_code\":\"900\",\"native_code\":[],\"target_sdk_version\":\"24\"}",
      			"contentType": "application/octet-stream",
      			"category": "PRIVATE"
              }
      	]
      }
    • android.aws.devicefarm.sahi_app_test.arn.
      To get the ARN of the sahi instrumentaion test app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
      infoReplace <<project_arn>> with the above project ARN
      {
      	"uploads": [
      		....
      		{
      			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:f8fa4d62-dec9-4be3-8bbf-f37be0249897/6142cb93-671a-437e-abd3-6f2a4e5c6948",
      			"name": "app-debug-androidTest.apk",
      			"created": "2020-04-17T03:57:51.722000+00:00",
      			"type": "INSTRUMENTATION_TEST_PACKAGE",
      			"status": "SUCCEEDED",
      			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3Af8fa4d62-dec9-4be3-8bbf-f37be0249897/6142cb93-671a-437e-abd3-6f2a4e5c6948/app-debug-androidTest.apk?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200417T050507Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74ENYFBITRQ%2F20200417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=805d11d2d673996750ed3aaa9156e0559cb8320765afde97268435ed3663187a",
      			"metadata": "{\"test_package_name\":null,\"instrumentation_runner\":\"android.support.test.runner.AndroidJUnitRunner\",\"error_type\":null,\"package_name\":\"in.co.sahi.android.test\"}",
      			"contentType": "application/octet-stream",
      			"category": "PRIVATE"
      		}
      	]
      }
  10. Configure android_devices.xml
    • Get AWS Device Pool ARN by executing aws devicefarm list-device-pools --arn <<arn_of_project>> in Command Prompt/Terminal.
      infoReplace <<project_arn>> with the above project ARN
      {
      	"devicePools": [
      		....
      		{
      			"arn": "arn:aws:devicefarm:us-west-2:332167198415:devicepool:f8fa4d62-dec9-4be3-8bbf-f37be0249897/9c614bb4-b950-48b4-b85e-2d54a4b677e0",
      			"name": "Google Pixel 2",
      			"type": "PRIVATE",
      			"rules": [
      				{
      					"attribute": "ARN",
      					"operator": "IN",
      					"value": "[\"arn:aws:devicefarm:us-west-2::device:5F20BBED05F74D6288D51236B0FB9895\"]"
      				}
      			]
      		}
      	]
      }
    • Configure device pool arn in android_devices.xml.
      <!-- Google pixel device with 5 capacity. -->
      <androidDevice>
      	<name>AWS-Google-Pixel</name>
      	<displayName>Google Pixel 2</displayName>
      	<deviceId aws="true" capacity="5">arn:aws:devicefarm:us-west-2:332167198415:devicepool:f8fa4d62-dec9-4be3-8bbf-f37be0249897/9c614bb4-b950-48b4-b85e-2d54a4b677e0</deviceId>
      </androidDevice>
  11. Open Sahi Pro Editor and execute demo/android/testSampleApp.sah script.
    • Click on Playback.
    • Select Android as Start Mode.
    • Select AWS-Google-Pixel as Android Device.
  12. Open Sahi Pro Script Report for the above executed script.

iOS Integration

  1. Goto: AWS Device Farm
  2. Create a new project under Mobile device testing projects.
  3. Click on Project settings.
  4. OPTIONAL: Set the maximum Execution Timeout according to the script execution timeout.
  5. Click on Device Pools.
  6. Click on Create a new device pool. Give a name to device pool and add one device in it. Click on Save device poolDone.
    infoNote: Add only ONE device in each device pool. Each device pool is considered as a Single Device for Sahi Pro Mobile Add-on.
  7. Create IPA files for Uploads in next step.
    1. Generate IPA file for your application which needs to test.
      • Open FinderDesktop, create a new folder and name it Payload.
      • warningFolder name should be Payload must required and no other files should be present in the Payload folder.
      • Copy the test-appication.app file, and paste it in the Payload folder.
      • Compress the Payload folder into .zip file.
      • Change the file name and extension of Payload.zip to test-application.ipa.
    2. Generate IPA file for SahiUITestAutomator-Runner.app.
      • Open FinderDesktop, create a new folder and name it Payload.
      • warningFolder name should be Payload must required and no other files should be present in the Payload folder.
      • Copy the SahiUITestAutomator-Runner.app file from <SahiPro>/iOS/build_output/DebugLocation/Device/<xcode_version>/Debug-iphoneos location, and paste it in the Payload folder.
        infoIf SahiUITestAutomator-Runner.app is not present then follow steps mentioned in Testing on a Device document.
        The iOS Version of device should be same as iOS version of device which we selected at the time of Device pool creation.
      • Compress the Payload folder into .zip file.
      • Change the file name and extension of Payload.zip to SahiUITestAutomator-Runner.ipa.
  8. Click on Uploads.
  9. Click on Upload a file and upload 2 IPAs.
    1. Test Application: Select Type as iOS App.
      infoUploading test app test-appication.app which is generated in Step - 7a
    2. Sahi SahiUITestAutomator-Runner App: Select Type as XCTest UI Test Package and upload SahiUITestAutomator-Runner.app which is generated in Step - 7b.
  10. Configure <SahiPro>/userdata/config/aws_devicefarm.properties file. Set properties:
    • ios.aws.devicefarm.project.arn.
      To get the ARN of the project, execute aws devicefarm list-projects in Command Prompt/Terminal.
      {
      	"projects": [
      		{
      			"arn": "arn:aws:devicefarm:us-west-2:3325467198415:project:16csd039-a132-4683-v3d7-cswsdadf7c093",
      			"name": "Sahi Pro - iOS",
      			"defaultJobTimeoutMinutes": 10,
      			"created": "2019-06-18T11:46:52.589000+00:00"
      		}
      	]
      }
    • ios.aws.devicefarm.main_app.arn.
      To get the ARN of the main test app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
      infoReplace <<project_arn>> with the above project ARN
      {
      	"uploads": [
      		....
      		{
      			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:16csd039-a132-4683-v3d7-cswsdadf7c093/c1bcddb2-be82-49fd-d632-eac5c31e4f77",
      			"name": "SahiTestApp.ipa",
      			"created": "2020-03-03T15:08:56.950000+00:00",
      			"type": "IOS_APP",
      			"status": "SUCCEEDED",
      			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Aupload:16csd039-a132-4683-v3d7-cswsdadf7c093/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3A16ccd039-a132-4683-a3d7-caedadf7c093/c1bcddb2-be82-49fd-d632-eac5c31e4f77/your-app.ipa?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200303T151908Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74ENYHBBSFBITRQ%2F20200303%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=f9a3c56126c60c3f5ad2e7f5fc70fb85153a2f31cf530d3e4c71d858eea42cd9",
      			"metadata": "{\"activity_name\":\"\",\"minimum_arm\":7,\"error_type\":null,\"package_name\":\"com.sahipro.ios.test.SahiTestApp\",\"sdk_version\":930,\"files\":{},\"warning_type\":null,\"supported_os\":\"9.3\",\"executable\":\"SahiTestApp\",\"platform\":[\"iPhoneOS\"],\"form_factor\":[1,2]}",
      			"contentType": "application/octet-stream",
      			"category": "PRIVATE"
      		}
      	]
      }
    • ios.aws.devicefarm.sahi_app.arn.
      To get the ARN of the sahi instrumentation app, execute aws devicefarm list-uploads --arn <<project_arn>> in Command Prompt/Terminal.
      infoReplace <<project_arn>> with the above project ARN
      {
      	"uploads": [
      		....
      		{
      			"arn": "arn:aws:devicefarm:us-west-2:332167198415:upload:upload:16csd039-a132-4683-v3d7-cswsdadf7c093/0d63fefc-2cc5-4acd-8fe4-88395512e2cd",
      			"name": "SahiUITestAutomator-Runner.ipa",
      			"created": "2020-03-03T15:16:47.256000+00:00",
      			"type": "XCTEST_UI_TEST_PACKAGE",
      			"status": "SUCCEEDED",
      			"url": "https://prod-us-west-2-uploads.s3-us-west-2.amazonaws.com/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aproject%3Aupload:16csd039-a132-4683-v3d7-cswsdadf7c093/uploads/arn%3Aaws%3Adevicefarm%3Aus-west-2%3A332167198415%3Aupload%3Ac1bcddb2-be82-49fd-d632-eac5c31e4f77/0d63fefc-2cc5-4acd-8fe4-88395512e2cd/SahiUITestAutomator-Runner.ipa?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200303T151908Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=86400&X-Amz-Credential=AKIAJSORV74EKMNSNYFBITRQ%2F20200303%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=c3d4e877361d5f76e11adacdbec188e30c5bef311661aadc28c7937111d91eb3",
      			"metadata": "{\"test_package_name\":\"com.sahipro.ios.UITestAutomator_ff5e470f0e28c043d20b48006a0f14e90ca3\",\"activity_name\":\"\",\"warning_type\":null,\"supported_os\":\"8.0\",\"executable\":\"SahiUITestAutomator-Runner\",\"platform\":[\"iPhoneOS\"],\"minimum_arm\":7,\"error_type\":null,\"package_name\":\"com.sahipro.ios.UITestAutomator_ff5e470f0e28c043d20b48006a0f14e90ca3.xctrunner\",\"sdk_version\":800,\"files\":{},\"test_executable\":\"SahiUITestAutomator\",\"form_factor\":[1,2]}",
      			"contentType": "application/octet-stream",
      			"category": "PRIVATE"
              }
      	]
      }
  11. Restart Sahi.
  12. Configure ios_devices.xml
    • Get AWS Device Pool ARN by executing aws devicefarm list-device-pools --arn <<arn_of_project>> in Command Prompt/Terminal.
      infoReplace <<project_arn>> with the above project ARN
      {
      	"devicePools": [
      		....
      		{
      			{
      			    "arn": "arn:aws:devicefarm:us-west-2:332167558415:devicepool:b83ss0c-84a8-4a4d-b9de-5fddsw3f64/9789465c-c2e2-4064-a043-83b977293c98",
      			    "name": "iPhone 11",
      			    "type": "PRIVATE",
      			    "rules": [
      			        {
      			            "attribute": "ARN",
      			            "operator": "IN",
      			            "value": "[\"arn:aws:devicefarm:us-west-2::device:A9AD8EC023394AC2BFC5148593BD6883\"]"
      			        }
      			    ]
      			}
      		}
      	]
      }
    • Configure device pool arn in ios_devices.xml.
      <!-- iPhone 11 device with 5 capacity. -->
      <iOSDevice>
      	<name>AWS-iPhone</name>
      	<displayName>iPhone 11</displayName>
      	<deviceId aws="true" capacity="5">arn:aws:devicefarm:us-west-2:332167558415:devicepool:b83ss0c-84a8-4a4d-b9de-5fddsw3f64/9789465c-c2e2-4064-a043-83b977293c98</deviceId>
      	<type>device</type>
      </iOSDevice>
  13. Open Sahi Pro Editor and execute testSampleApp.sah script.
    • Click on Playback.
    • Select iOS as Start Mode.
    • Select AWS-iPhone as iOS Device.
  14. Open Sahi Pro Script Report for the above executed script.