Visual Assist Accessor Functions
Sahi Pro Visual Assist Accessor functions provide Artificial Intelligence capabilities to automate applications by using computer vision and machine learning algorithms. It is supported since Sahi Pro Visual Assist Engine.
_visualImage
Section titled “_visualImage”- Since
- Visual Assist Engine 12.0.0.10
Modes BrowserWindowsAndroidiOS
_visualImage($imagePath[, $allowedPixelDifferencePercentage[, $allowedColorDifference]])Arguments
| Name | Type | Description |
|---|---|---|
$imagePath | string | Relative path of the image to be searched. This path is relative to the parent snips directory. |
$allowedPixelDifferencePercentage | integeroptional | This argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image. |
$allowedColorDifference | integeroptional | While matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200 |
Returns
null |
This is an Accessor function to identify an image on the screen using Image Recognition technology. To perform an action on _visualImage, use Native functions.
// Right click on save button on the screen_rightClickNative(_visualImage("save_btn.png"));
// Click on the 2nd instance of submit button.// To click on the 2nd instance of the submit button, the argument $imagePath should be appended with index 1 in square brackets._clickNative(_visualImage("submit_btn.png[1]"));
// Right click on the expand button._rightClick(_visualImage("expand_btn.png", 10));
// Move the cursor over the expand button._mouseOverNative(_visualImage("expand_btn.png", 20, 210));
// Click 50 pixels to the right of save button.var $a = _positionNative(_visualImage("save_btn.png"));var $x = $a[0];var $y = $a[1];_clickNativeXY($x + 50, $y);_visualText
Section titled “_visualText”- Since
- Visual Assist Engine 12.0.0.10
Modes BrowserWindowsAndroidiOS
_visualText($text[, $zoomLevel[, $props]])Arguments
| Name | Type | Description |
|---|---|---|
$text | string | Text to be searched on the screen. |
$zoomLevel | integeroptional | If a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300. |
$props | objectoptional | Additional properties to identify a text 1. level, string, optional, can be 'word', 'line' or 'para'. This property specifies the iterator level for identifying a text. By default, value is set as 'word' in "sahi.visual_assist.text.iterator.level" property. |
Returns
null |
This is an Accessor function to identify a text on the screen using Optical Character Recognition technology. To perform an action on _visualText, use Native functions.
// Click on the text 'Order'._clickNative(_visualText("Order"));
// Right click on the 3rd instance of text 'Order'.// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets._rightClickNative(_visualText("Order[2]"));
// Move the cursor over the text 'conditions'._mouseOverNative(_visualText("conditions", 400));
// Click at a position next to "PO_AZ3479" under table column "Select"var $x = _positionNative(_visualText("Select"))[1];var $y = _positionNative(_visualText("PO_AZ3479"))[0];_clickNativeXY($x, $y);
// Click on the text 'Show all'_clickNative(_visualText("Show all", null, {level:'line'}));_aiImage (Deprecated)
Section titled “_aiImage (Deprecated)”- Since
- Visual Assist Engine 10.0.0
Modes BrowserWindowsAndroid(11.1.0)iOS(11.1.0)
_aiImage($imagePath[, $allowedPixelDifferencePercentage[, $allowedColorDifference]])Arguments
| Name | Type | Description |
|---|---|---|
$imagePath | string | Relative path of the image to be searched. This path is relative to the parent snips directory. |
$allowedPixelDifferencePercentage | integeroptional | This argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image. |
$allowedColorDifference | integeroptional | While matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200 |
Returns
null |
This is an Accessor function to identify an image on the screen using Image Recognition technology. To perform an action on _aiImage, use Native functions.
// Right click on save button on the screen_rightClickNative(_aiImage("save_btn.png"));
// Click on the 2nd instance of submit button.// To click on the 2nd instance of the submit button, the argument $imagePath should be appended with index 1 in square brackets._clickNative(_aiImage("submit_btn.png[1]"));
// Right click on the expand button._rightClick(_aiImage ("expand_btn.png", 10));
// Move the cursor over the expand button._mouseOverNative(_aiImage("expand_btn.png", 20, 210));
// Click 50 pixels to the right of save button.var $a = _positionNative(_aiImage("save_btn.png"));var $x = $a[0];var $y = $a[1];_clickNativeXY($x + 50, $y);_aiText (Deprecated)
Section titled “_aiText (Deprecated)”- Since
- Visual Assist Engine 10.0.0
Modes BrowserWindowsAndroid(11.1.0)iOS(11.1.0)
_aiText($text[, $zoomLevel[, $props]])Arguments
| Name | Type | Description |
|---|---|---|
$text | string | Text to be searched on the screen. |
$zoomLevel | integeroptional | If a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300. |
$props | objectoptional | Additional properties to identify a text 1. level, string, optional, can be 'word', 'line' or 'para'. This property specifies the iterator level for identifying a text. By default, value is set as 'word' in "sahi.ai_assist.text.iterator.level" property. |
Returns
null |
This is an Accessor function to identify a text on the screen using Optical Character Recognition technology. To perform an action on _aiText, use Native functions.
// Click on the text 'Order'._clickNative(_aiText("Order"));
// Right click on the 3rd instance of text 'Order'.// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets._rightClickNative(_aiText("Order[2]"));
// Move the cursor over the text 'conditions'._mouseOverNative(_aiText("conditions", 400));
// Click at a position next to "PO_AZ3479" under table column "Select"var $x = _positionNative(_aiText("Select"))[1];var $y = _positionNative(_aiText("PO_AZ3479"))[0];_clickNativeXY($x, $y);
// Click on the text 'Show all'_clickNative(_aiText("Show all", null, {level:'line'}));_byImageRecognition (Deprecated)
Section titled “_byImageRecognition (Deprecated)”Modes BrowserWindows
_byImageRecognition($imagePath[, $index[, $allowedPixelDifferencePercentage[, $allowedColorDifference]]])Arguments
| Name | Type | Description |
|---|---|---|
$imagePath | string | Relative path of the image to be searched. This path is relative to the parent snips directory |
$index | integeroptional | Zero based index to find the match. This argument is useful when same image/icon appears more than once |
$allowedPixelDifferencePercentage | integeroptional | This argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image. |
$allowedColorDifference | integeroptional | While matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200 |
Returns
array of integers | An array [x,y] of the image's position in pixels |
This function finds an image on the screen using Image Recognition technology. It returns the position of the given image.
// Finds save button image on the screen_byImageRecognition("save_btn.png");
// Click on the 2nd instance of submit button._clickNative(_byImageRecognition("submit_btn.png", 1));
// Right click on the expand button._rightClickNative(_byImageRecognition("expand_btn.png", 0, 10));
// Move the cursor over the expand button._mouseOverNative(_byImageRecognition("expand_btn.png", 0, 20, 210));
// Click 50 pixels to the right of save button.var $a = _positionNative(_byImageRecognition("save_btn.png"));var $x = $a[0];var $y = $a[1];_clickNativeXY($x + 50, $y);_byOCRText (Deprecated)
Section titled “_byOCRText (Deprecated)”Modes BrowserWindows
_byOCRText($text[, $zoomLevel])Arguments
| Name | Type | Description |
|---|---|---|
$text | string | Text to be searched on the screen. |
$zoomLevel | integeroptional | If a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300. |
Returns
array of integers | An array [x,y] of the text's position in pixels |
This function finds the given text on the current screen using Optical Character Recognition technology. It returns the position of the given text.
// Click on the text 'Order'._clickNative(_byOCRText("Order"));
// Right click on the 3rd instance of text 'Order'.// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets._rightClickNative(_byOCRText("Order[2]"));
// Move the cursor over the text 'conditions'._mouseOverNative(_byOCRText("conditions", 400));
// Click at a position next to "PO_AZ3479" under table column "Select"var $x = _positionNative(_byOCRText("Select"))[1];var $y = _positionNative(_byOCRText("PO_AZ3479"))[0];_clickNativeXY($x, $y);
