AI Assist Accessor APIs
abstract
Sahi Pro AI Assist Accessor APIs provide Artificial Intelligence capabilities to automate applications by using computer vision and machine learning algorithms. It is supported since Sahi Pro AI Assist Add-on.
_aiImage
Since: | Sahi Pro | Desktop Add-On | Mobile Add-On | SAP Add-On | AI Assist Add-On |
NA | NA | NA | NA | 10.0.0 |
Available for modes: Browser | Windows | Android(11.1.0) | iOS(11.1.0)
_aiImage($imagePath[, $allowedPixelDifferencePercentage[, $allowedColorDifference]])
Arguments
$imagePath | string | Relative path of the image to be searched. This path is relative to the parent snips directory. |
$allowedPixelDifferencePercentage | integer optional | This argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image. |
$allowedColorDifference | integer optional | While matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200 |
Returns
null |
Sahi Pro Flowcharts Action :Ai Image
Details
This is an Accessor API to identify an image on the screen using Image Recognition technology. To perform an action on _aiImage, use Native APIs.
This is an Accessor API to identify an image on the screen using Image Recognition technology. To perform an action on _aiImage, use Native APIs.
// Right click on save button on the screen
_rightClickNative(_aiImage("save_btn.png"));
// Click on the 2nd instance of submit button.
// To click on the 2nd instance of the submit button, the argument $imagePath should be appended with index 1 in square brackets.
_clickNative(_aiImage("submit_btn.png[1]"));
// Right click on the expand button.
_rightClick(_aiImage ("expand_btn.png", 10));
// Move the cursor over the expand button.
_mouseOverNative(_aiImage("expand_btn.png", 20, 210));
// Click 50 pixels to the right of save button.
var $a = _positionNative(_aiImage("save_btn.png"));
var $x = $a[0];
var $y = $a[1];
_clickNativeXY($x + 50, $y);
_aiText
Since: | Sahi Pro | Desktop Add-On | Mobile Add-On | SAP Add-On | AI Assist Add-On |
NA | NA | NA | NA | 10.0.0 |
Available for modes: Browser | Windows | Android(11.1.0) | iOS(11.1.0)
_aiText($text[, $zoomLevel[, $props]])
Arguments
$text | string | Text to be searched on the screen. |
$zoomLevel | integer optional | If a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300. |
$props | object optional | Additional properties to identify a text
1. level, string, optional, can be 'word', 'line' or 'para'. This property specifies the iterator level for identifying a text. By default, value is set as 'word' in "sahi.ai_assist.text.iterator.level" property. |
Returns
null |
Sahi Pro Flowcharts Action :Ai Text
Details
This is an Accessor API to identify a text on the screen using Optical Character Recognition technology. To perform an action on _aiText, use Native APIs.
This is an Accessor API to identify a text on the screen using Optical Character Recognition technology. To perform an action on _aiText, use Native APIs.
// Click on the text 'Order'.
_clickNative(_aiText("Order"));
// Right click on the 3rd instance of text 'Order'.
// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets.
_rightClickNative(_aiText("Order[2]"));
// Move the cursor over the text 'conditions'.
_mouseOverNative(_aiText("conditions", 400));
// Click at a position next to "PO_AZ3479" under table column "Select"
var $x = _positionNative(_aiText("Select"))[1];
var $y = _positionNative(_aiText("PO_AZ3479"))[0];
_clickNativeXY($x, $y);
// Click on the text 'Show all'
_clickNative(_aiText("Show all", null, {level:'line'}));
_byImageRecognition
Since: | Sahi Pro | Desktop Add-On | Mobile Add-On | SAP Add-On | AI Assist Add-On |
NA | NA | NA | NA | NA |
Available for modes: Browser | Windows
_byImageRecognition($imagePath[, $index[, $allowedPixelDifferencePercentage[, $allowedColorDifference]]])
Arguments
$imagePath | string | Relative path of the image to be searched. This path is relative to the parent snips directory |
$index | integer optional | Zero based index to find the match. This argument is useful when same image/icon appears more than once |
$allowedPixelDifferencePercentage | integer optional | This argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image. |
$allowedColorDifference | integer optional | While matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200 |
Returns
array of integers | An array [x,y] of the image's position in pixels |
Sahi Pro Flowcharts Action :By Image Recognition
Details
This API finds an image on the screen using Image Recognition technology. It returns the position of the given image.
This API finds an image on the screen using Image Recognition technology. It returns the position of the given image.
// Finds save button image on the screen
_byImageRecognition("save_btn.png");
// Click on the 2nd instance of submit button.
_clickNative(_byImageRecognition("submit_btn.png", 1));
// Right click on the expand button.
_rightClickNative(_byImageRecognition("expand_btn.png", 0, 10));
// Move the cursor over the expand button.
_mouseOverNative(_byImageRecognition("expand_btn.png", 0, 20, 210));
// Click 50 pixels to the right of save button.
var $a = _positionNative(_byImageRecognition("save_btn.png"));
var $x = $a[0];
var $y = $a[1];
_clickNativeXY($x + 50, $y);
dangerAPI _byImageRecognition was available only with the early access version of AI Assist Add-on. This API is deprecated since Sahi Pro 10.0.0. Use _aiImage instead.
_byOCRText
Since: | Sahi Pro | Desktop Add-On | Mobile Add-On | SAP Add-On | AI Assist Add-On |
NA | NA | NA | NA | NA |
Available for modes: Browser | Windows
_byOCRText($text[, $zoomLevel])
Arguments
$text | string | Text to be searched on the screen. |
$zoomLevel | integer optional | If a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300. |
Returns
array of integers | An array [x,y] of the text's position in pixels |
Sahi Pro Flowcharts Action :By OCRText
Details
This API finds the given text on the current screen using Optical Character Recognition technology. It returns the position of the given text.
This API finds the given text on the current screen using Optical Character Recognition technology. It returns the position of the given text.
// Click on the text 'Order'.
_clickNative(_byOCRText("Order"));
// Right click on the 3rd instance of text 'Order'.
// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets.
_rightClickNative(_byOCRText("Order[2]"));
// Move the cursor over the text 'conditions'.
_mouseOverNative(_byOCRText("conditions", 400));
// Click at a position next to "PO_AZ3479" under table column "Select"
var $x = _positionNative(_byOCRText("Select"))[1];
var $y = _positionNative(_byOCRText("PO_AZ3479"))[0];
_clickNativeXY($x, $y);
dangerAPI _byOCRText was available only with the early access version of AI Assist Add-on. This API is deprecated since Sahi Pro 10.0.0. Use _aiText instead.