AI Assist Accessor APIs

abstract Sahi Pro AI Assist Accessor APIs provide Artificial Intelligence capabilities to automate applications by using computer vision and machine learning algorithms. It is supported since Sahi Pro AI Assist Add-on.

_aiImage

Since: Sahi ProDesktop Add-OnMobile Add-OnSAP Add-OnAI Assist Add-On
NANANANA10.0.0

Available for modes: Browser | Windows | Android(11.1.0) | iOS(11.1.0)

_aiImage($imagePath[, $allowedPixelDifferencePercentage[, $allowedColorDifference]])

Arguments
$imagePathstring Relative path of the image to be searched. This path is relative to the parent snips directory.
$allowedPixelDifferencePercentageinteger optionalThis argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image.
$allowedColorDifferenceinteger optionalWhile matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200

Returns
null

Sahi Pro Flowcharts Action :Ai Image

Details

This is an Accessor API to identify an image on the screen using Image Recognition technology. To perform an action on _aiImage, use Native APIs.
// Right click on save button on the screen
_rightClickNative(_aiImage("save_btn.png"));

// Click on the 2nd instance of submit button.
// To click on the 2nd instance of the submit button, the argument $imagePath should be appended with index 1 in square brackets.
_clickNative(_aiImage("submit_btn.png[1]"));

// Right click on the expand button.
_rightClick(_aiImage ("expand_btn.png", 10));

// Move the cursor over the expand button.
_mouseOverNative(_aiImage("expand_btn.png", 20, 210));

// Click 50 pixels to the right of save button.
var $a = _positionNative(_aiImage("save_btn.png"));
var $x = $a[0];
var $y = $a[1];
_clickNativeXY($x + 50, $y);




_aiText

Since: Sahi ProDesktop Add-OnMobile Add-OnSAP Add-OnAI Assist Add-On
NANANANA10.0.0

Available for modes: Browser | Windows | Android(11.1.0) | iOS(11.1.0)

_aiText($text[, $zoomLevel[, $props]])

Arguments
$textstring Text to be searched on the screen.
$zoomLevelinteger optionalIf a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300.
$propsobject optionalAdditional properties to identify a text

1. level, string, optional, can be 'word', 'line' or 'para'.
This property specifies the iterator level for identifying a text.
By default, value is set as 'word' in "sahi.ai_assist.text.iterator.level" property.

Returns
null

Sahi Pro Flowcharts Action :Ai Text

Details

This is an Accessor API to identify a text on the screen using Optical Character Recognition technology. To perform an action on _aiText, use Native APIs.
// Click on the text 'Order'.
_clickNative(_aiText("Order"));

// Right click on the 3rd instance of text 'Order'.
// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets.
_rightClickNative(_aiText("Order[2]"));

// Move the cursor over the text 'conditions'.
_mouseOverNative(_aiText("conditions", 400));

// Click at a position next to "PO_AZ3479" under table column "Select"
var $x = _positionNative(_aiText("Select"))[1];
var $y = _positionNative(_aiText("PO_AZ3479"))[0];
_clickNativeXY($x, $y);

// Click on the text 'Show all'
_clickNative(_aiText("Show all", null, {level:'line'}));




_byImageRecognition

Since: Sahi ProDesktop Add-OnMobile Add-OnSAP Add-OnAI Assist Add-On
NANANANANA

Available for modes: Browser | Windows

_byImageRecognition($imagePath[, $index[, $allowedPixelDifferencePercentage[, $allowedColorDifference]]])

Arguments
$imagePathstring Relative path of the image to be searched. This path is relative to the parent snips directory
$indexinteger optionalZero based index to find the match. This argument is useful when same image/icon appears more than once
$allowedPixelDifferencePercentageinteger optionalThis argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image.
$allowedColorDifferenceinteger optionalWhile matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200

Returns
array of integersAn array [x,y] of the image's position in pixels

Sahi Pro Flowcharts Action :By Image Recognition

Details

This API finds an image on the screen using Image Recognition technology. It returns the position of the given image.
// Finds save button image on the screen
_byImageRecognition("save_btn.png");

// Click on the 2nd instance of submit button.
_clickNative(_byImageRecognition("submit_btn.png", 1));

// Right click on the expand button.
_rightClickNative(_byImageRecognition("expand_btn.png", 0, 10));

// Move the cursor over the expand button.
_mouseOverNative(_byImageRecognition("expand_btn.png", 0, 20, 210));

// Click 50 pixels to the right of save button.
var $a = _positionNative(_byImageRecognition("save_btn.png"));
var $x = $a[0];
var $y = $a[1];
_clickNativeXY($x + 50, $y);


dangerAPI _byImageRecognition was available only with the early access version of AI Assist Add-on. This API is deprecated since Sahi Pro 10.0.0. Use _aiImage instead.


_byOCRText

Since: Sahi ProDesktop Add-OnMobile Add-OnSAP Add-OnAI Assist Add-On
NANANANANA

Available for modes: Browser | Windows

_byOCRText($text[, $zoomLevel])

Arguments
$textstring Text to be searched on the screen.
$zoomLevelinteger optionalIf a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300.

Returns
array of integersAn array [x,y] of the text's position in pixels

Sahi Pro Flowcharts Action :By OCRText

Details

This API finds the given text on the current screen using Optical Character Recognition technology. It returns the position of the given text.
// Click on the text 'Order'.
_clickNative(_byOCRText("Order"));

// Right click on the 3rd instance of text 'Order'.
// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets.
_rightClickNative(_byOCRText("Order[2]"));

// Move the cursor over the text 'conditions'.
_mouseOverNative(_byOCRText("conditions", 400));

// Click at a position next to "PO_AZ3479" under table column "Select"
var $x = _positionNative(_byOCRText("Select"))[1];
var $y = _positionNative(_byOCRText("PO_AZ3479"))[0];
_clickNativeXY($x, $y);


dangerAPI _byOCRText was available only with the early access version of AI Assist Add-on. This API is deprecated since Sahi Pro 10.0.0. Use _aiText instead.