Sahi Documentation

AI Assist Accessor APIs

abstract Sahi Pro AI Assist Accessor APIs provide Artificial Intelligence capabilities to automate applications by using computer vision and machine learning algorithms. It is supported since Sahi Pro AI Assist Add-on.

_aiImage

Since: Sahi ProSahi OSSahi Pro StarterDesktop Add-OnMobile Add-OnSAP Add-OnAI Assist Add-On
NANANANANANA10.0.0

Available for modes: Browser | Windows

_aiImage($imagePath[, $allowedPixelDifferencePercentage[, $allowedColorDifference]])

Arguments
$imagePathstring Relative path of the image to be searched. This path is relative to the parent snips directory.
$allowedPixelDifferencePercentageinteger optionalThis argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image.
$allowedColorDifferenceinteger optionalWhile matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200

Returns
null

Details

This is an Accessor API to identify an image on the screen using Image Recognition technology. To perform an action on _aiImage, use Native APIs.
// Right click on save button on the screen
_rightClickNative(_aiImage("save_btn.png"));

// Click on the 2nd instance of submit button.
// To click on the 2nd instance of the submit button, the argument $imagePath should be appended with index 1 in square brackets.
_clickNative(_aiImage("submit_btn.png[1]"));

// Right click on the expand button.
_rightClick(_aiImage ("expand_btn.png", 10));

// Move the cursor over the expand button.
_mouseOverNative(_aiImage("expand_btn.png", 20, 210));

// Click 50 pixels to the right of save button.
var $a = _positionNative(_aiImage("save_btn.png"));
var $x = $a[0];
var $y = $a[1];
_clickNativeXY($x + 50, $y);




_aiText

Since: Sahi ProSahi OSSahi Pro StarterDesktop Add-OnMobile Add-OnSAP Add-OnAI Assist Add-On
NANANANANANA10.0.0

Available for modes: Browser | Windows

_aiText($text[, $zoomLevel])

Arguments
$textstring Text to be searched on the screen.
$zoomLevelinteger optionalIf a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300.

Returns
null

Details

This is an Accessor API to identify a text on the screen using Optical Character Recognition technology. To perform an action on _aiText, use Native APIs.
// Click on the text 'Order'.
_clickNative(_aiText("Order"));

// Right click on the 3rd instance of text 'Order'.
// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets.
_rightClickNative(_aiText("Order[2]"));

// Move the cursor over the text 'conditions'.
_mouseOverNative(_aiText("conditions", 400));

// Click at a position next to "PO_AZ3479" under table column "Select"
var $x = _positionNative(_aitext("Select"))[1];
var $y = _positionNative(_aiText("PO_AZ3479"))[0];
_clickNativeXY($x, $y);




_byImageRecognition

Since: Sahi ProSahi OSSahi Pro StarterDesktop Add-OnMobile Add-OnSAP Add-OnAI Assist Add-On
NANANANANANANA

Available for modes: Browser | Windows

_byImageRecognition($imagePath[, $index[, $allowedPixelDifferencePercentage[, $allowedColorDifference]]])

Arguments
$imagePathstring Relative path of the image to be searched. This path is relative to the parent snips directory
$indexinteger optionalZero based index to find the match. This argument is useful when same image/icon appears more than once
$allowedPixelDifferencePercentageinteger optionalThis argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image.
$allowedColorDifferenceinteger optionalWhile matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200

Returns
array of integersAn array [x,y] of the image's position in pixels

Details

This API finds an image on the screen using Image Recognition technology. It returns the position of the given image.
// Finds save button image on the screen
_byImageRecognition("save_btn.png");

// Click on the 2nd instance of submit button.
_clickNative(_byImageRecognition("submit_btn.png", 1));

// Right click on the expand button.
_rightClickNative(_byImageRecognition("expand_btn.png", 0, 10));

// Move the cursor over the expand button.
_mouseOverNative(_byImageRecognition("expand_btn.png", 0, 20, 210));

// Click 50 pixels to the right of save button.
var $a = _positionNative(_byImageRecognition("save_btn.png"));
var $x = $a[0];
var $y = $a[1];
_clickNativeXY($x + 50, $y);


dangerAPI _byImageRecognition was available only with the early access version of AI Assist Add-on. This API is deprecated since Sahi Pro 10.0.0. Use _aiImage instead.


_byOCRText

Since: Sahi ProSahi OSSahi Pro StarterDesktop Add-OnMobile Add-OnSAP Add-OnAI Assist Add-On
NANANANANANANA

Available for modes: Browser | Windows

_byOCRText($text[, $zoomLevel])

Arguments
$textstring Text to be searched on the screen.
$zoomLevelinteger optionalIf a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300.

Returns
array of integersAn array [x,y] of the text's position in pixels

Details

This API finds the given text on the current screen using Optical Character Recognition technology. It returns the position of the given text.
// Click on the text 'Order'.
_clickNative(_byOCRText("Order"));

// Right click on the 3rd instance of text 'Order'.
// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets.
_rightClickNative(_byOCRText("Order[2]"));

// Move the cursor over the text 'conditions'.
_mouseOverNative(_byOCRText("conditions", 400));

// Click at a position next to "PO_AZ3479" under table column "Select"
var $x = _positionNative(_byOCRText("Select"))[1];
var $y = _positionNative(_byOCRText("PO_AZ3479"))[0];
_clickNativeXY($x, $y);


dangerAPI _byOCRText was available only with the early access version of AI Assist Add-on. This API is deprecated since Sahi Pro 10.0.0. Use _aiText instead.