Skip to content
Sahi Pro Flowcharts is retired. These pages document the last release (v12) and are no longer updated. Go to the current documentation.Flowcharts is retired. Current docs
Get Free Trial
Sahi Pro Flowcharts is retired. These pages document the last release (v12) and are no longer updated. Go to the current documentation.

Visual Assist Accessor APIs

Sahi Pro Visual Assist Accessor APIs provide Artificial Intelligence capabilities to automate applications by using computer vision and machine learning algorithms. It is supported since Sahi Pro Visual Assist Engine.

Since
Visual Assist Engine 12.0.0.10

Modes BrowserWindowsAndroidiOS

_visualImage($imagePath[, $allowedPixelDifferencePercentage[, $allowedColorDifference]])

Arguments

NameTypeDescription
$imagePathstringRelative path of the image to be searched. This path is relative to the parent snips directory.
$allowedPixelDifferencePercentageintegeroptionalThis argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image.
$allowedColorDifferenceintegeroptionalWhile matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200

Returns

null

This is an Accessor API to identify an image on the screen using Image Recognition technology. To perform an action on _visualImage, use Native APIs.

// Right click on save button on the screen
_rightClickNative(_visualImage("save_btn.png"));
// Click on the 2nd instance of submit button.
// To click on the 2nd instance of the submit button, the argument $imagePath should be appended with index 1 in square brackets.
_clickNative(_visualImage("submit_btn.png[1]"));
// Right click on the expand button.
_rightClick(_visualImage("expand_btn.png", 10));
// Move the cursor over the expand button.
_mouseOverNative(_visualImage("expand_btn.png", 20, 210));
// Click 50 pixels to the right of save button.
var $a = _positionNative(_visualImage("save_btn.png"));
var $x = $a[0];
var $y = $a[1];
_clickNativeXY($x + 50, $y);
Since
Visual Assist Engine 12.0.0.10

Modes BrowserWindowsAndroidiOS

_visualText($text[, $zoomLevel[, $props]])

Arguments

NameTypeDescription
$textstringText to be searched on the screen.
$zoomLevelintegeroptionalIf a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300.
$propsobjectoptionalAdditional properties to identify a text

1. level, string, optional, can be 'word', 'line' or 'para'.
This property specifies the iterator level for identifying a text.
By default, value is set as 'word' in "sahi.visual_assist.text.iterator.level" property.

Returns

null

This is an Accessor API to identify a text on the screen using Optical Character Recognition technology. To perform an action on _visualText, use Native APIs.

// Click on the text 'Order'.
_clickNative(_visualText("Order"));
// Right click on the 3rd instance of text 'Order'.
// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets.
_rightClickNative(_visualText("Order[2]"));
// Move the cursor over the text 'conditions'.
_mouseOverNative(_visualText("conditions", 400));
// Click at a position next to "PO_AZ3479" under table column "Select"
var $x = _positionNative(_visualText("Select"))[1];
var $y = _positionNative(_visualText("PO_AZ3479"))[0];
_clickNativeXY($x, $y);
// Click on the text 'Show all'
_clickNative(_visualText("Show all", null, {level:'line'}));
Since
Visual Assist Engine 10.0.0

Modes BrowserWindowsAndroid(11.1.0)iOS(11.1.0)

_aiImage($imagePath[, $allowedPixelDifferencePercentage[, $allowedColorDifference]])

Arguments

NameTypeDescription
$imagePathstringRelative path of the image to be searched. This path is relative to the parent snips directory.
$allowedPixelDifferencePercentageintegeroptionalThis argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image.
$allowedColorDifferenceintegeroptionalWhile matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200

Returns

null

This is an Accessor API to identify an image on the screen using Image Recognition technology. To perform an action on _aiImage, use Native APIs.

// Right click on save button on the screen
_rightClickNative(_aiImage("save_btn.png"));
// Click on the 2nd instance of submit button.
// To click on the 2nd instance of the submit button, the argument $imagePath should be appended with index 1 in square brackets.
_clickNative(_aiImage("submit_btn.png[1]"));
// Right click on the expand button.
_rightClick(_aiImage ("expand_btn.png", 10));
// Move the cursor over the expand button.
_mouseOverNative(_aiImage("expand_btn.png", 20, 210));
// Click 50 pixels to the right of save button.
var $a = _positionNative(_aiImage("save_btn.png"));
var $x = $a[0];
var $y = $a[1];
_clickNativeXY($x + 50, $y);
Since
Visual Assist Engine 10.0.0

Modes BrowserWindowsAndroid(11.1.0)iOS(11.1.0)

_aiText($text[, $zoomLevel[, $props]])

Arguments

NameTypeDescription
$textstringText to be searched on the screen.
$zoomLevelintegeroptionalIf a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300.
$propsobjectoptionalAdditional properties to identify a text

1. level, string, optional, can be 'word', 'line' or 'para'.
This property specifies the iterator level for identifying a text.
By default, value is set as 'word' in "sahi.ai_assist.text.iterator.level" property.

Returns

null

This is an Accessor API to identify a text on the screen using Optical Character Recognition technology. To perform an action on _aiText, use Native APIs.

// Click on the text 'Order'.
_clickNative(_aiText("Order"));
// Right click on the 3rd instance of text 'Order'.
// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets.
_rightClickNative(_aiText("Order[2]"));
// Move the cursor over the text 'conditions'.
_mouseOverNative(_aiText("conditions", 400));
// Click at a position next to "PO_AZ3479" under table column "Select"
var $x = _positionNative(_aiText("Select"))[1];
var $y = _positionNative(_aiText("PO_AZ3479"))[0];
_clickNativeXY($x, $y);
// Click on the text 'Show all'
_clickNative(_aiText("Show all", null, {level:'line'}));

Modes BrowserWindows

_byImageRecognition($imagePath[, $index[, $allowedPixelDifferencePercentage[, $allowedColorDifference]]])

Arguments

NameTypeDescription
$imagePathstringRelative path of the image to be searched. This path is relative to the parent snips directory
$indexintegeroptionalZero based index to find the match. This argument is useful when same image/icon appears more than once
$allowedPixelDifferencePercentageintegeroptionalThis argument specifies the pixel difference percentage that can be allowed. Default value is 20%. Recommended range is 0 to 20%. An optimal value for this argument may vary per image.
$allowedColorDifferenceintegeroptionalWhile matching the image pixel by pixel, there could be a color difference in the corresponding pixels. This argument specifies the color difference that can be allowed. Default value is 200. Recommended range is 0 to 200

Returns

array of integersAn array [x,y] of the image's position in pixels

This API finds an image on the screen using Image Recognition technology. It returns the position of the given image.

// Finds save button image on the screen
_byImageRecognition("save_btn.png");
// Click on the 2nd instance of submit button.
_clickNative(_byImageRecognition("submit_btn.png", 1));
// Right click on the expand button.
_rightClickNative(_byImageRecognition("expand_btn.png", 0, 10));
// Move the cursor over the expand button.
_mouseOverNative(_byImageRecognition("expand_btn.png", 0, 20, 210));
// Click 50 pixels to the right of save button.
var $a = _positionNative(_byImageRecognition("save_btn.png"));
var $x = $a[0];
var $y = $a[1];
_clickNativeXY($x + 50, $y);

Modes BrowserWindows

_byOCRText($text[, $zoomLevel])

Arguments

NameTypeDescription
$textstringText to be searched on the screen.
$zoomLevelintegeroptionalIf a text appears too small on the screen, an internal zoom would be necessary to identify such a text. This argument specifies the internal zoom percentage. Default value is 300.

Returns

array of integersAn array [x,y] of the text's position in pixels

This API finds the given text on the current screen using Optical Character Recognition technology. It returns the position of the given text.

// Click on the text 'Order'.
_clickNative(_byOCRText("Order"));
// Right click on the 3rd instance of text 'Order'.
// To right click on the 3rd instance of text 'Order', the argument $text should be appended with index 2 in square brackets.
_rightClickNative(_byOCRText("Order[2]"));
// Move the cursor over the text 'conditions'.
_mouseOverNative(_byOCRText("conditions", 400));
// Click at a position next to "PO_AZ3479" under table column "Select"
var $x = _positionNative(_byOCRText("Select"))[1];
var $y = _positionNative(_byOCRText("PO_AZ3479"))[0];
_clickNativeXY($x, $y);