AIPageCommand defines the JSON command protocol supported by executeAIPageCommand, including the input parameter format and response format of different commands. Before calling this API, the app needs to serialize the command object into a JSON string.
- command must be a JSON object string.
- The method field value is case-sensitive. Use the values listed in Command Overview.
- If the return value is not empty, it is a JSON string. The app can parse it using JSON.parse.
- When the page is unavailable, the command cannot be executed, or no result is returned, the API returns an empty string.
getFullDom
Obtains the complete DOM tree structure of the current web page. This command traverses from the document root node and returns a tree node list.
[h2]FullDomCommand
{
"method": "getFullDom",
"params": {
"wants": [
"rect",
"visible",
"xpath"
]
}
} [h2]Field Value Description of params.wants in getFullDom
| Value | Return Field | Return Type | Description |
| id | id | string | Node identifier generated by ArkWeb and returned on request. It does not indicate filtering nodes by the HTML id attribute. This value is encoded from the frame identifier, document scope identifier, and DOM node identifier, and is used to distinguish nodes in the returned result. It may change after page reload, frame rebuild, or DOM rebuild. It is returned only when a node identifier can be generated. To read the HTML id attribute, see attributes.id of the returned node. |
| tag | tag | string | Node tag name. Element nodes return the lowercase HTML tag name, text nodes return #text, and ShadowRoot nodes return #shadowRoot. |
| text | text | string | Node text content. This field is not returned when its value is empty. |
| title | title | string | Value of the node title attribute. This field is not returned when its value is empty. |
| aria-label | aria-label | string | Value of the node aria-label attribute. This field is not returned when its value is empty. |
| role | role | string | Node semantic role. This field is not returned when its value is empty. |
| aria-description | aria-description | string | Value of the node aria-description attribute. This field is not returned when its value is empty. |
| rect | rect | Object | Node rectangle information, including x, y, width, and height. |
| bounds | bounds | Object | Node rectangle information, including x, y, left, top, right, bottom, width, and height. |
| visible | visible | boolean | Node visibility. true indicates visible, and false indicates invisible. |
| isInViewport | isInViewport | boolean | Whether the node is within the current viewport. true indicates within the current viewport, and false indicates not within the current viewport. |
| clickable | clickable | boolean | Whether the node is clickable. true indicates clickable, and false indicates not clickable. |
| touchable | touchable | boolean | Whether the node is touchable. true indicates touchable, and false indicates not touchable. The current judgment logic is the same as that of clickable. |
| scrollable | scrollable | boolean | Whether the node is scrollable. true indicates scrollable, and false indicates not scrollable. |
| inputable | inputable | boolean | Whether the node accepts input. true indicates that input is accepted, and false indicates that input is not accepted. |
| url | url | string | Node associated URL. It is read in the order of href, src, action, data, and poster and converted to a complete URL. This field is not returned when its value is empty. |
| xpath | xpath | string | Node XPath. This field is not returned when its value is empty. |
| hover | hover | string | Value of the node cursor style. This field is not returned when its value is empty. |
| mouseover | mouseover | boolean | Whether the node declares the mouseover inline event. true indicates declared, and false indicates not declared. |
| mouseenter | mouseenter | boolean | Whether the node declares the mouseenter inline event. true indicates declared, and false indicates not declared. |
| value | value | string | When the node is a select element, returns the currently selected value. |
| options | options | Array<Object> | When the node is a select element, returns the option list. |
| value_text | value_text | Array<Object> | When the node is a select element, returns the option list. |
| focusable | focusable | boolean | Whether the node can obtain focus. true indicates that focus can be obtained, and false indicates that focus cannot be obtained. |
| editable | editable | boolean | Whether the node is editable. true indicates editable, and false indicates not editable. |
| settable | settable | boolean | Whether the node value can be set. true indicates that the value can be set, and false indicates that the value cannot be set. |
| checked | checked | boolean | Whether the node is in the checked state. true indicates checked, and false indicates not checked. |
| expanded | expanded | boolean | Whether the node is in the expanded state. true indicates expanded, and false indicates not expanded. |
| pressed | pressed | boolean | Whether the node is in the pressed state. true indicates pressed, and false indicates not pressed. |
| selected | selected | boolean | Whether the node is in the selected state. true indicates selected, and false indicates not selected. |
| required | required | boolean | Whether the node is required. true indicates required, and false indicates not required. |
| autocomplete | autocomplete | string | Node autocomplete information. The aria-autocomplete attribute is read first, and then the autocomplete attribute. This field is not returned when its value is empty. |
| keyshortcuts | keyshortcuts | string | Value of the node aria-keyshortcuts attribute. This field is not returned when its value is empty. |
[h2]FullDomResult
| Field | Subfield | Field Item | Type | Description |
| url | - | - | string | Current page URL. |
| title | - | - | string | Current page title. |
| children_nodes | - | - | Array<Object> | List of root nodes of the DOM tree. |
| children_nodes | - | id | string | Node identifier generated by ArkWeb, not the HTML id attribute. Returned only when wants contains id and the node identifier can be generated. |
| children_nodes | - | tag | string | Node tag name. |
| children_nodes | - | text | string | Node text content. |
| children_nodes | - | title | string | Value of the node's title attribute. |
| children_nodes | - | aria-label | string | Value of the node's aria-label attribute. |
| children_nodes | - | role | string | Node semantic role. |
| children_nodes | - | aria-description | string | Value of the node's aria-description attribute. |
| children_nodes | - | rect | Object | Node rectangle information. |
| children_nodes | rect | x | number | X coordinate of the upper left corner of the node rectangle. Unit: px. |
| children_nodes | rect | y | number | Y coordinate of the upper left corner of the node rectangle, relative to the upper left corner of the viewport of the frame to which the current node belongs. Unit: px. |
| children_nodes | rect | width | number | Width of the node rectangle. Unit: px. |
| children_nodes | rect | height | number | Height of the node rectangle. Unit: px. |
| children_nodes | - | bounds | Object | Node rectangle information. |
| children_nodes | bounds | x | number | X coordinate of the upper left corner of the node rectangle. Unit: px. |
| children_nodes | bounds | y | number | Y coordinate of the upper left corner of the node rectangle. Unit: px. |
| children_nodes | bounds | left | number | Left boundary of the node rectangle. Unit: px. |
| children_nodes | bounds | top | number | Top boundary of the node rectangle. Unit: px. |
| children_nodes | bounds | right | number | Right boundary of the node rectangle. Unit: px. |
| children_nodes | bounds | bottom | number | Bottom boundary of the node rectangle. Unit: px. |
| children_nodes | bounds | width | number | Width of the node rectangle. Unit: px. |
| children_nodes | bounds | height | number | Height of the node rectangle. Unit: px. |
| children_nodes | - | visible | boolean | Whether the node is visible. The value true means visible, and false means invisible. |
| children_nodes | - | isInViewport | boolean | Whether the node is in the current viewport. The value true means in the current viewport, and false means not in the current viewport. |
| children_nodes | - | clickable | boolean | Whether the node is clickable. The value true means clickable, and false means not clickable. |
| children_nodes | - | touchable | boolean | Whether the node is touchable. The value true means touchable, and false means not touchable. |
| children_nodes | - | scrollable | boolean | Whether the node is scrollable. The value true means scrollable, and false means not scrollable. |
| children_nodes | - | inputable | boolean | Whether the node is inputable. The value true means inputable, and false means not inputable. |
| children_nodes | - | url | string | Node associated URL. |
| children_nodes | - | xpath | string | Node XPath. |
| children_nodes | - | hover | string | Value of the node's cursor style. |
| children_nodes | - | mouseover | boolean | Whether the node declares the mouseover inline event. The value true means declared, and false means not declared. |
| children_nodes | - | mouseenter | boolean | Whether the node declares the mouseenter inline event. The value true means declared, and false means not declared. |
| children_nodes | - | value | string | Currently selected value of the select element. |
| children_nodes | - | options | Array<Object> | List of options of the select element. |
| children_nodes | - | value_text | Array<Object> | List of options of the select element. |
| children_nodes | options/value_text | value | string | Option value of the select element. |
| children_nodes | options/value_text | text | string | Option text of the select element. |
| children_nodes | - | focusable | boolean | Whether the node can obtain focus. The value true means focusable, and false means not focusable. |
| children_nodes | - | editable | boolean | Whether the node is editable. The value true means editable, and false means not editable. |
| children_nodes | - | settable | boolean | Whether the node is settable. The value true means settable, and false means not settable. |
| children_nodes | - | checked | boolean | Whether the node is in the checked state. The value true means in the checked state, and false means not in the checked state. |
| children_nodes | - | expanded | boolean | Whether the node is in the expanded state. The value true means in the expanded state, and false means not in the expanded state. |
| children_nodes | - | pressed | boolean | Whether the node is in the pressed state. The value true means in the pressed state, and false means not in the pressed state. |
| children_nodes | - | selected | boolean | Whether the node is in the selected state. The value true means in the selected state, and false means not in the selected state. |
| children_nodes | - | required | boolean | Whether the node is required. The value true means required, and false means not required. |
| children_nodes | - | autocomplete | string | Node autocomplete information. |
| children_nodes | - | keyshortcuts | string | Value of the node's aria-keyshortcuts attribute. |
| children_nodes | - | attributes | Object | Set of HTML attributes. |
| children_nodes | attributes | <attributeName> | string | HTML attribute name and its corresponding value. |
| children_nodes | - | children_nodes | Array<Object> | List of child nodes of the current node. |
- The node fields in children_nodes are determined by both the default fields and wants. When a field value is empty, some string fields and the attributes field are not returned.
- getFullDom skips the subtrees of script, noscript, style, template, and slot elements.
- getFullDom traverses open or closed author Shadow DOM, and does not return user-agent Shadow DOM.
- For parseable child frames, the root node of the child frame is merged into the children_nodes of the corresponding frame owner node.
[h2]Request Example
{
"method": "getFullDom",
"params": {
"wants": [
"rect",
"visible",
"xpath"
]
}
} [h2]Response Example
{
"url": "https://www.example.com/",
"title": "Example",
"children_nodes": [
{
"tag": "html",
"attributes": {
"lang": "en"
},
"rect": {
"x": 0,
"y": 0,
"width": 360,
"height": 640
},
"visible": true,
"xpath": "/html[1]",
"children_nodes": [
{
"tag": "body",
"children_nodes": [
{
"tag": "button",
"attributes": {
"id": "submit"
},
"rect": {
"x": 24,
"y": 36,
"width": 96,
"height": 40
},
"visible": true,
"xpath": "/html[1]/body[1]/button[1]",
"children_nodes": [
{
"tag": "#text",
"text": "Submit",
"children_nodes": []
}
]
}
]
}
]
}
]
} getLiteDom
Obtains the lightweight DOM node list of the current web page. This command first filters nodes based on rules, and then returns the specified fields based on wants.
[h2]LiteDomCommand
{
"method": "getLiteDom",
"params": {
"rules": {
"tags": ["button", "a"],
"isInViewport": true
},
"wants": [
"id",
"tag",
"text",
"rect",
"clickable",
"xpath",
{
"attributes": ["id", "class", "href"]
}
]
}
} [h2]Field Value Description of params.wants in getLiteDom
| Value | Return Field | Return Type | Description |
| id | id | string | Requests the node identifier generated by ArkWeb. It does not indicate filtering nodes by the HTML id attribute. This value is encoded by combining the frame identifier, document scope identifier, and DOM node identifier, and is used to distinguish nodes in the returned result. It may change after the page is reloaded, the frame is rebuilt, or the DOM is rebuilt. It is returned only when a node identifier can be generated. To read the HTML id attribute, request the id attribute through an attributes object item in wants, and check attributes.id of the returned node. |
| tag | tag | string | Node tag name. Returns the lowercase HTML tag name. |
| text | text | string | Node text content. This field is not returned when the field value is empty. |
| title | title | string | Value of the node title attribute. This field is not returned when the field value is empty. |
| aria-label | aria-label | string | Value of the node aria-label attribute. This field is not returned when the field value is empty. |
| role | role | string | Node semantic role. This field is not returned when the field value is empty. |
| aria-description | aria-description | string | Value of the node aria-description attribute. This field is not returned when the field value is empty. |
| rect | rect | Object | Node rectangle information, including x, y, width, and height. |
| bounds | bounds | Object | Node rectangle information, including x, y, left, top, right, bottom, width, and height. |
| visible | visible | boolean | Node visibility. true indicates visible, and false indicates invisible. |
| isInViewport | isInViewport | boolean | Whether the node is in the current viewport. true indicates in the current viewport, and false indicates not in the current viewport. |
| clickable | clickable | boolean | Whether the node is clickable. true indicates clickable, and false indicates non-clickable. |
| touchable | touchable | boolean | Whether the node is touchable. true indicates touchable, and false indicates non-touchable. The current judgment logic is the same as that of clickable. |
| scrollable | scrollable | boolean | Whether the node is scrollable. true indicates scrollable, and false indicates non-scrollable. |
| inputable | inputable | boolean | Whether the node is inputable. true indicates inputable, and false indicates non-inputable. |
| url | url | string | Node associated URL. It is read in the order of href, src, action, data, and poster and converted to a complete URL. This field is not returned when the field value is empty. |
| xpath | xpath | string | Node XPath. This field is not returned when the field value is empty. |
| hover | hover | string | Value of the node cursor style. This field is not returned when the field value is empty. |
| mouseover | mouseover | boolean | Whether the node declares the mouseover inline event. true indicates declared, and false indicates not declared. |
| mouseenter | mouseenter | boolean | Whether the node declares the mouseenter inline event. true indicates declared, and false indicates not declared. |
| value | value | string | When the node is a select element, returns the currently selected value. |
| options | options | Array<Object> | When the node is a select element, returns the option list. |
| value_text | value_text | Array<Object> | When the node is a select element, returns the option list. |
| focusable | focusable | boolean | Whether the node can obtain focus. true indicates that it can obtain focus, and false indicates that it cannot obtain focus. |
| editable | editable | boolean | Whether the node is editable. true indicates editable, and false indicates non-editable. |
| settable | settable | boolean | Whether the node can be set with a value. true indicates that it can be set with a value, and false indicates that it cannot be set with a value. |
| checked | checked | boolean | Whether the node is in the checked state. true indicates checked, and false indicates not checked. |
| expanded | expanded | boolean | Whether the node is in the expanded state. true indicates expanded, and false indicates not expanded. |
| pressed | pressed | boolean | Whether the node is in the pressed state. true indicates pressed, and false indicates not pressed. |
| selected | selected | boolean | Whether the node is in the selected state. true indicates selected, and false indicates not selected. |
| required | required | boolean | Whether the node is a required item. true indicates required, and false indicates not required. |
| autocomplete | autocomplete | string | Node autocomplete information. The aria-autocomplete attribute is read first, and then the autocomplete attribute. This field is not returned when the field value is empty. |
| keyshortcuts | keyshortcuts | string | Value of the node aria-keyshortcuts attribute. This field is not returned when the field value is empty. |
[h2]LiteDomResult
| Field | Subfield | Field Item | Type | Description |
| url | - | - | string | Current page URL. |
| title | - | - | string | Current page title. |
| nodes | - | - | Array<Object> | Node list that meets the filtering rules. |
| nodes | - | id | string | Node identifier generated by ArkWeb, not the HTML id attribute. Returned only when wants contains id and a node identifier can be generated. |
| nodes | - | tag | string | Node tag name. |
| nodes | - | text | string | Node text content. |
| nodes | - | title | string | Value of the node title attribute. |
| nodes | - | aria-label | string | Value of the node aria-label attribute. |
| nodes | - | role | string | Node semantic role. |
| nodes | - | aria-description | string | Value of the node aria-description attribute. |
| nodes | - | rect | Object | Node rectangle information. |
| nodes | rect | x | number | X coordinate of the upper-left corner of the node rectangle. Unit: px. |
| nodes | rect | y | number | Y coordinate of the upper-left corner of the node rectangle, relative to the upper-left corner of the viewport of the frame to which the current node belongs. Unit: px. |
| nodes | rect | width | number | Width of the node rectangle. Unit: px. |
| nodes | rect | height | number | Height of the node rectangle. Unit: px. |
| nodes | - | bounds | Object | Node rectangle information. |
| nodes | bounds | x | number | X coordinate of the upper-left corner of the node rectangle. Unit: px. |
| nodes | bounds | y | number | Y coordinate of the upper-left corner of the node rectangle. Unit: px. |
| nodes | bounds | left | number | Left boundary of the node rectangle. Unit: px. |
| nodes | bounds | top | number | Top boundary of the node rectangle. Unit: px. |
| nodes | bounds | right | number | Right boundary of the node rectangle. Unit: px. |
| nodes | bounds | bottom | number | Bottom boundary of the node rectangle. Unit: px. |
| nodes | bounds | width | number | Width of the node rectangle. Unit: px. |
| nodes | bounds | height | number | Height of the node rectangle. Unit: px. |
| nodes | - | visible | boolean | Whether the node is visible. true indicates visible, and false indicates invisible. |
| nodes | - | isInViewport | boolean | Whether the node is in the current viewport. true indicates in the current viewport, and false indicates not in the current viewport. |
| nodes | - | clickable | boolean | Whether the node is clickable. true indicates clickable, and false indicates not clickable. |
| nodes | - | touchable | boolean | Whether the node is touchable. true indicates touchable, and false indicates not touchable. |
| nodes | - | scrollable | boolean | Whether the node is scrollable. true indicates scrollable, and false indicates not scrollable. |
| nodes | - | inputable | boolean | Whether the node accepts input. true indicates that input is accepted, and false indicates that input is not accepted. |
| nodes | - | url | string | Node associated URL. |
| nodes | - | xpath | string | Node XPath. |
| nodes | - | hover | string | Value of the node cursor style. |
| nodes | - | mouseover | boolean | Whether the node declares the mouseover inline event. true indicates declared, and false indicates not declared. |
| nodes | - | mouseenter | boolean | Whether the node declares the mouseenter inline event. true indicates declared, and false indicates not declared. |
| nodes | - | value | string | Currently selected value of the select element. |
| nodes | - | options | Array<Object> | Option list of the select element. |
| nodes | - | value_text | Array<Object> | Option list of the select element. |
| nodes | options/value_text | value | string | Option value of the select element. |
| nodes | options/value_text | text | string | Option text of the select element. |
| nodes | - | focusable | boolean | Whether the node can obtain focus. true indicates that focus can be obtained, and false indicates that focus cannot be obtained. |
| nodes | - | editable | boolean | Whether the node is editable. true indicates editable, and false indicates not editable. |
| nodes | - | settable | boolean | Whether a value can be set for the node. true indicates that a value can be set, and false indicates that a value cannot be set. |
| nodes | - | checked | boolean | Whether the node is in the checked state. true indicates checked, and false indicates not checked. |
| nodes | - | expanded | boolean | Whether the node is in the expanded state. true indicates expanded, and false indicates not expanded. |
| nodes | - | pressed | boolean | Whether the node is in the pressed state. true indicates pressed, and false indicates not pressed. |
| nodes | - | selected | boolean | Whether the node is in the selected state. true indicates selected, and false indicates not selected. |
| nodes | - | required | boolean | Whether the node is a required item. true indicates required, and false indicates not required. |
| nodes | - | autocomplete | string | Node autocomplete information. |
| nodes | - | keyshortcuts | string | Value of the node aria-keyshortcuts attribute. |
| nodes | - | attributes | Object | Set of HTML attributes. |
| nodes | attributes | <attributeName> | string | HTML attribute name and its corresponding attribute value. |
- The node fields in nodes are determined by both the default fields and wants. When a field value is empty, some string fields and the attributes field are not returned.
- getLiteDom returns only element nodes, not text nodes.
- getLiteDom skips the subtrees of script, noscript, style, template, and slot elements.
- getLiteDom traverses open or closed author Shadow DOM, and does not return user-agent Shadow DOM.
- The result includes nodes in resolvable child frames. Nodes maintain the DOM traversal order within each frame, and child frame nodes are appended as a whole after the current frame node list. The merge order of multiple child frames is determined by the internal frame token set.
[h2]Request Example
{
"method": "getLiteDom",
"params": {
"rules": {
"tags": ["button", "a"],
"isInViewport": true
},
"wants": [
"id",
"tag",
"text",
"rect",
"clickable",
"xpath",
{
"attributes": ["id", "class", "href"]
}
]
}
} [h2]Response Example
{
"url": "https://www.example.com/",
"title": "Example",
"nodes": [
{
"id": "frameToken|documentToken|12",
"tag": "button",
"rect": {
"x": 24,
"y": 36,
"width": 96,
"height": 40
},
"clickable": true,
"xpath": "/html[1]/body[1]/button[1]",
"attributes": {
"id": "submit",
"class": "primary"
}
}
]
} screenCapture
Obtains a viewport screenshot of the current page or a screenshot of a target element within the viewport, and returns the image data encoded in Base64.
[h2]ScreenCaptureResult
On success, returns a Base64-encoded string in PNG format.
- If the format of nodeid is invalid, {"code": 392, "message": "invalid param: nodeid"} is returned. If the frameToken or documentToken in nodeid does not match the current page, {"code": 392, "message": "invalid param: nodeid, token mismatch"} is returned.
- If the target element is not found on the page based on nodeid or xpath, {"code": 352, "message": "element not found"} is returned.
[h2]Example
Obtain a screenshot of the target element by node identifier:
{
"method": "screenCapture",
"params": {
"nodeid": "frameToken|documentToken|12"
}
} Obtain a screenshot of the target element by XPath:
{
"method": "screenCapture",
"params": {
"xpath": "/html/body/div/p[2]/a"
}
}
- Developers need to replace nodeid or xpath with actual values. These can be obtained from the id field or xpath field returned by getFullDom or getLiteDom.
// xxx.ets
import { webview } from '@kit.ArkWeb';
import { BusinessError } from '@kit.BasicServicesKit';
interface CaptureParams {
xpath?: string;
nodeid?: string;
}
interface PageCommand {
method: string;
params?: CaptureParams;
}
@Entry
@Component
struct Index {
private controller: webview.WebviewController = new webview.WebviewController();
@State imgData: string = '';
@State statusMsg: string = '';
async capture() {
this.imgData = '';
this.statusMsg = '';
try {
const cmd: PageCommand = {
method: 'screenCapture',
params: {
xpath: "/html/body/div/p[2]/a"
}
};
const res = await this.controller.executeAIPageCommand(JSON.stringify(cmd)) as string;
if (res.includes('"code"')) {
this.statusMsg = `Screenshot failed: ${res} `;
return;
}
this.imgData = res;
this.statusMsg = '✅ Screenshot succeeded';
} catch (e) {
const error = e as BusinessError;
this.statusMsg = `Screenshot failed: ${error.message}`;
}
}
build() {
Row() {
Web({ src: 'https://www.example.com', controller: this.controller })
.width('75%')
.height('100%')
Column({ space: 10 }) {
Button('Take screenshot')
.width('100%')
.onClick(() => this.capture())
Image(this.imgData ? `data:image/png;base64,${this.imgData}` : '')
.width('100%')
.aspectRatio(1)
.backgroundColor('#F0F0F0')
.objectFit(ImageFit.Contain)
.border({ width: 1, color: '#DCDCDC' })
if (this.statusMsg) {
Text(this.statusMsg)
.width('100%')
.fontSize(12)
.fontColor(this.statusMsg.includes('✅') ? '#4CAF50' : '#F44336')
.textAlign(TextAlign.Center)
.padding(8)
.backgroundColor(this.statusMsg.includes('✅') ? '#E8F5E9' : '#FFEBEE')
.borderRadius(4)
}
}
.width('25%')
.padding(10)
.height('100%')
}
.width('100%')
.height('100%')
}
} getZoomLevel
Obtains the scale ratio of the current web page. This command is a pure query operation and is not restricted by zoomControlAccess.
[h2]GetZoomLevelCommand
{
"method": "getZoomLevel",
"params": {}
} [h2]ZoomLevelResult
| Name | Type | Description |
| code | number | Result code. For details, see Command Execution Result Code Description. |
| message | string | Result description. The value is "success" when the command succeeds. |
| zoomLevel | number | Scale ratio of the current page. Returned on success. |
[h2]Test Page
<!-- index.html -->
<!DOCTYPE html>
<html>
<body>
<h1>Zoom Level Demo</h1>
<p>The current zoom level can be queried via getZoomLevel and modified via setZoomLevel (see AIPageInteraction for details).</p>
</body>
</html> [h2]Request Example
{
"method": "getZoomLevel",
"params": {}
} [h2]Response Example
On success (the current page scale ratio is 1.5):
{
"code": 10,
"message": "success",
"zoomLevel": 1.5
} On failure:
132 (browser or host is null, usually indicating that the Web instance is unavailable):
{
"code": 132,
"message": "browser or host is null"
}