Visual-Tree Payload Contract

Consume the compact visual-tree v2 contract returned by native, MAUI, React Native, Flutter, and DOM providers.

Ansight visual-tree providers return a compact v2 payload for live ui.get_visual_tree calls and recorded session visual-tree evidence. Native, MAUI, React Native, Flutter, and DOM sources share the same platform-neutral node shape while retaining source-specific format, platform, and optional properties values.

Top-Level Shape

{
  "format": "ansight.native.visual-tree.compact.v2",
  "platform": "android",
  "source": "native",
  "snapshotId": "native:42:3f1e...",
  "revision": 42,
  "nodeIdentity": {
    "scope": "snapshot",
    "source": "native",
    "staleAfterRevision": 42
  },
  "capturedAtUtc": "2026-08-21T01:02:03.456Z",
  "types": [
    "android.widget.FrameLayout",
    "android.widget.Button"
  ],
  "root": {
    "id": "root",
    "typeId": 0,
    "role": "container",
    "supportedActions": [],
    "interactable": false,
    "visible": true,
    "enabled": true,
    "focusable": false,
    "childCount": 1,
    "visual": { "opacity": 1 },
    "children": [
      {
        "id": "root.0",
        "typeId": 1,
        "automationId": "checkout.submit",
        "label": "Pay now",
        "role": "button",
        "supportedActions": ["tap"],
        "interactable": true,
        "visible": true,
        "enabled": true,
        "focusable": true,
        "childCount": 0,
        "bounds": { "x": 24, "y": 640, "width": 320, "height": 48 },
        "visual": {
          "foreground": "#FFFFFFFF",
          "background": "#FF0066CC",
          "opacity": 1,
          "text": "Pay now"
        },
        "children": []
      }
    ]
  }
}

The exact format identifies the provider, for example ansight.native.visual-tree.compact.v2, ansight.react.visual-tree.compact.v2, ansight.flutter.visual-tree.compact.v2, or ansight.dom.visual-tree.compact.v2.

Type Registry

types is a capture-local registry of type names. Each node’s required typeId is a zero-based index into that array:

const typeName = payload.types[node.typeId];

Do not persist a typeId independently or compare it across captures; registry order can change. Consumers migrating from the legacy payload must replace node-local type, kind, or styleId reads with types[typeId] and the fields below.

Node Fields

Every serialized node has id and typeId. Non-truncated nodes then expose the common semantic and presentation fields supported by their provider. Some state fields are provider-specific: for example, native Android reports focused, while other providers can report focusable and childCount.

Field	Meaning
`id`	Provider-owned identifier for the captured node. Refresh the tree before reusing capture-oriented ids.
`typeId`	Index into the top-level `types` array.
`automationId`	Stable platform automation/test identifier when one exists, such as an accessibility identifier, Android resource name, React Native `testID`, Flutter string `ValueKey`, or DOM id/test attribute.
`label`	Best-effort visible or accessibility label.
`role`	Platform-neutral semantic role such as `button`, `text`, `input`, or `container`.
`supportedActions`	Semantic actions exposed by the provider, such as `tap`, `focus`, or `setValue`.
`interactable`	True when the node is visible, enabled, and exposes a supported action.
`visible`, `enabled`, `focusable`, `focused`	Normalized state flags when supplied by the provider.
`childCount`	Optional number of direct children, including children omitted because of depth or node limits.
`bounds`	Optional screen/window-relative `x`, `y`, `width`, and `height`.
`visual`	Small presentation snapshot containing required `opacity` and optional `foreground`, `background`, `text`, and `value`. Colors use `#AARRGGBB`.
`z`	Optional non-default stacking priority relative to siblings.
`properties`	Optional provider-specific data requested through computed-style or detailed-inspection options.
`children`	Nested child nodes in the provider’s structural sibling order.

Child-array order is the default sibling order. Use z only when present; do not sort every tree solely by z. A screenshot remains authoritative for final clipping, transforms, effects, and compositing.

Bounds, Text, and Sensitive Values

Bounds are diagnostic geometry rather than a cross-platform layout engine. Coordinate spaces, transforms, window insets, and framework adapters can affect interpretation.

Displayed text and value strings are bounded. Providers omit secure text-entry values, but ordinary labels, accessibility text, DOM text, widget diagnostics, and implementation-specific properties can still contain personal or app-sensitive data. Request computed properties only when needed and apply the same development-only policy used for screenshots and remote tools.

Capture Limits

maxDepth, maxNodes, provider limits, or a subtree rootNodeId can truncate a response. Use childCount when present and provider-specific truncated metadata to distinguish a leaf from an omitted subtree. A truncated placeholder may contain only id, typeId, and truncation/stacking metadata. ui.inspect_node returns the same types registry and compact node shape for the requested node, ancestors, and descendants.

Snapshot Identity and Generic Operations

Every live capture has a provider source, opaque snapshotId, and increasing revision. ui.query_nodes returns a complete reference for each match:

{
  "source": "native",
  "snapshotId": "native:42:3f1e...",
  "revision": 42,
  "nodeId": "root.0"
}

Pass the reference to ui.inspect_node or ui.perform_action. References are snapshot-scoped: after a newer tree is captured for the same source, an old reference returns stale_node_reference with refreshWith: "ui.query_nodes". This prevents a framework from silently resolving a recycled node id to the wrong control.

The generic tools work across native, maui, react, flutter, and dom providers:

Tool	Purpose
`ui.query_nodes`	Capture or reuse a snapshot and select by node id, automation id, role, type, text, action, visibility, enabled state, or bounds.
`ui.perform_action`	Invoke a provider-supported action such as `tap`, `focus`, `unfocus`, `setValue`, `typeText`, or `toggle`.
`ui.wait`	Re-query until `exists`, `notExists`, `visible`, or `enabled` is satisfied.

Framework-prefixed tools remain useful for framework-specific diagnostics, but Host, CLI, and agent workflows should prefer these ui.* operations for portable automation.

Tool calls can include an after request for a visual tree, screenshot, or both. Use tool.batch for up to 32 ordered calls when an inspect–act–verify workflow should run in one app round trip. See Remote Tool Protocol for request, response, catalog revision, and large-payload encoding details.