JPEG Screenshots

Ansight exposes screenshots through two .NET SDK surfaces:

• session JPEG capture, a Core runtime feature that streams periodic JPEG frames while a paired host session is open
• on-demand screenshot tools, a VisualTree tool-suite feature that lets the local player or a CLI client call ui.get_screenshot

Use session capture when you want timeline evidence for replay, QA handoff, or later AI analysis. Use the on-demand tool when an agent needs the current screen at a specific point in an investigation.

Important: Screen capture will result in an FPS drop while frames are captured, encoded, and transported. Use conservative interval, quality, and max-width settings, and disable session JPEG capture for trend-focused runs unless visual evidence is required.

Feature Suite Defaults

The all-in-one setup APIs enable the usual local-development feature suite for you.

Remote tools are still gated by ToolGuard. The all-in-one setup grants full tool access for local development; call WithReadOnlyToolAccess(), WithReadWriteToolAccess(), or WithToolsDisabled() inside the setup callback if the workflow should be narrower.

Session JPEG Capture

Session JPEG capture is a runtime feature. It captures the current app surface on an interval, encodes it as JPEG, and sends it over the paired host connection as live session evidence.

using Ansight;

var options = Options.CreateBuilder()
    .WithSessionJpegCapture(intervalMilliseconds: 2000, quality: 60, maxWidth: 720)
    .Build();

Runtime.InitializeAndActivate(options);

You can also pass a complete SessionJpegCaptureOptions instance:

using Ansight;

var options = Options.CreateBuilder()
    .WithSessionJpegCapture(new SessionJpegCaptureOptions
    {
        IntervalMilliseconds = 3000,
        Quality = 60,
        MaxWidth = 720,
        Mode = SessionJpegCaptureMode.ScreenshotAndVisualTree,
        CaptureKeyboardPresence = true
    })
    .Build();

Disable periodic capture when a run does not need screenshot history:

using Ansight;

var options = Options.CreateBuilder()
    .WithoutSessionJpegCapture()
    .Build();

For all-in-one setup, make the same call inside the setup callback:

using Ansight.Maui;

builder.UseAnsight<App>(ansight =>
{
    ansight.WithoutSessionJpegCapture();
});

SessionJpegCaptureOptions properties:

The all-in-one feature suite uses the more conservative 2000ms, quality 60, and max width 480 defaults.

Visual-Tree Capture Modes

Tree capture requires a provider registered with SessionVisualTreeCaptureRegistry. Touch-triggered mode also requires touch capture. The Ansight and Ansight.Maui packages register their standard providers; a core-only app must register its provider explicitly. Payloads follow the compact visual-tree v2 contract.

Keyboard-presence capture is opt-in and reports presence only. It does not collect typed text or keyboard geometry, and the metadata remains useful when the host owns simulator/emulator screenshots.

Simulator and Emulator Capture Ownership

The runtime advertises sessionJpegCaptureControlVersion: 1 in device.profile. The resident host can acknowledge the profile with sessionJpegCapture.mode: "host" and an optional source such as simctl or adb.

In host mode, the SDK suspends its periodic in-app JPEG loop for that live session so the host can capture the simulator or emulator externally. A missing response or mode: "app" keeps the configured SDK capture behavior. This negotiation is automatic and does not affect on-demand ui.get_screenshot calls.

On-Demand Screenshot Tool

On-demand screenshots are exposed by the VisualTree tool suite. This is the path used when the local player or ansight app call <session-id> ui.get_screenshot asks the running app for a screenshot at a specific moment.

When you use Ansight or Ansight.Maui, the suite is registered for you. With Ansight.Core, add the package and register it explicitly:

dotnet add package Ansight.Tools.VisualTree --prerelease

using Ansight;
using Ansight.Tools.VisualTree;

var options = Options.CreateBuilder()
    .WithVisualTreeTools()
    .WithReadOnlyToolAccess()
    .Build();

The tool id is ui.get_screenshot.

Arguments:

• format: png or jpeg
• quality: JPEG quality 1-100
• maxWidth: optional width cap
• annotateNodeIds: overlay node ids on the screenshot
• afterScreenUpdates: Apple platforms wait for pending screen updates before rendering; default true

Example request:

{
  "toolId": "ui.get_screenshot",
  "arguments": {
    "format": "jpeg",
    "quality": 80,
    "maxWidth": 1080,
    "annotateNodeIds": false,
    "afterScreenUpdates": true
  }
}

The response contains screenshot metadata such as format, width, height, transferId, downloadId, fileName, mimeType, and sizeBytes. Image bytes are delivered out-of-band over Ansight’s binary transfer protocol rather than inline base64.

Choosing a Capture Path

Trends and Privacy

Screenshots can contain user data, credentials, notifications, and other sensitive rendered content. Keep screenshotting scoped to local development and QA workflows.

JPEG capture adds rendering, encoding, and transport work every time a frame is captured. Prefer conservative intervals, moderate quality, and a max width cap. For a trend-focused session, disable periodic capture or rely on single on-demand screenshots.