Turn repeatable app workflows into reusable tasks

Capture a workflow once so you or your coding agent can repeat it without working out the same steps each time. Repository tasks run known operations against a live app and check the outcome with named assertions. You can write a task directly or extract a draft from a recorded session and add the checks that establish success.

A task is trusted local TypeScript, invoked explicitly by a user, agent, or CLI through its exact ID. It never runs in the background.

Use a task when the required steps and tool calls are already understood—for example, query every map annotation and assert that the expected areas exist. Use a workspace test with taskId and no prompt to run that task with test-runner app launch, suites, and reporting. Add prompt when a model should navigate and reason through a complete user journey.

Create a task

Task modules live beneath ansight/tasks. A module’s relative path becomes its ID: ansight/tasks/map/validate-areas.ts becomes map.validate-areas.

Only a .ts file that exports the static task descriptor is discovered as a task. Other .ts and .d.ts files beneath ansight/tasks are support modules, so task modules can import shared domain types and helper functions without creating phantom tasks or discovery warnings. Session extraction also makes these support modules available to its agent, compiler, and TypeScript editor so generated tasks can reuse the workspace’s existing APIs.

Create a starter task with the CLI:

ansight workspace add task . home.verify --app-id com.example.app

This also initializes the workspace support files if they do not exist yet. Edit the generated TypeScript to define the steps and the check that proves the result. For a custom title or description in the starter file, add --title or --description when creating it.

You can also review discovered tasks from the local player’s Tests area.

Review the complete TypeScript declarations and task descriptor schema online, or download them from the source viewer.

Extract a task from a session

Use timeline extraction when a recorded session contains a specific interaction you want to preserve as a repeatable task. The extraction is a starting draft, not an assertion that the recorded behavior was correct.

Find the recorded session and inspect the evidence around the behavior:

ansight session list --app-id com.example.app
ansight session show <session-id>
ansight session touches <session-id>
ansight session trees <session-id>
ansight session images <session-id>

Select a tight interval and pass its bounds as non-negative seconds from session start or as ISO-8601 timestamps:

ansight task extract <session-id> \
  --start 12.5 \
  --end 24.8 \
  --workspace . \
  --title "Complete checkout"

By default, the CLI writes a slugged TypeScript file to ansight/tasks/<suggested-name>.ts. Use --output <path> to choose an exact destination. Existing files are preserved unless you explicitly pass --force (--overwrite is an alias). --repository <path> is accepted as an alias for --workspace <path>.

The extractor:

• groups the selected touch events into gestures;
• generates taps only when nearby visual-tree evidence provides a stable automation ID or text selector;
• converts drags into directional swipe actions;
• reports long presses, multi-touch, incomplete gestures, ambiguous targets, and missing selectors as REVIEW: diagnostics; and
• adds a provisional final UI-stability assertion.

Open the generated file and resolve every diagnostic. Confirm or encode the required starting state, simplify selectors where necessary, and replace the provisional stability check with named assertions for the product outcome you actually care about. Extraction reads the stored session; the live app is only needed when you test the generated task.

npx tsc -p ansight/tasks/tsconfig.json
ansight task run <generated-task-id> \
  --app-id com.example.app \
  --repository . \
  --session-id <connected-session-id>

The extraction command and every option are also listed in the CLI command reference.

Export a Maestro flow from a session

Select a recorded interval and export a reviewable Maestro YAML draft:

ansight task extract <session-id> \
  --start 12.5 --end 24.8 \
  --format maestro \
  --workspace . \
  --title "Complete checkout"

The default destination is .maestro/<suggested-name>.yaml. Use --output for another path and --force to replace an existing file. Add --ai to refine the draft through Ansight’s existing signed-in model gateway; --reasoning fast, balanced, or deep selects the configured model tier. The local session player offers Generate Maestro draft for a selected timeline period; you can review and edit the YAML, optionally refine it with the signed-in extraction model, then save it to the app’s registered workspace.

The exporter maps captured taps to Maestro ID or text selectors, drags to relative swipes, and visual-tree input changes to text-entry commands. It reports omitted gestures, weak selectors, and uncertain text changes. Review the required starting state and add a product-outcome assertion before treating the draft as a test. Check syntax with maestro check-syntax .maestro/<suggested-name>.yaml, then run it on a connected device with maestro test .maestro/<suggested-name>.yaml.

Export an Appium script from a session

Select a recorded interval and export a reviewable JavaScript test using WebdriverIO and Appium:

ansight task extract <session-id> \
  --start 12.5 --end 24.8 \
  --format appium \
  --workspace . \
  --title "Complete checkout"

The default destination is appium/<suggested-name>.test.mjs. In the local session player, select a timeline range, choose Extract agent task, then click Generate Appium draft. You can edit, download, or save the JavaScript without starting agent extraction.

The draft maps recorded automation IDs to accessibility ID or Android resource ID lookups, recorded labels to native text lookups, and taps and swipes without stable targets to relative screen coordinates. Review the marked coordinates and selectors, set the starting state, and add an outcome assertion. Set APPIUM_PLATFORM if the capture’s platform was unknown, and optionally set APPIUM_SERVER_URL, APPIUM_UDID, or APPIUM_APP_ACTIVITY for your target. After installing WebdriverIO and the appropriate Appium driver, run the script with node --test appium/<suggested-name>.test.mjs.

Complete task example

Every task exports a static task descriptor and one default function:

import type { TaskDefinition, TaskInvocation } from "./ansight-task.d.ts";

type VerifyHomeInput = { automationId: string };

export const task = {
  "schemaVersion": 1,
  "title": "Verify signed-in home",
  "description": "Checks that the signed-in home screen is visible.",
  "feature": "onboarding",
  "keywords": ["login", "home"],
  "platforms": ["ios", "android"],
  "deviceKinds": ["virtual", "physical"],
  "frameworks": ["dotnet-maui"],
  "inputSchema": {
    "type": "object",
    "properties": {
      "automationId": {
        "type": "string",
        "default": "signed-in-home"
      }
    },
    "additionalProperties": false
  },
  "outputSchema": {
    "type": "object",
    "properties": {
      "matchCount": { "type": "number" }
    },
    "required": ["matchCount"],
    "additionalProperties": false
  },
  "timeoutSeconds": 30,
  "maximumActions": 1
} satisfies TaskDefinition;

export default async function verifySignedInHome(
  { input, ansight, expect }: TaskInvocation<VerifyHomeInput>
): Promise<{ matchCount: number }> {
  const result = await ansight.ui.assert({
    automationId: input.automationId,
    exists: true,
    expectedVisible: true
  });

  expect(result.passed, { id: "signed-in-home-visible" }).toBe(true);
  return { matchCount: result.matchCount };
}

The descriptor must remain a JSON-compatible object literal. The host extracts it without importing or executing the module, so do not use variables, spreads, functions, or computed values inside the descriptor.

Descriptor fields

outputSchema is descriptive in contract version 1. The host exposes it during discovery but does not reject a returned value that differs from the schema.

Target metadata is not task input. A workspace test links to the task with taskId; the CLI runner validates these optional declarations before agent execution. Framework values are dotnet-maui, dotnet-ios, dotnet-android, react-native, flutter, capacitor, ios-uikit, ios-swiftui, android-views, android-compose, and native-unknown.

Focused task discovery through ansight task list matches every query word against the task ID, title, description, feature, keywords, and serialized declared input schema. This includes property names, enum values, and defaults, so an agent can find a task by a known input value even when that value is not repeated in the title or keywords. Keep those values concise and domain-specific; every query word must match the same task.

Input validation

Task input is always an object. Contract version 1 validates the root properties, required, and additionalProperties fields, plus scalar property type, enum, default, minimum, and maximum values. Defaults are applied before the function starts.

Use a TypeScript type with TaskInvocation<TInput> so the editor understands the validated input shape. The input and run objects passed to the function are read-only.

Function API

The default function receives { run, input, secrets, ansight, app, expect }:

• run identifies the run, task, App ID, exact session, repository, timeout, and action limit.
• input contains validated input with defaults applied.
• secrets.get(alias) returns a value explicitly granted to the run, or undefined; secrets.require(alias) fails if the alias was not granted. Do not return or log secret values from a task.
• ansight exposes session-pinned resident-host methods.
• app exposes standardized app methods and callTool for user-defined tools.
• expect provides Playwright-style named matchers that determine whether the task passed.

Tasks can discover and compose other tasks from the same workspace through ansight.tasks. Discovery is read-only; execution reuses the caller’s exact repository, App ID, and live session:

type OpenGuideOutput = { guideVisible: boolean };

const catalog = await ansight.tasks.list({ query: "open area 3D guide" });
const child = catalog.tasks.find(candidate => candidate.taskId === "map.open-guide");
if (!child) throw new Error("The guide task is unavailable.");

const result = await ansight.tasks.run<OpenGuideOutput>({
  taskId: child.taskId,
  input: { area: "Secret Garden" }
});
expect(result.output?.guideVisible, { id: "guide-opened" }).toBe(true);

The call rejects unless the child task passes. Child tasks keep independent assertions, timeouts, action budgets, audit trails, and persisted run results; the parent’s discovery or execution call counts as one parent action. The parent must still make its own named assertion. Self-calls, recursive cycles, and call chains deeper than eight task execution levels are rejected.

Call a host-owned task tool through its feature API:

await ansight.ui.assert({ automationId: "home", exists: true });

Current-session inspection and running-app controls marked as task-callable are grouped under ansight.session, ansight.ui, ansight.logs, ansight.artifacts, ansight.screenshots, ansight.telemetry, and the other feature APIs. Workspace composition is exposed separately as ansight.tasks.list(...) and ansight.tasks.run(...); it does not require a hostTools declaration. For example, ansight_wait_for_ui is exposed as ansight.ui.waitFor(...) without descriptor boilerplate, while ansight.session.getProperties<TProperties>() returns the selected session’s custom properties in a named SessionPropertiesResult<TProperties> contract. App registration, pairing, cloud, repository management, session mutation, audit, host diagnostics, import/export, and profiling remain host-only. ansight.callTool(name, args) remains available for extra host tools named in hostTools; the host validates those names against the active task registry while loading the repository.

Call a custom tool published by the live app with:

await app.callTool("example.map.query_surface_contents", {
  annotationKind: "point"
});

Standard app tools instead have discoverable API suites such as app.artifacts.request(...), app.data.query(...), app.maui.getVisualTree(...), app.react.getComponentTree(...), app.flutter.inspectWidget(...), and app.capacitor.querySelector(...). These methods and custom app.callTool(...) IDs need no descriptor declaration.

For an externally monitored session, use schema version 2 and the available host APIs, including ansight.files for sandbox access. Check app.available or declare required app-tool IDs before using SDK suites. Missing capabilities reject execution. See Execution capabilities for a portable descriptor and the runtime capability API.

Task-callable host methods discard caller-supplied sessionId, appId, deviceId, and bundleIdentifier values and inject the task session. App tools are also pinned to the selected live session and must be live, authorized, and available there. Destructive app tools use their normal authorization and approval policies.

Await calls serially. An undeclared ansight.callTool host call, overlapping call, failed child task, failed tool guard, or exceeded maximumActions limit rejects the task. Calls retain normal audit and timeout handling.

Diagnostic result retention

When a task runs inside an agent execution captured with --trace, its API calls retain input arguments and returned outputs or errors alongside start and completion times, duration, and status. Composed task calls retain their child call entries. In the local player’s test history, select the task in the execution trace to open its results table, then select a call to inspect its inputs and outputs. Payloads have bounded capture limits and report truncation; without tracing, calls retain lightweight timing and outcome metadata.

A successful file_descriptors.list_open call also creates a durable open-file-descriptors.json artifact snapshot on the selected session timeline. The artifact preserves the complete app-tool response. The compact file_descriptors.count_open and file_descriptors.get_usage calls remain ordinary app-tool responses, so a task can return only their raw count or usage summary. Continuous open-handle and JNI-reference counts remain telemetry; JNI reference graphs are not currently exposed as an app tool.

Named assertions and status

The expect API uses a required metadata object so every result keeps its stable Ansight assertion ID:

• expect(actual, { id, message? }).toBe(expected) for strict Object.is equality
• .toEqual(expected) for strict deep equality
• .toBeTruthy(), .toBeFalsy(), .toBeDefined(), .toBeUndefined(), and .toBeNull()
• .toContain(expected) and .toContainEqual(expected)
• .not.<matcher> to negate a matcher
• expect.soft(actual, metadata).<matcher> to record failure and continue

Use stable, unique assertion IDs. A hard failed assertion stops the function; a soft failed assertion does not. Any recorded failure still fails the task, including a hard assertion error caught by task code. Returning output is not enough to pass: a task is Passed only when at least one named assertion passes and none fail. A normal return without assertions is Inconclusive.

Other terminal statuses are Failed, Error, TimedOut, Cancelled, and Rejected. The result includes validated input, returned output, duration, assertions, standard error, and a per-tool-call audit.

Test a task with the local host

1. Open the Test workspace and select the app beneath the active workspace.
2. Open or create a module under ansight/tasks.
3. Review the host-reported descriptor, schema, API, and assertion snippets.
4. Select an open simulator session for the same App ID.
5. Choose Save & test task.
6. Review the terminal status, named assertions, tool calls, and standard error.

The task editor validates the static module shape before saving. A live task run uses the saved source and the exact selected session.

Discover and run from the CLI

Inspect tasks without running them:

ansight task list --app-id com.example.app --repository .

Task execution requires a resident host and an already connected app session:

ansight host run

ansight task run home.verify \
  --app-id com.example.app \
  --repository . \
  --input '{"automationId":"signed-in-home"}' \
  --secret login_password

Use --device-id or --session-id to select an exact connected target, and --input-file for larger input. Without either target option, Ansight selects the most recently updated connected session whose App ID matches exactly.

Pass --headless to task run or repo task run if any permitted host tools invoked by the task should start virtual devices without opening native windows. Windows are shown by default for new launches. This flag does not launch an initial app session for the task, close existing windows, or restart a running emulator. The connected-session prerequisite still applies. See headless device launches.

Execution and security boundaries

Each task run starts with fresh module state. Module startup has a separate five-second limit. The execution environment is reduced, and hosted execution credentials are never added to it. Tool results, returned output, diagnostics, action count, and total duration are bounded.

Capability declarations restrict calls back into Ansight; they do not sandbox filesystem, process, or network access available to trusted local workspace code. Review a repository before running its tasks.

A repository can expose up to 256 tasks, and each executable module is limited to 1 MiB.

Related docs

• Workspace Getting Started
• Task API Reference
• Host API Methods
• App Tool API
• Workspace Execution Security
• Tests
• Triggers
• Tooling and Validation
• CLI Command Reference