Task Device Control API

Use the task’s session-bound host APIs to control its application and native device. These operations need the appropriate host tools even when the app has an SDK connection. Calls cannot select a different session, app, or device.

Prerequisites and diagnostics

Start with System Requirements and Dependencies for installation instructions, tool paths, and troubleshooting. Workspace tasks need Node.js in the host’s environment, plus the tools for their target:

Run diagnostics as the same user and from the same launcher/environment as the resident host:

ansight doctor
ansight doctor --json

Doctor reports missing tools and installation guidance. Install the dependencies for your workflow, then rerun the checks. For a support report, use ansight doctor --full --json --output system-report.json.

Microphone audio and network conditioning need additional setup and a check against the selected live session:

ansight audio capabilities --session <live-session-id> --json
ansight device network capabilities --session <live-session-id> --json

Use the command for the feature you need and inspect available and reason. Machine readiness does not prove that audio reaches the app or that network traffic is being conditioned. The feature guides explain how to verify those outcomes. The support matrix below lists the task API’s platform and mode limits.

Support matrix

Virtual means iOS Simulator or Android Emulator. Physical means a real iPhone/iPad or Android device. “Yes” below still requires a connected target, platform dependencies, and any operation-specific setup.

The current host rejects location, motion, audio, and network-conditioning task calls in device mode. Their native backends running outside the app does not make them available to an external task session. The ansight namespace alone is not a promise of SDK-free execution. Check this separately from schemaVersion: 2 and the session’s broad capability snapshot.

Application lifecycle

ansight.lifecycle.launch() starts the selected app; foreground() brings it forward; background() presses Home; terminate() ends its process. All return AppLifecycleResult; inspect success and the resulting app state. Backgrounding requires a live session and supports Android devices and iOS Simulator. Some declarations still label it virtual-only; physical Android is supported by the current host driver.

Permission changes, termination, or backgrounding an externally watched physical app can end its recording. Calls remain pinned to the original session; they do not automatically follow a replacement session. See external recording lifecycle.

Native app permissions

Permission APIs operate on the task’s enforced application and native device in SDK or device mode. No hostTools declaration is needed. Await calls serially:

import { Permission, IosPermission, AndroidPermission } from "./ansight-task.js";

await ansight.permissions.grant({ permission: Permission.Microphone });
const access = await ansight.permissions.query({ permission: Permission.Microphone });
expect(access.status, { id: "microphone-granted" }).toBe("granted");
await ansight.permissions.revoke({ permission: Permission.Microphone });

// Use these within tasks for the corresponding platform.
await ansight.permissions.ios.reset({ permission: IosPermission.Photos });
await ansight.permissions.android.grant({ permission: AndroidPermission.Camera });

Named constants are exported by the generated ansight-task.js runtime module with readonly literal types in ansight-task.d.ts. Permission.Camera, IosPermission.LocationAlways, and AndroidPermission.RecordAudio provide completion and compile-time validation. Their matching type names can annotate variables. Strings and the lowercase .names catalogs remain supported, including app-defined Android identifiers.

The Permission API Reference documents every constant’s platform availability, exact native mapping, OS/tool restrictions, and links to the official Apple and Android documentation. Every constant also includes these details in its editor hover documentation.

PermissionResult reports supported, isSuccess, status, backend, nativePermissions, and the enforced session/app/device identifiers. Status is granted, denied, notDetermined, limited, unsupported, or unknown. Mixed native states produce limited; missing or unreadable state is unknown. Query returns unsupported results for unavailable resources. Unsupported mutations and native command failures throw. A mutation may change some native permissions before failing; its recorded result preserves individual states. Successful commands report the subsequently observed state, which may be unknown if the native store cannot be read.

Android identifiers above have the android.permission. prefix. Mappings are resource approximations: Android contacts are read access, calendar and location use the permissions the app declares, photos cover declared visual media types, and media library covers shared audio. nativePermissions exposes the exact subset. On Android 14+, selected photo access is reported as limited and is also revoked by shared revoke({ permission: "photos" }). Background location grants first grant declared foreground dependencies; revocation removes background access only on API 29+. On iOS, revoking location-always denies location access entirely, following simctl behavior.

Android supports emulators and authorized physical devices through ADB, uses the active Android user, and requires declared permissions changeable through pm. Special access and app account roles are outside this API. iOS supports Simulator only. Native iOS service names also include photos-add, contacts-limited, reminders, and siri. Mutations use the installed simctl privacy service list; reset restores prompting on the next request. Queries read Simulator TCC and location authorization stores without changing them. Unrecognized store schemas return unknown. These describe app authorization, not global device switches or the ability of a particular hardware resource to function in Simulator.

Permission changes can terminate the running application. Subsequent calls still use the recorded session target; use ansight.lifecycle.launch() if the workflow needs to relaunch it. Native grants can bypass usage-description checks, so keep separate tests for the app’s actual permission request flow.

Simulated location

ansight.device.setLocation({ latitude, longitude }) sets a coordinate on a virtual device. ansight.device.clearLocation() is iOS Simulator only; Android has no reliable clear operation, so set a new coordinate instead. Neither operation changes physical-device GPS. These task calls currently require SDK mode.

Location route playback

ansight.device.playLocation(args) starts a GPX (or KML) route on the task’s live session device. Supply XML in routeContent, rather than a local path:

const playback = await ansight.device.playLocation({
  sourceFileName: "approach.gpx",
  routeContent: `<gpx version="1.1"><trk><trkseg>
    <trkpt lat="-35.094" lon="150.804" />
    <trkpt lat="-35.095" lon="150.805" />
  </trkseg></trk></gpx>`,
  mode: "fixed-speed",
  fixedSpeedKph: 20,
});
// Continue interacting with the app while the device moves.

The promise resolves after the first location is set and rejects if playback cannot start. The result includes runId, deviceId, platform, pointCount, and the effective timing settings. The device must be a booted simulator or emulator.

The default mode: "recorded" uses track timestamps. speed scales recorded timing from 0.1 to 100 (default 1). Without timestamps, playback falls back to fixedSpeedKph (0.5–500, default 30); speed does not scale fixed speed. loop: true repeats the route until stopped or replaced.

Playback shares the CLI’s single host-managed replay service. Starting a route replaces any active route, and playback continues after the task ends. Coordinate tasks that use location playback. Use ansight device location status to inspect progress or later failures, and ansight device location stop to stop it. clearLocation() clears the simulated coordinate but does not stop a running route.

Motion simulation

Use ansight.device.shake() to post a UIKit shake gesture to the task’s iOS Simulator, or send alternating accelerometer pulses to its Android Emulator. On Android, set intensity (5–50 m/s²), repetitions (1–8 pulse pairs), and intervalMs (40–500 ms per pulse) when the default shake is unsuitable:

await ansight.device.shake({ intensity: 24, repetitions: 3, intervalMs: 90 });
// Assert the app's shake response separately.

On iOS Simulator, call shake() without options. The gesture is fixed and received through UIKit’s responder chain. Ansight posts it to the selected Simulator through CoreSimulator; it does not inject Core Motion sensor samples.

For a controlled motion trace on Android Emulator, use playAccelerometer. It is unavailable on iOS Simulator and all physical devices. Each value is held for holdMs before the next sample. Values are in m/s², each axis is limited to −100–100, each hold is 10–1000 ms, and a sequence can have 1–100 samples and last at most 10 seconds:

await ansight.device.playAccelerometer({
  samples: [
    { x: 0, y: 9.81, z: 0, holdMs: 100 },
    { x: 18, y: 0, z: 0, holdMs: 80 },
    { x: -18, y: 0, z: 0, holdMs: 80 },
  ],
});

Android calls finish after all samples have been sent and the emulator’s previous acceleration has been restored. The result reports the sample or gesture count, duration, target device, and delivery backend; it does not confirm that the app handled the motion. Android Emulator supports sensor injection through its console. Physical-device motion is not supported by these methods.

Microphone audio input

Complete the microphone setup checklist before running an audio task. iOS audio tasks require one selected simulator and serial execution of the entire recording workflow; the injection lock alone does not protect concurrent recordings. See audio concurrency restrictions.

ansight.device.audioCapabilities() inspects the task’s existing live session without playing audio. Both audio APIs reject with audio-parallel-execution-unsupported in parallel test batches (including multi-device matrix batches); the error is logged in test progress and the task call audit. Run audio tests serially on one device without --parallel.

On an allowed serial run, audioCapabilities() returns available, a diagnostic code and message, provider details, the shared route scope, and accepted format/operation limits.

ansight.device.injectMicrophoneAudio({ file, timeoutMs?, waitForMicrophoneMs? }) delivers a speech fixture to the session’s virtual microphone. The app must already be recording through its normal flow and have microphone permission. This method does not launch an app, press Record, or supply a file directly to a recognizer.

// The app in this task's pinned session is already recording.
const injection = await ansight.device.injectMicrophoneAudio({
  file: "fixtures/audio/quote.wav",
  timeoutMs: 30_000,
  waitForMicrophoneMs: 0,
});
expect(injection.status, { id: "audio-delivery-completed" }).toBe("completed");
// Assert the app's transcript or saved recording separately.

• file is relative to run.repositoryRootPath. Absolute paths, traversal outside that root, and symlinks resolving outside it are rejected. The host validates and snapshots the exact bytes before delivery. This API rule does not make trusted repository TypeScript a filesystem sandbox.
• Input is RIFF/WAVE PCM16, mono, 16 kHz, at most 15 seconds and 1 MiB. Compressed audio is not converted automatically.
• timeoutMs defaults to 30000, accepts 100–60000, and is capped by the task’s remaining deadline. waitForMicrophoneMs defaults to provider preflight without waiting (0) and accepts 0–10000. Android checks guest microphone readiness; iOS at zero wait verifies its route only, so the app must already be recording. Unsupported or unhealthy readiness is reported explicitly.
• Calls are serial and awaited. Injection counts as one task action and blocks through delivery and cleanup. Task cancellation/deadline cancellation stops the provider; partial or uncertain delivery is never retried automatically.
• Success returns AudioInjectionResult with fixture SHA-256 and format, submitted frames, delivery timing/provider diagnostics, and a retained evidenceId. captureVerified and transcriptionVerified are false: provider delivery does not establish what the app recorded or recognized.
• iOS requires a separately installed loopback driver, exactly one booted simulator, and a verified input route in the configured Xcode’s running Simulator UI. The provider currently requires English audio-input menus and Accessibility access. It serializes injections across the shared host route. Readiness observes a host input client where supported, not the target app’s private recording stream. Android requires an authenticated, healthy emulator microphone provider. Emulator 37.1.11 has an observed native injection race that can crash the emulator; readiness guards do not guarantee stability. Ansight does not restart or retry automatically. Physical devices are unsupported.

The same operation is available through ansight audio inject. These standard methods require no hostTools declaration and cannot target a session, app, or device outside the task’s enforced session.

Network conditioning

ansight.device.network applies bounded TCP conditions to a virtual target. Current task calls require SDK mode, a macOS host, and the tested mitmproxy 12.2.3 installation (mitmdump, or ANSIGHT_MITMDUMP_PATH). macOS may require approval for mitmproxy’s Network Extension. The iOS Simulator app or Android Emulator must already be running. See network-conditioning setup and troubleshooting for installation, host configuration, and the readiness check.

Profiles, scope, and limits

iOS scope is the selected app process. Android scope includes all apps in the selected emulator. TLS remains encrypted; no CA installation is needed. Ansight control ports bypass conditioning. UDP/QUIC, packet loss, DNS, radio state, and OS link quality are not simulated. Loopback destinations bypass macOS local capture; use a LAN address for a local fixture.

A lease lasts 1,000–300,000 ms, defaulting to 120,000 ms. Updating a profile does not extend expiry or change destination ports. destinationPorts accepts up to 32 ports from 1–65,535. A scenario contains 1–50 steps, each at least 1,000 ms, with at most 300,000 ms total duration. The host permits one active conditioning lease; parallel conditioning is unsupported.

Apply, inspect, and restore

This fragment runs inside an SDK task with a virtual target. It checks provider availability, verifies an empty starting screen, applies a condition, and explicitly restores it:

const support = await ansight.device.network.capabilities();
if (!support.available) throw new Error(support.reason ?? "Conditioning unavailable");
const before = await ansight.ui.assert({ automationId: "results", exists: false });
expect(before.passed, { id: "results-not-loaded-yet" }).toBe(true);

const lease = await ansight.device.network.set({
  profile: { downloadKbps: 512, latencyMs: 200 },
  durationMs: 30_000
});
try {
  // This app's observed refresh control starts a request to the test service.
  await ansight.ui.tap({ automationId: "refresh" });
  const loaded = await ansight.ui.waitFor({ automationId: "results", condition: "visible" });
  expect(loaded.satisfied, { id: "results-loaded-on-slow-link" }).toBe(true);
  const observed = await ansight.device.network.get({ leaseId: lease.leaseId });
  expect(observed.counters.downloadBytes > 0, { id: "conditioned-traffic-observed" }).toBe(true);
} finally {
  await ansight.device.network.reset({ leaseId: lease.leaseId });
}

Replace selectors and conditions with the app’s real refresh workflow; ensure the result condition proves the new request completed. Availability is a preflight only: routingVerified is always false. Check lease traffic counters and app behavior to establish that the intended traffic was affected. Provider success alone does not prove an app’s loading or error handling.

Task-owned leases are also cleaned up when the task exits or is cancelled, and leases expire automatically. Reset closes proxied connections. If the app process restarts, reset and create a lease for the new process. Location route playback has different lifetime rules and continues after task completion.

For retained HTTP requests use ansight.network; for recorded OS connection state use ansight.connectivity.