Test Pencil input in Capacitor
Add the optional Pencil package, bind your Capacitor drawing surface, and play captured Simulator input with minimal app changes.
Requires Ansight CLI 1.6.0 or later and matching core SDK and Pencil packages at version 1.9.0 or later.
Dependencies and startup
Requires Capacitor 8 and iOS deployment target 15+. Add @ansight/capacitor-pencil alongside the matching @ansight/capacitor. Installing this optional package brings the transitive @ansight/pencil-runtime dependency and native AnsightPencil module. Run npx cap sync ios and rebuild the native app; web hot reload alone does not add the native module.
Install both packages at their latest matching release:
npm install @ansight/capacitor@latest @ansight/capacitor-pencil@latest
npx cap sync ios
The shared runtime is a transitive dependency. npx cap sync ios updates the native package graph; rebuild the app afterward.
import Ansight, { createOptionsBuilder } from '@ansight/capacitor';
import { AnsightPencil } from '@ansight/capacitor-pencil';
const options = createOptionsBuilder()
.withAnsightDefaults()
.withPlugin(AnsightPencil)
.build();
await Ansight.initializeAndActivate(options);
The Pencil builder opt-in registers and authorizes only its exact playback tools on iOS Simulator. No ReadWrite setting or separate registration/arming call is needed. Existing grants for other tools stay unchanged; an explicit global tool disable still blocks Pencil, regardless of builder order. Capture settings are independent: touch/screenshot recording can be enabled for verification and replay, but is not required for playback.
Bind the existing DOM canvas
After the element is mounted, pass its actual element reference:
const canvas = document.querySelector<HTMLCanvasElement>('#drawing-canvas');
if (!canvas) throw new Error('Drawing canvas is not mounted');
const disposePencil = AnsightPencil.bindSurface('drawing-canvas', canvas);
// Keep your existing addEventListener/onpointerdown/onpointermove handlers.
// Before removing/replacing the element or leaving this screen:
// await disposePencil();
Register once per mounted element; dispose in the screen/component cleanup. The DOM ID is only used by the consumer’s query above. The explicit binding is what connects the element to the playback target; no native accessibility identifier is required in sdk mode.
Existing handlers receive DOM PointerEvents:
canvas.addEventListener('pointermove', event => {
if (event.pointerType !== 'pen') return;
const pressure = event.pressure;
const tiltX = event.tiltX;
const tiltY = event.tiltY;
const twist = event.twist;
const distance = event.ansightPencil?.distance;
// Feed the existing drawing model.
});
tiltX/Y and twist are degrees. The optional ansightPencil property preserves exact source angles in radians and normalized hover distance; real events do not carry it. The adapter dispatches down/move/up/cancel and enter/leave on the selected element. It checks DOM hit testing, connectivity, visibility, unzoomed viewport, geometry and concurrent real pointers before delivery.
During its synthetic contact, setPointerCapture, releasePointerCapture and hasPointerCapture on the bound element recognize only the owned synthetic pointer. Other pointer IDs call the original browser methods. Those instance methods are restored after contact/cleanup, without replacing a method subsequently changed by the app. This does not patch DOM prototypes or create browser-level native pointer capture.
Synthetic DOM events have isTrusted: false. They cannot create browser user activation, native scrolling/gestures, or hardware coalesced/predicted events. Code requiring trusted browser input is outside this adapter’s contract. Existing DOM listeners and normal bubbling remain in place.
Optional recording
The startup example above adds playback without changing recording preferences. Keep existing capture configuration when integrating into an app. Pencil never enables recording as a side effect of playback.
Add these calls before .build() in the startup builder when retaining touch evidence and replay images:
.withTouchCapture()
.withSessionJpegCapture({ intervalMilliseconds: 250 })
Recording is independent of the permission granted by the Pencil opt-in.
Rebuilds and updates
After adding or updating Pencil, install matching npm packages, run npx cap sync ios and rebuild/relaunch the native app. A web asset reload alone cannot add or update the native module.
Changing the .pen.json file, its samples or host task arguments requires no application rebuild or package update. Startup options and surface-binding changes follow the app’s normal code lifecycle. See builds and package updates for the full cross-platform contract. Keep the CLI and optional SDK package compatible; protocol mismatches are rejected before input injection.
Connect, play, verify and finish
Start a matching CLI host before launching the Simulator app:
ansight host run
Build and launch the app on an iOS Simulator, open the drawing screen, then select its exact connected SDK session:
ansight session list --json
Confirm that the session belongs to this app and Simulator. For a recorded verification run, also confirm touch capture is active. Playback itself works with capture disabled and does not enable recording. A successful transport call alone is not proof that the app received the requested properties; test tasks must assert captured evidence and the app outcome.
Save this file as ansight/fixtures/drawing.pen.json in the app repository:
{
"schema": "ansight.pen-input/v1",
"samples": [
{ "offsetMs": 0, "phase": "down", "x": 0.3, "y": 0.5, "pressure": 0.2, "altitudeRadians": 0.5, "azimuthRadians": 0.7, "rollRadians": 0 },
{ "offsetMs": 150, "phase": "move", "x": 0.5, "y": 0.5, "pressure": 0.8, "altitudeRadians": 1.2, "azimuthRadians": 1.4, "rollRadians": 2 },
{ "offsetMs": 300, "phase": "up", "x": 0.5, "y": 0.5 }
]
}
Adjust coordinates so every point hits the selected surface. They are normalized to the app window in native mode, or the framework viewport in sdk mode, not to the target’s local bounds. Keep the drawing view stationary during playback.
From the repository root:
ansight ui play-pen --session <session-id> --file ansight/fixtures/drawing.pen.json --mode sdk --target drawing-canvas
Inside an Ansight TypeScript task (this is the host automation SDK, not the application SDK):
const playback = await ansight.ui.playPen({
file: "ansight/fixtures/drawing.pen.json",
mode: "sdk",
target: "drawing-canvas",
});
expect(playback.performed, { id: "pen-delivered" }).toBe(true);
await ansight.screenshots.take();
CLI paths resolve against the command’s working directory; task fixture paths resolve against the registered repository root.
For hover, extend the file using hoverEnter, hoverMove, and hoverExit with pressure zero and distance between 0 and 1. An up leaves proximity active: follow it with hoverMove or hoverExit, not a fresh hoverEnter. End contact with up or cancel. Files allow 2–2000 samples, strictly increasing offsetMs, at most 30 seconds and 1 MiB.
Verify three things separately:
- The command succeeds and the expected contact/hover samples appear in the retained session. Source labels distinguish simulated input.
- The app’s existing handler receives pressure, orientation and hover values; assert its drawing/model output, not only
performedor requested input. - Real input still works afterward. For native mode, a following HID-only control must have no override source label.
Open the retained session in the host’s player. Pencil-tip direction follows azimuth, length changes with tilt, the barrel highlight follows roll, hover lifts/fades, and contact shows pressure/tap feedback. Recording/rendering is independent of optional simulation registration.
When the run is complete, terminate the dedicated test app, retain its recording and confirm the exact session is disconnected:
ansight device terminate ios <simulator-udid> <bundle-id> --json
ansight session show <session-id> --json
Leave the host and Simulator running for replay. Do not terminate an unrelated/shared app instance.
SDK capture uses simulator-sdk-framework. It records the adapter-delivered fields, and must be checked against the app’s own drawing output. A native Capacitor drawing plugin can instead use UIKit hybrid setup; a DOM canvas now has the explicit binding above.
Lifecycle and failure behavior
Importing the package and building options do not install hooks. Normal SDK initialization registers the optional tools; there is no separate register() step. Unsupported platforms skip registration. Both Debug and Release Simulator builds are supported; physical-device and Mac Catalyst interception are excluded. There is no dependency on the consumer’s DEBUG symbol.
SDK framework playback uses ui.simulate_pen_framework and does not install UIKit hooks or inject HID. Native hybrid playback uses ui.simulate_pen. An active run expires after 35 seconds if abandoned. Explicit finish, errors and surface disposal release only the simulated pointer/hover owned by that run. Real input blocks or interrupts framework playback and continues to the app’s ordinary handlers.
If playback fails, check that the package is linked, initialization was awaited, the target is mounted and unique, tools are not globally disabled, the SDK is connected, and every point hits the target. Target movement, resizing, covering, unmounting or conflicting input stops playback. There is no silent fallback between backends.