Trends

Detect performance drift with policies embedded on quality measurements and distinct historical decisions.

CLI v0.23.1

A quality measurement can answer two related questions: does this run meet its fixed expectation, and is it meaningfully worse than its historical reference? Add trend to the measurement when the second question matters. There is no separate authoring resource or second ID to keep synchronized; the runtime trend ID is the measurement key, such as login-responsiveness.fps-p10.

This is an authoring decision, not a removal of the Trends concept. Quality evaluation computes the value and applies its fixed expectation. Trend evaluation separately selects a historical reference, records a decision, publishes trend events, and supplies trend history to the CLI and Trends view. Co-location is appropriate because the policy has no useful identity or lifecycle apart from the measurement it monitors.

The embedded model intentionally represents one historical policy per measurement. A separate policy resource would become useful only if policies needed independent ownership or reuse, several policies per measurement, or decisions composed from multiple measurements.

Add a trend policy

{
  "id": "fps-p10",
  "channel": { "type": "fps" },
  "statistic": "p10",
  "expect": { "gte": 55 },
  "minimumSamples": 10,
  "maximumSampleGapMs": 1000,
  "trend": {
    "regressionPercent": 10,
    "confirmRuns": 2
  },
  "display": {
    "title": "Login frame rate",
    "unit": "fps",
    "fractionDigits": 0,
    "minimum": 0,
    "includeZero": true
  }
}

The expectation normally supplies the direction. A gte measurement gets worse when it falls; an lte measurement gets worse when it rises. For an ambiguous expectation such as absoluteLte, set worseWhen to higher or lower.

regressionPercent and regressionAbsolute are optional individually, but at least one is required. When both are present, both thresholds must be crossed. This avoids reporting a large percentage change that is insignificant in absolute terms, or a large absolute change that is normal for a large baseline.

Reference behavior

The defaults use the median of 10 reference runs and wait for 5 before making a decision. Override them with baselineRuns and minimumBaselineRuns.

A versioned session makes one release decision against a pinned earlier app version. Once selected, that reference version does not change because another version happens to be observed. An unversioned session uses its earliest comparable runs as a fixed reference. Ansight does not continually move the reference forward, so a sustained degradation cannot normalize itself.

Only successful functional runs enter a reference. One latest flow instance per run is used so a busy session cannot dominate the median.

Decisions

confirmRuns defaults to 2. The first breach is Suspect; repeated evidence changes it to Regressed. A later good run is Recovered. Until enough reference runs exist the signal is Warmup.

Historical regressions are informative by default. Set blocking: true when a confirmed regression should fail the quality report and its workspace test.

Comparable series

By default, Ansight keeps platform, device model, and OS-major series separate. Set seriesBy to any combination of buildNumber, platform, deviceModel, operatingSystemMajor, and buildConfiguration. Missing configured metadata is an explicit error rather than an invitation to mix unlike runs. Use "seriesBy": [] only when the values are genuinely comparable everywhere.

SDK event details grouping is retained too. A “Secret Garden” load compares with other “Secret Garden” loads, not with a different guide using the same flow.

Inspect history

The local explorer’s Trends view lists each separated signal once and opens one focused detail view with current value, reference, delta, decision, context, and time-scaled history.

ansight test quality-history --app-id com.example.app
ansight test quality-history \
  --measurement login-responsiveness.fps-p10
ansight test quality-history \
  --trend login-responsiveness.fps-p10 \
  --group "Secret Garden" \
  --json

If a signal remains in Warmup or Error, check that its measurement produced a numeric value, enough successful reference runs exist, configured series fields are present, the app profile reports the expected version, and grouped flows use consistent details.

Rebuild history with current rules

Rebuild the Trends decisions after changing baseline, regression, series, or confirmation rules. The command replays the stored numeric measurements in chronological order; it does not need to rerun the app.

# Preview every registered app with stored measurements.
ansight test quality-history rebuild --dry-run

# Rebuild every eligible app.
ansight test quality-history rebuild

# Rebuild one app, or only one version of that app.
ansight test quality-history rebuild --app-id com.example.app
ansight test quality-history rebuild \
  --app-id com.example.app \
  --app-version 2.4.0

The Host uses each app’s registered codebase as the source of current rules. Use --workspace <path> with --app-id to override that location. Measurement definition hashes continue to separate values that were extracted under incompatible measurement definitions.

Local quality history stays free and account-independent. To share persistent history with an organisation, first sign in and register the exact App ID, then upload the local history:

ansight auth login --email you@example.com --otp
ansight cloud app register com.example.app \
  --team-id <organisation-uuid> \
  --name "Example App"
ansight cloud trend sync com.example.app --team-id <organisation-uuid>

The organisation must have Cloud Trends enabled through a current trial, paid plan, or explicit grant. The portal then shows the uploaded trend decisions by registered app. Repeating an upload updates matching evaluation records rather than duplicating them.