Reading Results

A run result answers three questions:

Did tests pass?
How fast and stable were endpoint calls?
Did candidate endpoints match the reference?

Run lifecycle

Run status transitions:

queued
running
Done: success, failed, partial, timeout, or infra_failed

Runs can also be superseded if a newer run replaces a queued one. A success run can still include warnings in comparison verdicts. A partial run indicates some targets completed while others failed.

Where to inspect

Results are available at several levels of detail in the dashboard.

Test Suites list (/chains/:chainId/suites) for latest status
Suite detail (/suites/:suiteId) for run history trends
Run detail (/runs/:runId) for assertions, samples, and verdict

Result sections

Each run result is broken into sections.

1) Run header

Shows run status, timing, and triggered-by source. If the comparison verdict is fail or warn, a verdict badge appears next to the status badge linking to the comparison section.

2) Endpoint test outcomes

Per endpoint you see pass/fail/skip counts and failed assertion details. Acknowledged failures are marked with a badge but do not affect the overall status.

3) RPC samples

Per-method call samples include:

method name
success/failure
latency in ms
response excerpt or error payload

4) Comparison verdict

For multi-endpoint suites, the verdict card shows individual checks:

latency_p95: absolute p95 latency against threshold
latency_vs_reference: candidate p95 vs reference p95
correctness: response match rate (when enabled)
latency_vs_baseline: candidate p95 vs previous baseline run

Each check shows its own pass/warn/fail status with a tooltip explaining what it measures.

Interpreting failures quickly

When a run fails, here’s a quick triage process.

Identify failing endpoint(s) first.
Check which verdict checks failed and read their messages.
Check method-level failures and error codes.
Compare latency drift vs previous runs.
For mismatches, inspect reference vs candidate fields.
Consider acknowledging known-benign differences.
Confirm whether failures are deterministic by re-running manually.

CI usage

In CI pipelines, you typically only need the exit code from the verdict endpoint.

0: pass
1: fail

This is documented in CI Integration.