Acknowledgements

Acknowledgements let you mark known, expected failures so they stop affecting run verdicts while remaining visible for audit.

Why acknowledge instead of fixing?

Some failures are expected and benign:

RPC error differences between client implementations (e.g. different revert message formats)
Known quirks in specific methods on certain providers
Temporary differences during a planned migration

Threshold tuning is too blunt for these cases. Removing assertions loses coverage. Acknowledgements give you a third option: keep the check, suppress its verdict impact, track it explicitly.

How it works

You create an acknowledgement rule scoped to a check, target endpoint, and error pattern.
On future runs, failures are matched against your active rules automatically.
Matched failures are marked as acknowledged and remain visible, but no longer count as failures.
If all failures on a target endpoint are acknowledged, that target is promoted to success.
The overall run status updates to reflect the new target statuses.

Creating rules

Rules are created from the suite’s Acknowledgements tab in the dashboard. Each rule specifies:

which check failed (e.g. “receipt field match”, “correctness”)
an error pattern to match (substring or regex)
optionally, which endpoint the failure occurs on
optionally, which response field diverged (for comparison mismatches)

Assertion failures

When you acknowledge assertion failures, matched samples no longer count toward the target’s failure tally. If all failures on a target endpoint are acknowledged, the target is promoted to success and the run status updates accordingly.

Comparison divergences

For cross-endpoint comparison mismatches, rules match by field path. A divergence is only suppressed when all mismatched fields have matching rules. Partially acknowledged comparisons still count as failures.

Good to know

New rules apply to future runs only. Existing runs keep their stored results. Trigger a new run to see the effect.
Deleting a rule does not retroactively change past runs. Historical samples that were acknowledged stay that way.

When to use

Use acknowledgements for:

known client implementation differences you plan to address later
transient issues with a known timeline for resolution
expected divergences after a planned change (e.g. new field added to receipts)

Do not use acknowledgements to:

hide failures you don’t understand (investigate first)
permanently suppress checks (tune thresholds or remove the assertion instead)
work around flaky infrastructure (fix the root cause)

Pending rules indicator

In the dashboard, rules created after the most recent run show a “pending” badge. This means the rule exists but has not been tested against a new run yet. Trigger a new run to see its effect.