Skip to content
OnchainProbe Docs

Alerting Setup

This guide configures production-ready alerting for suite failures and latency regressions.

Prerequisites

Section titled “Prerequisites”
  • Pro plan
  • at least one active suite
  • destination credentials (Slack webhook URL, email list, or webhook endpoint)

See Alerting.

Step 1: Create destination

Section titled “Step 1: Create destination”

First, set up where alerts should go.

Navigate to Alerts (/settings/alerts).

Choose destination type:

  • webhook: generic HTTP webhook
  • email: one or more email addresses
  • slack: Slack incoming webhook

Step 2: Create rule

Section titled “Step 2: Create rule”

Next, define what conditions trigger an alert and where to send it.

  • scope_type: tenant, chain, or suite (with chain_id/suite_id as needed)
  • trigger_type and trigger_config: what triggers the alert
  • cooldown_minutes: minimum time between repeated alerts
  • notify_on_recovery: whether to send a recovery notification
  • destination_ids: target destination(s)

Step 3: Test alert path

Section titled “Step 3: Test alert path”

Before relying on alerts in production, verify the full delivery path end-to-end.

  1. Temporarily lower a threshold to force trigger.
  2. Click Run Now on the suite.
  3. Confirm destination receives notification.
  4. Restore production threshold.

Step 4: Review alert events

Section titled “Step 4: Review alert events”

After alerts have been firing for a while, review the event log to confirm delivery is reliable.

  • trigger time
  • destination delivery status
  • dedup/cooldown behavior
  • recovery event emission
Section titled “Recommended initial policy”

Starting with fewer, well-tuned rules avoids alert fatigue. You can always add more as you learn your baseline.

  • suite failure rule: threshold 2 failures / 10m
  • latency rule: p95 threshold based on baseline + buffer
  • cooldown: 10-20 minutes for paging channels
Section titled “Related”