Test Suites

A test suite is a reusable configuration that executes one catalog template against selected endpoints.

Suite anatomy

A suite combines five parts:

template: defines checks and expected output model
targets: endpoint list for execution
reference endpoint: the endpoint marked as reference on the Endpoints page, used as source of truth for comparisons
schedule: manual, interval, on_every_n_blocks, or on_timestamp
params: template-specific values

Suites are scoped to a single chain.

Lifecycle

Create from catalog template
Configure endpoints and parameters
Enable schedule or run manually
Review runs and tune thresholds
Disable, update, or delete as needed

What templates provide

Templates in Suite Catalog (/suites/catalog) define:

which RPC methods are called
assertion rules
optional comparison logic
required/optional parameters

Execution behavior

When a suite run starts:

A worker receives run config snapshot
Calls are executed per target endpoint
Assertions evaluate pass/fail/skip
Comparison verdict is computed if multi-endpoint
Metrics and samples are stored

Status and enablement

Suite-level enabled controls scheduled execution.

enabled=true: scheduler may trigger runs
enabled=false: only manual/API trigger if supported by permissions

Auto-disable

If a suite fails to resolve any valid targets for 3 consecutive scheduler evaluations, it is automatically disabled. This protects against suites running indefinitely against misconfigured or empty target sets.

Common causes: all matched endpoints were deleted, tag selectors match nothing, or targets are misconfigured. The suite stores a structured disabled_reason with a breakdown of what failed, and an suite_disabled alert is fired.

Fix the target configuration and re-enable the suite to resume scheduling.

Typical suite patterns

Common configurations include:

Smoke checks every 5 minutes on production RPC
Deep regression checks hourly or on block events
Pre/post upgrade baseline comparisons
Provider A vs provider B consistency checks

See Configure Suites and Upgrade Testing.

Operating guidance

Keep suites focused and purpose-specific
Use clear names including env and interval
Separate canary suites from deployment gates
Review failing assertions before widening thresholds