Webhooks
Configure HTTP webhook notifications for probe health status changes, verify HMAC-SHA256 signatures, handle retries, and integrate with Slack, PagerDuty, and Microsoft Teams.
Webhooks notify external systems when a probe's health status transitions between states (e.g., Healthy to Degraded, Degraded to Unhealthy). Use them to trigger alerts in incident management tools, chat platforms, or custom monitoring systems. This guide covers configuration, security verification, retry behavior, and integration examples.
How Webhooks Work
When a probe's health status changes after an execution, CallMeter sends an HTTP POST request to each configured webhook URL with a JSON payload describing the transition. The webhook fires only on status changes -- if the probe remains in the same state across multiple executions, no webhook is sent.
Trigger conditions:
| Transition | Fires Webhook |
|---|---|
| HEALTHY to DEGRADED | Yes |
| HEALTHY to UNHEALTHY | Yes |
| DEGRADED to UNHEALTHY | Yes |
| DEGRADED to HEALTHY | Yes |
| UNHEALTHY to HEALTHY | Yes |
| UNHEALTHY to DEGRADED | Yes |
| Any to UNKNOWN | Yes |
| HEALTHY to HEALTHY (no change) | No |
| DEGRADED to DEGRADED (no change) | No |
Configuring a Webhook
- Open your probe's detail page
- Navigate to the Settings tab
- Scroll to the Webhooks section
- Enter the Webhook URL -- must be a publicly accessible HTTPS endpoint
- Click Save
The platform generates a webhook secret when you create the webhook. Copy and store this secret securely -- it is used to verify webhook signatures.
HTTPS required
Webhook URLs must use HTTPS. HTTP endpoints are rejected. This ensures the payload (which includes your probe name and metric data) is encrypted in transit.
Webhook Payload
When a status transition occurs, CallMeter sends the following JSON payload:
{
"event": "probe.status_changed",
"probe_id": "clx1abc2def3ghi4jkl5",
"probe_name": "Production PBX Health",
"project_id": "clx9mno6pqr7stu8vwx0",
"project_name": "Production Environment",
"previous_status": "HEALTHY",
"current_status": "DEGRADED",
"timestamp": "2026-01-15T10:30:00.000Z",
"execution_id": "clx2yz1abc2def3ghi4",
"metrics": {
"mos": 3.2,
"jitter_ms": 45.3,
"packet_loss_pct": 2.1,
"rtt_ms": 180,
"setup_time_ms": 2100
},
"thresholds_breached": [
{
"metric": "mos",
"value": 3.2,
"warning_threshold": 3.5,
"critical_threshold": 2.5,
"level": "warning"
}
]
}Payload fields:
| Field | Type | Description |
|---|---|---|
event | string | Always "probe.status_changed" |
probe_id | string | Unique identifier of the probe |
probe_name | string | Human-readable probe name |
project_id | string | The project this probe belongs to |
project_name | string | Human-readable project name |
previous_status | string | The probe's status before this transition |
current_status | string | The probe's new status after this transition |
timestamp | string | ISO 8601 timestamp of the status change |
execution_id | string | The probe execution that triggered the change |
metrics | object | Current metric values from the latest execution |
thresholds_breached | array | Details of which thresholds were breached and at which level |
HMAC-SHA256 Signature Verification
Every webhook request includes cryptographic signatures that let you verify the request genuinely came from CallMeter. Always verify signatures before processing webhook payloads to prevent spoofing.
Signature Headers
| Header | Description |
|---|---|
X-CallMeter-Signature | HMAC-SHA256 signature of the payload |
X-CallMeter-Timestamp | Unix timestamp (milliseconds) when the webhook was sent |
Verification Process
- Extract the
X-CallMeter-TimestampandX-CallMeter-Signatureheaders - Construct the signing string:
{timestamp}.{raw_request_body} - Compute HMAC-SHA256 of the signing string using your webhook secret
- Compare the computed signature with the provided signature using timing-safe comparison
- Optionally, reject requests where the timestamp is older than 5 minutes (replay prevention)
Example: Node.js Verification
const crypto = require('crypto');
function verifyWebhook(req, webhookSecret) {
const signature = req.headers['x-callmeter-signature'];
const timestamp = req.headers['x-callmeter-timestamp'];
const body = req.rawBody; // raw request body as string
// Replay prevention: reject requests older than 5 minutes
const age = Date.now() - parseInt(timestamp, 10);
if (age > 5 * 60 * 1000) {
return false;
}
// Compute expected signature
const signingString = `${timestamp}.${body}`;
const expected = crypto
.createHmac('sha256', webhookSecret)
.update(signingString)
.digest('hex');
// Timing-safe comparison (CRITICAL: prevents timing attacks)
const sig = `sha256=${expected}`;
return crypto.timingSafeEqual(
Buffer.from(sig),
Buffer.from(signature)
);
}Example: Python Verification
import hmac
import hashlib
import time
def verify_webhook(headers, body, webhook_secret):
signature = headers.get('X-CallMeter-Signature')
timestamp = headers.get('X-CallMeter-Timestamp')
# Replay prevention
age = int(time.time() * 1000) - int(timestamp)
if age > 5 * 60 * 1000:
return False
# Compute expected signature
signing_string = f"{timestamp}.{body}"
expected = hmac.new(
webhook_secret.encode(),
signing_string.encode(),
hashlib.sha256
).hexdigest()
# Timing-safe comparison
return hmac.compare_digest(
f"sha256={expected}",
signature
)Never use simple string comparison
Always use timing-safe comparison functions (like crypto.timingSafeEqual in Node.js or hmac.compare_digest in Python) when comparing signatures. Simple string comparison (=== or ==) is vulnerable to timing attacks that can leak your webhook secret.
Retry Policy
If a webhook delivery fails (the endpoint returns a non-2xx HTTP status code or the connection times out), CallMeter retries with exponential backoff:
| Attempt | Delay After Failure |
|---|---|
| 1st retry | 1 second |
| 2nd retry | 5 seconds |
| 3rd retry | 30 seconds |
| 4th retry | 5 minutes |
After all retry attempts are exhausted, the webhook delivery is marked as failed. If the webhook endpoint fails consistently (10 or more failures within 24 hours), the webhook is automatically disabled and you receive a notification.
Your webhook endpoint should:
- Return a 2xx status code (200, 201, 202, 204) to acknowledge receipt
- Respond within 10 seconds to avoid timeout
- Process the payload asynchronously if heavy processing is needed (acknowledge immediately, process in a background queue)
Integration Examples
Slack Integration
Use Slack's Incoming Webhooks to post probe alerts to a channel:
- In Slack, go to Apps and search for Incoming Webhooks
- Add a new webhook and select the target channel
- Copy the webhook URL (e.g.,
https://hooks.slack.com/services/T00/B00/xxx)
To format the CallMeter payload for Slack, you need a small middleware service that transforms the payload into Slack's message format:
// Middleware that receives CallMeter webhook and forwards to Slack
app.post('/callmeter-to-slack', verifyWebhookMiddleware, async (req, res) => {
const { probe_name, previous_status, current_status, metrics, timestamp } = req.body;
const color = current_status === 'HEALTHY' ? '#36a64f'
: current_status === 'DEGRADED' ? '#daa520'
: '#ff0000';
await fetch(process.env.SLACK_WEBHOOK_URL, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
attachments: [{
color,
title: `Probe Alert: ${probe_name}`,
text: `Status changed from ${previous_status} to ${current_status}`,
fields: [
{ title: 'MOS', value: metrics.mos?.toFixed(1) || 'N/A', short: true },
{ title: 'Jitter', value: `${metrics.jitter_ms?.toFixed(0) || 'N/A'}ms`, short: true },
{ title: 'Packet Loss', value: `${metrics.packet_loss_pct?.toFixed(1) || 'N/A'}%`, short: true },
{ title: 'RTT', value: `${metrics.rtt_ms?.toFixed(0) || 'N/A'}ms`, short: true },
],
ts: new Date(timestamp).getTime() / 1000,
}],
}),
});
res.sendStatus(200);
});PagerDuty Integration
Use PagerDuty's Events API v2 to create and resolve incidents:
app.post('/callmeter-to-pagerduty', verifyWebhookMiddleware, async (req, res) => {
const { probe_id, probe_name, current_status, metrics } = req.body;
const action = current_status === 'HEALTHY' ? 'resolve' : 'trigger';
const severity = current_status === 'UNHEALTHY' ? 'critical' : 'warning';
await fetch('https://events.pagerduty.com/v2/enqueue', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
routing_key: process.env.PAGERDUTY_ROUTING_KEY,
event_action: action,
dedup_key: `callmeter-probe-${probe_id}`,
payload: {
summary: `CallMeter probe "${probe_name}" is ${current_status}`,
severity,
source: 'callmeter.io',
custom_details: metrics,
},
}),
});
res.sendStatus(200);
});The dedup_key ensures that multiple DEGRADED or UNHEALTHY notifications for the same probe update the existing incident instead of creating duplicates. When the probe recovers to HEALTHY, the resolve action closes the incident automatically.
Microsoft Teams Integration
Use Teams' Incoming Webhook connector to post adaptive cards:
app.post('/callmeter-to-teams', verifyWebhookMiddleware, async (req, res) => {
const { probe_name, previous_status, current_status, metrics, timestamp } = req.body;
const themeColor = current_status === 'HEALTHY' ? '36a64f'
: current_status === 'DEGRADED' ? 'daa520'
: 'ff0000';
await fetch(process.env.TEAMS_WEBHOOK_URL, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
'@type': 'MessageCard',
themeColor,
summary: `Probe Alert: ${probe_name}`,
sections: [{
activityTitle: `Probe: ${probe_name}`,
activitySubtitle: new Date(timestamp).toISOString(),
facts: [
{ name: 'Previous Status', value: previous_status },
{ name: 'Current Status', value: current_status },
{ name: 'MOS', value: metrics.mos?.toFixed(1) || 'N/A' },
{ name: 'Jitter', value: `${metrics.jitter_ms?.toFixed(0) || 'N/A'}ms` },
{ name: 'Packet Loss', value: `${metrics.packet_loss_pct?.toFixed(1) || 'N/A'}%` },
{ name: 'RTT', value: `${metrics.rtt_ms?.toFixed(0) || 'N/A'}ms` },
],
}],
}),
});
res.sendStatus(200);
});Troubleshooting Webhooks
| Problem | Cause | Solution |
|---|---|---|
| Webhook never fires | Probe status is not changing between executions | Check probe execution history -- if status is always HEALTHY, no webhook is sent |
| 403 Forbidden response | Your endpoint is rejecting the request | Ensure your server accepts POST requests from external sources |
| Timeout errors | Your endpoint takes too long to respond | Respond with 200 immediately and process the payload asynchronously |
| Signature verification fails | Incorrect secret or payload manipulation | Verify you are using the correct webhook secret and computing the signature over the raw request body (not parsed JSON) |
| Webhook disabled automatically | 10+ consecutive failures in 24 hours | Fix the endpoint issue and re-enable the webhook in probe settings |
| Duplicate notifications | Your endpoint returned a non-2xx status but still processed the payload | Implement idempotency using the execution_id field to deduplicate |
Best Practices
- Always verify signatures -- never process webhook payloads without HMAC verification
- Implement replay prevention -- reject webhooks with timestamps older than 5 minutes
- Respond quickly -- return 2xx within a few seconds; do heavy processing asynchronously
- Use idempotency -- use
execution_idto handle potential duplicate deliveries gracefully - Monitor webhook health -- set up your own monitoring for the webhook receiver endpoint
- Keep secrets secure -- store webhook secrets in environment variables, never in source code
- Test with manual status changes -- some integrations have test webhook features to verify your endpoint before going live
Next Steps
- Threshold Configuration -- Control when webhooks fire by adjusting thresholds
- Creating a Probe -- Set up a probe with webhook notifications
- Status Pages -- Public health dashboards as an alternative to webhook alerts
- Probe Health States -- Complete reference for health state transitions