CI/CD Integration

Integrate CallMeter into your CI/CD pipeline for automated VoIP quality gates, regression testing, and deployment validation.

CallMeter integrates with your CI/CD pipeline to automatically validate SIP infrastructure quality before, during, or after deployments. By triggering tests via the API and enforcing quality gates on metrics like MOS, packet loss, and jitter, you can catch VoIP regressions before they reach production.

Use Cases

Pre-Deployment Gate

Run a baseline SIP test before deploying changes to your PBX, SIP proxy, or session border controller. If the test fails or quality metrics fall below thresholds, the deployment is blocked.

Post-Deployment Validation

After deploying, trigger a test to verify that call quality has not degraded. This catches issues that pre-deployment testing might miss, such as configuration errors or network path changes.

Nightly Regression Testing

Schedule a comprehensive test suite to run every night. Detect gradual degradation (increasing jitter, declining MOS) before it affects users.

Canary Testing

Run a quick test against a canary environment before routing production traffic. If the canary passes quality gates, proceed with the full rollout.

Pull Request Validation

Trigger tests on PR branches that modify SIP server configurations. Ensure that configuration changes do not break call quality before merging.

API Workflow

The CI/CD integration follows a three-step pattern:

Trigger -- Start a test run via POST request
Poll -- Check the test run status until completion
Evaluate -- Query metrics and enforce quality thresholds

Step 1: Trigger a Test Run

TEST_RUN_ID=$(curl -s -X POST \
  -H "Authorization: Bearer $CALLMETER_API_KEY" \
  https://callmeter.io/api/v1/projects/$PROJECT_ID/tests/$TEST_ID/run \
  | jq -r '.data.id')

echo "Started test run: $TEST_RUN_ID"

The TEST_ID refers to a pre-configured test in your CallMeter project. Create the test once through the web interface with your desired endpoints, codecs, and duration, then reference it by ID in your pipeline.

Step 2: Poll for Completion

echo "Waiting for test to complete..."
while true; do
  RESPONSE=$(curl -s \
    -H "Authorization: Bearer $CALLMETER_API_KEY" \
    https://callmeter.io/api/v1/projects/$PROJECT_ID/test-runs/$TEST_RUN_ID)

  STATUS=$(echo $RESPONSE | jq -r '.data.status')

  case $STATUS in
    COMPLETED)
      echo "Test completed successfully"
      break
      ;;
    FAILED|ERROR)
      echo "Test failed with status: $STATUS"
      exit 1
      ;;
    *)
      echo "Status: $STATUS - waiting..."
      sleep 15
      ;;
  esac
done

Polling interval recommendation

Use a 10-15 second polling interval. Shorter intervals waste API quota; longer intervals delay your pipeline unnecessarily. For long-running tests, start with 30-second intervals.

Step 3: Evaluate Quality Metrics

After the test completes, query the test run summary and enforce your quality thresholds:

SUMMARY=$(curl -s \
  -H "Authorization: Bearer $CALLMETER_API_KEY" \
  https://callmeter.io/api/v1/projects/$PROJECT_ID/test-runs/$TEST_RUN_ID)

# Extract key metrics from the test run summary
MOS=$(echo $SUMMARY | jq -r '.data.summary.avgMos')
PACKET_LOSS=$(echo $SUMMARY | jq -r '.data.summary.avgPacketLossFraction')
JITTER=$(echo $SUMMARY | jq -r '.data.summary.avgJitterMs')
ASR=$(echo $SUMMARY | jq -r '.data.summary.asr')

echo "MOS: $MOS | Loss: $PACKET_LOSS% | Jitter: ${JITTER}ms | ASR: $ASR%"

# Enforce quality gates
FAILED=0

if (( $(echo "$MOS < 4.0" | bc -l) )); then
  echo "FAIL: MOS $MOS is below threshold 4.0"
  FAILED=1
fi

if (( $(echo "$PACKET_LOSS > 1.0" | bc -l) )); then
  echo "FAIL: Packet loss $PACKET_LOSS% exceeds threshold 1.0%"
  FAILED=1
fi

if (( $(echo "$JITTER > 20.0" | bc -l) )); then
  echo "FAIL: Jitter ${JITTER}ms exceeds threshold 20ms"
  FAILED=1
fi

if (( $(echo "$ASR < 95.0" | bc -l) )); then
  echo "FAIL: ASR $ASR% is below threshold 95%"
  FAILED=1
fi

if [ $FAILED -eq 1 ]; then
  echo "Quality gate FAILED - blocking deployment"
  exit 1
fi

echo "All quality gates PASSED"

Quality Gate Thresholds

Recommended thresholds for different environments:

Metric	Production	Staging	Minimum Viable
MOS (Mean Opinion Score)	4.0 or higher	3.8 or higher	3.5 or higher
Packet Loss	Below 1%	Below 2%	Below 5%
Jitter	Below 20ms	Below 30ms	Below 50ms
ASR (Answer Seizure Ratio)	Above 95%	Above 90%	Above 80%
PDD (Post-Dial Delay)	Below 3s	Below 5s	Below 8s

Adjust these thresholds based on your specific SLA commitments and infrastructure characteristics. Stricter thresholds catch regressions earlier but may produce false positives in high-variance environments.

Set realistic thresholds

Overly strict thresholds cause false pipeline failures. Start with lenient thresholds and tighten them gradually as you establish your baseline quality. Use CallMeter dashboards to understand your typical metric ranges before defining CI/CD gates.

GitHub Actions Example

name: VoIP Quality Gate

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  voip-quality-check:
    runs-on: ubuntu-latest
    steps:
      - name: Trigger CallMeter Test
        id: trigger
        run: |
          RESPONSE=$(curl -s -X POST \
            -H "Authorization: Bearer ${{ secrets.CALLMETER_API_KEY }}" \
            https://callmeter.io/api/v1/projects/${{ vars.CALLMETER_PROJECT_ID }}/tests/${{ vars.CALLMETER_TEST_ID }}/run)
          echo "run_id=$(echo $RESPONSE | jq -r '.data.id')" >> $GITHUB_OUTPUT

      - name: Wait for Completion
        id: wait
        run: |
          RUN_ID="${{ steps.trigger.outputs.run_id }}"
          echo "Waiting for test run $RUN_ID..."

          for i in $(seq 1 60); do
            STATUS=$(curl -s \
              -H "Authorization: Bearer ${{ secrets.CALLMETER_API_KEY }}" \
              https://callmeter.io/api/v1/projects/${{ vars.CALLMETER_PROJECT_ID }}/test-runs/$RUN_ID \
              | jq -r '.data.status')

            echo "Attempt $i: Status = $STATUS"

            if [ "$STATUS" = "COMPLETED" ]; then
              echo "status=COMPLETED" >> $GITHUB_OUTPUT
              exit 0
            fi

            if [ "$STATUS" = "FAILED" ] || [ "$STATUS" = "ERROR" ]; then
              echo "Test failed with status: $STATUS"
              exit 1
            fi

            sleep 15
          done

          echo "Timeout waiting for test completion"
          exit 1

      - name: Evaluate Quality Gates
        if: steps.wait.outputs.status == 'COMPLETED'
        run: |
          RUN_ID="${{ steps.trigger.outputs.run_id }}"

          SUMMARY=$(curl -s \
            -H "Authorization: Bearer ${{ secrets.CALLMETER_API_KEY }}" \
            https://callmeter.io/api/v1/projects/${{ vars.CALLMETER_PROJECT_ID }}/test-runs/$RUN_ID)

          MOS=$(echo $SUMMARY | jq -r '.data.summary.avgMos')
          LOSS=$(echo $SUMMARY | jq -r '.data.summary.avgPacketLossFraction')
          JITTER=$(echo $SUMMARY | jq -r '.data.summary.avgJitterMs')

          echo "## VoIP Quality Report" >> $GITHUB_STEP_SUMMARY
          echo "| Metric | Value | Threshold | Status |" >> $GITHUB_STEP_SUMMARY
          echo "|--------|-------|-----------|--------|" >> $GITHUB_STEP_SUMMARY

          PASS=true

          if (( $(echo "$MOS < 4.0" | bc -l) )); then
            echo "| MOS | $MOS | >= 4.0 | FAIL |" >> $GITHUB_STEP_SUMMARY
            PASS=false
          else
            echo "| MOS | $MOS | >= 4.0 | PASS |" >> $GITHUB_STEP_SUMMARY
          fi

          if (( $(echo "$LOSS > 1.0" | bc -l) )); then
            echo "| Packet Loss | $LOSS% | < 1% | FAIL |" >> $GITHUB_STEP_SUMMARY
            PASS=false
          else
            echo "| Packet Loss | $LOSS% | < 1% | PASS |" >> $GITHUB_STEP_SUMMARY
          fi

          if (( $(echo "$JITTER > 20.0" | bc -l) )); then
            echo "| Jitter | ${JITTER}ms | < 20ms | FAIL |" >> $GITHUB_STEP_SUMMARY
            PASS=false
          else
            echo "| Jitter | ${JITTER}ms | < 20ms | PASS |" >> $GITHUB_STEP_SUMMARY
          fi

          if [ "$PASS" = false ]; then
            echo "Quality gate failed"
            exit 1
          fi

          echo "All quality gates passed"

GitLab CI Example

voip-quality-gate:
  stage: test
  image: alpine:latest
  before_script:
    - apk add --no-cache curl jq bc
  variables:
    CALLMETER_PROJECT_ID: "your-project-id"
    CALLMETER_TEST_ID: "your-test-id"
  script:
    # Trigger test
    - |
      RUN_ID=$(curl -s -X POST \
        -H "Authorization: Bearer $CALLMETER_API_KEY" \
        https://callmeter.io/api/v1/projects/$CALLMETER_PROJECT_ID/tests/$CALLMETER_TEST_ID/run \
        | jq -r '.data.id')
      echo "Test run ID: $RUN_ID"

    # Poll for completion
    - |
      for i in $(seq 1 60); do
        STATUS=$(curl -s \
          -H "Authorization: Bearer $CALLMETER_API_KEY" \
          https://callmeter.io/api/v1/projects/$CALLMETER_PROJECT_ID/test-runs/$RUN_ID \
          | jq -r '.data.status')
        echo "Poll $i: $STATUS"
        if [ "$STATUS" = "COMPLETED" ]; then break; fi
        if [ "$STATUS" = "FAILED" ] || [ "$STATUS" = "ERROR" ]; then
          echo "Test failed"; exit 1
        fi
        sleep 15
      done

    # Evaluate metrics
    - |
      SUMMARY=$(curl -s \
        -H "Authorization: Bearer $CALLMETER_API_KEY" \
        https://callmeter.io/api/v1/projects/$CALLMETER_PROJECT_ID/test-runs/$RUN_ID)
      MOS=$(echo $SUMMARY | jq -r '.data.summary.avgMos')
      LOSS=$(echo $SUMMARY | jq -r '.data.summary.avgPacketLossFraction')
      echo "MOS: $MOS | Loss: $LOSS%"
      if (( $(echo "$MOS < 4.0" | bc -l) )); then
        echo "MOS below threshold"; exit 1
      fi
      if (( $(echo "$LOSS > 1.0" | bc -l) )); then
        echo "Packet loss above threshold"; exit 1
      fi
      echo "Quality gates passed"
  only:
    - main
    - merge_requests

#!/bin/bash
set -euo pipefail

API_KEY="${CALLMETER_API_KEY:?Set CALLMETER_API_KEY environment variable}"
PROJECT_ID="${1:?Usage: $0 <project-id> <test-id> [mos-threshold]}"
TEST_ID="${2:?Usage: $0 <project-id> <test-id> [mos-threshold]}"
MOS_THRESHOLD="${3:-4.0}"
API_BASE="https://callmeter.io/api/v1"

# Trigger
echo "Triggering test $TEST_ID..."
RUN_ID=$(curl -sf -X POST \
  -H "Authorization: Bearer $API_KEY" \
  "$API_BASE/projects/$PROJECT_ID/tests/$TEST_ID/run" | jq -r '.data.id')

echo "Run ID: $RUN_ID"

# Poll
echo "Waiting for completion..."
while true; do
  STATUS=$(curl -sf \
    -H "Authorization: Bearer $API_KEY" \
    "$API_BASE/projects/$PROJECT_ID/test-runs/$RUN_ID" | jq -r '.data.status')

  case $STATUS in
    COMPLETED) break ;;
    FAILED|ERROR) echo "FAIL: $STATUS"; exit 1 ;;
    *) sleep 15 ;;
  esac
done

# Evaluate
SUMMARY=$(curl -sf \
  -H "Authorization: Bearer $API_KEY" \
  "$API_BASE/projects/$PROJECT_ID/test-runs/$RUN_ID")

MOS=$(echo $SUMMARY | jq -r '.data.summary.avgMos')
echo "MOS: $MOS (threshold: $MOS_THRESHOLD)"

if (( $(echo "$MOS < $MOS_THRESHOLD" | bc -l) )); then
  echo "QUALITY GATE FAILED"
  exit 1
fi

echo "QUALITY GATE PASSED"

Usage:

# Run with default MOS threshold (4.0)
./callmeter-test.sh your-project-id your-test-id

# Run with custom MOS threshold
./callmeter-test.sh your-project-id your-test-id 3.8

Best Practices

Test Configuration

Create dedicated CI/CD tests -- Do not reuse manual testing configurations. Create purpose-built tests with appropriate duration and endpoint counts for pipeline execution.
Keep tests short -- CI/CD tests should complete in 1-3 minutes. Use longer tests for nightly regressions.
Test one thing at a time -- Separate tests for audio quality, video quality, call setup reliability, and load capacity.

Pipeline Design

Fail fast -- Check test run status before evaluating metrics. A FAILED status means the test itself could not complete, which is a deployment blocker.
Timeout handling -- Always include a maximum wait time. If a test does not complete within a reasonable window, fail the pipeline rather than blocking indefinitely.
Artifact results -- Store metric outputs as pipeline artifacts for later analysis and trending.

Environment Management

Store API keys in secrets -- Never hardcode keys in pipeline files
Use environment variables -- Reference CALLMETER_API_KEY from your CI/CD secret store
Separate test IDs per environment -- Use different test configurations for staging vs. production validation

Threshold Tuning

Start lenient, tighten over time -- Begin with thresholds that rarely fail, then gradually tighten as you establish baselines
Track trends -- A single test run may have variance; look at trends over multiple runs
Different thresholds per environment -- Production gates should be stricter than staging

Next Steps

API Authentication -- Create and manage API keys
API Endpoints -- Complete API reference
Analyzing Results -- Understand the metrics returned by the API