Webhook Security
Securing your webhook endpoints is critical to protecting your application from unauthorized access, data tampering, and malicious attacks. This guide provides comprehensive security practices for implementing and maintaining secure webhook integrations with Dasha BlackBox.
Overview
Webhook security involves multiple layers of protection to ensure that:
- Only legitimate Dasha BlackBox webhooks reach your application
- Webhook payloads haven’t been tampered with during transmission
- Replay attacks are prevented
- Sensitive data is protected in transit and at rest
- Your webhook endpoint can’t be overwhelmed by malicious traffic
Security is not optional: Failing to properly secure webhook endpoints can lead to data breaches, service disruptions, unauthorized access, and compliance violations. Treat webhook security with the same rigor as user authentication.
Why Webhook Security Matters
Webhooks are publicly accessible endpoints that accept data from external sources. Without proper security measures, your application is vulnerable to:
Spoofing Attacks
Malicious actors can send fake webhook requests pretending to be from Dasha BlackBox, potentially:
- Injecting false data into your systems
- Triggering unauthorized actions
- Bypassing business logic validation
- Corrupting analytics and reporting
Tampering Attacks
Without integrity verification, attackers can:
- Modify webhook payload data in transit
- Change call outcomes or transcripts
- Manipulate financial or user data
- Alter system state maliciously
Replay Attacks
Captured legitimate webhook requests can be:
- Resent multiple times to duplicate actions
- Used to overwhelm your systems
- Exploited to trigger unintended side effects
- Leveraged to bypass rate limiting
Data Exposure
Insecure webhook endpoints can:
- Expose sensitive call data over unencrypted connections
- Leak personally identifiable information (PII)
- Violate GDPR, HIPAA, or other compliance requirements
- Compromise user privacy
HTTPS Requirement
All webhook endpoints MUST use HTTPS with valid TLS/SSL certificates. HTTP is not supported and will be rejected by Dasha BlackBox.
Why HTTPS is Required
Encryption in Transit
- Prevents eavesdropping on webhook payloads
- Protects sensitive call data (transcripts, user info, metadata)
- Ensures data integrity during transmission
- Meets compliance requirements (PCI DSS, HIPAA, GDPR)
Server Authentication
- Validates your server’s identity via SSL certificates
- Prevents man-in-the-middle attacks
- Ensures webhooks reach the intended destination
- Provides chain of trust verification
SSL Certificate Requirements
Production Requirements
Development Testing
Troubleshooting SSL
Required for Production:
- Valid SSL certificate from trusted Certificate Authority (CA)
- Certificate not expired or self-signed
- Certificate matches your domain exactly
- TLS 1.2 or higher (TLS 1.3 recommended)
- Strong cipher suites enabled
- No mixed content (all resources over HTTPS)
Recommended Certificate Authorities:
- Let’s Encrypt (free, automated renewal)
- DigiCert
- Sectigo
- GlobalSign
- AWS Certificate Manager (for AWS-hosted endpoints)
For Local Development:
- Use ngrok or similar tunneling service for HTTPS
- Localtunnel with —local-https flag
- Use self-signed certificates ONLY for isolated testing
- Never use self-signed certificates in production
ngrok Example:# Tunnel local port 3000 with HTTPS
ngrok http 3000
# Use the HTTPS URL in webhook configuration
# https://abc123.ngrok.io/webhooks/blackbox
Localtunnel Example:# Expose local port with HTTPS
npx localtunnel --port 3000
# Use the provided HTTPS URL
# https://random-subdomain.loca.lt
Common SSL Issues:Certificate Expired:
- Automated renewal failed (Let’s Encrypt)
- Manual renewal forgotten
- Fix: Renew certificate immediately, update configuration
Certificate Mismatch:
- Certificate issued for different domain
- Subdomain not covered by wildcard
- Fix: Issue new certificate for correct domain
Weak Cipher Suites:
- Older TLS versions enabled
- Weak encryption algorithms allowed
- Fix: Update server configuration, disable TLS 1.0/1.1
Mixed Content:
- HTTP resources on HTTPS page
- Insecure redirects
- Fix: Ensure all resources use HTTPS
Automated Certificate Management: Use Let’s Encrypt with automated renewal (Certbot) to eliminate manual certificate management and prevent expiration issues.
Webhook Signatures
Dasha BlackBox signs all webhook requests using HMAC-SHA256 to ensure authenticity and integrity. Always verify webhook signatures before processing payloads.
How Webhook Signatures Work
-
Dasha BlackBox generates signature:
- Creates HMAC-SHA256 hash of webhook payload
- Uses your secret key as the signing key
- Includes signature in
X-Dasha BlackBox-Signature header
-
Your server verifies signature:
- Receives webhook with signature header
- Computes HMAC-SHA256 of received payload with same secret
- Compares computed signature with received signature
- Accepts webhook only if signatures match
Signature Verification Algorithm
Extract Signature from Header
const receivedSignature = request.headers['x-blackbox-signature'];
if (!receivedSignature) {
// Reject unsigned requests
return res.status(401).json({ error: 'Missing signature' });
}
Get Raw Request Body
// CRITICAL: Use raw request body, not parsed JSON
// Signature is computed from exact bytes received
const rawBody = request.rawBody; // Express: use raw-body middleware
// or
const rawBody = await request.text(); // Next.js/modern frameworks
Compute HMAC-SHA256 Hash
const crypto = require('crypto');
const secret = process.env.BLACKBOX_WEBHOOK_SECRET;
const computedSignature = crypto
.createHmac('sha256', secret)
.update(rawBody)
.digest('hex');
Compare Signatures Securely
// Use timing-safe comparison to prevent timing attacks
const crypto = require('crypto');
const isValid = crypto.timingSafeEqual(
Buffer.from(receivedSignature, 'hex'),
Buffer.from(computedSignature, 'hex')
);
if (!isValid) {
return res.status(401).json({ error: 'Invalid signature' });
}
Process Webhook Payload
// Signature verified - safe to process
const payload = JSON.parse(rawBody);
await processWebhook(payload);
return res.status(200).json({ success: true });
Complete Signature Verification Examples
Node.js (Express)
Python (Flask)
Go
const express = require('express');
const crypto = require('crypto');
const bodyParser = require('body-parser');
const app = express();
// CRITICAL: Use raw body for signature verification
app.use(bodyParser.json({
verify: (req, res, buf) => {
req.rawBody = buf.toString('utf8');
}
}));
app.post('/webhooks/blackbox', async (req, res) => {
try {
// Step 1: Extract signature header
const receivedSignature = req.headers['x-blackbox-signature'];
if (!receivedSignature) {
console.error('Missing webhook signature');
return res.status(401).json({ error: 'Unauthorized' });
}
// Step 2: Get raw request body
const rawBody = req.rawBody;
if (!rawBody) {
console.error('Missing request body');
return res.status(400).json({ error: 'Bad Request' });
}
// Step 3: Compute expected signature
const secret = process.env.BLACKBOX_WEBHOOK_SECRET;
const computedSignature = crypto
.createHmac('sha256', secret)
.update(rawBody)
.digest('hex');
// Step 4: Timing-safe comparison
const signatureBuffer = Buffer.from(receivedSignature, 'hex');
const computedBuffer = Buffer.from(computedSignature, 'hex');
if (signatureBuffer.length !== computedBuffer.length) {
console.error('Signature length mismatch');
return res.status(401).json({ error: 'Invalid signature' });
}
const isValid = crypto.timingSafeEqual(signatureBuffer, computedBuffer);
if (!isValid) {
console.error('Signature verification failed');
return res.status(401).json({ error: 'Invalid signature' });
}
// Step 5: Process webhook (signature verified)
const payload = JSON.parse(rawBody);
console.log('Webhook verified:', payload.event);
// Process webhook based on event type
await handleWebhookEvent(payload);
return res.status(200).json({ success: true });
} catch (error) {
console.error('Webhook processing error:', error);
return res.status(500).json({ error: 'Internal server error' });
}
});
async function handleWebhookEvent(payload) {
switch (payload.type) {
case 'StartWebHookPayload':
await handleCallStarted(payload);
break;
case 'CompletedWebHookPayload':
await handleCallCompleted(payload);
break;
case 'FailedWebHookPayload':
await handleCallFailed(payload);
break;
default:
console.warn('Unknown payload type:', payload.type);
}
}
app.listen(3000, () => {
console.log('Webhook endpoint listening on port 3000');
});
from flask import Flask, request, jsonify
import hmac
import hashlib
import os
app = Flask(__name__)
@app.route('/webhooks/blackbox', methods=['POST'])
def blackbox_webhook():
try:
# Step 1: Extract signature header
received_signature = request.headers.get('X-Dasha BlackBox-Signature')
if not received_signature:
app.logger.error('Missing webhook signature')
return jsonify({'error': 'Unauthorized'}), 401
# Step 2: Get raw request body
raw_body = request.get_data(as_text=True)
if not raw_body:
app.logger.error('Missing request body')
return jsonify({'error': 'Bad Request'}), 400
# Step 3: Compute expected signature
secret = os.environ.get('BLACKBOX_WEBHOOK_SECRET').encode('utf-8')
computed_signature = hmac.new(
secret,
raw_body.encode('utf-8'),
hashlib.sha256
).hexdigest()
# Step 4: Timing-safe comparison
is_valid = hmac.compare_digest(
received_signature,
computed_signature
)
if not is_valid:
app.logger.error('Signature verification failed')
return jsonify({'error': 'Invalid signature'}), 401
# Step 5: Process webhook (signature verified)
payload = request.get_json()
app.logger.info(f"Webhook verified: {payload.get('event')}")
# Process webhook based on event type
handle_webhook_event(payload)
return jsonify({'success': True}), 200
except Exception as e:
app.logger.error(f'Webhook processing error: {str(e)}')
return jsonify({'error': 'Internal server error'}), 500
def handle_webhook_event(payload):
payload_type = payload.get('type')
if payload_type == 'StartWebHookPayload':
handle_call_started(payload)
elif payload_type == 'CompletedWebHookPayload':
handle_call_completed(payload)
elif payload_type == 'FailedWebHookPayload':
handle_call_failed(payload)
else:
app.logger.warning(f'Unknown payload type: {payload_type}')
def handle_call_started(payload):
# Implement your call started logic
pass
def handle_call_completed(payload):
# Implement your call completed logic
pass
def handle_call_failed(payload):
# Implement your call failed logic
pass
if __name__ == '__main__':
app.run(port=3000)
package main
import (
"crypto/hmac"
"crypto/sha256"
"crypto/subtle"
"encoding/hex"
"encoding/json"
"io/ioutil"
"log"
"net/http"
"os"
)
type WebhookPayload struct {
Type string `json:"type"`
Status string `json:"status"`
CallID string `json:"callId"`
AgentID string `json:"agentId"`
OrgID string `json:"orgId"`
Data map[string]interface{} `json:"data"`
}
func main() {
http.HandleFunc("/webhooks/blackbox", blackboxWebhookHandler)
log.Println("Webhook endpoint listening on port 3000")
log.Fatal(http.ListenAndServe(":3000", nil))
}
func blackboxWebhookHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
return
}
// Step 1: Extract signature header
receivedSignature := r.Header.Get("X-Dasha BlackBox-Signature")
if receivedSignature == "" {
log.Println("Missing webhook signature")
http.Error(w, "Unauthorized", http.StatusUnauthorized)
return
}
// Step 2: Get raw request body
rawBody, err := ioutil.ReadAll(r.Body)
if err != nil {
log.Printf("Error reading request body: %v", err)
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
defer r.Body.Close()
// Step 3: Compute expected signature
secret := os.Getenv("BLACKBOX_WEBHOOK_SECRET")
mac := hmac.New(sha256.New, []byte(secret))
mac.Write(rawBody)
computedSignature := hex.EncodeToString(mac.Sum(nil))
// Step 4: Timing-safe comparison
receivedBytes, err := hex.DecodeString(receivedSignature)
if err != nil {
log.Println("Invalid signature format")
http.Error(w, "Invalid signature", http.StatusUnauthorized)
return
}
computedBytes, _ := hex.DecodeString(computedSignature)
if subtle.ConstantTimeCompare(receivedBytes, computedBytes) != 1 {
log.Println("Signature verification failed")
http.Error(w, "Invalid signature", http.StatusUnauthorized)
return
}
// Step 5: Process webhook (signature verified)
var payload WebhookPayload
err = json.Unmarshal(rawBody, &payload)
if err != nil {
log.Printf("Error parsing webhook payload: %v", err)
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
log.Printf("Webhook verified: %s", payload.Type)
// Process webhook based on payload type
handleWebhookEvent(payload)
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(map[string]bool{"success": true})
}
func handleWebhookEvent(payload WebhookPayload) {
switch payload.Type {
case "StartWebHookPayload":
handleCallStarted(payload)
case "CompletedWebHookPayload":
handleCallCompleted(payload)
case "FailedWebHookPayload":
handleCallFailed(payload)
default:
log.Printf("Unknown payload type: %s", payload.Type)
}
}
func handleCallStarted(payload WebhookPayload) {
// Implement your call started logic
}
func handleCallCompleted(payload WebhookPayload) {
// Implement your call completed logic
}
func handleCallFailed(payload WebhookPayload) {
// Implement your call failed logic
}
Common Mistakes to Avoid:
- Using parsed JSON body: Signature must be computed from raw bytes, not re-serialized JSON
- String comparison: Use timing-safe comparison to prevent timing attacks
- Wrong secret: Ensure secret matches what’s configured in Dasha BlackBox dashboard
- Missing middleware: Some frameworks parse body before signature verification runs
- Character encoding: Ensure consistent UTF-8 encoding throughout
Timestamp Validation
Prevent replay attacks by validating webhook timestamps. Reject webhooks that are too old to prevent attackers from reusing captured requests.
Why Timestamp Validation Matters
Replay Attack Prevention
- Captured legitimate webhooks can’t be resent hours/days later
- Limits the window of opportunity for attackers
- Prevents duplicate processing of old events
- Protects against time-based manipulation
Clock Skew Tolerance
- Accounts for small time differences between servers
- Typical tolerance: 5 minutes (300 seconds)
- Prevents false rejections due to network latency
Implementation Guide
function validateWebhookTimestamp(payload) {
const TOLERANCE_SECONDS = 300; // 5 minutes
// Extract timestamp from payload
const webhookTimestamp = new Date(payload.timestamp);
const currentTimestamp = new Date();
// Calculate age of webhook
const ageSeconds = (currentTimestamp - webhookTimestamp) / 1000;
// Reject if too old
if (ageSeconds > TOLERANCE_SECONDS) {
console.error(`Webhook too old: ${ageSeconds}s (max ${TOLERANCE_SECONDS}s)`);
return false;
}
// Reject if from future (clock skew)
if (ageSeconds < -TOLERANCE_SECONDS) {
console.error(`Webhook from future: ${ageSeconds}s`);
return false;
}
return true;
}
// In your webhook handler:
app.post('/webhooks/blackbox', async (req, res) => {
// ... signature verification ...
const payload = JSON.parse(req.rawBody);
// Validate timestamp
if (!validateWebhookTimestamp(payload)) {
return res.status(401).json({ error: 'Webhook timestamp invalid or expired' });
}
// Process webhook
await handleWebhookEvent(payload);
return res.status(200).json({ success: true });
});
from datetime import datetime, timezone, timedelta
def validate_webhook_timestamp(payload):
TOLERANCE_SECONDS = 300 # 5 minutes
# Extract timestamp from payload
webhook_timestamp_str = payload.get('timestamp')
if not webhook_timestamp_str:
app.logger.error('Missing timestamp in payload')
return False
# Parse ISO 8601 timestamp
webhook_timestamp = datetime.fromisoformat(
webhook_timestamp_str.replace('Z', '+00:00')
)
current_timestamp = datetime.now(timezone.utc)
# Calculate age of webhook
age = (current_timestamp - webhook_timestamp).total_seconds()
# Reject if too old
if age > TOLERANCE_SECONDS:
app.logger.error(
f'Webhook too old: {age}s (max {TOLERANCE_SECONDS}s)'
)
return False
# Reject if from future (clock skew)
if age < -TOLERANCE_SECONDS:
app.logger.error(f'Webhook from future: {age}s')
return False
return True
# In your webhook handler:
@app.route('/webhooks/blackbox', methods=['POST'])
def blackbox_webhook():
# ... signature verification ...
payload = request.get_json()
# Validate timestamp
if not validate_webhook_timestamp(payload):
return jsonify({
'error': 'Webhook timestamp invalid or expired'
}), 401
# Process webhook
handle_webhook_event(payload)
return jsonify({'success': True}), 200
import (
"time"
"log"
)
func validateWebhookTimestamp(payload WebhookPayload) bool {
const toleranceSeconds = 300 // 5 minutes
// Parse timestamp from payload
webhookTimestamp, err := time.Parse(time.RFC3339, payload.Timestamp)
if err != nil {
log.Printf("Invalid timestamp format: %v", err)
return false
}
currentTimestamp := time.Now().UTC()
// Calculate age of webhook
age := currentTimestamp.Sub(webhookTimestamp).Seconds()
// Reject if too old
if age > toleranceSeconds {
log.Printf("Webhook too old: %.0fs (max %d s)", age, toleranceSeconds)
return false
}
// Reject if from future (clock skew)
if age < -toleranceSeconds {
log.Printf("Webhook from future: %.0fs", age)
return false
}
return true
}
// In your webhook handler:
func blackboxWebhookHandler(w http.ResponseWriter, r *http.Request) {
// ... signature verification ...
var payload WebhookPayload
json.Unmarshal(rawBody, &payload)
// Validate timestamp
if !validateWebhookTimestamp(payload) {
http.Error(w, "Webhook timestamp invalid or expired",
http.StatusUnauthorized)
return
}
// Process webhook
handleWebhookEvent(payload)
w.WriteHeader(http.StatusOK)
json.NewEncoder(w).Encode(map[string]bool{"success": true})
}
Adjusting Tolerance: If you experience legitimate webhook rejections due to clock skew, you can increase the tolerance to 10 minutes (600 seconds), but never exceed 15 minutes for security reasons.
IP Whitelisting
Restrict webhook delivery to known Dasha BlackBox IP ranges for additional security.
Dasha BlackBox Webhook IP Ranges
Configure your firewall or application to only accept webhook requests from these IP ranges:
Production IP Ranges (CIDR notation):
- 52.202.195.162/32
- 34.226.14.86/32
- 3.218.180.13/32
Development/Staging IP Ranges:
- 54.210.32.45/32
- 18.206.107.29/32
IP Ranges May Change: Dasha BlackBox IP ranges may be updated occasionally. Subscribe to the Dasha BlackBox Status Page for notifications about infrastructure changes.
AWS Security Groups
Nginx
Express.js Middleware
Cloudflare
# Add inbound rules to security group
aws ec2 authorize-security-group-ingress \
--group-id sg-12345678 \
--protocol tcp \
--port 443 \
--cidr 52.202.195.162/32
aws ec2 authorize-security-group-ingress \
--group-id sg-12345678 \
--protocol tcp \
--port 443 \
--cidr 34.226.14.86/32
aws ec2 authorize-security-group-ingress \
--group-id sg-12345678 \
--protocol tcp \
--port 443 \
--cidr 3.218.180.13/32
# In your server block
location /webhooks/blackbox {
# Allow Dasha BlackBox production IPs
allow 52.202.195.162;
allow 34.226.14.86;
allow 3.218.180.13;
# Deny all other IPs
deny all;
# Proxy to application
proxy_pass http://localhost:3000;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
const ALLOWED_IPS = [
'52.202.195.162',
'34.226.14.86',
'3.218.180.13'
];
function ipWhitelistMiddleware(req, res, next) {
// Get client IP (consider X-Forwarded-For if behind proxy)
const clientIP = req.headers['x-forwarded-for']?.split(',')[0]
|| req.connection.remoteAddress
|| req.socket.remoteAddress;
// Remove IPv6 prefix if present
const normalizedIP = clientIP.replace(/^::ffff:/, '');
// Check if IP is whitelisted
if (!ALLOWED_IPS.includes(normalizedIP)) {
console.error(`Rejected request from unauthorized IP: ${normalizedIP}`);
return res.status(403).json({ error: 'Forbidden' });
}
next();
}
// Apply to webhook endpoint
app.post('/webhooks/blackbox', ipWhitelistMiddleware, async (req, res) => {
// ... webhook processing ...
});
// Cloudflare Workers script
const ALLOWED_IPS = [
'52.202.195.162',
'34.226.14.86',
'3.218.180.13'
];
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
});
async function handleRequest(request) {
// Get client IP
const clientIP = request.headers.get('CF-Connecting-IP');
// Check IP whitelist
if (!ALLOWED_IPS.includes(clientIP)) {
return new Response('Forbidden', {
status: 403,
headers: { 'Content-Type': 'text/plain' }
});
}
// Forward to origin
return fetch(request);
}
Behind a Proxy? If your application is behind a reverse proxy (Nginx, Cloudflare, AWS ALB), use the X-Forwarded-For header to get the real client IP. Always validate the proxy is trusted before trusting this header.
In addition to signature verification, use custom authentication headers for defense-in-depth.
Custom Token Authentication
Include a secret token in webhook configuration that must be present in all requests.
In Dasha BlackBox Dashboard:
- Navigate to agent configuration
- Go to Webhooks tab
- Set webhook URL with authentication token:
https://api.yourapp.com/webhooks/blackbox?token=your_secret_token_here
Or use custom header (if supported):URL: https://api.yourapp.com/webhooks/blackbox
Headers:
X-Auth-Token: your_secret_token_here
app.post('/webhooks/blackbox', async (req, res) => {
// Validate authentication token from query parameter
const providedToken = req.query.token;
const expectedToken = process.env.WEBHOOK_AUTH_TOKEN;
if (!providedToken || providedToken !== expectedToken) {
console.error('Invalid or missing authentication token');
return res.status(401).json({ error: 'Unauthorized' });
}
// Continue with signature verification and processing
// ... rest of webhook handler ...
});
Token Security:
- Generate cryptographically random tokens (minimum 32 characters)
- Store tokens securely (environment variables, secrets manager)
- Rotate tokens periodically (every 90 days recommended)
- Never commit tokens to version control
- Use different tokens for different environments
Rate Limiting
Protect your webhook endpoint from abuse by implementing rate limiting.
Why Rate Limiting Matters
- DoS Protection: Prevent denial-of-service attacks
- Resource Management: Protect application resources
- Cost Control: Prevent runaway processing costs
- Fair Usage: Ensure system stability under load
Rate Limiting Strategies
const rateLimit = require('express-rate-limit');
// Configure rate limiter
const webhookLimiter = rateLimit({
windowMs: 60 * 1000, // 1 minute window
max: 100, // Max 100 requests per window per IP
message: 'Too many webhook requests, please try again later',
standardHeaders: true, // Return rate limit info in headers
legacyHeaders: false,
// Custom key generator (use IP or other identifier)
keyGenerator: (req) => {
return req.headers['x-forwarded-for']?.split(',')[0]
|| req.connection.remoteAddress;
},
// Skip rate limiting for successful requests
skipSuccessfulRequests: false,
// Skip rate limiting for failed requests
skipFailedRequests: false,
});
// Apply to webhook endpoint
app.post('/webhooks/blackbox', webhookLimiter, async (req, res) => {
// ... webhook processing ...
});
# Define rate limit zone (10MB memory, ~160k IPs)
limit_req_zone $binary_remote_addr zone=webhook_limit:10m rate=100r/m;
server {
listen 443 ssl;
server_name api.yourapp.com;
location /webhooks/blackbox {
# Apply rate limit (burst allows temporary spikes)
limit_req zone=webhook_limit burst=20 nodelay;
# Custom response for rate limit exceeded
limit_req_status 429;
# Proxy to application
proxy_pass http://localhost:3000;
}
}
// Use Cloudflare Durable Objects for distributed rate limiting
const RATE_LIMIT = {
requests: 100,
window: 60 // seconds
};
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
});
async function handleRequest(request) {
const clientIP = request.headers.get('CF-Connecting-IP');
const key = `webhook_limit:${clientIP}`;
// Get current count from KV
const currentCount = await WEBHOOK_KV.get(key);
const count = currentCount ? parseInt(currentCount) : 0;
if (count >= RATE_LIMIT.requests) {
return new Response('Rate limit exceeded', {
status: 429,
headers: {
'Retry-After': RATE_LIMIT.window,
'Content-Type': 'text/plain'
}
});
}
// Increment counter
await WEBHOOK_KV.put(key, (count + 1).toString(), {
expirationTtl: RATE_LIMIT.window
});
// Forward request
return fetch(request);
}
Choosing Rate Limits: Set limits based on expected legitimate traffic plus safety margin. For most applications, 100-1000 requests per minute per IP is reasonable. Monitor actual traffic patterns and adjust accordingly.
Idempotency
Handle duplicate webhook deliveries gracefully to prevent unintended side effects.
Why Idempotency Matters
Dasha BlackBox may deliver the same webhook multiple times due to:
- Network retries: Temporary connection failures
- Timeout retries: Your endpoint didn’t respond quickly enough
- System failures: Infrastructure issues during delivery
- Manual retries: Support team testing or investigation
Without idempotency, duplicate deliveries can cause:
- Duplicate charges or transactions
- Multiple database inserts
- Repeated notifications to users
- Incorrect analytics or reporting
Idempotency Implementation
Database-Based (Recommended)
Redis-Based (Fast Lookups)
In-Memory (Development Only)
const crypto = require('crypto');
// Database schema for processed webhooks
// CREATE TABLE processed_webhooks (
// id VARCHAR(255) PRIMARY KEY,
// event_type VARCHAR(100) NOT NULL,
// processed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
// payload JSONB,
// INDEX idx_processed_at (processed_at)
// );
async function processWebhookIdempotent(payload) {
// Generate unique webhook ID from payload
const webhookId = generateWebhookId(payload);
// Check if already processed
const existing = await db.query(
'SELECT id FROM processed_webhooks WHERE id = $1',
[webhookId]
);
if (existing.rows.length > 0) {
console.log(`Webhook ${webhookId} already processed, skipping`);
return { success: true, duplicate: true };
}
// Begin transaction
await db.query('BEGIN');
try {
// Record webhook as processed
await db.query(
'INSERT INTO processed_webhooks (id, event_type, payload) VALUES ($1, $2, $3)',
[webhookId, payload.event, JSON.stringify(payload)]
);
// Process webhook logic
await handleWebhookEvent(payload);
// Commit transaction
await db.query('COMMIT');
return { success: true, duplicate: false };
} catch (error) {
// Rollback on error
await db.query('ROLLBACK');
throw error;
}
}
function generateWebhookId(payload) {
// Create deterministic ID from payload
// Use event ID if available, otherwise hash payload
if (payload.eventId) {
return payload.eventId;
}
// Generate hash from stable payload fields
const stable = {
event: payload.event,
timestamp: payload.timestamp,
callId: payload.data?.callId,
agentId: payload.data?.agentId
};
return crypto
.createHash('sha256')
.update(JSON.stringify(stable))
.digest('hex');
}
// Webhook handler
app.post('/webhooks/blackbox', async (req, res) => {
// ... signature verification ...
const payload = JSON.parse(req.rawBody);
try {
const result = await processWebhookIdempotent(payload);
if (result.duplicate) {
console.log('Duplicate webhook processed successfully');
}
return res.status(200).json({ success: true });
} catch (error) {
console.error('Webhook processing failed:', error);
return res.status(500).json({ error: 'Internal server error' });
}
});
const redis = require('redis');
const client = redis.createClient();
const IDEMPOTENCY_TTL = 86400; // 24 hours
async function processWebhookIdempotent(payload) {
const webhookId = generateWebhookId(payload);
const key = `webhook:processed:${webhookId}`;
// Try to set key (NX = only if not exists)
const wasSet = await client.set(key, '1', {
NX: true,
EX: IDEMPOTENCY_TTL
});
if (!wasSet) {
console.log(`Webhook ${webhookId} already processed, skipping`);
return { success: true, duplicate: true };
}
try {
// Process webhook
await handleWebhookEvent(payload);
return { success: true, duplicate: false };
} catch (error) {
// Remove key if processing failed (allow retry)
await client.del(key);
throw error;
}
}
function generateWebhookId(payload) {
// Same as database-based example
if (payload.eventId) {
return payload.eventId;
}
const stable = {
event: payload.event,
timestamp: payload.timestamp,
callId: payload.data?.callId,
agentId: payload.data?.agentId
};
return crypto
.createHash('sha256')
.update(JSON.stringify(stable))
.digest('hex');
}
// WARNING: Not suitable for production (not shared across instances)
const processedWebhooks = new Set();
const CLEANUP_INTERVAL = 3600000; // 1 hour
// Periodic cleanup of old IDs
setInterval(() => {
if (processedWebhooks.size > 10000) {
processedWebhooks.clear();
console.log('Cleared processed webhooks cache');
}
}, CLEANUP_INTERVAL);
function processWebhookIdempotent(payload) {
const webhookId = generateWebhookId(payload);
if (processedWebhooks.has(webhookId)) {
console.log(`Webhook ${webhookId} already processed, skipping`);
return { success: true, duplicate: true };
}
// Mark as processed
processedWebhooks.add(webhookId);
// Process webhook
handleWebhookEvent(payload);
return { success: true, duplicate: false };
}
Idempotency Key TTL: Store idempotency keys for at least 24 hours to handle delayed retries. For critical financial transactions, consider 7-30 days retention.
Secret Management
Store webhook secrets securely to prevent unauthorized access.
Environment Variables (Basic)
# .env file (never commit to version control)
BLACKBOX_WEBHOOK_SECRET=your_webhook_secret_here
WEBHOOK_AUTH_TOKEN=your_custom_auth_token_here
# Load in application
require('dotenv').config();
const secret = process.env.BLACKBOX_WEBHOOK_SECRET;
AWS Secrets Manager (Recommended for Production)
const { SecretsManagerClient, GetSecretValueCommand } =
require('@aws-sdk/client-secrets-manager');
const client = new SecretsManagerClient({ region: 'us-east-1' });
async function getWebhookSecret() {
try {
const command = new GetSecretValueCommand({
SecretId: 'blackbox/webhook-secret'
});
const response = await client.send(command);
if (response.SecretString) {
const secrets = JSON.parse(response.SecretString);
return secrets.WEBHOOK_SECRET;
}
throw new Error('Secret not found');
} catch (error) {
console.error('Error retrieving secret:', error);
throw error;
}
}
// Cache secret (refresh periodically)
let cachedSecret = null;
let secretLastFetched = null;
const SECRET_CACHE_TTL = 3600000; // 1 hour
async function getCachedWebhookSecret() {
const now = Date.now();
if (!cachedSecret || (now - secretLastFetched) > SECRET_CACHE_TTL) {
cachedSecret = await getWebhookSecret();
secretLastFetched = now;
}
return cachedSecret;
}
import boto3
import json
from datetime import datetime, timedelta
client = boto3.client('secretsmanager', region_name='us-east-1')
def get_webhook_secret():
try:
response = client.get_secret_value(
SecretId='blackbox/webhook-secret'
)
if 'SecretString' in response:
secrets = json.loads(response['SecretString'])
return secrets['WEBHOOK_SECRET']
raise Exception('Secret not found')
except Exception as e:
print(f'Error retrieving secret: {e}')
raise
# Cache secret (refresh periodically)
cached_secret = None
secret_last_fetched = None
SECRET_CACHE_TTL = timedelta(hours=1)
def get_cached_webhook_secret():
global cached_secret, secret_last_fetched
now = datetime.now()
if (not cached_secret or
not secret_last_fetched or
(now - secret_last_fetched) > SECRET_CACHE_TTL):
cached_secret = get_webhook_secret()
secret_last_fetched = now
return cached_secret
HashiCorp Vault (Enterprise)
const vault = require('node-vault')({
apiVersion: 'v1',
endpoint: 'https://vault.yourcompany.com',
token: process.env.VAULT_TOKEN
});
async function getWebhookSecret() {
try {
const result = await vault.read('secret/data/blackbox/webhooks');
return result.data.data.WEBHOOK_SECRET;
} catch (error) {
console.error('Vault error:', error);
throw error;
}
}
Secret Rotation: Implement automated secret rotation every 90 days. When rotating, support both old and new secrets for 24 hours to prevent downtime during transition.
Security Best Practices Checklist
Use this checklist to ensure your webhook security implementation is complete.
Critical Security Requirements
Important Security Measures
Recommended Enhancements
Common Security Mistakes
Avoid these frequent webhook security pitfalls.
Using HTTP Instead of HTTPS
Problem: Webhook data transmitted in plaintext, exposing sensitive information.
Solution: Always use HTTPS with valid SSL certificates. Configure redirects from HTTP to HTTPS.
# Nginx redirect
server {
listen 80;
server_name api.yourapp.com;
return 301 https://$server_name$request_uri;
}
Skipping Signature Verification
Problem: Accepting all webhook requests without verifying authenticity.
Solution: Implement HMAC-SHA256 signature verification for every request. Reject unsigned or invalid requests.
Computing Signature from Parsed JSON
Problem: Re-serializing JSON changes byte representation, causing signature mismatch.
Solution: Compute signature from raw request body bytes before parsing JSON.
// WRONG
const payload = req.body;
const signature = crypto.createHmac('sha256', secret)
.update(JSON.stringify(payload)) // Don't do this!
.digest('hex');
// CORRECT
const rawBody = req.rawBody; // Raw bytes
const signature = crypto.createHmac('sha256', secret)
.update(rawBody) // Use raw body
.digest('hex');
Using String Comparison for Signatures
Problem: Vulnerable to timing attacks that can leak signature information.
Solution: Use timing-safe comparison functions.
// WRONG
if (receivedSignature === computedSignature) { }
// CORRECT
const isValid = crypto.timingSafeEqual(
Buffer.from(receivedSignature, 'hex'),
Buffer.from(computedSignature, 'hex')
);
Storing Secrets in Code
Problem: Secrets committed to version control or exposed in application code.
Solution: Use environment variables or secrets management services.
// WRONG
const secret = 'hardcoded_secret_12345'; // Never do this!
// CORRECT
const secret = process.env.BLACKBOX_WEBHOOK_SECRET;
No Idempotency Handling
Problem: Duplicate webhooks cause duplicate actions (charges, emails, database inserts).
Solution: Implement idempotency using database or cache-based deduplication.
Exposing Webhook URLs
Problem: Webhook URLs discoverable through public repositories, logs, or error messages.
Solution:
- Use non-obvious webhook paths
- Never log full webhook URLs
- Add authentication tokens to query parameters or headers
- Review public repositories for exposed secrets
Insufficient Error Handling
Problem: Security errors expose internal details to attackers.
Solution: Return generic error messages, log detailed errors securely.
// WRONG
return res.status(401).json({
error: 'Signature mismatch',
expected: computedSignature, // Don't expose this!
received: receivedSignature // Don't expose this!
});
// CORRECT
console.error('Signature verification failed', {
expected: computedSignature,
received: receivedSignature
});
return res.status(401).json({ error: 'Unauthorized' });
Compliance Considerations
Ensure webhook security meets regulatory and compliance requirements.
GDPR (General Data Protection Regulation)
Applies to: EU users or businesses operating in EU
Key Requirements:
- Data Encryption: All webhook data encrypted in transit (HTTPS)
- Data Minimization: Only collect necessary data in webhooks
- Access Control: Restrict who can configure webhooks
- Right to Erasure: Ability to delete webhook data on request
- Data Processing Agreement: Ensure Dasha BlackBox has GDPR-compliant DPA
Implementation:
// Redact PII from webhook logs
function logWebhook(payload) {
const sanitized = {
...payload,
data: {
...payload.data,
// Redact PII fields
phoneNumber: '[REDACTED]',
email: '[REDACTED]',
transcript: '[REDACTED]'
}
};
console.log('Webhook received:', sanitized);
}
HIPAA (Health Insurance Portability and Accountability Act)
Applies to: Healthcare applications handling PHI (Protected Health Information)
Key Requirements:
- Encryption: End-to-end encryption for PHI data
- Access Logs: Audit logs of all webhook access
- Business Associate Agreement (BAA): Signed agreement with Dasha BlackBox
- Breach Notification: Plan for reporting security incidents
Implementation:
// Audit logging for HIPAA compliance
function auditWebhookAccess(payload, result) {
const auditLog = {
timestamp: new Date().toISOString(),
event: 'webhook_received',
source: 'blackbox',
eventType: payload.event,
callId: payload.data?.callId,
result: result.success ? 'success' : 'failure',
ipAddress: req.headers['x-forwarded-for'] || req.connection.remoteAddress
};
// Store in secure, append-only audit log
await auditLogger.log(auditLog);
}
PCI DSS (Payment Card Industry Data Security Standard)
Applies to: Applications processing payment card data
Key Requirements:
- No PAN Storage: Never store full card numbers in webhook payloads
- Encrypted Transmission: TLS 1.2+ for all webhook traffic
- Access Control: Restrict webhook configuration to authorized personnel
- Logging: Log all webhook access for forensic analysis
Implementation:
// Tokenize sensitive data before logging
function processPCIWebhook(payload) {
if (payload.data.paymentMethod) {
// Replace with token, never log full card number
payload.data.paymentMethod = {
type: payload.data.paymentMethod.type,
last4: payload.data.paymentMethod.last4,
token: payload.data.paymentMethod.token
};
}
return payload;
}
CCPA (California Consumer Privacy Act)
Applies to: Businesses serving California residents
Key Requirements:
- Data Disclosure: Inform users about data collected via webhooks
- Right to Delete: Ability to delete user data from webhook storage
- Opt-Out: Allow users to opt out of data collection
Troubleshooting Security Issues
Common security-related webhook problems and solutions.
Signature Verification Failing
Symptoms: All webhooks rejected with “Invalid signature” error
Common Causes:
- Wrong secret: Using incorrect webhook secret
- Character encoding: UTF-8 encoding mismatch
- Body parsing: Signature computed from parsed JSON instead of raw bytes
- Whitespace: Extra whitespace in secret or body
Debugging Steps:
// Enable detailed signature debugging
function debugSignatureVerification(req) {
const receivedSignature = req.headers['x-blackbox-signature'];
const rawBody = req.rawBody;
const secret = process.env.BLACKBOX_WEBHOOK_SECRET;
console.log('=== Signature Debug ===');
console.log('Received signature:', receivedSignature);
console.log('Raw body length:', rawBody.length);
console.log('Raw body (first 100 chars):', rawBody.substring(0, 100));
console.log('Secret length:', secret.length);
console.log('Secret (first 10 chars):', secret.substring(0, 10));
const computedSignature = crypto
.createHmac('sha256', secret)
.update(rawBody)
.digest('hex');
console.log('Computed signature:', computedSignature);
console.log('Signatures match:', receivedSignature === computedSignature);
console.log('======================');
}
Solutions:
- Verify secret matches Dasha BlackBox dashboard configuration
- Ensure raw body middleware configured correctly
- Check for whitespace or encoding issues
- Test with Dasha BlackBox webhook test feature
Webhooks from Future or Past
Symptoms: Legitimate webhooks rejected due to timestamp validation
Common Causes:
- Clock skew: Server clock incorrect
- Timezone issues: Timestamp parsing in wrong timezone
- Tolerance too strict: 5-minute window too narrow
Solutions:
// Diagnose timestamp issues
function debugTimestamp(payload) {
const webhookTimestamp = new Date(payload.timestamp);
const serverTimestamp = new Date();
console.log('=== Timestamp Debug ===');
console.log('Webhook timestamp:', payload.timestamp);
console.log('Parsed webhook time:', webhookTimestamp.toISOString());
console.log('Server time:', serverTimestamp.toISOString());
console.log('Difference (seconds):',
(serverTimestamp - webhookTimestamp) / 1000);
console.log('Server timezone:',
Intl.DateTimeFormat().resolvedOptions().timeZone);
console.log('======================');
}
Fixes:
- Sync server clock with NTP
- Increase tolerance to 10-15 minutes if needed
- Ensure timestamps parsed as UTC
IP Whitelist Blocking Legitimate Webhooks
Symptoms: Webhooks rejected with 403 Forbidden
Common Causes:
- Outdated IP ranges: Dasha BlackBox IPs changed
- Proxy issues: Getting proxy IP instead of original IP
- IPv6 vs IPv4: IP format mismatch
Solutions:
// Debug IP detection
function debugIPDetection(req) {
console.log('=== IP Debug ===');
console.log('X-Forwarded-For:', req.headers['x-forwarded-for']);
console.log('X-Real-IP:', req.headers['x-real-ip']);
console.log('Remote Address:', req.connection.remoteAddress);
console.log('Socket Address:', req.socket.remoteAddress);
console.log('================');
}
Fixes:
- Verify Dasha BlackBox IP ranges on status page
- Check reverse proxy configuration
- Use X-Forwarded-For if behind trusted proxy
Rate Limiting Blocking Legitimate Traffic
Symptoms: Webhooks rejected with 429 Too Many Requests
Solutions:
- Review rate limit thresholds
- Whitelist Dasha BlackBox IPs from rate limiting
- Increase limits for webhook endpoints specifically
- Monitor actual traffic patterns and adjust
Security Monitoring and Alerts
Proactive monitoring catches security issues before they become critical.
Metrics to Monitor
Security Events:
- Invalid signature attempts
- Timestamp validation failures
- IP whitelist rejections
- Rate limit violations
- Authentication failures
Performance Metrics:
- Webhook processing time
- Error rates by error type
- Duplicate webhook rate
- Storage usage for idempotency keys
Alert Configuration
CloudWatch (AWS)
Datadog
Application Logs
const { CloudWatchClient, PutMetricDataCommand } =
require('@aws-sdk/client-cloudwatch');
const cloudwatch = new CloudWatchClient({ region: 'us-east-1' });
async function trackSecurityEvent(eventType) {
const params = {
Namespace: 'Webhooks/Security',
MetricData: [
{
MetricName: 'SecurityEvents',
Dimensions: [
{
Name: 'EventType',
Value: eventType
}
],
Value: 1,
Unit: 'Count',
Timestamp: new Date()
}
]
};
await cloudwatch.send(new PutMetricDataCommand(params));
}
// Track security events
if (!isValidSignature) {
await trackSecurityEvent('InvalidSignature');
}
const { StatsD } = require('node-dogstatsd');
const dogstatsd = new StatsD();
function trackSecurityEvent(eventType) {
dogstatsd.increment('webhook.security.events', 1, {
event_type: eventType,
environment: process.env.NODE_ENV
});
}
// Track security events
if (!isValidSignature) {
trackSecurityEvent('invalid_signature');
}
const winston = require('winston');
const logger = winston.createLogger({
level: 'info',
format: winston.format.json(),
transports: [
new winston.transports.File({
filename: 'webhook-security.log',
level: 'warn'
})
]
});
function logSecurityEvent(eventType, details) {
logger.warn('Webhook security event', {
eventType,
timestamp: new Date().toISOString(),
...details
});
}
// Log security events
if (!isValidSignature) {
logSecurityEvent('invalid_signature', {
receivedSignature: receivedSignature.substring(0, 10),
ipAddress: req.connection.remoteAddress
});
}
Recommended Alerts
Configure alerts for these security scenarios:
Critical (Immediate Response):
- Above 10 invalid signatures per minute
- Above 5 IP whitelist violations per minute
- Sudden spike in authentication failures (above 100% baseline)
Warning (Investigation Needed):
- Above 5 timestamp validation failures per hour
- Rate limit hits increasing trend
- Idempotency key storage above 80% capacity
Informational (Monitor Trends):
- Daily summary of security events
- Weekly security metrics report
- Monthly compliance audit summary
Next Steps
After implementing webhook security:
API Cross-Refs
Webhook security-related endpoints: