Raxx · internal docs

internal · gated

FreeScout Audit Webhook Setup

Endpoint: POST /api/internal/freescout-webhook/audit Feature flag: FLAG_FREESCOUT_WEBHOOK_RECEIVE (default OFF) Issues: #4258 (primary), #1484 (SC-A4), rescopes #3286

Purpose

This webhook keeps freescout_crm_ticket_cache current within seconds of a FreeScout conversation event. The CRM customer-profile panel (#4254) and audit writer adapters (SC-A5) read from this cache synchronously so they never need a live FreeScout API call on every load.

Events handled: conversation.created, conversation.updated, conversation.status_changed. All others are 200 no-ops.

Cache miss = fail-closed = ticket_state_at_read='none' = Path B (security incident notification). Keeping this webhook healthy is operationally important.

CF / BFM caveat

Read this before enabling in any environment.

FreeScout outbound webhook POSTs traverse Cloudflare WAF before reaching Raptor. When Bot Fight Mode (BFM) is active, Cloudflare may challenge or block these POST requests before HMAC verification runs — meaning the webhook fails before Raptor even sees the payload.

Resolution (required): Add a WAF skip rule in the Cloudflare dashboard keyed on the X-FreeScout-Signature header presence. FreeScout always sends this header; legitimate traffic without it is rejected by Raptor's HMAC check anyway.

Alternatively, if FreeScout uses static egress IPs, add those IPs to the WAF allowlist (check FreeScout server curl ifconfig.me).

This WAF rule must be verified in staging before enabling the flag in production. BFM state can be checked with:

# BFM status — raxx-dev-bot has Zone:Bot Management:Write
curl -s -X GET "https://api.cloudflare.com/client/v4/zones/${CF_ZONE_ID}/bot_management" \
  -H "Authorization: Bearer ${CLOUDFLARE_RAXX_AUTOMATION_API_TOKEN}" | jq '.result.fight_mode'

If fight_mode is true, disable BFM (ops-authorized) or add the WAF skip rule before proceeding. The WAF skip rule is tracked separately from this runbook.

Pre-flight checklist

Before enabling FLAG_FREESCOUT_WEBHOOK_RECEIVE=1:

Step 1: Generate the HMAC secret

python3 -c "import secrets; print(secrets.token_urlsafe(48))"

Store the output in Infisical at: - Path: /MooseQuest/freescout/ - Key: FREESCOUT_AUDIT_WEBHOOK_SECRET

Do not set this via heroku config:set — vault is the source of truth.

Step 2: Register the webhook in FreeScout admin UI

  1. Log in to the FreeScout admin panel.
  2. Navigate to Manage → Apps and find the Webhooks app (install if not present).
  3. Click New Webhook and configure: - Payload URL: https://api.raxx.app/api/internal/freescout-webhook/audit (use https://api-staging.raxx.app/... for staging) - Content Type: application/json - Secret: the HMAC secret from Infisical (paste once; FreeScout stores its own copy) - Events: select all Conversation events (status changed, created, updated). The handler no-ops on unknown events, so a broad subscription is safe.
  4. Save and note the webhook ID.

The URL ...freescout-webhook/audit is the audit cache path. The RBAC auto-revocation webhook (RV-4, future) will use ...freescout-webhook/rbac — register separately when that ships.

Step 3: Enable the feature flag

Staging:

heroku config:set FLAG_FREESCOUT_WEBHOOK_RECEIVE=1 --app raxx-api-staging >/dev/null 2>&1

Prod (after 48-hour staging soak and CF/BFM verification):

heroku config:set FLAG_FREESCOUT_WEBHOOK_RECEIVE=1 --app raxx-api-prod >/dev/null 2>&1

Note: when the flag is OFF, the endpoint returns HTTP 503 (not 404) so FreeScout retries the delivery rather than deactivating the webhook subscription.

Step 4: Verify delivery

After registering the webhook, change a ticket status in FreeScout (e.g., close a test ticket). Then query the cache:

SELECT conversation_id, mailbox_id, customer_email, subject, status,
       last_reply_at, thread_count, raxx_user_id, synced_at
FROM freescout_crm_ticket_cache
ORDER BY synced_at DESC
LIMIT 10;

The row should appear within ~5 seconds of the conversation event. If it doesn't:

  1. Check Heroku logs: heroku logs --tail --app raxx-api-staging | grep freescout_audit
  2. Verify FreeScout webhook delivery log (Manage → Apps → Webhooks → delivery history). - HTTP 401 = HMAC secret mismatch (re-check Infisical + FreeScout UI). - HTTP 503 = flag is still off (set FLAG_FREESCOUT_WEBHOOK_RECEIVE=1). - Connection refused / timeout = CF/BFM blocking (see CF/BFM caveat above).
  3. Confirm FREESCOUT_AUDIT_WEBHOOK_SECRET is correctly set in Infisical and propagated to the dyno.

HMAC secret rotation

2026-07-21 correction: Step 4 below ("update the FreeScout webhook secret in admin UI") is not possible on this FreeScout instance. FreeScout's ApiWebhooks module (v1.0.101) has no configurable per-webhook or global secret — Modules/ApiWebhooks/Resources/views/settings.blade.php only displays the derived key (\Webhook::getSecretKey() = md5(APP_KEY . 'webhook_key')) as a read-only label; there is no input field, DB column, or artisan command to set it to an arbitrary value. Rotating FREESCOUT_AUDIT_WEBHOOK_SECRET in Infisical/Heroku only ever changes the Raptor side. There is no FreeScout-side action for this rotation — see docs/ops/runbooks/freescout.md §"Failure mode K" for the full finding (confirmed via live webhook_logs evidence 2026-07-21: real FreeScout deliveries 401 regardless of the vault secret value, because FreeScout signs with HMAC-SHA1+base64 using its own derived key, while the Raptor receiver verifies HMAC-SHA256+hex against the vault secret — these can never match). Until the receiver code is fixed to implement FreeScout's actual signing scheme, this rotation procedure only serves to keep the Raptor-side secret current for synthetic/test signed requests; it does not affect real-webhook verification success either way.

Original procedure (still valid for the Raptor/vault side only):

  1. Generate a new secret (Step 1 above).
  2. Update Infisical at /MooseQuest/freescout/ FREESCOUT_AUDIT_WEBHOOK_SECRET.
  3. Restart dynos to pick up the new vault value: heroku ps:restart --app raxx-api-staging >/dev/null 2>&1
  4. ~~Immediately update the FreeScout webhook secret in admin UI (same session).~~ Not applicable — no such control exists on FreeScout's side (see correction above).
  5. Verify delivery (Step 4) before considering the rotation complete. Note: a synthetic signed request (built with the Raptor code's own HMAC-SHA256/hex scheme) will verify; a real FreeScout-originated delivery will not, per Failure mode K.

If webhook deliveries fail during rotation (brief window): FreeScout will retry. The freescout_crm_ticket_cache may be briefly stale; this is acceptable during planned rotation. Document the rotation window in the ops channel.

Retention

The freescout_crm_ticket_cache table has a 2-year retention cap enforced by:

  1. Read-side filtering at the support-history endpoint layer (#4260) — queries exclude rows where last_reply_at < NOW() - INTERVAL '2 years'.
  2. Nightly cleanup sweep keyed on the ftc_retention index ((last_reply_at NULLS LAST)) — removes stale rows. The sweep job is filed as a follow-on card under #4264. Enable the sweep after it lands.

Ingestion accepts all event ages — no ingestion-time age drop.

Polling fallback (post-launch, if needed)

If webhook delivery proves unreliable (e.g., during FreeScout restarts or network partitions), a polling fallback can be activated separately. The polling interval should be ≤5 minutes to meet the 5-second-at-best SLA in steady state. See SC-A4 issue #1484 Phase 2+ for implementation.

Webhook delivery gaps

FreeScout may miss delivering a webhook during a restart or network partition. If you suspect a gap:

  1. Query the cache for tickets known to have changed status: sql SELECT conversation_id, status, synced_at FROM freescout_crm_ticket_cache WHERE synced_at < '<gap_start_utc>';
  2. For any stale rows, manually re-trigger by updating the ticket status in FreeScout admin UI (a no-op status change triggers a fresh webhook).
  3. The polling fallback (if activated) handles this automatically.

Endpoint path coordination with RV-4

Both this card (SC-A4) and the RBAC auto-revocation card (RV-4) receive FreeScout events. They use separate endpoint paths to avoid routing collisions:

Purpose Path
Audit cache /api/internal/freescout-webhook/audit
RBAC revoke /api/internal/freescout-webhook/rbac

Both paths share the same HMAC validation helper (api.routes.freescout_audit_webhook.verify_freescout_hmac). If RV-4 uses a separate HMAC secret, update its registration separately.