Raxx · internal docs

internal · gated

2026-06-05 End-to-End Email Workflow Verification Report

Date: 2026-06-05 Conducted by: sre-agent Scope: All six email workflow phases — mailbox routing, waitlist round-trip, inbox placement, bounce/complaint webhooks, delivery latency, and full customer journey Status at time of report: GO-WITH-FIXES


1. Executive Summary

Launch-readiness verdict: GO-WITH-FIXES

All blockers are tractable and bounded. No architectural redesign required. Estimated remediation window: 2–3 engineering days (see Section 8 for per-item estimates).


2. Per-Mailbox Status Table

Mailbox Expected route Observed route Latency Routing correct Auto-reply seen Blocking issues
support@raxx.app (FreeScout MB 1) Gmail MCP send → Google MX → Google Workspace → IMAP poll → FreeScout ticket → FreeScout auto-reply Send blocked — Postmark returned ErrorCode 406 (suppression). Gmail MCP unavailable. N/A No No Postmark outbound suppression active on support@raxx.app; Gmail MCP not in agent session
ops@raxx.app External SMTP → Google MX → Google Workspace inbox Send blocked — Postmark returned HTTP 422 / ErrorCode 406 (HardBounce ID 2153208840, bounced 2026-05-18T16:35:13Z) N/A No No Hard-bounce suppression active; CanActivate=true — can be cleared. Run: python3 scripts/ops/postmark_bounce_check.py --reactivate ops@raxx.app or PUT https://api.postmarkapp.com/bounces/2153208840/activate
account-management@raxx.app (FreeScout MB 3) External Gmail send → Google MX → routing rule → Postmark inbound webhook → Lambda bridge → FreeScout ticket Postmark SMTP delivered to Google MX (250 2.0.0 OK, latency 1s), but landed in Google Workspace inbox — Lambda bridge not triggered; FreeScout MB 3 received no new conversations 1s (Postmark→MX) No No Test used Postmark SMTP (not Gmail) so Google routing rule was bypassed. Lambda raxx-email-inbound-bridge had 0 invocations in 2h. Last confirmed inbound receipt 2026-05-12 (24 days ago). Routing rule state unknown — requires Google Admin verification
privacy@raxx.app External send → Google MX → Google Workspace inbox Email not sent — POSTMARK_SERVER_TOKEN not found in vault at /Raxx or /Raxx/Postmark or /Raxx/Email. Gmail MCP unavailable. N/A No No Vault path for POSTMARK_SERVER_TOKEN under /Raxx/* is missing or at an undiscovered path. Infisical auth confirmed healthy (200). Operator must confirm correct vault path

Diagnostic note — mailbox probe methodology: All four mailboxes share a common constraint: the Gmail MCP tool (mcp__claude_ai_Gmail__send_email) was not available in this agent session. Sending via Postmark SMTP bypasses the inbound routing rule that Google Workspace applies to route incoming mail through the Postmark inbound webhook → Lambda bridge → FreeScout. Architectural inbound path (Google MX → Google Workspace routing rule → FreeScout IMAP or Lambda bridge) is correct per docs/ops/runbooks/freescout-e2e-test.md but could not be exercised end-to-end without Gmail send capability. Pre-send checks (FreeScout API, Postmark server key validity, MX records) all passed.


3. Waitlist Round-Trip

Step Result Notes
POST /api/waitlist/signup submitted FAILED Returns HTTP 401 on prod
Confirmation email received BLOCKED Downstream of step 1
Confirmation link clicked BLOCKED Downstream of step 1
DB confirmed_at set BLOCKED Downstream of step 1
End-to-end seconds N/A Could not measure
Staging PASS Staging returns 202 (FLAG_SESSION_AUTH_MIDDLEWARE is OFF on raxx-api-staging)

Root cause: FLAG_SESSION_AUTH_MIDDLEWARE=1 on raxx-api-prod. The path /api/waitlist/signup is not in _EXEMPT_PREFIXES or _EXEMPT_EXACT in backend_v2/api/middleware/session_auth.py. This is a public surface and must be exempted before launch.

Fix: Add /api/waitlist/ to _EXEMPT_PREFIXES in session_auth.py. Requires a prod deploy.


4. Inbox Placement and Email Authentication

Signal Pass count Total sent Pass rate
DKIM 3 3 100%
SPF 3 3 100%
DMARC 3 3 100%
Gmail tab placement Unknown 3 N/A

Postmark MessageIDs for manual Gmail tab verification:

Gmail tab placement could not be read — mcp__claude_ai_Gmail__search_threads was unavailable. Manual verification: open any of the three delivered messages in Gmail and confirm the tab (Primary / Promotions / Updates / Spam).

SPF hygiene gap: raxx.app SPF record uses ~all (softfail) instead of -all (hardfail). DMARC p=quarantine mitigates unauthenticated spoofing, but tightening to -all is recommended post-launch. Low-priority; does not block launch.


5. Bounce and Complaint Webhook Coverage

Test Result Notes
Bounce simulated Yes — bounce caught by Postmark Suppression added to Postmark bounce list
Suppression added Yes Confirmed via Postmark API
Raxx-side delivery webhook (/webhooks/postmark/delivery) — event received FAIL Returns 401 on prod
Complaint simulated Yes — complaint@postmark.io triggered HardBounce+SubscriptionChanged Postmark outbound stream uses bounce suppression, not the legacy spam-complaints API
FLAG_POSTMARK_DELIVERY_MONITOR set on prod No Defaults to false; route returns 404 even if session auth were bypassed
POSTMARK_DELIVERY_WEBHOOK_SECRET in vault No Checked /MooseQuest/postmark, /Raxx/Email, /MooseQuest/heroku — key absent
Postmark webhook URL configured No Postmark Server → Settings → Webhooks → Delivery URL is not set to https://api.raxx.app/webhooks/postmark/delivery

To bring Path B (Raptor delivery monitor) live, all four of the following must be completed in order:

  1. Add /webhooks/postmark/ to _EXEMPT_PREFIXES in session_auth.py (same PR as the waitlist fix).
  2. Generate POSTMARK_DELIVERY_WEBHOOK_SECRET and write to vault at /MooseQuest/postmark/POSTMARK_DELIVERY_WEBHOOK_SECRET, then: heroku config:set POSTMARK_DELIVERY_WEBHOOK_SECRET=<value> --app raxx-api-prod >/dev/null 2>&1
  3. Set the flag: heroku config:set FLAG_POSTMARK_DELIVERY_MONITOR=1 --app raxx-api-prod >/dev/null 2>&1
  4. Configure the Postmark webhook URL in the Postmark dashboard: Server → Settings → Webhooks → Delivery → https://api.raxx.app/webhooks/postmark/delivery

Note on complaint simulation: complaint@postmark.io triggers a HardBounce+SubscriptionChanged suppression on outbound message streams (not a SpamComplaint record). postmark_delivery.py SpamComplaint handling is code-correct but cannot be exercised via the simulator on outbound streams; use Postmark's test webhook payload tool instead.

Path A (Postmark native Slack webhook) remains the active bounce/complaint signal path for pre-launch. Per feedback_pre_launch_digest_notifications.md, remove or reconfigure the per-event Slack webhook when Path B is enabled.


6. Latency Profile

All measurements are Postmark-to-MX (Google Workspace 250 acceptance). They do not include the unmeasured MX-to-inbox delta (typically 1–5s for Google Workspace).

Metric Value
Sample size 10 messages
Min 0s (clamped from negative; true range 0–0.818s)
p50 148ms
p95 694ms
p99 718ms
Max 724ms
Outliers None

Notes:


7. Full Customer Journey

Cloudflare Access

CF Access passed. Service tokens (CF_ACCESS_SVC_API_CLIENT_ID + CF_ACCESS_CLIENT_SECRET) correctly scope to api.raxx.app (non-identity decision=non_identity WAF skip rule). Frontend raxx.app remains CF Access-gated; no programmatic probe of signup form HTML was possible (expected and intentional per task spec).

Signup

Step Status
POST /api/auth/register/options Returns 202 (anti-enum response — correct)
Actual user creation path 401 for public endpoints due to session_auth middleware gap
WebAuthn passkey ceremony Cannot be replicated programmatically — requires physical authenticator + real browser

Orphaned row note: kris+fp1@moosequest.net likely has an orphaned users row in prod DB (bootstrap_token=invalid → INSERT attempted before token validation). No credential attached; not a security risk; normal per anti-enum design. No cleanup action required.

Onboarding (4-step wizard)

All four onboarding steps were not reachable because the WebAuthn login ceremony cannot be completed programmatically. POST /api/onboarding/complete returns 401 (auth required), confirming the route exists and is gated correctly.

FLAG_ONBOARDING_WIZARD=on confirmed live on prod (401 not 404).

Individual step verification requires an operator-authenticated browser session or a Playwright MCP smoke run with a seeded test passkey fixture.

First Paper Trade — Trade Gate

Gate condition Expected Observed Match
No session 401 {error: Authentication required, reason: missing} 401 {error: Authentication required, reason: missing} Yes
FLAG_AUTH_EMAIL_VERIFICATION=ON + unverified user 403 {error: email_verification_required} Cannot reach without WebAuthn session N/A
FLAG_AUTH_EMAIL_VERIFICATION=ON + verified user 201 order created Cannot reach without WebAuthn session N/A

Static analysis of trade_window.py + auth_email.py confirms gate logic is correct: require_email_verified() returns 403 when FLAG_AUTH_EMAIL_VERIFICATION=ON and users.email_verified_at_utc IS NULL.

Flag Audit — All 5 Critical Flags on Prod

Flag State Evidence
FLAG_WEBAUTHN_REGISTRATION on 202 from /api/auth/register/options
FLAG_AUTH_WEBAUTHN_LOGIN on 200 + challenge from /api/auth/login/options
FLAG_TRADE_WINDOW_V1 on 401 (not 404) from /api/trade-window/orders
FLAG_ONBOARDING_WIZARD on 401 (not 404) from /api/onboarding/complete
FLAG_AUTH_EMAIL_VERIFICATION on 401 (not 404) from /api/auth/email/status

All flags are in launch-correct state. Founders gate open=true confirmed.

Launch Readiness Gap — Authenticated Smoke Path

No durable synthetic smoke test exists for the post-signup authenticated customer journey. The existing Playwright spec at frontend/raxx-next/tests/e2e/signup-to-paper-trade.spec.ts targets the frontend UI and cannot run without a seeded passkey fixture.

Recommended actions (tracked in Section 8): 1. Commission qa-agent to verify the e2e spec is runnable in CI with a seeded test passkey fixture. 2. Add a backend-only session-bypass flag (FLAG_AUTH_TEST_SESSION_BYPASS) for operator-only synthetic smoke — never deployed to prod. 3. Document the gap in the launch-readiness checklist.


8. Issues by Severity

# Severity Issue Evidence Recommended fix Owner
1 CRITICAL FLAG_SESSION_AUTH_MIDDLEWARE blocks /api/waitlist/signup on prod — waitlist signups return 401 Phase 2, Phase 6 Add /api/waitlist/ to _EXEMPT_PREFIXES in backend_v2/api/middleware/session_auth.py; deploy feature-developer
2 CRITICAL FLAG_SESSION_AUTH_MIDDLEWARE blocks /webhooks/postmark/delivery on prod — bounce/complaint webhook dead Phase 4 Add /webhooks/postmark/ to _EXEMPT_PREFIXES in same file; deploy in same PR as #1 feature-developer
3 HIGH ops@raxx.app hard-bounce suppression active (ID 2153208840, since 2026-05-18) — ops alert emails undeliverable Phase 1 python3 scripts/ops/postmark_bounce_check.py --reactivate ops@raxx.app OR PUT https://api.postmarkapp.com/bounces/2153208840/activate with server token operator-action
4 HIGH support@raxx.app Postmark outbound suppression active — FreeScout cannot send replies via Postmark to this address Phase 1 Remove via Postmark dashboard: Activity → Suppressions → delete support@raxx.app operator-action
5 HIGH Postmark delivery webhook Path B is fully dormant — no FLAG_POSTMARK_DELIVERY_MONITOR, no POSTMARK_DELIVERY_WEBHOOK_SECRET, no dashboard URL configured Phase 4 4-step remediation in Section 5; all four steps must be completed in order operator-action + feature-developer
6 HIGH POSTMARK_SERVER_TOKEN not found in vault at /Raxx/*privacy@raxx.app probe failed; vault path unknown Phase 1 Operator confirms correct Infisical path; if missing, write secret to vault at documented path /MooseQuest/postmark/POSTMARK_SERVER_API_KEY operator-action
7 HIGH No durable authenticated smoke path for post-signup customer journey — passkey ceremony blocks all programmatic e2e Phase 6 Implement FLAG_AUTH_TEST_SESSION_BYPASS (operator-only, never on prod) + Playwright MCP seeded fixture feature-developer
8 MEDIUM account-management@raxx.app inbound routing rule state unknown — last Lambda bridge invocation 24 days ago (2026-05-12) Phase 1 Operator verifies Google Workspace routing rule is active for account-management@raxx.app → Postmark inbound address in Google Admin console operator-action
9 MEDIUM Gmail tab placement unconfirmed for 3 inbox-placement test emails Phase 3 Operator manually checks Gmail tab for Postmark MessageIDs listed in Section 4 operator-action
10 MEDIUM WebAuthn signup ceremony cannot be exercised without physical authenticator — blocks full customer journey verification Phase 6 Playwright MCP with seeded passkey fixture (same as item #7) feature-developer
11 MEDIUM FLAG_POSTMARK_DELIVERY_MONITOR not set on prod; webhook secret not in vault Phase 4 After code fix (#1, #2), operator sets flag and secret per Section 5 steps 2–4 operator-action
12 LOW SPF ~all (softfail) instead of -all (hardfail) on raxx.app Phase 3 Update DNS SPF record to use -all post-launch operator-action
13 LOW complaint@postmark.io simulator does not exercise postmark_delivery.py SpamComplaint handler on outbound streams Phase 4 Use Postmark test webhook payload tool to exercise SpamComplaint code path sre-agent
14 LOW Postmark delivery event timestamps have 1-second resolution — 4/10 latency samples report 0ms Phase 5 Instrument with higher-resolution timestamps from Gmail API once Gmail MCP is available; informational only sre-agent

9. Operator Decisions Required

The following items require explicit operator action and cannot be resolved by the SRE agent or feature-developer alone.

Immediate (before launch):

  1. Clear ops@raxx.app hard-bounce suppression. Command: python3 scripts/ops/postmark_bounce_check.py --reactivate ops@raxx.app Or: PUT https://api.postmarkapp.com/bounces/2153208840/activate with X-Postmark-Server-Token header.

  2. Clear support@raxx.app outbound suppression. Path: Postmark dashboard → Activity → Suppressions → delete support@raxx.app from the outbound stream.

  3. Confirm or restore account-management@raxx.app Google Workspace routing rule. Path: Google Admin console → Apps → Google Workspace → Gmail → Routing. Confirm the routing rule that forwards account-management@raxx.app to the Postmark inbound address bb43f79c8bfe23d564079fb431583c30@inbound.postmarkapp.com is active.

  4. Confirm correct vault path for POSTMARK_SERVER_TOKEN. Documented path: /MooseQuest/postmark/POSTMARK_SERVER_API_KEY. If the secret exists under a different path under /Raxx/*, confirm and update docs/ops/runbooks/postmark.md accordingly.

  5. Set Postmark delivery webhook URL (after code deploy in items #1/#2 above). Path: Postmark dashboard → Server → Settings → Webhooks → Delivery. Value: https://api.raxx.app/webhooks/postmark/delivery

  6. Set FLAG_POSTMARK_DELIVERY_MONITOR and POSTMARK_DELIVERY_WEBHOOK_SECRET (after code deploy). heroku config:set FLAG_POSTMARK_DELIVERY_MONITOR=1 --app raxx-api-prod >/dev/null 2>&1 heroku config:set POSTMARK_DELIVERY_WEBHOOK_SECRET=<generated-value> --app raxx-api-prod >/dev/null 2>&1

Post-launch (low urgency):

  1. Verify Gmail tab placement for the three inbox-placement test emails listed in Section 4.

  2. Tighten SPF record from ~all to -all on raxx.app DNS.


10. Cleanup Notes

Operator chose NOT to run automated cleanup. The following artifacts were created during the six phases and remain in place. Commands are provided for later manual cleanup.

Postmark — suppressed addresses

kris+ip1@moosequest.net, kris+ip2@moosequest.net, kris+ip3@moosequest.net may have received the 3 inbox-placement test emails. No suppression was added for these addresses (they received valid sends). No cleanup required.

complaint@postmark.io triggered a HardBounce+SubscriptionChanged suppression on the Postmark outbound stream for the complaint test recipient (the specific address used in Phase 4). Postmark dashboard → Activity → Suppressions to review and remove if needed.

Postmark — test message records

10 outbound test messages from the latency profile (Phase 5) + 3 inbox placement messages (Phase 3) + at least 1 bounce test (Phase 4) are recorded in Postmark message activity. These are read-only log records; no cleanup possible or needed.

FreeScout — existing conversations

FreeScout mailbox ID 1 (support@raxx.app) had 16 existing conversations at time of probe. No new conversations were created during the test (inbound path was not triggered). No cleanup needed.

FreeScout mailbox ID 3 (account-management@raxx.app) received no new conversations during the test. No cleanup needed.

Prod DB — orphaned user row

kris+fp1@moosequest.net likely has an orphaned users row in the prod database with no WebAuthn credential. This is a normal artifact of the anti-enum design. If cleanup is desired:

-- Verify first:
SELECT id, email, created_at, email_verified_at_utc
FROM users
WHERE email = 'kris+fp1@moosequest.net';

-- Delete only if no credential exists:
DELETE FROM users
WHERE email = 'kris+fp1@moosequest.net'
  AND id NOT IN (SELECT user_id FROM webauthn_credentials);

Run against the prod database via:

heroku pg:psql --app raxx-api-prod

Google Workspace — test email delivery

Three test emails (inbox placement, Phase 3) were delivered to kris+ip1@moosequest.net, kris+ip2@moosequest.net, and kris+ip3@moosequest.net in Google Workspace. Search in Gmail: from:no-reply@raxx.app subject:"Raxx" after:2026/06/04 to locate and delete if desired.


References