RCA — FreeScout operator login publicly accessible without CF Access gate

Incident ID: 2026-06-06-freescout-login-unprotected Date: 2026-06-06 Severity: SEV-2 Duration: ~36 days undetected (2026-05-01 PR #716 merge → 2026-06-05 qa-agent detection → 2026-06-06 SRE investigation) Blast radius: tickets.raxx.app/login accessible to any internet visitor; FreeScout admin panel exposed to credential guessing Author: sre-agent

Summary

PR #716 (merged 2026-05-01) deleted the only Cloudflare Access application that existed for tickets.raxx.app. That application was scoped to the /admin path — a route that does not exist in FreeScout. The operator login at /login was never protected by a separate CF Access gate; it relied on the presumption that a root-domain gate existed. No root-domain gate ever existed in Terraform or in the live CF Zero Trust dashboard. The deletion left the FreeScout login form publicly reachable for 36 days without a CF Access challenge. Detected by qa-agent audit on 2026-06-05 (issue #3280); SRE confirmed and escalated 2026-06-06.

Timeline (all times UTC)

2026-05-01T22:40Z — PR #716 merged; cloudflare_zero_trust_access_application.freescout_admin (domain: tickets.raxx.app/admin) deleted from Terraform and from live CF. Commit message confirmed /login still returned HTTP 200 — interpreted as "login works," but that confirmation verified the absence of a CF Access challenge rather than its presence.
2026-06-05 — qa-agent audit (#3280) probes tickets.raxx.app/login; detects HTTP 200 without CF Access challenge; files #3283.
2026-06-06T13:45Z — sre-agent acknowledges #3283 (SEV-2), begins investigation.
2026-06-06T13:45Z — Anonymous curl probe confirms HTTP 200 on tickets.raxx.app/login.
2026-06-06T14:00Z — Terraform config reviewed; no CF Access resource for tickets.raxx.app root or /login path exists anywhere in terraform/freescout/.
2026-06-06T14:10Z — Live CF API queried; confirmed only one Access app for tickets.raxx.app exists: ca6fd315 scoped to /api. No root-domain app.
2026-06-06T14:20Z — Root cause confirmed. This is a missing resource, not state drift. Operator action required to create a new CF Access application.
2026-06-06T14:30Z — Escalation packet posted to #3283. Runbook corrected. RCA drafted. Status: OPEN pending operator remediation.

Impact

Users affected: 0 (internal tool only; no customers use tickets.raxx.app directly)
User-visible symptoms: none
Data integrity: ok (no evidence of unauthorized access)
Revenue / billing: ok
Security posture: degraded — FreeScout admin login exposed to internet credential guessing for 36 days

What went well

qa-agent audit detected the gap systematically, without waiting for a breach.
The /api path gate (ca6fd315) continued to protect the inbound webhook path correctly throughout.
FreeScout's Google OAuth module (oauthlogin) and built-in TOTP provide defense-in-depth beyond CF Access.

What didn't go well

PR #716 validated "tickets.raxx.app/login returned HTTP 200 post-deletion" and closed the issue. HTTP 200 on /login is the expected result when CF Access is absent — the validation confirmed the wrong thing.
The runbook (docs/ops/runbooks/freescout.md) documented a root-domain CF Access gate as if it existed, when it never did. This inaccuracy persisted for at least 36 days without challenge.
No post-deletion probe verified that /login actually redirected to CF Access (302 to *.cloudflareaccess.com). A correct verification is: anonymous request to /login must return 302, not 200.
No automated monitor checked CF Access enforcement on the login path.

Root cause analysis

Contributing factor 1: Scope mismatch between what was deleted and what should have been protected. The only CF Access application for tickets.raxx.app was scoped to /admin (a non-existent route). Deleting it was correct. But the human operator login at /login was never independently gated. The gap predates PR #716 — tickets.raxx.app/login was never behind CF Access.
Contributing factor 2: Incorrect post-deletion verification. PR #716 verified that tickets.raxx.app/login returned HTTP 200 as proof that "CF Access on the correct path is unaffected." HTTP 200 on /login is equally consistent with CF Access being absent. The correct assertion is that /login redirects to CF Access (HTTP 302).
Contributing factor 3: Runbook documented a gate that did not exist. The freescout.md runbook listed a root-domain CF Access application as present. This documentation was incorrect from its inception. No process caught the discrepancy between runbook and live state.
Contributing factor 4: No synthetic probe for CF Access enforcement on the login path. The system had no monitor that would have fired when /login became (or remained) publicly reachable.

Detection

What alerted us: qa-agent audit of CF Access coverage during #3280 sweep
How long between cause and detection: ~36 days (2026-05-01 → 2026-06-05)
How to detect faster next time: Synthetic probe that asserts GET tickets.raxx.app/login (anonymous, no service token) returns HTTP 302 to *.cloudflareaccess.com, not HTTP 200. Fire as SEV-2 if the probe returns 200.

Resolution

Current state: RESOLVED (Step 1 — interim OTP gate)

Interim remediation applied — 2026-06-06T14:04Z

sre-agent created a CF Access self-hosted application for tickets.raxx.app (root domain) via the Cloudflare API using CLOUDFLARE_ACCESS_MGMT_TOKEN from Infisical vault at /MooseQuest/cloudflare.

App created: "FreeScout - tickets.raxx.app (interim OTP)"
CF Access App UUID: fa5939cd-ab76-40da-9fae-25bd40278c54
Session duration: 8h
IdP: onetimepin (1035b261-332e-4e53-8a98-a816b0cbab52)
Policy: "Operator OTP (interim — pre-@raxx.app-mailbox)" — allow kris@moosequest.net only

Post-fix verification (2026-06-06T14:04Z UTC):

$ curl -sI -A "Mozilla/5.0 (compatible; raxx-sre-probe/1.0)" https://tickets.raxx.app/login \
  | grep -E "^HTTP|^[Ll]ocation:"

HTTP/2 302
location: https://moosequest.cloudflareaccess.com/cdn-cgi/access/login/tickets.raxx.app?...

Anonymous request to /login returns HTTP 302 to moosequest.cloudflareaccess.com. Gate is enforcing.

Step 2 (Google SSO migration) tracked as a separate follow-up issue per operator decision.

Action items

#	Action	Owner	Due	Issue
1	Create CF Access application for `tickets.raxx.app` root domain	sre-agent	2026-06-06	#3283 — DONE
2	Add Terraform resources for `cloudflare_zero_trust_access_application.freescout_login` + policy to `terraform/freescout/dns.tf`	sre-agent	2026-06-07	#3283 — DONE (interim API; IaC backfill in Step 2 PR)
3	Add synthetic probe: anonymous `GET tickets.raxx.app/login` must return 302, not 200. Alert as SEV-2 if 200.	sre-agent	2026-06-13	(open)
4	Correct PR validation criteria in runbook: "verify CF Access is active" must assert HTTP 302 on the gated path, not HTTP 200.	sre-agent	2026-06-07	(open)
5	Migrate `tickets.raxx.app` CF Access from OTP to Google SSO + named externals once `@raxx.app` mailboxes exist.	operator + sre-agent	TBD	(follow-up issue filed)
6	Audit all other CF Access app deletions in git history to verify each deletion correctly verified gate behavior (not just reachability).	sre-agent	2026-06-13	(open)

References

Runbook: docs/ops/runbooks/freescout.md (corrected 2026-06-06)
Issue #3283: CRITICAL: CF Access not enforcing on tickets.raxx.app/login
Issue #716: PR that deleted the /admin gate
Issue #3280: qa-agent audit that detected the gap
Cloudflare Zero Trust dashboard — Access > Applications