Signup E2E Smoke Runbook

The signup smoke harness exercises the full Option C bootstrap-link flow against live prod without a real iPhone or Face ID. It exists to catch one-shot bugs (parser compat, boolean cast, challenge round-trip regressions) that are only visible against the real production stack before handing the URL to the operator.

When to run

Run the smoke before any admin merge of a PR that touches:

• backend_v2/api/routes/auth.py
• backend_v2/api/services/webauthn_service.py
• backend_v2/api/services/bootstrap_token_service.py
• backend_v2/db/ (any migration that touches users, webauthn_credentials, or bootstrap_tokens_*)
• Any Heroku config change that could affect BOOTSTRAP_TOKEN_SIGNING_KEY, WEBAUTHN_RP_ID, WEBAUTHN_ORIGIN, or REDIS_URL

The smoke is also the final gate before tapping the signup URL on a real device. If the smoke passes, the chain is confirmed good at the data layer. If it fails, there is a backend regression to fix before any device testing.

What it does (and does not) test

The harness confirms:

1. Token can be minted via heroku run on prod
2. /api/auth/register/begin-with-token returns a valid WebAuthn challenge
3. A synthetic EC P-256 credential (CBOR attestation format none) can be constructed and accepted by py-webauthn's verify_registration_response
4. /api/auth/register/verify-with-token returns {status: ok, user_id: N}
5. The users row and webauthn_credentials row are present in the prod DB after verification (confirms DB write + commit, not just API 200)

It does not test:

• Real Face ID biometrics (by design — this is the point of the harness)
• Antlers frontend rendering
• iOS native flow
• Login (separate path; not exercised here)
• Backup-code generation (it queries backup code count but doesn't assert on it)

Prerequisites

• Heroku CLI installed and authenticated (heroku whoami returns your user)
• Python 3.11+ with cbor2, cryptography, pytest, and webauthn installed
• Access to raxx-api-prod Heroku app (operator account)
• A terminal with internet access to api.raxx.app

Install Python deps if missing:

pip install cbor2 cryptography pytest webauthn

Running the smoke

via Makefile (recommended)

RAXX_SMOKE_ALLOW_PROD=1 make smoke-signup-prod

via bin script

RAXX_SMOKE_ALLOW_PROD=1 ./bin/smoke-signup-prod.sh

via pytest directly

RAXX_SMOKE_ALLOW_PROD=1 python -m pytest backend_v2/tests/smoke/test_signup_e2e_prod.py -s -v

Environment variables

Prod-write safety

The harness writes exactly one users row and one webauthn_credentials row. Both are deleted in the teardown step (step 6) regardless of pass or fail.

The teardown is fenced with an explicit email check:

if email == "kris@moosequest.net": skip cleanup

This prevents accidental deletion of the operator's real account even if RAXX_SMOKE_EMAIL is set incorrectly. The test also uses a timestamped default email (smoke-test+<ts>@moosequest.net) to guarantee a fresh nonce on every run.

If a run fails before teardown completes (e.g. heroku run timeout), orphaned rows may remain under the smoke email. These are harmless but can be cleaned manually:

heroku run --app raxx-api-prod -- python -c "
import sys; sys.path.insert(0, '/app'); import os; os.chdir('/app')
from backend_v2.db.engine import get_engine
from sqlalchemy import text
email = 'smoke-test+...@moosequest.net'  # replace with the actual email
with get_engine().connect() as conn:
    row = conn.execute(text('SELECT id FROM users WHERE email = :em'), {'em': email}).mappings().fetchone()
    if row:
        conn.execute(text('DELETE FROM webauthn_credentials WHERE user_id = :uid'), {'uid': row['id']})
        conn.execute(text('DELETE FROM users WHERE id = :uid'), {'uid': row['id']})
        conn.commit()
        print('cleaned up')
    else:
        print('no rows found')
"

Reading the output

A passing run looks like:

============================================================
RAXX SIGNUP E2E SMOKE — 2026-05-25T14:32:10Z UTC
  API:        https://api.raxx.app
  Heroku app: raxx-api-prod
  Test email: smoke-test+1748181130@moosequest.net
============================================================

[step 1] minting bootstrap token for 'smoke-test+1748181130@moosequest.net' via raxx-api-prod...
[step 1] OK  token=eyJ2IjoxLCJlbWFp...abc12345

[step 2] POST /api/auth/register/begin-with-token ...
[step 2] OK  challenge=rsBS7cZK4GWOm8-V... user.id=dXNlcl8x...

[step 3] building synthetic credential (EC P-256, fmt=none)...
[step 3] OK  cred_id=Xk1wa-j68frNNJYB... clientData challenge matches options challenge

[step 4] POST /api/auth/register/verify-with-token ...
[step 4] OK  user_id=42 status=ok

[step 5] confirming DB persistence via heroku run (raxx-api-prod)...
[step 5] OK  user.id=42 email=smoke-test+1748181130@moosequest.net cred_count=1

[teardown] deleting smoke rows for user_id=42 email='smoke-test+1748181130@moosequest.net'...
[teardown] OK  deleted 1 credential row(s), 1 user row(s)

============================================================
SMOKE VERDICT: PASS
  user_id:              42
  credential_id prefix: Xk1wa-j68frNNJYB...
  DB cred_count:        1
  recovery codes:       0
============================================================

A failing run will print the full HTTP response body and a suggested next step for each failure mode.

Interpreting failures