Stripe Keys Verification + Vault Copy — 2026-05-12
Incident ID: 2026-05-12-stripe-keys-verified Date: 2026-05-12 UTC Severity: SEV-4 (infrastructure provisioning — no outage, no user impact) Executed by: sre-agent Authorized by: Operator (2026-05-12 ~01:15 UTC; webhook secret confirmed ~01:25 UTC)
Summary
Operator added Stripe test-mode keys to Infisical at /MooseQuest/stripe/ (dev environment) at approximately 01:15 UTC on 2026-05-12, and created a webhook signing secret at approximately 01:25 UTC. SRE confirmed keys present, validated formats, copied the primary restricted key to the architect's planned path at /Raxx/Queue/Billing/Stripe/STRIPE_RESTRICTED_KEY (prod environment), and documented outstanding items. Card #1681 (Queue Stripe service layer) is now unblocked for the key it needs. The webhook secret exists at the operator staging path and must be promoted to the service path before #1682 (webhook handler) can use it.
Task 1 — Keys Found at /MooseQuest/stripe/ (dev environment)
Vault lookup performed 2026-05-12 ~01:30 UTC via Infisical REST API (/api/v3/secrets/raw) with CF-Access-authenticated universal auth token.
| Secret Name | Value Length | Stripe Key Format | Notes |
|---|---|---|---|
STRIPE_RAXX_DEV_BOT_KEY |
107 chars | rk_test_... (restricted test key) |
Operator named "full-access dev bot key"; actual Stripe format is restricted (rk_), not full-access (sk_). See Task 2. |
STRIPE_SECRET_KEY |
107 chars | sk_test_... (secret test key) |
Standard Stripe secret key (full access, test mode) |
STRIPE_PUBLISHABLE_KEY |
107 chars | pk_test_... (publishable test key) |
Frontend/Stripe Elements key; not a server secret |
All three keys are in the dev environment under /MooseQuest/stripe/. The prod and staging environments at that path are empty — this is expected for vendor credentials in the MooseQuest namespace (operator-side credentials live in dev; the Raxx service paths carry the promoted copies).
Task 2 — Key Selection for Queue's Stripe API Calls
Finding: STRIPE_RAXX_DEV_BOT_KEY is an rk_test_... key — a Stripe restricted key in test mode, not a full-access secret key as the operator's description suggested. This is architecturally better than a full-access key: it already has scoped permissions. The exact permission set on this restricted key is not inspectable via API; operator should verify in the Stripe dashboard that it includes: Customers (write), Subscriptions (write), Invoices (write), Charges (read), Webhooks (read) — the permissions called for in ADR-0076.
Recommendation — v1 (test mode): Use STRIPE_RAXX_DEV_BOT_KEY (now copied to STRIPE_RESTRICTED_KEY). It is already a restricted key (rk_test_...), which matches the architect's intent exactly. No permission gap vs. using a full-access key; if anything, this is better than expected.
Recommendation — live mode: When the operator is ready to go live:
1. Create a new restricted key in the Stripe dashboard (live mode toggle active) with the same permission set.
2. Write it to /Raxx/Queue/Billing/Stripe/STRIPE_RESTRICTED_KEY (prod), overwriting the test key.
3. Restart the raxx-queue-prod dyno so Queue picks up the new value at startup.
The operator also stored STRIPE_SECRET_KEY (sk_test_...) at /MooseQuest/stripe/. This is the full-access key and should NOT be promoted to the Queue service path — Queue should always use the restricted key. The sk_test_ key is available as a break-glass for dashboard operations but should not be in service config.
Task 3 — Copy Operation
Source: /MooseQuest/stripe/STRIPE_RAXX_DEV_BOT_KEY (dev environment)
Destination: /Raxx/Queue/Billing/Stripe/STRIPE_RESTRICTED_KEY (prod environment)
Status: COMPLETED 2026-05-12 ~01:35 UTC
Verification after write:
- Secret name: STRIPE_RESTRICTED_KEY
- Path: /Raxx/Queue/Billing/Stripe/
- Environment: prod
- Value length: 107 chars
- Format: rk_test_... — confirmed Stripe restricted test key format
Name mismatch resolution
The operator named the key STRIPE_RAXX_DEV_BOT_KEY. ADR-0076 calls for STRIPE_RESTRICTED_KEY. These are two names for the same credential used in different contexts:
STRIPE_RAXX_DEV_BOT_KEY— operator's descriptive name at the vendor-credential staging path (/MooseQuest/stripe/). Accurate: it is the dev-bot's Stripe key.STRIPE_RESTRICTED_KEY— architect's functional name at the service runtime path (/Raxx/Queue/Billing/Stripe/). Accurate: it is a restricted key used by the Queue service.
Resolution chosen: keep both names, both paths. The /MooseQuest/stripe/ path is the operator's staging area for new credentials (human-readable names, organized by vendor). The /Raxx/Queue/Billing/Stripe/ path is the service's read path (functional names that match code). This two-path pattern is already established for Heroku, Postmark, and other vendors. No renaming needed. The copy operation is the correct routing step.
Publishable key — Antlers frontend
STRIPE_PUBLISHABLE_KEY (pk_test_...) is present at /MooseQuest/stripe/ but has NOT been copied to any service path. Per ADR-0076, Antlers frontend Stripe usage (Stripe Elements for signup forms) is post-Queue Phase 1. When that work is scheduled, the publishable key should be promoted to /Raxx/Antlers/Billing/STRIPE_PUBLISHABLE_KEY (or equivalent frontend config path) — it is not a server secret and can be embedded in frontend build config, but it should still flow through vault for auditability. This is a non-blocker today.
Task 4 — Webhook Secret Status
Updated 2026-05-12 UTC — operator confirmed creation ~01:25 UTC. SRE verified.
Verification result
SRE queried vault 2026-05-12 UTC via Infisical REST API with CF-Access authentication:
| Path | Environment | Secret Name | Result |
|---|---|---|---|
/MooseQuest/stripe/ |
dev | STRIPE_WEBHOOK_SECRET |
FOUND — length 38, whsec_ prefix confirmed |
/Raxx/Queue/Billing/Stripe/ |
prod | STRIPE_WEBHOOK_SECRET |
NOT FOUND — not yet promoted |
/Raxx/Queue/Billing/Stripe/ |
staging | STRIPE_WEBHOOK_SECRET |
NOT FOUND — not yet promoted |
/MooseQuest/stripe/ |
prod | STRIPE_WEBHOOK_SECRET |
NOT FOUND — expected |
Format check: Length 38 is consistent with Stripe webhook signing secrets (whsec_ prefix + 32 chars of base64). The whsec_ prefix is confirmed. The secret is valid in format.
Where it needs to go
The secret is stored at the operator's staging path. Before QP-C5 / #1682 (webhook handler) can use it:
- The webhook handler must be deployed to
raxx-queue-staging. - The operator registers the staging endpoint in the Stripe dashboard (Developers → Webhooks → Add endpoint).
- Stripe generates the signing secret for that endpoint registration.
- That secret (which may differ from the one currently at
/MooseQuest/stripe/) gets written to/Raxx/Queue/Billing/Stripe/STRIPE_WEBHOOK_SECRET(prod env).
The key at /MooseQuest/stripe/STRIPE_WEBHOOK_SECRET was likely created when the operator walked the Stripe webhook creation flow. If a specific endpoint URL was used during that flow, the secret is tied to that endpoint registration. If no endpoint URL was used (Stripe CLI local testing flow), the secret is a CLI-generated test secret. The operator should note which flow was used — this determines whether the secret can be reused for the staging endpoint or whether a new one must be generated.
This is a non-blocker for #1681 — QP-C4 (service layer) uses only STRIPE_RESTRICTED_KEY for outbound calls. The webhook secret is consumed only by the inbound handler (QP-C5 / #1682).
What #1681 needs vs. what it does NOT need
Card #1681 (Stripe service layer — outbound API calls) uses only STRIPE_RESTRICTED_KEY to make outbound calls to Stripe's API (create customer, create subscription, get subscription status). It does NOT need STRIPE_WEBHOOK_SECRET — that is consumed only by the inbound webhook handler (QP-C5 / #1682). STRIPE_RESTRICTED_KEY is now present at the service path. #1681 is unblocked.
Task 5 — ADR-0076 Path Reconciliation
ADR-0076 (docs/architecture/adr/0076-queue-phase1-cpp-billing-v1.md) calls for:
- STRIPE_RESTRICTED_KEY at /Raxx/Queue/Billing/Stripe/ — present (prod env, 107 chars, rk_test_...)
- STRIPE_WEBHOOK_SECRET at /Raxx/Queue/Billing/Stripe/ — at staging path only (promotion pending QP-C5 deploy)
The ADR describes STRIPE_RESTRICTED_KEY as needing least-privilege permissions: Customers (write), Subscriptions (write), Invoices (write), Charges (read), Webhooks (read). The key now at that path is an rk_test_... Stripe restricted key — it is structurally a restricted key, which matches the ADR's intent. Operator should verify the specific permission set in the Stripe dashboard matches the list above.
A short addendum has been appended to ADR-0076 clarifying the v1 key type, the live-mode swap procedure, and the webhook secret staging status.
What Is Now Unblocked
| Card | Title | Was Blocked On | Status After This Action |
|---|---|---|---|
| #1681 (QP-C4) | Stripe service layer in C++ (libcurl + nlohmann/json) | STRIPE_RESTRICTED_KEY missing from vault |
UNBLOCKED — key is at /Raxx/Queue/Billing/Stripe/STRIPE_RESTRICTED_KEY (prod) |
| #1682 (QP-C5) | Stripe webhook handler (HMAC verify + idempotent upsert) | Depends on #1681 + schema (QP-C3) + webhook secret at service path | Still blocked on #1681 + QP-C3 completion + secret promotion |
The immediate next claimable card in the Queue billing chain is #1681 — assuming QP-C1 (#1678, Queue C++ scaffold) and QP-C2 (#1679, GH Actions deploy pipeline) are verified deployed.
Operator Actions Remaining
| Action | When | Path |
|---|---|---|
Verify STRIPE_RAXX_DEV_BOT_KEY permission set in Stripe dashboard matches ADR-0076 list |
Before #1681 claims this card | Stripe dashboard → Developers → API keys → find restricted key |
| ~~Clarify which Stripe flow generated the webhook secret (dashboard endpoint URL vs. Stripe CLI)~~ | DONE 2026-05-12 ~01:33 UTC — Operator confirmed Option 1 (Stripe dashboard with actual endpoint URL); secret is tied to a real registered endpoint | — |
| ~~Promote webhook secret to service path~~ | DONE 2026-05-12 UTC — SRE promoted STRIPE_WEBHOOK_SECRET to /Raxx/Queue/Billing/Stripe/ (prod env) |
See Task 5 below |
| Create live-mode restricted key + promote | At live-mode cutover | Stripe dashboard → live mode; then update /Raxx/Queue/Billing/Stripe/STRIPE_RESTRICTED_KEY + restart Queue dyno |
Promote STRIPE_PUBLISHABLE_KEY for Antlers |
When Antlers Stripe Elements work is scheduled (post-Queue Phase 1) | /Raxx/Antlers/Billing/STRIPE_PUBLISHABLE_KEY or equivalent |
| Configure Postmark inbound webhook URL in Postmark dashboard | After Terraform apply completes (API Gateway URL pending) | Postmark → Inbound → Webhook URL |
Task 5 — Follow-up: Webhook Secret Promotion + Postmark Inbound (2026-05-12 UTC)
Executed by: sre-agent Authorized by: Operator confirmation 2026-05-12 ~01:33 UTC — "I used Option 1." (Stripe dashboard with actual endpoint URL)
5a — STRIPE_WEBHOOK_SECRET promoted to service path
Operator confirmed the webhook secret was created in the Stripe dashboard against a real registered endpoint (Option 1), meaning the secret is endpoint-bound and safe to use in production config.
SRE read source and wrote to destination via Infisical REST API with CF-Access authentication.
| Field | Value |
|---|---|
| Source path | /MooseQuest/stripe/ (dev env) |
| Destination path | /Raxx/Queue/Billing/Stripe/ (prod env) |
| Secret name | STRIPE_WEBHOOK_SECRET |
| Source length | 38 chars |
| Source prefix | whsec_ — confirmed |
| Destination length after write | 38 chars — matches source |
| Destination prefix after write | whsec_ — confirmed |
| Source unchanged | Yes — still 38 chars at /MooseQuest/stripe/ (dev env) |
Two-path pattern preserved: operator staging copy at /MooseQuest/stripe/ (dev env) is intentionally kept. No deletion.
5b — Postmark inbound address stored in vault
Operator generated a Postmark Inbound stream. The unique inbound address is a Postmark-public URL-shaped identifier (not a secret credential).
Stored at: /Raxx/Email/POSTMARK_INBOUND_EMAIL_ADDRESS (prod env)
| Field | Value |
|---|---|
| Secret name | POSTMARK_INBOUND_EMAIL_ADDRESS |
| Path | /Raxx/Email/ (prod env) |
| Value | bb43f79c8bfe23d564079fb431583c30@inbound.postmarkapp.com |
| Length | 56 chars |
| Verification | Read-back confirmed exact match |
| Purpose | Single source of truth for downstream automation (Google forwarding setup, Terraform module references) |
5c — What is now unblocked
| Card | Title | Was Blocked On | Status |
|---|---|---|---|
| #1682 (QP-C5) | Stripe webhook handler (HMAC verify + idempotent upsert) | STRIPE_WEBHOOK_SECRET not at service path |
Secret data-ready — still blocked on #1681 (service layer) + #1680 schema upstream |
| Postmark inbound webhook configuration | Configure Postmark dashboard to POST to API Gateway URL | POSTMARK_INBOUND_EMAIL_ADDRESS stored, API Gateway URL pending Terraform apply |
Unblocked once Terraform apply completes |
5d — What remains
- #1682 (QP-C5) is data-ready but still blocked on #1681 (QP-C4 service layer) and #1680 (schema). No action needed on vault until those cards complete.
- Postmark inbound webhook URL in the Postmark dashboard requires the API Gateway URL from the in-flight Terraform apply. Once that completes, configure the webhook endpoint in the Postmark dashboard to point to the generated URL.
References
- ADR-0076:
docs/architecture/adr/0076-queue-phase1-cpp-billing-v1.md - Runbook:
docs/ops/runbooks/rotation/stripe-restricted-key.md - Feedback:
feedback_vault_folder_must_exist.md,feedback_no_inline_secrets_in_repo.md