Raxx · internal docs

internal · gated ↑ index

Rotation SOP — Alpaca Live Trading API Keys

Mode: operator-assisted Last validated: 2026-04-24 UTC Validation method: read-only-docs (DO NOT live-rotate without operator presence + paused trading) Average duration: 15m (includes pause + verify + resume cycle) Required role: superadmin

Applies to: ALPACA_LIVE_API_KEY_ID, ALPACA_LIVE_API_SECRET_KEY. Live keys move real money. Treat every rotation as a planned change with a maintenance window.

Critical: Live trading must be paused before rotation. Open orders may need to be canceled or completed. Algorithmic strategies that depend on these keys must be halted, then resumed with the new keys after validation. Do not rotate live keys without the operator on console.

When to run

Prerequisites

Steps

1. Pre-rotation checks

# Pause live trading via the existing kill-switch
heroku config:set TRADING_LIVE_DISABLED=1 -a raxx-api-prod
# Wait for dyno restart and confirm
sleep 30
curl -sS https://api.raxx.app/health/trading | jq '.live_enabled'
# Expect: false

Confirm no in-flight orders:

curl -sS -H "APCA-API-KEY-ID: $CURRENT_KEY_ID" \
     -H "APCA-API-SECRET-KEY: $CURRENT_SECRET" \
     https://api.alpaca.markets/v2/orders?status=open | jq '. | length'
# Expect: 0 (or each one will be canceled by the rotation)

Snapshot live account state (audit trail):

curl -sS -H "APCA-API-KEY-ID: $CURRENT_KEY_ID" \
     -H "APCA-API-SECRET-KEY: $CURRENT_SECRET" \
     https://api.alpaca.markets/v2/account > /tmp/alpaca-live-pre-rotate-$(date -u +%Y%m%dT%H%M%SZ).json

2. Generate the new credential

  1. Navigate to https://app.alpaca.markets/live/dashboard/overview.
  2. CONFIRM the upper-left environment switcher is on Live Trading, not Paper. Mistakes here have cost real customers real money industry-wide.
  3. In the right sidebar API Keys section, click "Regenerate" (or "Generate New Keys" if no Regenerate option).
  4. The dashboard will require email verification or 2FA confirmation for live key regeneration.
  5. Copy both Key ID and Secret Key immediately. Secret is shown once.

3. Validate the new credential

NEW_KEY_ID="..."
NEW_SECRET="..."
curl -sS -H "APCA-API-KEY-ID: $NEW_KEY_ID" \
     -H "APCA-API-SECRET-KEY: $NEW_SECRET" \
     https://api.alpaca.markets/v2/account | jq '{status, account_number, equity, cash, buying_power}'

Sanity-check: account_number and equity MUST match the pre-rotation snapshot exactly. If account_number differs, the operator clicked into the wrong environment; STOP, do not propagate.

4. Store in Infisical

infisical secrets set ALPACA_LIVE_API_KEY_ID="$NEW_KEY_ID" \
  --projectId="$INFISICAL_PROJECT_ID" --env=prod
infisical secrets set ALPACA_LIVE_API_SECRET_KEY="$NEW_SECRET" \
  --projectId="$INFISICAL_PROJECT_ID" --env=prod

5. Propagate to downstream consumers

Consumer How
Raptor live runtime heroku config:set ALPACA_LIVE_API_KEY_ID=... ALPACA_LIVE_API_SECRET_KEY=... -a raxx-api-prod
MQ-A algo layer per its own config-var path (Heroku app name varies)

Do NOT propagate live keys to staging, GitHub Actions, or local dev. Live keys belong only to production runtime.

6. Verify downstream

# Account check via Raptor
curl -sS https://api.raxx.app/api/trading/account?mode=live | jq '{status, account_number}'
# account_number must match snapshot from step 1

# Place a tiny check order at a price far from market (no fill expected) to validate auth
curl -sS -X POST https://api.raxx.app/api/trading/orders \
  -H "Content-Type: application/json" \
  -d '{"symbol":"AAPL","qty":1,"side":"buy","type":"limit","limit_price":1.00,"time_in_force":"day","mode":"live"}'
# Capture the order ID, then immediately cancel:
curl -sS -X DELETE https://api.raxx.app/api/trading/orders/<id>
# Expect: 204 / order status = canceled

7. Revoke the old credential

The dashboard regeneration already invalidated the old key pair atomically. Confirm:

curl -sS -o /dev/null -w "%{http_code}\n" \
     -H "APCA-API-KEY-ID: $OLD_KEY_ID" \
     -H "APCA-API-SECRET-KEY: $OLD_SECRET" \
     https://api.alpaca.markets/v2/account
# Expect: 401 / 403

8. Resume trading + audit log

heroku config:set TRADING_LIVE_DISABLED=0 -a raxx-api-prod
# Wait for dyno restart, confirm
sleep 30
curl -sS https://api.raxx.app/health/trading | jq '.live_enabled'
# Expect: true

action: secret.rotate.completed
actor: <superadmin_id>
context: {
  "secret_name": "ALPACA_LIVE_API_KEY_ID",
  "method": "operator-assisted-dashboard",
  "trading_paused_at": "...",
  "trading_resumed_at": "...",
  "downtime_seconds": <delta>,
  "account_number_verified": true
}

Rollback

If validation in step 3 or 6 fails:

  1. Keep trading paused (TRADING_LIVE_DISABLED=1).
  2. Old keys are already dead — there is no "revert key" path.
  3. Regenerate again from the dashboard until you have a working key pair.
  4. Repeat steps 4–6 until validation passes.
  5. Resume trading only after step 6 verification succeeds.

If the wrong environment was rotated (paper instead of live or vice versa), pause both trading modes and consult both runbooks. Audit-log the mis-rotation as a separate event.

Vendor doc references

Known gotchas