GHA Deliberate-Hold Workflows — Cutover Design

Date: 2026-07-09 UTC Author: software-architect Status: Accepted and executed — see §10 for the as-built status (verified against develop on 2026-10-30 UTC) ADR: docs/architecture/adr/0136-gha-deliberate-holds-cutover.md Parent epic: #4066 (cutover epic #4106) Related: docs/architecture/woodpecker-migration-table-2026-07-04.md

Reading note. §§1–9 are the design as accepted on 2026-07-09 and are kept as the decision record. Where the implementation differs, §10 is authoritative. A few plain factual errors in the original text (ECR region, key-rotation tooling, the CloudTrail alert claim) are corrected inline and marked "as built".

1. Context

Eight GitHub Actions workflows carry a "deliberate GHA hold" classification in the Wave D migration (ADR-0135). They were not ported in Waves A–C because each faces a specific technical blocker:

This document designs the cutover for each: what stays on GHA, what moves to Woodpecker, and the prerequisite shared-infra cards that unblock per-workflow ports.

2. Invariants

From ADR-0135 and the project-level constraints:

• No stored credentials between jobs. WP agent workspaces are ephemeral. SSH keys, AWS keys, and CF tokens must not persist to disk beyond the step that needs them.
• Credentials into infra, not code. All secrets flow from vault (Infisical via wpc_load_vault_secrets.py) or AWS SSM at pipeline runtime.
• Same-step source pattern. WOODPECKER_ENV_FILE is not set in WP v3.16.0 step context (confirmed build #6). Load vault secrets and consume them in the same step. See docs/architecture/woodpecker-wpc-step-usage.md.
• Audit trail for every state change affecting money, permissions, or data access. Every deploy and apply pipeline logs actor, ref SHA, target, and outcome (90-day CloudWatch retention per ADR-0135).
• Paper-first gating. Prod execution paths must be at least as protected as the current GHA model (WP tag + secret event restriction per ADR-0135 §3).
• External vantage for monitoring is non-negotiable. A monitor that watches Woodpecker must run on infrastructure independent from Woodpecker.

3. Per-Workflow Verdicts

3.1 woodpecker-healthcheck.yml

Current mechanism: GHA schedule: */15 * * * *. Probes ci.moosequest.net for reachability, TLS cert expiry, and pipeline health. Edge-triggered Postmark email + GitHub anchor issue on state transitions.

Verdict: GENUINELY STAYS ON GHA

A Woodpecker-hosted pipeline cannot alert when Woodpecker is down — that is the failure mode this monitor catches. This is a sound architectural boundary, not a workaround. The external vantage requirement rules out all WP options.

Alternative assessment: - SaaS uptime monitor (Better Uptime, UptimeRobot): Valid secondary layer for multi-region vantage. Does not replace the GHA pipeline because the GH anchor-issue dedup and edge-triggered Postmark logic would require custom webhook wiring. - Prometheus blackbox exporter on monitoring EC2 (#1022): Same AWS region and availability zone as ci.moosequest.net. Same-failure-domain; does not satisfy the independent vantage requirement. Valuable for latency dashboards, not the watchdog role.

Dependencies: None. Effort: XS — documentation only. Risk: None.

3.2 deploy-antlers-cutover.yml

Current mechanism: workflow_dispatch only. Typed inputs: confirm_cutover (string magic gate), dry_run (boolean, default true), rollback (boolean), cra_project_name (string), auto_rollback (boolean). GH Environment production-nextjs approval gate (currently no required reviewers per ADR-0020 waiver). 15-step sequence: validate → vault load → 3 pre-flights → CF Pages ops → DNS update → health checks → summary.

Verdict: PORT TO WP MANUAL PIPELINE (or retire immediately if cutover already ran)

Open question OQ-5: verify whether the CRA→Next.js cutover has already executed. If yes, retire the workflow without porting. If no, port to WP and fire from there.

WP design: - Trigger: event: manual. Operator sets CONFIRM_CUTOVER, DRY_RUN=true|false, and MODE=forward|rollback as WP pipeline variables via Console Ops Dispatch or POST /api/repos/1/pipelines. - Confirm gate: First step asserts CONFIRM_CUTOVER == "CUTOVER raxx.app TO NEXT.JS" (forward) or CONFIRM_CUTOVER == "ROLLBACK raxx.app TO CRA". Fails loudly on mismatch. WP manual pipelines have no typed boolean inputs; DRY_RUN is a string. - GH Environment gate: WP event: manual + Console RBAC is the equivalent. No secondary reviewer needed (ADR-0020 waiver in effect). - All 15 steps port directly. Vault init loads CF tokens via wpc_load_vault_secrets.py in the same step as the first CF API call. No GHA-specific constructs in the core steps (they are pure shell + curl). - $GITHUB_STEP_SUMMARY → stdout in WP build log.

Dependencies: None beyond Wave C WP pipeline infra. Effort: M (port 15 steps, translate typed inputs, update Console Ops Dispatch). Risk: Low — one-shot; testable with DRY_RUN=true before live execution.

3.3 deploy-failure-streak-alert.yml

Current mechanism: workflow_run: trigger fires after every deploy workflow completion. Queries GH Actions API for last N runs on main; detects 2+ consecutive failures within 30 minutes; posts SEV2 Slack alert. Also: deploy-queue-failure-monitor.yml (a companion workflow using an unbounded streak window, 3+ consecutive failures, no time gate, specific to the Queue pipeline).

Verdict: PORT TO WP CRON (poll WP API); absorb deploy-queue-failure-monitor

workflow_run: has no WP equivalent (ADR-0135 §4). Replace the reactive trigger with a 30-minute cron that polls the WP API.

WP design: - Trigger: event: cron (30-minute interval, registered as deploy-failure-streak-check via POST /api/cron). - Logic: Replace GET /repos/{repo}/actions/workflows/{name}/runs with GET /api/repos/1/builds?limit=20 filtered by pipeline name and status=failure. WP build API returns status and created timestamp. Same two-failure-in-30-min streak logic applies. - Pipeline list: Update DEPLOY_WORKFLOWS from GHA workflow names to WP pipeline filenames (e.g., deploy-heroku.yaml, deploy-antlers-next-prod.yaml). - WP API auth: Vault-loaded WOODPECKER_API_TOKEN (existing WP user API token, already used by woodpecker-healthcheck.yml). - Absorb deploy-queue-failure-monitor.yml: Add a separate check block for the Queue pipeline with threshold 3 and no time window. Retire deploy-queue-failure-monitor.yml as part of this card. - Slack alert: Unchanged (vault-loaded SLACK_WEBHOOK_URL).

Dependencies: None (WP API token already in vault). Effort: M (port Python streak logic; update API calls from GH to WP; absorb queue monitor). Risk: Low — alert-only, no deploy action.

3.4 freescout-apply.yml

Current mechanism: workflow_dispatch. Two jobs: (1) Terraform plan + apply with AWS credentials loaded from GHA repo secrets; (2) SSH bootstrap smoke test — fetches SSH key from SSM (/raxx/freescout/ssh_key), SSHes to 54.146.13.200, waits for cloud-init completion. Artifact upload: smoke log to GH artifact store (7-day retention). Slack DM on failure. No actual GH Environment approval gate (the job has no environment: block; ADR-0020 waiver is in effect).

Verdict: PORT TO WP MANUAL PIPELINE

WP design: - Trigger: event: manual. Safety gate: first step asserts APPLY_CONFIRM == "APPLY-FREESCOUT-NOW". For module-only or dry-run: MODULE_ONLY=true or DRY_RUN=true variables. - AWS credentials: Vault-loaded IAM access keys replacing GHA repo secrets AWS_TF_ACCESS_KEY_ID / AWS_TF_SECRET_ACCESS_KEY. Requires Prerequisite-P2. - SSH key from SSM: Smoke step runs aws ssm get-parameter --name /raxx/freescout/ssh_key --with-decryption using vault-loaded AWS creds. Key written to /tmp/lightsail_key_XXXXXX.pem with chmod 600, used in the same step, then rm -f before step exit. No inter-step key persistence (same-step pattern). Multi-line PEM key is handled correctly: WP auto-redacts from_secret values; the SSM-fetched key lives only in the step's process memory, never in WP secrets. - Artifact upload: Stream smoke log to stdout (preserved in WP build log). For durable 7-day archive, upload to s3://raxx-ci-artifacts/freescout-smoke-logs/ using vault-loaded AWS creds in the same step. S3 lifecycle rule: 7-day expiry. Requires Prerequisite-P2 S3 bucket setup. - FreeScout module license keys: Loaded from vault (stay in Infisical per feedback_aws_workloads_use_ssm_not_vault.md). - Slack failure DM: when: status: failure inline step, vault-loaded SLACK_BOT_TOKEN. - Terraform binary: image: hashicorp/terraform:1.7 step image; no hashicorp/setup-terraform@v3 GHA action available in WP.

Dependencies: Prerequisite-P2 (vault IAM keys + S3 artifact bucket). Effort: M (port 3 jobs × ~5 steps; SSH key handling is the highest-risk step). Risk: Medium — SSH key same-step cleanup is non-trivial; cloud-init 20-min wait must be reflected in WP timeout: setting.

3.5 security-zap.yml

Current mechanism: Three triggers: (A) pull_request on release/main branches with path filter (frontend/raxx-next/** or backend_v2/api/**); (B) weekly schedule Monday 09:07 UTC; (C) workflow_dispatch. Uses zaproxy/action-baseline@v0.12.0 (a GHA composite action that wraps docker run internally). Two scan jobs: zap-antlers and zap-api. Uploads HTML/JSON reports as GH artifacts; posts PR comment; files GH issues for HIGH findings.

Verdict: PORT TO WP — ZAP image as WP step image (no DinD required)

Key insight: zaproxy/action-baseline wraps Docker internally because GHA steps cannot specify a container image. In WP, each step specifies a Docker image directly. Run ZAP as image: zaproxy/zap-stable — the WP agent runs this container natively, exactly as it runs python:3.12-slim for vault init. No Docker-in-Docker needed.

WP design: - Triggers: event: cron (weekly Monday 09:07 UTC, registered as security-zap-weekly) + event: pull_request, branch: [release, main] (ADR-0115 promotion-PR gate) + event: manual override. - ZAP step: image: zaproxy/zap-stable. Commands: zap-baseline.py -t ${TARGET_URL} -r /tmp/report.html -J /tmp/report.json. Same .zap/rules.tsv suppression file (present in repo, mounted via WP workspace clone). - Pin: zaproxy/zap-stable:2.14.0 (or latest stable at port time). Port the existing pin-check step as a WP step that greps the pipeline YAML for @latest. - PR preview URL resolution: Same branch-name-to-pages-dev derivation. CF Access service-token headers loaded from vault in the same step. - PR comments: image: python:3.12-slim step; vault-loaded raxx-ops-bot GH token; GH REST API call replaces actions/github-script@v7. - Artifact storage: Upload report files to s3://raxx-ci-artifacts/zap-reports/$CI_BUILD_NUMBER/ using vault-loaded AWS creds. 30-day lifecycle. Requires Prerequisite-P2 S3 bucket. - HIGH issue filing: python3 .github/scripts/zap_file_issues.py in a python:3.12-slim step with vault-loaded raxx-ops-bot GH token. - Monitoring stack note: The weekly scan could move to monitoring EC2 (#1022) as a cron job probing staging. Valid future direction; start with WP cron (in Wave B plan) and revisit once #1022 is operational.

Dependencies: Prerequisite-P2 (S3 bucket for artifacts, raxx-ops-bot token already in vault). Effort: S (ZAP as step image is simpler than the GHA composite; PR comments and issue filing are the main porting effort). Risk: Low — no privileged modes required; WP native image execution is the standard pattern.

3.6 terraform-email-delivery-stack.yml

Current mechanism: Three triggers: pull_request (plan-only), push to main (auto-apply), daily schedule (drift detection). AWS OIDC short-lived federation with two IAM roles: raxx-tf-email-delivery-stack-plan (read-only) and raxx-tf-email-delivery-stack-apply (read-write). GH Environment production-tf-destroy approval gate when plan includes destroy actions. 5 jobs: preflight → plan → check-destroy-gate → apply → notify-failure.

Verdict: PORT TO WP — vault-loaded long-lived IAM keys + destroy confirm gate

This is the most significant security trade-off in these 8 workflows. WP cannot act as an OIDC issuer or assume IAM roles via federated identity. See §4.3 and ADR-0136 for the formal decision and mitigations.

WP design: - Triggers: event: pull_request, branch: [develop, release] (plan-only); event: push, branch: main (plan + apply); event: cron (06:00 UTC daily, drift); event: manual (operator-triggered apply or plan). - AWS auth: Vault-loaded long-lived IAM access keys (two separate key pairs): - Plan keys: /MooseQuest/aws/terraform/email-delivery-stack-plan/ (ACCESS_KEY_ID, SECRET_ACCESS_KEY) - Apply keys: /MooseQuest/aws/terraform/email-delivery-stack-apply/ (ACCESS_KEY_ID, SECRET_ACCESS_KEY) Plan steps source plan keys; apply steps source apply keys. Same-step pattern. Requires Prerequisite-P2. - Destroy gate: WP has no GH Environment reviewer UI equivalent. Design: When destroy_count > 0, the apply step is skipped and the pipeline emits an explicit error. Operator re-triggers manually with DESTROY_CONFIRM=ALLOW-DESTROY-EMAIL-STACK to proceed. The apply step checks this variable before executing. The WP build log is the audit trail. See §4.4 for the general approval-gate pattern. - PR plan comment: image: python:3.12-slim step; vault-loaded raxx-ops-bot GH token; sticky-comment logic (find existing comment by HTML marker, update or create) ports directly from the actions/github-script@v7 JavaScript to Python. - Plan artifact: Replace actions/upload-artifact@v4 with S3 upload to s3://raxx-ci-artifacts/tf-plans/email-delivery-stack/$CI_COMMIT_SHA.tfplan (1-day lifecycle). The apply job re-plans from scratch anyway (same as current GHA design); the S3 artifact is for audit reference only. - Nightly drift: event: cron plan-only run; post to WP build log. - Terraform binary: image: hashicorp/terraform:1.7 step image.

Dependencies: Prerequisite-P2 (vault IAM keys + S3 bucket). Effort: L (5 jobs, OIDC→vault migration, destroy gate redesign, PR comments, Terraform image, S3 artifact migration). Risk: High — OIDC→long-lived key is a security posture downgrade; destroy gate UX change reduces operator friction protection.

3.7 deploy-queue.yml

Current mechanism: Push to release/main (auto-staging) and workflow_dispatch (manual prod confirm gate). Jobs: lint-sqitch-plan → build-test → build-container → deploy-staging or deploy-prod. build-test uses actions/cache@v4 for vcpkg binary cache + docker run --mount inside gcc:13-bookworm. build-container builds the production Docker image and pushes to Heroku Container Registry via docker build + docker push. Both jobs require the Docker daemon.

Verdict: PORT TO WP — Docker socket mount + ECR layer cache

WP has no native cache action (actions/cache@v4 equivalent does not exist). The GHA layer cache (type=gha) must be replaced with ECR layer cache (type=registry). This requires Prerequisite-P1.

WP design: - Triggers: event: push, branch: [release, main] (staging/prod auto); event: manual (prod confirm gate: DEPLOY_CONFIRM=deploy-prod-now). - Docker access: WP agent has /var/run/docker.sock:/var/run/docker.sock in WOODPECKER_VOLUMES (Prerequisite-P1). Steps use image: docker:24-cli or any image with Docker CLI. - vcpkg binary cache: GHA actions/cache@v4 → ECR layer cache. Strategy: docker buildx build --cache-from type=registry,ref=<ECR>:vcpkg-cache-bookworm --cache-to type=registry,ref=<ECR>:vcpkg-cache-bookworm,mode=max. ECR repo: raxx-queue-vcpkg-cache (created in P1). Cache key semantics: image tag includes vcpkg.json SHA digest (vcpkg-cache-bookworm-<sha256[:8]>). Tag changes only when vcpkg.json changes, preserving the GHA cache-key semantics. Warm build: ~5 min (ECR hit). Cold build: ~25-30 min (ECR miss or first run). ECR warm bootstrap: Run one full build before WP cutover (on GHA or from a dev machine with ECR push rights) to populate the ECR cache. Avoids cold-build penalty at cutover time. - Sanitizer build + tests: docker run --mount type=bind,src=./queue,dst=/src inside gcc:13-bookworm. Logic unchanged; requires Docker socket. - Production Docker build: docker buildx build --push to Heroku Container Registry (registry.heroku.com/raxx-queue-{staging,prod}/web). Login: docker login registry.heroku.com --username=_ --password=$HEROKU_API_KEY. Vault-loaded Heroku key. - Prod secret restriction: HEROKU_API_KEY_PROD stored in WP with event: ["tag", "manual"] restriction (ADR-0135 §3 pattern). Staging key unrestricted. - Prod confirm gate: WP event: manual + DEPLOY_CONFIRM variable check (same pattern as existing deploy-prod.yaml). - sqitch.plan lint: image: python:3.12-slim step, no Docker needed. - Audit emit and Slack DM: wpc_emit_audit_event.py and when: status: failure Slack step, unchanged from other migrated deploy pipelines.

Dependencies: Prerequisite-P1 (Docker socket + ECR repo + agent IAM). Effort: L (4+ jobs, cache strategy migration, Docker socket reliance, Heroku CR auth, prod confirm gate, audit emit). Risk: High — vcpkg cold builds are 25-30 min; ECR cache warmth must be validated before WP cutover. A cold build on first WP run would block a staging deploy for ~30 min.

3.8 queue-docker-smoke.yml

Current mechanism: pull_request on release/main branches touching queue/**. Jobs: detect-changes → queue-docker-build. Build job uses docker/setup-buildx-action@v3 + docker/build-push-action@v5 with GHA layer cache (type=gha). Then docker run -d smoke to verify the binary loads at runtime.

Verdict: PORT TO WP — Docker socket mount + ECR layer cache

Shares Docker socket and ECR infrastructure with deploy-queue. Prerequisite-P1 is the same blocker.

WP design: - Trigger: event: pull_request, branch: [release, main] (ADR-0115 promotion-PR gate). - Detect-changes: image: python:3.12-slim step; git diff --name-only $CI_COMMIT_BEFORE_SHA...$CI_COMMIT_SHA to detect queue/** changes. If no changes: step sets SKIP=true, remaining steps exit early. - Docker build: docker buildx build --cache-from type=registry,ref=<ECR>:queue-smoke-layer --cache-to type=registry,ref=<ECR>:queue-smoke-layer,mode=max. GHA type=gha → ECR type=registry. The smoke layer cache is separate from the vcpkg cache layer to allow independent invalidation. - Smoke: docker run -d --name queue-smoke-$CI_BUILD_NUMBER <image>. Wait 5 s. docker logs queue-smoke-$CI_BUILD_NUMBER. Assert startup log contains "Starting raxx-queue" or "Binding to 0.0.0.0". Cleanup: docker rm -f queue-smoke-$CI_BUILD_NUMBER. - No secrets required (matches current GHA version). - Cold build: same 25-30 min concern. ECR cache from P1 applies.

Dependencies: Prerequisite-P1. Effort: M (simpler than deploy-queue; no deploy step; Docker build + run only). Risk: Medium — ECR cache warmth same concern as deploy-queue.

4. Cross-Cutting Shared Infrastructure

4.1 Prerequisite P1 — Docker-capable WP agent + ECR layer cache

Scope: 1. Configure WP agent: Add /var/run/docker.sock:/var/run/docker.sock to WOODPECKER_VOLUMES in the agent launch config. Preferred over DinD (WOODPECKER_PRIVILEGED=true) because socket mount avoids a nested daemon and its startup latency on spot instances. See OQ-1 for operator decision. 2. Create ECR repository raxx-queue-vcpkg-cache in us-east-2 (as built: the region is locked to us-east-2 by infra/ci/variables.tf; the original text said us-east-1). 3. Expand WP agent IAM role to add ECR push/pull permissions scoped to raxx-queue-vcpkg-cache: - ecr:GetAuthorizationToken, ecr:BatchCheckLayerAvailability, ecr:GetDownloadUrlForLayer, ecr:BatchGetImage, ecr:PutImage, ecr:InitiateLayerUpload, ecr:UploadLayerPart, ecr:CompleteLayerUpload. 4. Validate: Manually trigger a test WP pipeline that runs docker buildx build with --cache-to type=registry,ref=<ECR>:test-layer and confirm push succeeds. 5. ECR cache bootstrap: Run one full vcpkg build (on GHA or from a dev machine with ECR rights) and push the vcpkg cache layer to ECR before WP cutover, to avoid a cold build on the first WP run.

Unblocks: W7 (deploy-queue), W8 (queue-docker-smoke).

4.2 Prerequisite P2 — Vault-loaded AWS IAM keys + S3 artifact bucket

Scope: 1. Create three least-privilege IAM users (long-lived access keys, not roles): - raxx-tf-email-stack-plan-ci: read-only policy on email-delivery-stack resources. - raxx-tf-email-stack-apply-ci: read+write policy on email-delivery-stack resources. - raxx-freescout-ssm-reader-ci: ssm:GetParameter on /raxx/freescout/* + s3:PutObject on raxx-ci-artifacts/freescout-smoke-logs/*. 2. Create vault folders before writing secrets (Infisical returns 404 on missing folder — see feedback_vault_folder_must_exist.md): POST /api/v1/folders for /MooseQuest/aws/terraform/ and /MooseQuest/aws/freescout/. 3. Store access keys in vault at: - /MooseQuest/aws/terraform/email-delivery-stack-plan/ (ACCESS_KEY_ID, SECRET_ACCESS_KEY) - /MooseQuest/aws/terraform/email-delivery-stack-apply/ (ACCESS_KEY_ID, SECRET_ACCESS_KEY) - /MooseQuest/aws/freescout/ssm-reader/ (ACCESS_KEY_ID, SECRET_ACCESS_KEY) 4. 90-day rotation schedule. As built: there is no velvet handler. Rotation is an operator-run script, scripts/ops/secrets/mint_ci_aws_keys.sh --rotate, documented in docs/ops/runbooks/ci-aws-credentials.md (the original text named a velvet handle_rotate_aws_iam_key handler and an aws-iam-rotation.md runbook; neither exists). 5. Create S3 bucket raxx-ci-artifacts (or confirm it exists) with lifecycle rules: - freescout-smoke-logs/ → 7-day expiry. - zap-reports/ → 30-day expiry. - tf-plans/email-delivery-stack/ → 1-day expiry. - Bucket policy: allow write from raxx-freescout-ssm-reader-ci and WP agent role.

Unblocks: W4 (freescout-apply), W5 (ZAP artifact storage), W6 (terraform-email).

4.3 AWS Auth: OIDC → long-lived key security trade-off

GHA used OIDC short-lived tokens (15-30 min TTL, cryptographically bound to the specific workflow run, repository, and branch ref). WP cannot issue OIDC tokens or assume IAM roles via federated identity.

The WP replacement uses long-lived IAM access keys. This is a posture downgrade. Mitigations: - Separate plan (read-only) and apply (read-write) IAM users. A leaked plan key cannot create or destroy infrastructure. - 90-day rotation. As built this is operator-run (mint_ci_aws_keys.sh --rotate), not automated by velvet, so it depends on the operator keeping to the 90-day cadence. - CloudTrail alerting on unexpected API calls (wrong IP, unexpected hours). As built: not yet implemented in IaC; infra/ci/ defines no CloudTrail alarm for these users. The runbook only describes a manual CloudTrail lookup. - Least-privilege scoped-per-resource policies, not AdministratorAccess. - Keys in vault only, never in repo files or WP YAML.

See ADR-0136 §OIDC for the formal decision record.

4.4 Approval gates: WP confirm-variable pattern

GHA Environments provide a required reviewer approval UI and a GH deployment audit trail. WP has no equivalent.

The established WP pattern (from .woodpecker/deploy-prod.yaml and ADR-0135 §3): 1. Trigger with event: manual. 2. First step asserts a confirm variable equals a magic string. Fails loudly on mismatch. 3. Console Ops Dispatch RBAC controls who can trigger event: manual pipelines. 4. WP build log is the audit trail.

Apply this pattern to freescout-apply (APPLY_CONFIRM=APPLY-FREESCOUT-NOW) and terraform-email destroy gate (DESTROY_CONFIRM=ALLOW-DESTROY-EMAIL-STACK). The UX difference is acceptable for a single-operator team and is consistent with ADR-0135.

5. Migrations

State changes required before first WP pipeline runs:

Rollback per workflow:

Each WP pipeline runs alongside its GHA counterpart during a 7-day soak. If the WP pipeline fails soak, disable it in the WP admin UI; the GHA workflow remains active. GHA workflows retire only after WP soak passes with at least two successful runs (ADR-0135 Wave entry criteria).

Exceptions: - woodpecker-healthcheck.yml: Never migrated; no rollback needed. - deploy-antlers-cutover.yml: Run exactly once. Retire the GHA workflow immediately after the live cutover is confirmed via smoke (it will never run again). - deploy-queue-failure-monitor.yml: Retire when W3 lands (absorbed into streak pipeline).

6. Rollout Plan

7. Security Considerations

8. Open Questions

Decisions required before the relevant cards are claimed:

1. OQ-1 (P1): Docker socket vs DinD. Socket mount (/var/run/docker.sock) is simpler and avoids nested-daemon startup latency. DinD (WOODPECKER_PRIVILEGED=true) is more isolated but slower on spot instances. Recommended: socket mount. Does the operator accept Docker socket mount on the WP EC2 agent?

2. OQ-2 (W6): Single vs two-pipeline Terraform model. Single pipeline with DESTROY_CONFIRM variable is operationally simpler. Two-pipeline (plan.yaml + apply.yaml) gives cleaner audit separation. Recommended: single pipeline + confirm variable (consistent with deploy-queue confirm gate). Operator preference?

3. OQ-3 (W5): ZAP weekly scan — WP cron or monitoring stack (#1022)? WP cron is Wave B plan and has no blocking dependency. Monitoring stack (#1022) is not yet operational. Recommended: WP cron now; revisit for #1022 later. Confirm?

4. OQ-4 (P2): S3 bucket raxx-ci-artifacts — does it already exist? If yes, confirm agent IAM role can write to it and that lifecycle rules can be appended without disrupting existing prefixes.

5. OQ-5 (W2): Has the CRA→Next.js cutover already been executed on production? If yes, retire deploy-antlers-cutover.yml without porting. If no, the Wave C WP port proceeds.

9. Proposed Sub-Card Breakdown

Ordered by dependency. P cards must land before the W cards they unblock.

Prerequisite (shared infra)

Per-workflow (ordered by dependency, then effort)

10. Implementation status (as of 2026-10-30 UTC)

Verified against develop at dd2f2d6d4. §§1–9 above are the design as accepted; this section records what was built and where it deviates.

10.1 Per-workflow outcome

Workflows still under .github/workflows/: woodpecker-healthcheck.yml (permanent) and billing-collector-cron.yml (paused 2026-05-15, kept as a fallback; it is not one of the eight holds). ios-ci.yml was retired in Wave D (#4064).

A stray copy of .woodpecker/deploy-queue-failure-monitor.yaml was on develop. #4135 deleted it as a duplicate of W3, and #4342 re-added it unrelated to this work. It was a flag-gated duplicate (never enabled) of the Queue profile in W3's pipeline and was removed 2026-10-30 once that profile was fixed to agree with deploy-queue.yaml. Its Woodpecker cron registration still has to be deleted by hand (docs/ops/runbooks/ci-woodpecker.md, cron 38).

10.2 Shared infrastructure

10.3 Open questions, resolved

10.4 Security mitigations not delivered as designed

ADR-0136 Decision 2 accepted the OIDC-to-long-lived-key downgrade on the strength of five mitigations. Two are not in place as written:

• Rotation is not automated. No velvet handler exists. Rotation is the operator-run mint_ci_aws_keys.sh --rotate, so the 90-day cadence depends on the operator.
• No CloudTrail alert. infra/ci/ defines no alarm on API calls by these users. Only the runbook's manual aws cloudtrail lookup-events check exists.

The .woodpecker/terraform-email-delivery-stack.yaml header still lists both as provided.