Runbook: Support portal S3 attachments

Related: Issue #668, Epic #651

What this is

S3 bucket (raxx-support-attachments-prod) that stores customer-uploaded attachments for the support.raxx.app portal. Files go directly from the customer's browser to S3 via a pre-signed POST URL — they never pass through the Raxx backend. Downloads are served via short-TTL (5-minute) pre-signed GET URLs.

All objects are encrypted at rest with a customer-managed KMS key (alias/raxx-support-attachments), separate from any other AWS resource, providing blast-radius isolation.

Provisioning (first time)

cd terraform/support-attachments
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars: set ops_break_glass_arn to your operator IAM user ARN

export AWS_PROFILE=raxx-prod   # or set AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY
terraform init
terraform plan    # review; verify no PUBLIC access resources
terraform apply

After apply, write outputs to vault:

# Create IAM access key for the long-lived Heroku user
aws iam create-access-key --user-name raxx-support-backend-user

# Store in Infisical
infisical secrets set \
  AWS_ACCESS_KEY_ID_SUPPORT="<AccessKeyId>" \
  AWS_SECRET_ACCESS_KEY_SUPPORT="<SecretAccessKey>" \
  S3_BUCKET_SUPPORT_ATTACHMENTS="$(terraform output -raw bucket_name)" \
  KMS_KEY_ARN_SUPPORT_ATTACHMENTS="$(terraform output -raw kms_key_arn)" \
  --path /MooseQuest/aws/support-attachments/

Set the same four vars in Heroku (prod + staging environments), then flip the feature flag:

# backend_v2/api/feature_flags.yaml
support_s3_attachments: true

Or via env override: FLAG_SUPPORT_S3_ATTACHMENTS=1

Object key scheme

s3://raxx-support-attachments-prod/<customer_uuid>/<ticket_uuid>/<file_uuid>.<ext>

The per-customer prefix is the isolation boundary. Future per-customer IAM scoping will use s3:prefix conditions on this path segment.

KMS key details

To check rotation status:

aws kms describe-key --key-id alias/raxx-support-attachments \
  --query 'KeyMetadata.{Enabled:Enabled,Rotation:KeyRotationEnabled}'

Lifecycle policy

Monitoring + audit

Every upload URL generation and download URL generation writes a structured audit log line via Python's logging module at INFO level:

AUDIT {'event': 'support.attachment.upload_url_issued', 'customer_uuid': '...', 'ticket_uuid': '...', ...}
AUDIT {'event': 'support.attachment.accessed', 'customer_uuid': '...', ...}

Pre-signed URLs are never included in audit log entries.

Incident response

Suspected data exfiltration

1. Disable the KMS key alias (makes all existing encrypted objects unreadable): sh aws kms disable-key --key-id alias/raxx-support-attachments
2. Rotate the IAM access key for raxx-support-backend-user: sh aws iam list-access-keys --user-name raxx-support-backend-user aws iam delete-access-key --user-name raxx-support-backend-user --access-key-id <old> aws iam create-access-key --user-name raxx-support-backend-user
3. Update the new key in Infisical and Heroku.
4. Re-enable the KMS key once the incident scope is understood.

Accidental object deletion

Versioning is enabled. To restore a deleted object:

# List versions + delete markers
aws s3api list-object-versions \
  --bucket raxx-support-attachments-prod \
  --prefix "<customer_uuid>/<ticket_uuid>/<file_uuid>"

# Remove the delete marker to restore
aws s3api delete-object \
  --bucket raxx-support-attachments-prod \
  --key "<object_key>" \
  --version-id "<DeleteMarkerVersionId>"

KMS key compromised

1. Schedule key deletion (14-day waiting period): sh aws kms schedule-key-deletion \ --key-id alias/raxx-support-attachments \ --pending-window-in-days 14
2. Provision a new KMS key via terraform apply with a new kms_key_alias.
3. Existing objects encrypted under the old key are unreadable; this is intentional blast-radius isolation.
4. Contact affected customers per the incident response playbook.

Cost estimate