Reasonator — Sentiment Scoring Service Design

Service codename: reasonator Service tagline: "Facts, not opinions" Date: 2026-05-09 UTC Status: Design — awaiting sub-card implementation Refs: #1385 (parent card, re-scoped), #89 (sentiment ingestion umbrella), #80 (epic) Design branch: design/reasonator-sentiment-service

1. Context

The data-scientist's research (branch research/sentiment-ingestion-89) validated that FinBERT (ProsusAI/finbert, MIT license) produces better financial sentiment scores than Alpaca's built-in vendor tags. FinBERT is ~400 MB of model weights and takes ~200 ms per headline on CPU.

The operator decision on 2026-05-09: FinBERT must NOT load inside the main Raxx app process (Raptor). Reasons:

• Raptor serves synchronous API requests; a 400 MB model load at startup or a 26-second batch score inside an HTTP worker are unacceptable.
• Scalability: sentiment demand will grow independently of Raptor's trading-execution load.
• Isolating the model allows independent deployment, scaling, and versioning.

Reasonator is a standalone scoring service. It lives alongside Raptor, Antlers, and Velvet. Raptor calls Reasonator; Reasonator scores; Reasonator returns results. Reasonator never touches execution, never predicts, never fires orders.

2. Invariants (Non-Negotiable)

These bind every design choice below. Violation requires stopping and surfacing the conflict.

3. Naming and Theme

Reasonator fits the existing codename theme: - Antlers theme — a "reasonator" is the full set of antlers on a buck. The frontend is Antlers; Reasonator decorates it with data. - Rick Sanchez wink — RICK → RACK single-letter swap (operator's R&M-themed scientist vibe). - Technical — server reasonator, scoring reasonator, reasonator of facts.

The operator noted this may be renamed. The ADR records the choice; rename is a find-and-replace when it happens.

4. Architectural Decisions

Decision 1: Deployment Target — Heroku (Tier B)

Options considered:

Decision: Heroku Standard-2X dyno, minimum 1 dyno always running.

Rationale: Heroku is already the established Tier B platform (ADR-0052). Adding Reasonator as another Heroku app follows the existing playbook with no new ops tooling. A single min-1 Standard-2X dyno (1 GB RAM) comfortably holds FinBERT (400 MB) plus Flask overhead. AWS ECS is the right call at sustained high scale (>10k req/day) — flag this as the scale-trigger migration path in the cost model. The design keeps Reasonator stateless enough to make that migration mechanical. Lambda provisioned concurrency costs more than Heroku at the Phase 1 load tier, and the ops model is materially more complex.

Scale trigger: When daily requests consistently exceed 5,000/day for 7 consecutive days, migrate to Fargate. A monitoring alert fires at 4,000/day as the advance warning.

ADR: docs/architecture/reasonator/adr/0054-reasonator-deployment-target.md

Decision 2: API Contract — REST (Sync + Async)

Decision: REST with two endpoint shapes:

1. POST /v1/score — synchronous, single-batch, Pro+ real-time path. Latency target: p99 < 2s for batches of ≤10 headlines.
2. POST /v1/score/batch — asynchronous, accepts a job reference (list of sentiment_events.id values), enqueues the batch for processing, returns a job_id. Raptor polls GET /v1/score/batch/{job_id} for status and results.
3. POST /v1/score/rescore — synchronous re-scoring with a new model SHA for the re-scoring sweep.

Full contract: docs/architecture/reasonator/api-contract.md

ADR: docs/architecture/reasonator/adr/0055-reasonator-api-contract-rest.md

Decision 3: Auth Between Raptor and Reasonator — Service Token in Vault

Decision: Bearer token, stored in Infisical at /reasonator/prod/REASONATOR_SERVICE_TOKEN. Raptor reads it at startup. Reasonator verifies it on every request. Token is 32 bytes random, base64url-encoded (256-bit entropy), rotatable without redeploy.

Vault folder /reasonator/ must be created in Infisical before the first secret write. The scaffold sub-card is responsible for this step.

ADR: docs/architecture/reasonator/adr/0056-reasonator-service-auth.md

Decision 4: Re-scoring Strategy — Model SHA as First-Class Provenance

Decision: Append-with-audit. sentiment_events stores the current (latest) score. Every score write is appended to sentiment_score_audit with both the new score and (for re-scores) the previous score and its model SHA.

POST /v1/score/rescore carries previous_score and previous_model_sha in the request; Reasonator echoes them in the response alongside the new scores.

FINBERT_MODEL_SHA is an env var in the Reasonator config, sourced from Infisical.

ADR: docs/architecture/reasonator/adr/0057-reasonator-rescoring-model-sha-provenance.md

Decision 5: Tier-Aware Rate Limiting

Design: Two tiers, one service, priority-annotated endpoints.

• POST /v1/score carries X-Raxx-Tier: pro_plus or X-Raxx-Tier: pro header.
• Reasonator's internal handler processes pro_plus requests first. Pro+ requests bypass the batch queue and go directly to the synchronous scorer thread pool. Pro requests join the async job queue.
• Rate limits enforced at Raptor side (before the call) using existing feature-flag middleware. Reasonator enforces secondary server-side limits: Pro+ max 10 concurrent sync requests; Pro max 5 concurrent batch jobs.
• Headers returned: X-Reasonator-RateLimit-Remaining, X-Reasonator-RateLimit-Reset.

Decision 6: Observability

Latency budget:

Metrics emitted to Sentry:

• reasonator.score.latency_ms — histogram, tagged tier, batch_size, model_sha
• reasonator.score.error_rate — counter, tagged error_type
• reasonator.model.load_ms — gauge, emitted once at startup
• reasonator.batch_job.duration_ms — histogram, tagged batch_size
• reasonator.queue.depth — gauge every 30s, tagged priority

Sentry DSN: In vault at /reasonator/prod/SENTRY_DSN. Not hardcoded.

Decision 7: Cold-Start and Warm Pool

• Minimum 1 dyno always running on both staging and prod (Heroku app config, not Procfile).
• Scheduled keep-alive ping: GitHub Actions cron */10 * * * * pings GET /v1/health every 10 minutes.
• Model load at startup, not at first request. Procfile: gunicorn reasonator.app:create_app() --preload --workers 2 --timeout 120. --preload loads the model before forking workers.
• Model cache: $TRANSFORMERS_CACHE in env, set to /tmp/hf_cache. After initial download, TRANSFORMERS_OFFLINE=1 prevents re-download on dyno restart where the cache persists.

Decision 8: Cost Model

Full table: docs/architecture/reasonator/cost-model.md

Heroku wins at Phase 1. ECS wins at >5k req/day sustained. Lambda never wins for this workload.

Decision 9: Failure Modes and Circuit Breaker

Circuit breaker (Raptor-side): - Threshold: 5 failures in 60 seconds → circuit opens. - Open state: all Reasonator calls return immediately with {"sentiment": null, "reason": "rack_unavailable"}. - Cache: Raptor caches the last successful Reasonator response per (symbol, window_start, window_end) for 15 minutes. On circuit open, stale cache is served with stale: true. - Half-open: after 120 seconds, one probe request. Success → closed. Failure → reset timer.

What is never blocked by Reasonator being down: Order entry, position display, backtesting, any non-sentiment surface.

Decision 10: Repo Organization

reasonator/
  app.py                  # Flask app factory
  config.py               # Env-driven config (no hardcoded values)
  scorer/
    finbert.py            # FinBERT load-once + score_batch
    rescore.py            # Re-scoring sweep logic
  routes/
    score.py              # POST /v1/score, /v1/score/batch, /v1/score/rescore
    health.py             # GET /v1/health
  middleware/
    auth.py               # Bearer token check
    rate_limit.py         # Tier-aware rate limits
    audit.py              # Audit log emitter
  jobs/
    batch_worker.py       # Background job processor
  tests/
    unit/
    integration/
  Procfile                # web: gunicorn reasonator.app:create_app() --preload ...
  requirements.txt        # Python deps for reasonator only
  runtime.txt             # python-3.11.x
  .env.example            # Documented env var names (no values)
.github/
  workflows/
    deploy-reasonator.yml       # Heroku deploy workflow (ADR-0053 structure)
    reasonator-keep-alive.yml   # Cron keep-alive ping

Separate Heroku apps: reasonator-raxx (prod), reasonator-raxx-staging (staging).

5. Data Model

Reasonator is stateless. All database writes go through Raptor (which owns sentiment_events).

Reasonator's in-process state: - Loaded FinBERT pipeline (module-level singleton, loaded at startup) - In-memory job queue (async batch jobs) - Job result cache (TTL 10 minutes, keyed by job_id)

New Raptor-side table: sentiment_score_audit

CREATE TABLE sentiment_score_audit (
    id                    INTEGER PRIMARY KEY AUTOINCREMENT,
    sentiment_event_id    INTEGER NOT NULL REFERENCES sentiment_events(id),
    scored_at             TEXT NOT NULL,          -- UTC ISO 8601
    scorer_model          TEXT NOT NULL,
    scorer_model_version  TEXT NOT NULL,
    sentiment_label       TEXT NOT NULL,
    sentiment_score       REAL NOT NULL,
    sentiment_confidence  REAL,
    score_source          TEXT NOT NULL           -- 'rack_sync' | 'rack_batch' | 'rack_rescore'
);
CREATE INDEX idx_ssa_event_id ON sentiment_score_audit (sentiment_event_id, scored_at);

Rollback: drop sentiment_score_audit. No impact on existing tables.

6. Tier Surface

Pro tier (retrospective, low priority)

Raptor's background scheduler calls POST /v1/score/batch with unscored sentiment_events row IDs. Reasonator processes in background; Raptor polls every 30 seconds.

User-facing: "During your AAPL position (2026-04-14 to 2026-04-18), sentiment was neutral (+0.08). 12 articles. Here are the top headlines."

Pro+ tier (real-time, high priority)

When a Pro+ user opens a trade ticket, Raptor calls POST /v1/score with the last N hours of ingested headlines for that symbol. Reasonator scores synchronously within 2s.

Pro+ alert rules: "Alert me when AAPL sentiment flips negative on positions over $10K." Raptor evaluates the rule against the Reasonator score. If triggered: email fact-surface ("AAPL sentiment is currently negative — you have $X in open positions"). No recommendation. No trade proposal. User decides.

7. Feature Flags

• rack_enabled — gates all Raptor→Reasonator calls. false → {"sentiment": null, "reason": "feature_disabled"}.
• rack_pro_realtime_enabled — gates the Pro+ sync path. false → Pro+ falls back to async path.

Both require console_flag_promotions migration rows per B1 enforcement. Scaffold sub-card creates these rows.

8. Migrations

The re-scoring sub-card adds sentiment_score_audit as migration 015. Migration 014 (sentiment_events) is the existing Phase 1 schema card (#1381).

9. Rollout Plan

Paper-profitability gate does not apply — Reasonator is a data surface, not an execution path.

10. Security Considerations

11. Open Questions