Credential-Boundary Test Pattern

Status: Adopted
Date: 2026-05-10 UTC
Refs: #1535 (immediate fix), PR #1501 (SC-A1), PR #1500 (RV-1), PR #1465 / #1502 / #1516 (SC-A*), #1552 (Postgres CI)

What credential-boundary tests are

A credential-boundary test exercises a hard isolation invariant: that one role, token, or identity cannot reach data or operations reserved for another.

Concrete test surfaces in this repo:

These tests need stricter hygiene than typical unit tests because:

• They mutate role state in shared or per-test databases.
• They handle credential strings (even test-only ones) that must not appear in CI logs, test output, or artifacts.
• A sloppy assertion can silently pass when the boundary is absent — the test must assert the negative case too (access denied).

The canonical pattern

Fixtures

# conftest.py (backend_v2/tests/integration/)

import os
import pytest
from contextlib import contextmanager


@pytest.fixture
def credential_env(monkeypatch, tmp_path):
    """
    Factory fixture: yields a context manager that injects test credential
    env-vars and returns a scratch path, then asserts full cleanup.

    Usage:
        def test_role_resolution(credential_env):
            with credential_env("raptor_app") as env:
                assert env["db_url"].startswith("postgres://raptor_app:")
    """
    @contextmanager
    def _make(role_name: str, extra_env: dict | None = None):
        db_url = f"postgres://{role_name}:testpass@localhost:5432/testdb"
        monkeypatch.setenv("RAPTOR_APP_DATABASE_URL", db_url)
        if extra_env:
            for k, v in extra_env.items():
                monkeypatch.setenv(k, v)
        scratch = tmp_path / role_name
        scratch.mkdir()
        try:
            yield {"db_url": db_url, "scratch": scratch}
        finally:
            # monkeypatch auto-restores env after the test.
            # Assert scratch is clean so tests don't silently leave state.
            remaining = list(scratch.iterdir())
            assert remaining == [], (
                f"credential_env({role_name!r}) left files in scratch: {remaining}"
            )
    return _make

Three rules, applied everywhere in SC-A and RV- test files:

1. tmp_path for all filesystem state. Never use a hardcoded /tmp/... path. tmp_path is pytest-managed, unique per test invocation, and cleaned up automatically. This resolves the B108 finding in #1535 directly.

2. monkeypatch.setenv for all env-var injection. Auto-restored after each test regardless of pass/fail. Never os.environ[k] = v directly.

3. pytest-postgresql or testcontainers for Postgres-specific behavior. Use when the test depends on RLS, partial-index predicates, Postgres-specific DDL, or role GRANTs that SQLite cannot emulate. Connects to #1552 which adds Postgres-aware migration CI. SQLite-only tests remain SQLite — don't over-provision.

Teardown assertions

Every credential-boundary test class or module should include an explicit negative assertion after teardown:

def test_raptor_app_cannot_read_owner_table(credential_env):
    with credential_env("raptor_app") as env:
        # Exercise the boundary
        with pytest.raises(PermissionError):
            _query_as_role(env["db_url"], "SELECT * FROM _owner_only_table")
        # Positive path still works
        result = _query_as_role(env["db_url"], "SELECT 1")
        assert result == 1
    # No additional teardown needed: monkeypatch + tmp_path handle it.

The negative assertion (the pytest.raises) is mandatory. A test that only checks the happy path does not prove the boundary exists.

Anti-patterns

1. Hardcoded /tmp/... paths.
The original finding in #1535. Bandit B108 flags these. The fix is tmp_path, not # nosec B108. Never paper over this with a suppression comment — fix the root issue.

2. Tests that mutate the dev database without explicit teardown.
Use pytest-postgresql or testcontainers for an ephemeral Postgres instance. The dev DATABASE_URL is not a test database.

3. Implicit test-order dependencies.
tmp_path makes each test hermetic. If a test relies on state from a previous test, it is broken by definition. Fixtures must be self-contained.

4. # nosec suppressions for B108 or B310.
These mask real violations. Bandit suppression comments are not acceptable in credential-boundary tests. Fix the path, don't suppress the scan.

5. Logging or printing raw credential strings.
Even "postgres://raptor_app:testpass@..." is a credential. Use pytest-redacted or mask at assertion time. Do not let credential strings appear in captured stdout/stderr (which CI logs and artifacts retain).

Invariants this pattern enforces

• No filesystem side-effects survive a test (tmp_path + teardown assertion).
• No env-var mutations survive a test (monkeypatch).
• No credential string appears in test output (masking rule).
• Every boundary test exercises both the positive and negative case.
• B108 / B310 bandit findings are zero-tolerance in this test surface.

Reference

Note for security-agent (#1535 implementation)

The tmp_path refactor in #1535 is fully consistent with this pattern — tmp_path is the exact fix for the B108 finding at line 41 of test_raptor_app_role_separation_1481.py. The credential_env factory fixture above is an optional follow-on abstraction for reuse across the SC-A and RV- test slates. Security-agent may adopt it now or file a follow-up sub-card; the immediate #1535 fix is just the tmp_path swap.

No structural divergence from security-agent's approach — this doc endorses the same fix. The factory fixture is additive, not a prerequisite.