ADR-013: audit.ndjson format + retention policy

Date: 2026-05-25 Status: Accepted Author: Rômulo Febasi

Context

The pending queue (SYN-60, BR-12.3) gives the human gate over AI writes. What it does not give is an immutable record: the user can syn pending reject --prune a row out of existence. For regulatory contexts (EU AI Act Article 50, internal compliance audits, post-hoc investigations) we need a tamper-evident, append-only log of every decision that ever was: distinct from the queue, which is working state.

Synapse already commits markdown writes inside Workspace::tx (BR-6.1 atomicity). The audit log is a different beast: it must be writable when the markdown write succeeds, and the user must not be able to silently edit it.

Decision

Every pending-queue decision (accept / reject / prune) writes one JSON object to .synapse/audit.ndjson before the corresponding queue mutation, except for accept where the workspace apply happens first (see the ordering trade-off below). The schema:

{
  "id": 7,
  "kind": "capture_fact",
  "decision": "accepted",
  "proposed_at": "2026-05-25T15:00:00Z",
  "decided_at": "2026-05-25T15:30:00Z",
  "source": "mcp:capture_fact",
  "reason": "user-supplied justification",
  "payload_sha256": "sha256:f8e2c1…"
}

Why ndjson and not a JSON array: append-only with O_APPEND, no tail bookkeeping, line-grep friendly, robust to crashes (a half-written trailing line never corrupts the rows above it).

Rotation kicks in on the next append after the live file ages past [audit] retention_days (default 90). Rotation moves stale entries into .synapse/audit-archive/<yyyy-mm>.ndjson then rewrites the live tail atomically (temp file + rename). The live file is never auto-deleted; syn audit prune --before YYYY-MM is the only path to physical removal, and it only touches the archive directory.

Ordering trade-off: “incomplete trail” beats “lying trail”

The trail must never claim a workspace mutation happened when it didn’t:

• accept: apply_* (workspace mutation) → audit::append → mark accepted. If the append fails after a successful apply, the queue row stays pending. Next syn pending list shows it; the user retries. The workspace is mutated but the trail is shorter than reality; that is recoverable. The inverse (trail says “accepted” but the workspace mutation failed) is not.
• reject and prune: audit::append → mark. There is no workspace mutation to fail; the trail comes first.

Payload hash, not payload

The audit row carries sha256:<hex> of the proposal payload, not the payload itself. The payload lives on pending.payload until the user prunes the queue row; the hash forever links the trail entry to its original content. syn audit replay <id> (SYN-113) joins the two when the pending row still exists; otherwise it reports unavailable and the hash alone proves what was decided without revealing it.

Unparseable rows are preserved, not dropped

Rotation reads lines, attempts to deserialise each one, and keeps anything it can’t parse in the live file. The audit trail must never silently lose bytes: a corrupted line stays around as a forensic artifact, not as a deletion.

Consequences

• + Tamper-evident audit history available as a single git diff .synapse/audit.ndjson (the file lives inside .synapse/ but is not excluded from git: workspace owner can choose to commit or gitignore based on their policy).
• + Rotation keeps the live file small without bloating the workspace; archives are partitioned by month for cheap grep.
• + Independent of the queue: syn pending reject --prune cannot erase the historical record.
• − Two write surfaces (queue row + ndjson line) introduce a small window where one succeeds and the other fails. Mitigated by the explicit ordering above; never silently incorrect.
• − Concurrent processes appending to the live file at the same millisecond rely on O_APPEND atomicity guarantees of the OS. POSIX guarantees this for line-sized writes (< PIPE_BUF). Documented; callers should not write multi-MB lines (BR-7.3 caps proposal size anyway).
• − No cryptographic chaining yet (each line is independent). A v0.5 extension can add a prev_hash field for a Merkle-tree audit trail if downstream tools demand it.

Alternatives considered

• JSON array file. Rejected: needs tail bookkeeping, breaks on crashes, harder to grep.
• SQLite table. Rejected: pending is already SQLite; the audit trail needs to be readable by non-Synapse tools (compliance reviewers running jq, archiving systems doing daily backups). A flat ndjson is the universal interchange.
• Auto-purge on retention. Rejected: the archive must survive until the user says otherwise. syn audit prune is explicit.
• Cryptographic Merkle chain in v0.3. Deferred: adds complexity without a current consumer asking for it. Slot reserved for a future ADR.