Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

ADR-028: Typed-but-extensible entity types: a validation registry

Date: 2026-06-12 Status: Accepted (owner-delegated review, 2026-06-12) Author: Rômulo Febasi

Context

Synapse has six hard-coded entity types (project, person, entity/org, api, task, note) wired across CLI, MCP, store, and index. The query tool (ADR-023) deliberately hard-codes the columns its group_by/filter may touch and binds all values, making SQL injection and internal-table exfiltration impossible by construction. Users and agents increasingly want new entity types (e.g. book, meeting, decision, client) without a code change.

The tension: add extensibility now, yet still freeze a stable schema at v1.0 (SYN-104), and without reopening the injection surface ADR-023 closed. This is a one-time cross-cutting refactor; doing it before bitemporal (ADR-027) and before the freeze avoids paying for it twice. Implemented in SYN-234.

Decision

Build a closed-world validation registry, not an open-world ontology. A type definition lives in .synapse/types.toml. Conceptually a type is a row-polymorphic open record over a closed core: a small fixed set of indexed/queryable columns (the part ADR-023 protects) plus an open frontmatter bag validated on read. “A function can only project fields it explicitly names”: enforced at the DB boundary.

schema_version = "1.0"

[types.book]
display = "Book"
extends = []                       # reserved for v1.x composition; [] = no-op in v1
store_dir = "books"
indexed = ["title", "author"]      # WHITELIST → these become bindable query keys

  [types.book.fields.title]
  type = "string"; required = true
  [types.book.fields.author]
  type = "ref"; target = "person"; required = false
  [types.book.fields.isbn]
  type = "string"; pattern = "^[0-9-]{10,17}$"

Injection safety is preserved by construction:

  • The SQLite entities table keeps its fixed columns; user fields never become new columns. Every entity is one row regardless of type.
  • A query’s kind is checked against the registered slug set (a HashSet lookup) and then bound as a parameter: never string-interpolated. Unknown kind → rejected before SQL.
  • indexed fields project into a narrow whitelisted entity_fields(entity_id, field_name, value) table; group_by/filter resolve field_name against the registry’s indexed list for that kind (a closed in-memory set) and bind it as a value parameter (WHERE field_name = ?1 AND value = ?2). Column identifiers stay static; only values are dynamic. ADR-023’s invariant is not reopened.
  • Non-indexed fields are schema-on-read only: validated on accept, never queryable via SQL, zero injection surface.

Validation rules (deterministic, offline, pure): reserved names (the six built-ins + self/tag/audit/relation); slug grammar ^[a-z][a-z0-9_]{0,31}$ (also a safe SQL/path token by construction); closed field-type enum (string|int|float|bool|date|ref|list<T>); ref target must be a registered type or *; constraints limited to deterministic ones (regex under a step budget, numeric/length bounds, enum value lists: no executable validators, no network); indexed ⊆ fields, |indexed| ≤ 16. Validation is a pure function of (types.toml, frontmatter), stamped like provenance (ADR-012) so accept/replay is reproducible.

Migration = expand-contract (Avro/Protobuf/Confluent discipline): adding a type is additive → MINOR, forever-legal post-1.0 (user-type space never bumps the major); a new field on an existing type must be optional-with-default; a retired slug is never reused (reserved). The v1.0 freeze covers only the closed core: the six built-in kinds’ semantics, the fixed entities columns, the query contract, and the types.toml format version. Freeze the spine, not the skin.

Consequences

  • Improves: extensibility now and a clean v1.0 freeze; nominal safety (kinds are validated names) + structural extensibility (open frontmatter) + determinism + a clean SemVer story; injection safety unchanged. Pays the enum→registry refactor once, before bitemporal and the freeze.
  • Worsens: one cross-cutting refactor (replace the hard-coded Kind enum with a registry-resolved kind loaded at startup); an EAV join for indexed fields; reserved-slug discipline to maintain.
  • Resolved (2026-06-13, SYN-234 slice 2): EAV vs per-type generated columns for indexed fields → EAV (entity_fields(type, id, field_name, value), indexed on (field_name, value)). The spike (docs/design/04-syn234-eav-spike.md) confirmed both query patterns (point filter, group-by) are index-backed at 10k scale and that binding field_name as a value preserves ADR-023 more strongly than columns would. EAV keeps types.toml purely additive (no DDL on a new type) and fits the disposable/rebuildable index; generated columns only win at a scale Synapse does not target and are a later, backward-compatible optimisation if SYN-227 ever shows a need.
  • Open: ref validation eager (target must exist) vs advisory (dangling allowed, flagged by doctor --lint-provenance): recommend advisory, consistent with current LSP dangling-ref diagnostics and bulk-import ordering; extends composition stays a reserved no-op in v1, single-level last-wins in v1.x; per-type governance policy (a type declaring its own write-policy / pii flag, composing with ADR-022) is split out as SYN-248.

Alternatives considered

  • OWL / Description-Logic reasoner. Open-world inference adds facts to conform rather than reporting errors: the opposite of what a bounded local KG needs. SHACL-style closed-world validation is the right paradigm. Rejected.
  • Arbitrary user SQL / a query DSL over dynamic columns. Exactly the surface ADR-023 closed. field_name never reaches SQL as an identifier. Rejected.
  • Executable / Turing-complete validators (user lambdas, plugin code, network-fetching constraints). Break determinism and the offline guarantee. Fixed constraint vocabulary only.
  • RDF triple-store rewrite. Markdown-as-truth + disposable SQLite is the architecture; a TOML registry suffices for typing without a triple store.
  • Full structural typing / multiple-inheritance diamonds at v1. Defer; linear single-extends is 90% of the value at 10% of the cost.
  • An Avro-grade automated data-migration engine. Expand-contract by convention + reindex is enough for a single-user local KG.

Research grounding

SHACL (closed-world validation) vs OWL (open-world inference); row polymorphism / extensible records (arXiv 1707.07872; ACM Rows by Any Other Name); nominal vs structural typing; Confluent/Avro/Protobuf schema-evolution rules; SemVer 2.0.0; Tana supertags (composable types, the extends north star); Anytype/Capacities (local-first user-defined types ship); Obsidian properties + Propsec (the freeform-drift cautionary tale).