Hyphae
GitHub

01 / Architecture

How Hyphae works

One process. One data directory. One catalog, WAL, MVCC snapshot space, commit sequence, scheduler and object-ID namespace underneath SQL, native structures, lexical search and vector search. This page walks the write path, the read path, durability and recovery, proofs, and the formal model that checks the commit protocol — then states plainly what Hyphae is not.

A — The shape of one process

Every engine — SQL, structures, lexical search, vector search — reads and writes through the same catalog, the same write-ahead log, the same MVCC root set, the same scheduler, and the same stable object-ID namespace. There is no per-engine commit path, no internal network hop between engines, and no separate database process for any of them.

A committed cross-engine transaction is visible at exactly one commit sequence number (CSN) in every engine, or in none of them — this is the system claim the rest of the product is built on.

One process, one commit sequence SQL, structures, lexical search and vector search sit over one shared catalog, WAL, MVCC snapshot space and commit sequence inside one Hyphae process. A cross-engine transaction commits at exactly one CSN in all of them or in none. ONE HYPHAE PROCESS SQL indexed, fail-closed core Structures keyspace, TTL, algebra Lexical BM25 / BM25F Vector exact + ANN ONE CATALOG · ONE WAL · ONE MVCC SNAPSHOT SPACE ONE COMMIT SEQUENCE (CSN) · ONE SCHEDULER · ONE RECOVERY A transaction across SQL + structure + search is one commit, not a saga — visible at one CSN everywhere, or nowhere. RESULTS COME WITH PROOFS VERIFIABLE OFFLINE

Source: canonical claims and non-claims, "the system claim".

B — The write path

A request stages its mutations privately: a relational overlay keyed by row, a scalar-structure overlay keyed by key, a lexical overlay keyed by search collection and document ID. None of this reads or copies unrelated rows, keys or documents — it resolves only the exact identities the transaction touches, from a captured immutable snapshot.

Commit re-resolves those staged identities against the currently admitted roots, applies the deltas with copy-on-write page mutation, encodes one canonical WAL transaction, applies the selected durability policy, and publishes every changed root once through one commit coordinator. The delta all-engine transaction contract's scaling sweep shows this cost stays flat — 9 page reads and 3 WAL appends per commit — from 1 prior version up to 1,024 prior versions of the same row.

The write path A request stages mutations privately against SQL, structure and search overlays, then one WAL append and one root publication make one commit sequence number visible in all engines at once. Request SQL + structure + search in one txn SQL overlay private, keyed by row Structure overlay private, keyed by key Lexical overlay private, keyed by doc ID One WAL append 9 page reads, 3 appends at any version depth Root publication copy-on-write CSN 42 CSN 42 IS VISIBLE TO SQL, STRUCTURE AND SEARCH AT ONCE — OR TO NONE OF THEM

Source: delta all-engine transaction v1, commit admission and publication; sweep numbers from the a443c52 receipt, §5.

C — The read path

A transaction begins by capturing one immutable snapshot: a visible CSN, a catalog version, a logical-time sample, and a root set. Every read inside that transaction — SQL, structure, or search — resolves against that same snapshot plus the transaction's own private writes. A later commit publishing a new CSN cannot change what an in-flight snapshot sees.

This is snapshot isolation with first-committer-wins over logical write identities and read-your-writes inside a transaction. It is not serializable: predicate and range conflicts are not detected, so write skew is admissible exactly as in any snapshot-isolation system.

The read path A transaction begins with one immutable snapshot fixed at a commit sequence number. SQL, structure and search reads all resolve against that same snapshot, so they see the same visible state. Snapshot visible_csn = 42 fixed at BEGIN root_set + catalog_version SQL read sees CSN 42 Structure read sees CSN 42 Search read sees CSN 42 SAME COMMIT SEQUENCE No engine can read ahead of or behind another. A later writer publishing CSN 43 never changes this snapshot's result mid-transaction. Snapshot isolation, first-committer- wins, read-your-writes inside a transaction. Not serializable: predicate/range conflicts are not detected.

Source: native MVCC and commit semantics v1 and the isolation claim.

D — Durability and recovery

Every commit picks one durability class. Strict acknowledges after this transaction's own WAL fsync. Group acknowledges after a shared cohort fsync. Memory acknowledges without any fsync — an acknowledged Memory commit can be lost by a crash, but is never torn: recovery only ever drops whole commits from the volatile WAL suffix.

On dedicated hardware, an identical single-SET commit measures 4.39 ms p50 Strict and 3.86 ms p50 Memory; a group-commit cohort of 8 producers reaches 12.4 ms p50 at 2.5× Strict's throughput. Recovery itself has two outcomes only: interrupted before PageSynchronized reopens the prior state; interrupted from WalAppended through RootPublished reopens the complete new state. No boundary in between is ever observable.

Durability and recovery boundaries A commit walks from WAL append through page synchronization to root publication. A crash before page synchronization reopens the prior state; a crash from WAL append through root publication reopens the complete new state. No boundary exposes a mixed engine state. WalAppended PageSynchronized RootPublished CRASH ANYWHERE IN THIS SPAN → Reopens the prior state Before PageSynchronized: replay truncates the volatile WAL suffix. The commit never happened. Strict fsync means an acknowledged commit is past this boundary before it acknowledges. Reopens the complete new state From WalAppended through RootPublished: recovery replays the WAL forward and republishes the same roots. No boundary in between is ever observable: the commit is whole in every engine, or absent.

Source: durability classes from the claims page; ablation numbers from the a443c52 receipt, §4; crash boundaries from delta all-engine transaction v1, "failure and crash semantics".

E — Proofs

An eligible read — catalog, SQL, or a native product read — can emit a canonical HYNPRF02 proof plus a witness. The producer checkpoints the exact root that produced the result while it holds the directory lock; the proof binds directory lineage, visible CSN, catalog version and root digest into a domain-separated envelope.

Verification is not self-consistency: it requires an ExternalTrustedAnchor obtained independently of the proof and witness. hyphae proof verify runs on another machine, with no directory access and no network call — it re-executes the operation under bounded reference semantics and compares the exact result.

Proof and offline verification A producer holding the directory lock generates a result proof and witness bound to the exact root that produced the result. A verifier on another machine, holding only an independently obtained trusted anchor, re-executes the operation and compares the exact result — no directory access and no network. Producer holds the exclusive directory lock checkpoints the root that produced the result Proof + witness HYNPRF02 envelope canonical request, ordered result, execution evidence BLAKE3 + CRC32C bound query.hynproof / .hynwitness TRANSPORTED ANYWHERE Verifier another machine, no directory access, no network holds only a 32-byte trusted anchor RE-EXECUTES UNDER BOUNDED REFERENCE SEMANTICS AND COMPARES STATUS: VERIFIED SELF-CONSISTENCY IS NOT TRUST — THE ANCHOR MUST COME FROM OUTSIDE THE PROOF

Source: native result proof v2 and the usage manual, "verifiable proofs".

F — The formal model

The cross-engine commit protocol has a machine-checked TLA+ model, HyphaeCommit.tla, run with TLC. It mirrors the ordered commit boundary walk, serialized writer admission with first-committer-wins, per-transaction durability classes, sequential WAL recovery with broken-tail truncation, and conflict-table reconstruction — the same state machine the physical crash matrix exercises directly.

The 2026-08-30 run generated 79,063,806 states (43,885,299 distinct, depth 30) and held all six checked invariants: TypeOk, Atomicity, StrictDurability, FirstCommitterWins, VisiblePrefixComplete, and CsnBounded. The 2026-09-03 re-measurement reproduced the identical state counts against the exact spec, config and tool SHA-256 digests. The model checks the protocol as specified — it is evidence about the design, not a proof of the Rust implementation; implementation fidelity is carried separately by the physical crash matrices.

G — What Hyphae is not

Bounded on purpose.

Not universal SQL The admitted grammar is versioned and bounded: no subqueries, no UNION, no outer joins, no expression arithmetic. A plan that cannot bind to an index fails closed (HYSQL011) instead of scanning.
Not distributed Single-node, single-writer publication. No replication, no clustering, no distributed transactions, no consensus, no multi-tenant kernel.
Not a RESP surface Structures share Hyphae's own transaction and TTL semantics. There is no Redis wire protocol in the native product.
Not serializable Snapshot isolation with first-committer-wins. Predicate and range conflicts are not detected; write skew is admissible exactly as in any snapshot-isolation system.
No embedded LLM The engine does not embed PostgreSQL, Valkey, OpenSearch, a cloud service, an embedding provider, or an LLM. Agent Memory stores explicit, conservative, PII-rejected memories — it does not run a model.
Fails closed, not open An unsupported SQL shape, an oversized batch, or a memory-ledger overrun is rejected before mutation — never silently downgraded to a scan or a partial write.

Wording authority: docs/product/claims.md, "prohibited claim shapes" and the topology, isolation and system claims.