In 2016, four researchers at TU Berlin published a short workshop paper with a plain title: "Bridging the Gap: Towards Optimization Across Linear and Relational Algebra." It describes a problem every data team knows. You clean and join your data in one kind of language, something like SQL, built on relational algebra. Then you train your model in another kind, built on linear algebra and matrices. Two languages, two systems, and a seam between them where good things go to die.
The paper names three failures at that seam. You author the pipeline in multiple languages and glue them with extra code. You materialize intermediate results at the boundary, and unless the staging format carries meta-information about the physical layout, that information is simply lost. And because each language keeps its own internal representation, optimizations that need to see both sides at once become impossible.
Their fix, a language called Lara, refuses to translate. It lifts both algebras into one intermediate representation where a single optimizer sees the whole program. Filters applied to a matrix get pushed down into the tables the matrix came from. A join and the matrix operation that follows it share one partitioning of the data instead of shuffling everything twice. The paper compiles to Apache Flink, which came out of the same Berlin lab and repeated the same unification at engine scale: one runtime for batch and streaming instead of two systems pretending to agree.
I read this paper the way I read most systems papers now: as a description of knowledge work. The three failures are not about databases. They are about boundaries. And they recur, layer by layer, all the way up the stack I live in.
The same seam, four more times
Documents. Every note in my vault carries YAML frontmatter: status, audience, conditions, publication targets. That frontmatter is where I declare intent about the document. And in a standard knowledge-graph intake pipeline, it's the first thing stripped. The layer where the user encoded meaning is discarded by the system that most needs it. That is the staging-format failure, moved from physical layout to meaning.
Prompts. Whatever voice and structure you tuned into an AI system last run tends to die before the next one, so teams re-paste rule blocks between workflows like plumbing code. Google's dotprompt format is the Lara response: a prompt file whose frontmatter carries the model config, the input schema, and the output schema, so the contract rides with the prompt and gets enforced at generation time rather than audited afterward.
Reports. We hit this seam building ShurIQ. The analysis layer and the editorial layer are different languages. For a while, the writing rules were a block of text re-pasted into every build, and violations were caught by auditors after rendering. This month we moved the structural half of every rubric into the modules' own execution contracts. A constraint declared once now enforces itself in every report that calls the module. Database people have a name for this: push-down.
Identity and participation. This is the layer Totem Protocol works on. What you would agree to, at what price, under which conditions, with what tolerance for other people's category systems: today that lives in your head, and every platform gets a two-line bio and a settings page instead. The declaration doesn't exist in any form a system could carry. So your agents can't act on it, your collaborators can't check it, and every new context makes you restate it by hand.
If you want proof this last layer is buildable with today's infrastructure, it's already running in mainstream data stacks. Flink SQL's MATCH_RECOGNIZE lets you declare a pattern once, and the engine watches an unbounded stream on your behalf, forever, without you re-asking. A standing query is declared intent compiled into an observable machine state. Twenty lines of SQL. The identity version is the same shape: your conditions, declared once, watched continuously, carried by agents into every operation that touches your stuff.
What the winners share
Look across the substrates that actually won and a pattern shows up three times.
The contract has to live in the same plain medium as the work. SOAP shipped the most contract-forward architecture in history and lost to REST, because its contracts were generated XML only specialists could read. Frontmatter, dotprompt files, and Kubernetes manifests are all human-editable text sitting next to, or inside, the thing they govern.
Something has to run on the declaration. Cory Doctorow called voluntary metadata "metacrap" in 2001, and he was right about every descriptive-only layer: the HTML meta tag died of spam, and the universal Semantic Web died of neglect. But schema.org markup stays accurate because Google executes it into search results. Kubernetes calls an object a "record of intent" and pays an army of controllers to reconcile declared state with observed state. A declaration nothing reads is decoration.
Revision has to be cheap. The DAO raised $150 million into an immutable contract in the spring of 2016, lost about $50 million to a recursive-call bug on 2016-06-17, and the only available edit was a hard fork that split the community. Intent changes. A substrate that can't absorb the change re-creates the original disease one level up: yesterday's declared intent pretending to be today's.
Legible contract, live enforcement, cheap revision. Miss any one and you get SOAP, or metacrap, or The DAO.
Propulsion instead of combustion
Here's why this compounds. If ten groups each keep a private ontology, connecting them pairwise costs about forty-five translations, and every translation burns meta-information at a boundary. If ten groups adopt one shared discourse grammar, it costs ten adoptions, and each new member makes the shared graph more valuable rather than more expensive. Economists formalized this in 1985 as network externalities. Combustion spends the difference between systems as heat. Propulsion turns a shared medium into motion.
That's what we're building toward with ShurIQ and Totem Protocol: knowledge graphs made of knowledge graphs, connected through a shared discourse grammar, where each brand, municipality, and NGO keeps custody of its own graph and declares the conditions of its participation. The conditions are the interesting part. Careful, domain-specific ontologies let rules become gates: I share these annotated sources and these scripts at these prices, under these constraints, and I'll accept your alternative taxonomy so long as it never requires a declaration from me beyond the scope I've listed. Precision about participation is what makes pooling resources safe enough to do at all.
None of the primitives are ours, and that's the point. The economic semantics were standardized as ISO/IEC 15944-4. Decentralized identifiers and verifiable credentials became W3C Recommendations in 2022 and 2025. Elinor Ostrom documented the governance pattern in commons that have run for centuries: clear boundary rules, rules matched to local conditions, watchers, and graduated consequences. What doesn't exist yet is the compiler that keeps all of it current while you live your life. That's the product. That's also, not coincidentally, the part the 2016 paper would recognize: one representation, one holistic view, declarations that survive every boundary.
Where this goes next
This post opens a pathway in the Declarative Intent series, and it comes with homework I actually want you to do. The Wayfinding Edition program works backwards from the far future to your laptop: you'll build a swarm simulation whose mission lives in a declaration file rather than in code, stand up an Apache Flink standing query that watches a stream on your behalf, turn a note's frontmatter into an enforced publishing gate, and write a machine-readable participation charter for a real collaboration. Four small artifacts. Each one is the same lesson at a different scale: encode the intent, keep it alive, and let the system carry it.
The paper's title was better than its authors knew. The gap was never just between two algebras.
---
Jonny Dubowsky is co-founder of Sense Collective and architect of Totem Protocol. He has been designing multi-agent systems and collective intelligence frameworks since 2004.