A method, and what it found in 262 tables

Rivering a database

Two hundred tables, and nobody can say what half of them are for. Rivering is deciding — once, on purpose — what each one is: every table turns out to be a noun or a river, taking one of a few named shapes, and that tells you what you may do to it.

A table that has been rivered answers for itself. An unrivered one can only be guessed at — and guessing is where the damage comes from, every time.

Four documents · which one you want
  1. Rivering, step by step
    the whole idea from nothing, in forty-two short steps. No prior knowledge. Start here if it is new to you.
  2. Rivers and Their Directionyou are here
    the reference. Every kind and archetype with its own rules, the ladder, the checklists. For looking things up once you know the idea.
  3. What Rivering Proves
    the technical companion. Which claims are theorems, which are empirical, which are only preference — and what would refute each.
  4. The Falsification Record
    what we tried to break, what survived 270 real tables, and the four defects the testing found.
01

The shop with two answers

A shop writes every sale in one notebook. Then someone also writes today's total on a whiteboard.

Now two things answer "how much did we sell today?" For a while they agree, and the whiteboard is genuinely useful — reading one number beats adding up a hundred lines.

Then one day they disagree. A sale went in the notebook while the whiteboard was being wiped. And here is the part that matters: nobody in the shop can tell which one is wrong. Both were written by hand. Both look equally official. The disagreement isn't the problem — the problem is that there's no way to settle it.

The idea everything follows from

Keep the notebook. Throw away the whiteboard. Add up the notebook when you need a total — and if you must keep a whiteboard for speed, it may never be written on by hand, only recopied from the notebook.

There is a second cure, and it is worth knowing because sometimes it is the only one available. The shop is stuck because nothing recorded the writing. If every write — to the notebook and to the whiteboard — were itself logged, the disagreement stops being symmetric: you can see which write landed and which didn't, and decide. That is what an audit trail buys, and it is why a system with a complete one survives a shape that would otherwise be unrecoverable. It is the more expensive cure. Prefer one original where you can have one.

02

Two things the word “direction” means

These get confused constantly — including by us, while working this out. They are unrelated ideas and both matter.

MEANING A — WHAT THE RIVER IS ABOUT the contact diary every line is about… a person the audit log every line is about… who did it the email log every line is about… one message Rivers sit side by side. Nothing flows between them. Each simply points at a different subject. MEANING B — WHICH WAY COPIES MOVE it happened the river the original totals · lists · maps NEVER BACK Everything on the right can be deleted and recomputed. Nothing on the right may be edited by hand — the moment it can, it stops being a copy and becomes a rival original.
Meaning A is what a river is for. Meaning B is a rule about copies. Using one word for both is what made this confusing the first time.

Meaning B is why the arrow matters. If copies only ever flow one way, disagreement has a cure: throw the copy away and recompute it. No investigation, no judgement, no meeting. If data can flow both ways, a disagreement has no cure — that's the shop with two answers, and it is the single most expensive shape in a database.

Meaning A is what tells you a new river is needed. When you find facts about some subject smeared across five tables with no log of its own, that subject wants a river.

On “the sea”

Worth saying plainly: the contact diary is not a sea that other rivers empty into. It is itself a river, pointed at a person. If the sea is anything, it's the whole system — the body of water the rivers run through. Rivers don't join. They run in parallel, each aimed at its own subject.

Which one is the original?

Meaning B tells you copies flow one way. It doesn't tell you which of two tables is the original — and standing in front of two things holding the same fact, that is the only question that matters.

You don't choose. What the rows are decides it.

“Levan entered the welcome sequence on Tuesday” has a time and a person. It's an event, so it's a river line. “Levan is currently in the welcome sequence” has no time and describes no occurrence — it's a state, and a state is what you get by reading events.

The rule that settles every one of these

The arrow always runs from the event to the state. An event is never derived from a state, because a state cannot say when, or how many times.

And the arrow holds for one reason: the derivation throws something away. “Is currently enrolled” is the same answer for someone who joined once and for someone who joined, left and rejoined — two different histories, one value, so nothing can run the arrow backwards. Which has a consequence worth stating outright: if a copy loses nothing, it is not derived at all. A table that retains everything the history contained is the history in other clothes, and it inherits a river's rules rather than a derived table's. Derivation presupposes loss.

This matters because the mistake is easy and expensive. If you decide the state table is the original, then the events look like a copy and get deleted — and once they're gone, nothing can rebuild them. The state can always be rebuilt from the events. Never the reverse. That asymmetry is what makes it a rule rather than a preference.

One thing the arrow does not promise: that a state is worked out from its own river. “Levan's last email open” is a fact about a person, but it comes from the river of email events — whose subject is a message, not a person. The chain runs: this message was opened, this message was sent to Levan, therefore Levan's last open. State is always worked out from events; not always from the events about itself.

02b

Which of two tables is the original? You do not choose.

When two tables hold overlapping answers, it feels like a judgement call about which one to trust. It is not. The direction is already fixed by what the rows say.

read the two sentences aloud

“Levan entered the welcome sequence on Tuesday” — has a time, has a person, describes an occurrence. An event.

“Levan is currently in the welcome sequence” — no time, no occurrence. A state.

The arrow always runs from the event to the state. An event is never worked out from a state, because a state cannot say when, or how many times. The state can always be rebuilt from the events; never the reverse. That asymmetry is what makes this a rule rather than a preference.

And it holds for one reason: the derivation throws something away

“Is currently enrolled” is the same answer for somebody who joined once and for somebody who joined, left, and rejoined. Two different histories, one value. Nothing can run the arrow backwards, because the information is simply gone.

The testDoes this copy lose anything the history had? If it loses nothing, it is not derived at all.

That consequence catches a whole class of mislabelled tables. A copy retaining everything the history contained is the history in other clothes, and it inherits a river's rules — never edited, kept for its retention — not a derived table's. Derivation presupposes loss. A table that has lost nothing has not derived anything; it has duplicated.

One thing the arrow does not promise

That a state is worked out from its own river. “Levan's last email open” is a fact about a person, but it comes from the river of email events, whose subject is a message.

State is always worked out from events — not always from the events about itself. Looking for the missing river next to the state, and concluding there isn't one, is how a perfectly ordinary derived table gets promoted to an original.

03

What a river needs

A river is not just "a table you only add to." Five decisions have to be made, and skipping any of them produces a log that looks healthy and cannot be used.

ONE
A subject

What every line is about. A person. An account. One message. This is the direction, in meaning A — and it is what decides whether you need a new river or not.

TWO
A unit

How big one line is. A conversation, or every keystroke in it? This is a real decision with no default.

THREE
A vocabulary

A fixed list of what a line may be called — and the identity of the thing kept in its own column, never inside the name.

FOUR
The rules

What may and may not be written in it. Below.

FIVE
Retention

How long it is kept. Three archetypes depend on this answer, and nobody writes it down.

Decision one — the subject, and how to settle it

When to add a river, and when not to

More detail goes into the river you already have, as more kinds of line. A new river is only ever for a different subject.

This one saves a lot of wasted work. Wanting to know more about a person — which lesson they opened, not just which course — is never a reason for a second river about them. It is a reason for more kinds of line in the one you have. You only reach for a new river when the subject itself changes: from what happened to a person, to what happened to an account, or to a message.

Which raises the obvious problem: “a different subject” depends entirely on what you call things. Call the subject “a message” and email and SMS are one. Call them “an email” and “an SMS” and they are two. A rule you can steer by choosing a word is not a rule.

The fix — don't name the subject, point at it

A river's subject is the noun its rows point at. Nothing is named, so nothing can be argued.

“Noun” here means a thing that exists in its own right — a person, a message, a course. It gets a full treatment as the first of the two kinds, in the next section; for now, a thing rather than an event.

Both of those message logs point at rows in the same notifications table. So the system already decided the subject, and it decided “a notification” — the moment somebody built one notifications table covering both channels. No word had to be chosen.

And it settles the harder cases the same way. Two campaign tables are two separate nouns, so campaigns are two subjects — and whether they should be one noun is a different question, about the nouns, which has to be answered first.

Two rivers, one subject → they are one river

If two logs point at the same noun, they are recording the same thing twice, in two places, with two vocabularies. Anyone asking “what happened to this?” has to remember to look in both — and one day somebody won't.

And it works in the other direction too

The events follow the noun. One noun, one river. To justify two rivers you must first argue the noun should be two tables — and win that on the noun's own terms.

Which is why the two message logs merge and the two campaign tables don't: there is one notifications table covering both channels, and there are two campaign tables, deliberately. Same rule, opposite answers, no special pleading needed.

One river, two subjects → they are two rivers

The reverse case, and it's subtler, because a single line legitimately mentions several nouns. “Levan paid £240 for the coaching package” touches a person and a product. That is not two subjects.

Telling the subject from a mention

Take the noun away and read the line again. Still means something? It was a reference. Means nothing at all? That was the subject.

Remove the product and “Levan paid £240” still says something. Remove Levan and “paid £240” says nothing — nobody paid. So the line has one subject and one reference, not two subjects.

It is genuinely two rivers when that test gives different answers for different lines — when some rows collapse without a person and others collapse without a domain. The tell is a subject column filled in for some rows and empty for the rest, where the empty ones are about something else entirely.

A pointer that sits on most tables is a scope, not a subject

In a multi-tenant system every row points at the tenant. “Rows per tenant” is multi-tenancy, not a unit.

Derive it rather than listing it: count how many tables each id column appears on. Here tenant_id sits on more than half of 270 tables, and no real subject comes close. This is not pedantry — measuring the unit against the scope read 595,089 rows per subject on the contact diary and classified a 1.3-million-row log as a heartbeat. Against its real subject it reads 83.

Decision two — the unit

Take a support conversation. Someone sends forty messages back and forth. Does the log get forty lines, or one?

The system this came from had already decided this, and decided well. It writes three — started, handed to a human, resolved — and each carries a count of how many messages there were.

Both extremes are wrong in ways that are easy to miss. Forty lines and one person's history is flooded; you can't see anything else that ever happened to them. No lines at all and you lose that they ever spoke to you. Three lines with a count keeps the shape of the conversation — you can still tell "asked once and left" from "went back and forth ten times" — without drowning everything around it.

The unit rule

The unit is the smallest thing a person would actually want to see on a timeline. For a conversation, that's the conversation — not the keystroke.

And it does not have to stay a matter of taste. A unit is too fine when one subject's rows crowd out everything else on that subject's timeline — and that is a number, not an argument.

Here is the scale, measured rather than guessed. Across 77 rivers and heartbeats in one real system, the 99th percentile of rows-per-subject ran: median 19, 75th percentile 72, 90th 202, 95th 953, worst 31,372. 83% sit under 100. Only three of the seventy-seven cleared 1,000 — and all three turned out to be heartbeats rather than rivers.

So: under 100 is ordinary. 100–1,000 is worth a look. Over 1,000 is almost certainly a heartbeat wearing a river's clothes. The contact diary runs at 83. Measure against the subject, never a scope.

Nothing else in this document answers the question for you. Every river needs its own answer, and it should be written down.

Decision three — the vocabulary

The contact diary has a fixed list of names — payment_recorded, zoom_joined, community_post_created — around 125 of them, each deliberately added. The audit log has no list at all. Its name column takes free text, and this is what got written into it:

A LINE THAT WORKS WHO WHAT HAPPENED KIND OF THING WHICH ONE levan SETTINGS_CHANGED TENANT a1f4…9c Countable — the name comes from a short list, so “how many settings changes this month?” has an answer. Filterable — the thing has its own column, so “everything that happened to this account” has an answer too. A LINE THAT DOESN'T levan POST /support/tickets/e4f5b5cc-fabd-485d-834e-3887fa2d66c0/typing empty empty
The columns for "which thing" exist and sit empty, while the ticket's identity is buried inside the name. Because every ticket has a different identity, every line is a different sentence — so nothing can be counted, and nothing can be filtered.

Decision four — the rules

  1. If it can be worked out from the river, don't store it. A second copy isn't a convenience — it's a second answer.
  2. A copy has no front door. It is only ever replaced by a rebuild — never edited, never nudged up by one.
  3. “It can't be worked out” is a symptom, not a verdict. It usually means the river is missing something. Fix the river, and the copy becomes ordinary again.
  4. Measurement is not story. The same thing sampled a hundred times belongs somewhere else. The river gets the conclusion.
  5. Only what happened to the person goes in. Ask whether they could have told you it happened themselves. “I registered” — yes. “I arrived at stage 3” — no; nobody arrives anywhere. That line is a conclusion the system drew, wearing an event's clothes.

Rule five is a fence around rule three. Rule three says that if a fact never reached the river, the fix is to write it — and that is exactly how stage_arrived got in. So write the missing fact only if it is a fact about the person. A conclusion the system drew is not a missing fact, and the river is the wrong place to keep it.

The cost of getting this wrong is not subtle. With those lines included, the thickest thread on the funnel map was arrived → left at 33,917 people, and its reverse right behind it at 31,169 — the engine talking to itself, drawn as though it were customer behaviour.

And the property that makes a river trustworthy at all: rows are never edited. You don't correct history — you add a line underneath. That's why withdrawing consent writes a new record rather than changing the old one. A row that can be overwritten cannot prove what was true last March.

Decision five — retention

How long the river is kept. This is the decision the archetypes quietly depend on, and the one almost nobody makes on purpose.

A summary inherits it: a rebuild reaches only as far back as the river still goes, so everything older than the window is permanent rather than derived. A heartbeat needs its own, never shared with a river. And a rule that appears nowhere else on this page: anyone proposing to prune a log has to ask what rebuilds depend on that window. Shortening retention looks like a storage decision and is secretly a correctness one.

Retention is decided per river — it never follows from the kind

Some heartbeats are binned within days; typing indicators are nobody's loss. Others are kept forever on purpose — the graph of who watched a webinar and when they left is the analytics. Same archetype, opposite retention, both correct. Which is why it has to be a decision rather than a property.

And it needs a default, or it will be skipped. A decision nobody is forced to make is a decision nobody makes. So: a river with no stated retention is kept forever, and counts as unrivered until somebody states it. Forever is the safe default — it destroys nothing — and calling it unrivered is what stops “forever” becoming a decision by accident.

A summary is allowed to outlive its river, and the industry does this deliberately: keep thirty days of raw rows and years of rollups. But past the river's window that summary is an original, not a copy, and nothing can recompute it. A fine design and a terrible accident — and the only difference between them is whether anyone wrote it down. Which is why the two retentions are always set together.

04

Two kinds, and their archetypes

If this is your first time here

This page is arranged for looking things up — every kind with its rules, the diagrams, the checklists. If you would rather be walked through the whole idea from nothing, in small steps that build on each other, start with the companion booklet: Rivering, step by step. Same material, forty-two steps, no prior knowledge assumed.

There are two kinds. Everything else is an archetype — a named shape that one of the two takes, carrying extra rules of its own. Knowing which turns a judgement call into a checklist, and a table that is two things at once is where nearly every real problem turns out to live.

the whole division

The river — what happened. A verb; ordered; never edited.
The noun — what is. A thing; current; may be replaced.

Of the river: the branch and the heartbeat. Of the noun: state, summary and the lock. Every rule an archetype carries is the rule its old kind carried — nothing changed but where the rules live.

Why this replaced a list (2026-09-10). The list had been arguing with itself in two places, both visible in its own text: it said “a heartbeat is a river” and then listed it separately, and it needed a footnote explaining that state and summary did not contradict each other, because their rules were identical but one. Two kinds and a set of archetypes settles both.

The example that shows this was not invented — double-entry bookkeeping

If any of this feels abstract, it has been practised since 1494. Accounting is this model, worked out over five centuries by people who could not afford to get it wrong.

ACCOUNTING
The general ledger

Every entry, in order, never erased. Sub-ledgers — receivables, payables — split off when the volume demands it.

HERE
The river, and its branches

Exactly that. A branch is a sub-ledger: the same water, in a channel of its own.

ACCOUNTING
Accounts and balances

The chart of accounts is the list of things. A balance is never typed — it is what the entries add up to.

HERE
Nouns and state

The account is a noun; the balance is state. “No noun is ever written directly”, stated as professional practice.

The rules match one for one, in the same order of importance. Never erase an entry — a mistake is corrected by a new, opposite entry, so the history of the mistake survives. A balance is derived, never written. A closed period is frozen even though it could still be recomputed — which is our sealed mark, arrived at independently.

the sharp version

An accountant who typed a balance straight into an account instead of deriving it from the ledger would be committing fraud — the number would no longer be answerable to the entries beneath it. In software the same act is called keeping a counter, and it ships without comment. Three counters were checked in one production system on 2026-09-10; all three had drifted, one by a factor of eight.

Which also settles the “is this general?” question. Accounting is not a special domain that happens to suit this shape. It is the one domain where getting it wrong was expensive enough, early enough, that the shape was found and then enforced by law. Everything else has the same structure and merely tolerates the drift.

The second axis — original or derived

Kind is one question. Original or derived is a different one, and they are independent: contacts is an original noun, a state row is a derived noun. Asking them as one question is how second originals get mistaken for caches, and caches for originals.

the rule that turned out to be about the noun

No noun is ever written directly. Every write goes to a river; every noun is derived. State, summary, the plain noun, contacts itself. Writes go to a river — the main one, a branch, or a heartbeat — and a branch is written to directly, because writing to a branch is writing to a river.

The lock is the archetype where this breaks, and that break is what earns it a name. A lock must REFUSE; refusing is atomic, at the moment of writing; and a derived thing can never refuse. So a lock is claimed by a direct write — the one noun-shaped thing that is written rather than derived.

Archetypes are open; the two kinds are not expected to grow. That asymmetry is the difference between a model and a list. Candidates already visible and recorded rather than adopted: the queue (rows arrive, are claimed, are deleted — not a river, since rows are removed), the link (a noun whose identity is a pair), and the draft (a thing that exists but has not happened yet).

Each kind and archetype below carries the detail that is genuinely its own. Some carry several rules like the river; some have two, because two is all there is. Nothing here is invented for symmetry.

First: three of these are the same subject, seen three ways

The archetypes read like independent things. Three of these shapes aren't. A noun, its river and its state are one subject described three times, and seeing that makes the rest easier.

WHAT IT IS
The noun

A person. Their name, their phone number. The things that are simply true of them.

WHAT HAPPENED TO IT
The river

They registered, opened an email, paid, cancelled. In order, never edited.

WHERE IT STANDS NOW
The state

They are a customer, on lesson four, in two sequences. Worked out from the river.

Read left to right and the arrow from section 02 is doing the work: the river produces the state; the state never produces the river. And the noun holds only what it is — never what happened to it, and never a state that was written by hand instead of worked out.

The order below is not arbitrary

They are arranged so that each one can be defined using only what came before it. Read straight down and nothing refers forward.

  1. The noun depends on nothing. It is the only shape you can define cold.
  2. The river needs a subject, and a subject is a noun.
  3. The branch — the river's own facts, split off for volume — is defined by the river it was split from, so the river has to exist first.
  4. The heartbeat — the same thing measured over and over — is defined by what it must be kept out of, so the river has to exist first.
  5. The state — where something stands now — is worked out from a river and usually lives on a noun. It needs both.
  6. The summary — what happened in a period that has finished — is state's sibling across a time boundary. It needs state.
  7. The lock — a row whose existence prevents something happening twice — is last for the strongest reason of all: every one of its rules mentions something else on this list. “Never on a river or a derived table.” “A noun's identity is not a lock.” Those sentences cannot be written until everything above it exists.

Which is also a useful thing to notice about the lock. It is the only archetype that cannot be explained on its own — and that is exactly why it is the one people put in the wrong place.

KIND

The noun

A thing that exists.

The testDoes a row describe something that is, rather than something that happened?
contactspagescoursesproductswebinars

A noun has an identity that survives change. Rename a course and it's still the same course, with the same students and the same history. That's what makes it a noun and not an event.

The grammar is not decoration, and it pairs: a noun is a thing; every line in a river is a verb. Paid. Registered. Cancelled. If a row reads as a thing, it belongs here; if it reads as something done, it belongs in a river. That one test settles most tables before you have looked at a single column.

Its own rules
  1. It holds what it is, never what happened to it. Date columns like pausedAt and rejectedAt stacked inside a noun are a diary crushed into one row — each slot holds only the most recent occurrence, and everything earlier is gone. The same defect has a larger form in document stores: an array of events embedded inside the parent document is a river living inside a noun. It is recommended practice there when the set is small, bounded and always read with its parent — and it becomes the crushed diary the moment it grows. The tell never changes: can you count and filter those events without loading every parent? But overwriting is not itself the defect — losing history nobody decided to lose is. Someone changes their phone number and you overwrite it; the old one is gone and nobody minds. Dimensional modelling names that deliberately — a “type 1” dimension — and treats discarding history as correct where it has no value. The sending-domain case was a defect because nobody decided: the second rejection reason vanished before anyone knew they wanted it. Overwrite on purpose and it is a decision; overwrite by default and it is a loss you discover much later.
  2. One noun, not several. When two tables differ by what a thing is used for rather than what it is, they are one noun with a field. The system this came from had already made this call twice, and both are written into its house rules: quizzes, polls and surveys are one form; every kind of access is one entitlement. The instinct to add a new table for each new use is the one to resist.
Seen in the wild The life of a sending domain — requested, verified, approved, rejected, paused, resumed — lives as fourteen date columns spread across five nouns. Get rejected twice and the first reason no longer exists anywhere.
KIND

The river

A record of something that happened.

The testDoes a row describe an event — something with a time and a subject?
contact_actionsaudit_logsemail_eventssms_eventsconsent_records

Rivers are the only genuinely original tables. Everything else on this page is either something a river is about, something computed from one, or something that isn't history at all.

A river needs the five things in the section before this one: a subject, a unit, a vocabulary, the rules, and a retention decision.

Its own rules
  1. Rows are never edited. You don't correct history — you add a line underneath. That's why withdrawing consent writes a new record instead of changing the old one. A row that can be overwritten cannot prove what was true last March.
  2. A line carries two times: when it happened, and when you wrote it down. With only one you can obey the rule above perfectly and still lose the ability to say what you believed last March — because a backdated line inserted today silently changes the answer to a question about the past, and no row was edited to do it. Banking, insurance and health records all keep both; the formal name is bitemporal. The test: “what did this system believe on the 5th?” If your river can't answer that, it has one time where it needs two.
  3. A line may only be removed if it duplicates something that still exists. Not something that existed once. This sounds pedantic and is the difference between tidying and destroying: the copy you were deferring to may have been deleted years ago, leaving the line you're about to remove as the only record.
  4. No locks in a river. Rivers get pruned, and pruning is decided for reasons — size, privacy — that have nothing to do with guarantees. A lock in here dies of somebody else's storage decision. If one row both happened and needs a guarantee, that is two rows in two tables.
Seen in the wild 41,765 lines were marked for deletion as duplicates of another table. 85% of the rows they duplicated no longer existed — the parent records had been deleted years earlier, and the diary lines were the last surviving copy. Section 06 walks through it.
ARCHETYPE OF RIVER

The branch

The same water, in a separate channel.

The testIs every row a fact that stands alone, about the same subject as the river — and is there simply far too much of it for a timeline?
page_viewsection_viewcontent_viewclick

“Levan viewed /pricing at 14:32” is a complete fact. It passes every test a river line passes; it is not a sample of anything. And there are 188 of them per person, which is why it cannot live in the timeline — one afternoon's browsing buries a year of somebody's history.

A branch is still the river. Same subject, same kind of content, ordered, never edited. It has been split off for volume and nothing else — which is exactly what makes it a branch rather than an extraction. The water is the same; only the channel is separate.

Its own rules
  1. Same subject as its river. A different subject is not a branch — it is another river. Split by volume; never by subject.
  2. Its rows are facts, and keep a river's rules. Never edited, ordered, each row readable on its own.
  3. The main river may want a conclusion, on the same terms as a heartbeat: a line saying “visited the pricing page” belongs on the timeline even when every individual view does not. Decide that interval; don't let it happen by accident.
  4. Its retention is its own decision — usually longer than people expect, because a branch is where the analytics live.
  5. A branch may have its own heartbeat. It is a river, so everything true of a river is true of it, including being sampled.

Branch or heartbeat — one question settles it. Does a single row state a fact? Yes, and only the quantity is the problem → branch. No, only the pile means anything → heartbeat. A branch is the river's own water in another channel; a heartbeat is a different substance taken out of it.

Seen in the wild Page views, section views and content views sat in a production timeline at 188, 106 and 24 rows per person — about 29% of the whole river. Nobody could call them heartbeats, because each row plainly is a fact. There was no other word, so they stayed where they did not belong. A thing with no name does not get moved — which is why the branch was named on 2026-09-10, the first archetype added since the list was written. The two kinds did not change; a new shape of river got a name.
ARCHETYPE OF RIVER

The heartbeat

The same thing, measured over and over.

The testDoes one row mean anything on its own?
“still watching” pingstyping indicatorsAI cost logstracking dispatch

Someone watching a webinar sends a signal every few seconds. That is not fifty events. It is one event — they watched — sampled fifty times.

“Levan paid £240 on Tuesday” is a complete fact standing alone. “Levan still watching at 14:32:05” means almost nothing alone — only the pile says “he watched for forty minutes.” A river line is a fact; a heartbeat row is a dot on a graph. Prefer that test to “is it the same situation sampled over and over”, which needs you to already understand the table. This one you can answer by reading a single row cold.

A heartbeat is a river. It has a subject, a unit, its rows are never edited, they arrive in order — it passes every structural test for one. And its subject is the same noun the river has: remove Levan from “Levan still watching at 14:32:05” and the line collapses, so the contact is the subject; remove the viewing session and the line still means something, so that was only a mention. It is extracted for volume, not subject — its unit is finer than a timeline wants, which is decision two, not a different river.

Its own rules
  1. It never enters a river. The river gets the conclusion — “watched for 40 minutes” — never the samples.
  2. The extraction is what creates the need for the conclusion. Had the detail stayed in the river, nobody would write a summary line at all — “30% watched to the end” would simply be derived, like any other question you ask of it. That line exists only because you pulled the detail out, so writing it is the debt the extraction incurs. Skip it and the extraction has silently destroyed the answer.
  3. Its retention is a decision, not a property. An earlier version of this rule said a heartbeat must have a lifespan and that deleting is correct rather than a loss. That over-claims. Typing indicators are binned; webinar watch data is kept deliberately and forever, because the graph of who left when is what the analytics is. Decide it — decision five — rather than assuming it.
  4. Its lifespan is its own. Never share a bin with a river. When they're mixed, the river's records get swept out on the heartbeat's schedule — and a record that mattered is deleted on a timer that was set for noise.
  5. The reporting interval is a named decision, made the day the heartbeat is born. Rule two says the river gets the conclusion. This says which conclusion, how often, and when that was decided — and it is the rule most often skipped, because a heartbeat works perfectly without it. Nothing errors. The hole only becomes visible later, from the other end, when somebody tries to ask the river a question.

Choose the interval from the questions, not from convenience. The interval you pick is the finest question anyone will ever be able to ask. Report at 25/50/75/100 and “who watched more than 90%” is unanswerable forever — not slow, unanswerable, because the samples that could have answered it were binned on the heartbeat's own schedule. Write the questions down first and the interval falls out of them.

Deciding late leaves a permanent hole

The heartbeat had been recording since 27 February. The conclusion was first written to the river on 13 May. Two and a half months in between, in which the samples were collected, aged out, and never summarised. The shape of the event was right; the history was simply gone.

79%of people with watch progress
had none of it in the river

Backfill reaches only as far back as the heartbeat's own retention. Past that the answer exists nowhere, and the honest move is to say so rather than to infer it.

auditing an existing heartbeat — ask in this order

What conclusion does it write to the river · at what interval · was that interval chosen from the questions or from convenience · does its coverage match the heartbeat's own age, counted per person · and if not, how far back can it still be replayed. Four of those five are counts. Do not accept prose for any of them.

Seen in the wild The audit log was being filled automatically with every web request, including typing indicators and background token refreshes. Real entries came to 0.17% of the table — and inherited the heartbeats' 90-day deletion. An admin logging into a customer's account is forgotten after three months, because it was stored in a bin sized for noise.
ARCHETYPE OF NOUN

The state

Where something stands right now — worked out, not decided.

Derived · never written to · rebuildable Shared with the summary — both are worked out from a river and may only ever be replaced by a rebuild.
The testCould it be rebuilt from a river or from a noun — but must it be correct the instant something changes?
the daily send capwho is enrolled in whata person's last email openwhere someone is in a course

In one line: where something stands now — worked out from a river or a noun, written only as a consequence of the thing it follows, and always rebuildable. That is the whole archetype. Everything below is detail: where it lives, how it gets written, and the two ways it goes wrong. This is the longest section on the page because it is the one that has absorbed every correction; you can stop after the first line and be right about most tables.

This archetype was called “settings” in an earlier draft, and that name was wrong twice over. A setting is one example of state, not the category — and most state is nothing like a setting.

State or summary? The question is whether the moment has finished

State and summary are the only two shapes derived data comes in, and the line between them is time.

A football scoreboard during the match reads 2–1. True right now, and about to be replaced — someone scores and it becomes 2–2. That is state: one answer, always current, always replaceable.

The referee blows the whistle at 3–1. The number hasn't changed, but something about it has: it can never be replaced again. That match ended 3–1, forever. That is a summary.

Which means the two are exhaustive, not a list somebody collected. Every derived answer describes some moment, and a moment is either finished or it isn't. There is no third kind of moment.

And “not finished” includes moments that haven't started. A forecast — projected revenue for next month — is state: it will be replaced as data arrives. Reading state as “about now” is narrower than the rule allows. The line is closed or not closed, and the future is not closed.

And the case that proves the line is real: “views this month, so far.” It looks like a summary, but the month isn't over, so the number still moves — it is state. The instant the month closes, the same row becomes a summary and will never change again. Same data, different kind, because time passed.

Where it lives: one question decides it

Can a thing have only ONE of these? Then it is a column on the noun. Levan has one “last email opened,” the way he has one phone number. Same shelf, no separate table.

Which is the normal case, and worth saying plainly: a noun ordinarily carries its own state, as columns. Giving state a table of its own is the exception, not the rule — reached for only when the noun cannot hold it. Everything on this page still applies to those columns: worked out from a river, never written by hand, rebuildable. They simply live on the noun rather than beside it.

Can it have MANY? Then it cannot fit on the noun and needs its own table. Levan is in five courses, at a different point in each — there is no way to write five answers in one column.

That is a question about shape, not about kind. Both are state; both follow every rule below. The library version: a member's address goes on their card, but a borrowing needs its own slip, because a slip is about a member and a book and neither one can hold it alone.

Chosen or produced — and why it is a warning label

State arrives two ways, and the difference tells you where history gets destroyed.

Produced state is safe by nature. Nobody decided “Levan is on lesson four” — finishing lesson three put him there. The events exist whether anyone planned for them or not, so the value can always be rebuilt.

Chosen state has no such guarantee. Somebody picked 500 from a dropdown. Unless somebody deliberately wrote that choice down, nothing recorded it — and the previous value is simply gone.

Where to look for missing history

Chosen state is where the log tends to be missing. Produced state is where it cannot be.

But it is a warning label, not a verdict. Chosen state tells you where to look; it does not decide what is lost. Auditing a real system, produced state turned out to go missing just as easily — a media file's uploads and repairs, a live event's every go-live. Nobody chose those, and nothing recorded them either.

What actually decides it is simpler, and it is the rule worth carrying:

What actually decides whether history survives

History survives exactly when a river exists whose subject is that thing. Chosen or produced only changes how likely it is that nobody built one.

And the thing that must be instant — three shapes, not two

State is read constantly, and when a person changes something they must see it change immediately. So it cannot wait for a rebuild — which raises the obvious worry: if it is written at the same instant, isn't that the whiteboard all over again?

No — and the difference is small enough to miss, which is why it needs a name.

  1. Write both, side by side. The change goes to the log and to the copy as peers. Always wrong. If one lands and the other fails they disagree with no cure.
  2. Write the log; the copy follows in the same indivisible action. One original. Right when somebody is waiting on the answer — a person sets their send cap to 500 and is looking at the screen.
  3. Write the log; the copy follows a moment later, in a separate step. Also one original — the copy is still only ever written from the log, it just arrives a second late. Right when nobody is waiting: a dashboard count, a search index, a nightly report. Ordinary practice in event-driven systems, and not the whiteboard mistake.

Shapes two and three are both consequence, not copy. What separates them is who is waiting, not the mechanism. Only the first is ever wrong — and the summary already grants the same permission when it says it “is allowed to lag”, so the two rules contradicted each other until this was written down.

If the river records “the cap changed from 200 to 500 on Tuesday”, then the current value isn't a fact needing storage. It is simply the most recent change. Settings are state — and you get “who changed this?” for free, instead of as a separate feature somebody has to remember to build.

WRONG — WRITE BOTH the change the log the settings Two originals, side by side. If one succeeds and the other fails they disagree — and you cannot tell which is wrong, because both were first-hand. Rebuilding cannot help you. RIGHT — WRITE ONE, THE OTHER FOLLOWS the change the log the settings ONE INDIVISIBLE ACTION — BOTH, OR NEITHER One original. The settings row is a consequence of writing the log, not a peer of it. Instant for the user, and if it ever drifts, the rebuild is the answer. Always repairable.
Both give the user an instant result. Only the right-hand shape can be repaired. The mechanism is ordinary — a database transaction treats two writes as one indivisible action: either both land, or neither does.
The state — its own rules
  1. Consequence, not copy. It may be written in the same breath as the river. It may never be written beside it.
  2. It must still be fully rebuildable. Instant updating is a convenience on top; it never replaces the rebuild. If there's no rebuild, it is not derived at all — a second original wearing a faster coat.
  3. One value per thing lives on the thing. A separate table is for state a noun can have many of. Making a table for a single value adds a join and buys nothing.
  4. Chosen state needs its change recorded deliberately. Produced state gets its history for free; chosen state does not, and that is where it goes missing.
The correction that reshaped this list

Can't that be derived? We can have a system that in parallel saves the settings as well — but shouldn't that be derived from the log we already talked about?

Yes — and that question is what collapsed “settings” into state, and state into something that mostly isn't a table at all.

A later one narrowed it further: “That case is not settings; it's something else.” Which is right. A setting is chosen. Where someone stands in a course is produced. Both are state; only one of them gets its history for free.

One honest wrinkle. A setting nobody has ever changed has no change to replay, so the very first value has to come from somewhere — the defaults the system ships with. That's a small, real exception, and it should stay small. If your settings tables are large, most of what's in them is not defaults. It's the current state of things people changed, and every one of those changes was an event that should have been written down.

ARCHETYPE OF NOUN

The summary

What happened during a period that has finished.

Derived · never written to · rebuildable Shared with the state — both are worked out from a river and may only ever be replaced by a rebuild.
The test — the sharpest one hereCould you delete this entire table right now and rebuild it from a river? If no, it is not derived at all, whatever it looks like.
views per page per dayrevenue per monthopens per campaign

Summaries are legitimate and necessary — nobody can replay a hundred thousand events on every page load. The danger isn't that they exist. It's that they look identical to originals right up until the day they disagree.

What makes it a summary rather than state: it describes a period that has closed. “This page got 412 views on 3 March” became true the moment March 3rd ended, and will be true in fifty years. Nothing will ever replace that row.

Its own rules
  1. No front door. Never edited, never incremented. Only ever recomputed.
  2. The rebuild must actually exist. Not "could be written" — written, today. Without one, this is not derived at all; it's a second original, and the day it drifts there is no way back.
  3. It is allowed to lag. A summary a minute behind is fine — its period has closed, so nothing is waiting on it. One that can't be repaired is not fine.
  4. It inherits its river's retention. A rebuild can only reach as far back as the river still goes. If the river is pruned at 30 days, the copy is permanent — not derived — for everything older than that.
  5. No locks here either. For the mirror image of the river's reason: this table is deliberately deleted every time it is rebuilt.

That last one is the sharp edge where this archetype meets the heartbeat, and it is easy to miss because both halves look correct on their own. A retention policy looks like a storage decision. It is also a correctness decision: shortening a river's retention silently freezes every copy built from it. Anyone proposing to prune a log has to ask what rebuilds depend on that window.

Seen in the wild Fourteen tables store counts a river could produce — campaign opens, community posts, link clicks. They're accurate today. But they're only ever nudged up by one, and not one of them has a rebuild. The six that are already wrong will stay wrong forever.
ARCHETYPE OF NOUN

The lock

A row whose existence is the point, not its contents.

The testIf you deleted this row, would you lose information — or would something happen twice?
one email address per personone active enrolment“webhook already handled”one send per campaign

Think of a “reserved” card on a restaurant table. It says nothing and holds no facts. Remove it and you haven't forgotten anything — you've caused two parties to be seated in the same seat.

And the name is not borrowed from software. A lock on a river is a real thing: a chamber with gates at both ends that lets one vessel through at a time, then closes behind it. That is precisely this kind's job — admit one claim, refuse the second — and it is worth picturing, because a physical lock makes the rule obvious in a way the software word never does.

First: not every uniqueness rule is a lock

“Two people cannot share an email address” looks like a lock and isn't. That is identity — a statement about what a contact is, permanent, part of what makes it that thing. It belongs on the noun and moving it elsewhere would break it.

A lock is a statement about what may happen. One active enrolment. One pending card per step. It is temporary, and it is released when the situation ends.

Telling them apart

Does it prevent a second thing, or a second action? A second thing is identity, and it lives on the noun. A second action is a lock, and it lives alone.

And a lock may never live on a river or on a derived table

Earlier we said a lock must not share rows with something rebuildable, because a rebuild deletes first. The same danger reaches further, and the reason is worth following.

Rivers get pruned — and the water word is the truer one: a river loses its old water at the far end, continuously, and nobody decides to keep the water from last March. Logs are the same. Old lines are deleted to save space, or because keeping personal data forever is a liability.

Both are correct and responsible decisions, and both are decisions about the records. Nobody in that conversation is thinking about guarantees.

So a lock sitting in a pruned river dies of a decision that was never about it. The guarantee expires as a side effect of somebody else's storage judgement, and nothing announces it.

Where a lock may not live

Never on anything that is routinely deleted or rebuilt — a river or a derived table. A noun is neither, which is why identity is safe there.

And when one thing is both, that is two rows in two tables. “We scheduled a call for Tuesday” genuinely happened — someone decided it, and it belongs in the river where it will stay. The guarantee that it cannot be scheduled twice is a separate row in its own table, where nothing will prune it by accident. Not a contradiction; just two different things that were being asked of one row.

Its own rules
  1. The claim is the check. A lock is a row you write in order to find out whether you're allowed to. Never look first and then act — see below for why.
  2. A refusal is an answer, not an error. When the database rejects the second write, nothing has gone wrong. That rejection is the information.
  3. It lives alone, and carries no cargo — no extra columns riding along on the lock row. Anything worth carrying is eventually worth rebuilding, and rebuilding a lock releases it.
  4. It never sits on a river or a derived table. Both are routinely deleted — one pruned, one rebuilt — for reasons that have nothing to do with guarantees.
  5. A noun's own identity is not a lock. Unique email addresses, unique slugs, unique codes: those say what the thing is, and they belong on the noun.

Be honest about what kind of rule this is. A rebuild serialised against everything the guarantee protects — an exclusive lock held across the whole operation — leaves no observable gap, and a selective rebuild that spares the lock rows avoids it too. So this is not a correctness theorem. It is a robustness rule: both safe variants work, and both fail silently the first time somebody empties the table the ordinary way — no error, no failing test, double-booking back with nothing to announce it. That is a sufficient reason for the rule. It is simply a different reason than “impossible”, and worth saying which one you mean.

Why a lock must live alone. Two of these archetypes give opposite instructions about deletion. Anything derived must be deletable — its test is literally “delete it and rebuild it.” A lock must not be — its existence is the guarantee.

So a row that is both cannot be rebuilt at all. A rebuild is delete-then-recompute, and for those seconds the lock is guarding nothing. Anyone rebuilding the state would quietly hold the door open while they did it.

And separate rows are not enough — they need separate tables. A rebuild empties a whole table, so claims sharing a table with state get tipped out with it. You could delete selectively and spare the claims, but that holds only until someone empties the table the obvious way, and then double-booking returns with no warning at all.

The simplest form of it

A table you are allowed to delete must not contain anything you are not allowed to delete.

The deeper reason

A lock carries no cargo — because anything worth carrying is eventually worth rebuilding, and rebuilding a lock releases it.

“Cargo duplicates truth” is also true, and it's the shallower reason. This is the one that explains why the collision is dangerous rather than merely untidy.

Seen in the wild Webinar registrations are locks — one per person per session. But six columns were hung on them, duplicating a record they already pointed at, including the token used to log people in. Measured where the two copies must agree, 149 people hold two different access tokens, and nothing can now say which is real.
04b

Two marks, which are not kinds

A mark is a sticker on a table that already has a kind. It is still a noun, or state, or a summary — one rule about it simply reads differently.

A red ball and a blue ball are both balls; you do not invent “red ball” as a new shape. Shape is one question, colour is another. Make a mark into a kind and you have to copy every rule of the underlying kind into it — and the day somebody edits one copy and not the other, they drift.

external — the source of truth is outside this database

Another company's system, your own repository's seed data, or the physical world. One rule changes: rebuild becomes refetch — and a refetch is not guaranteed to agree with what you had.

sealed — frozen by an act outside the database

A signature, an audit, a filing. Once an accounting period closes its figures may not be recomputed even though they still could be. One rule changes: rebuild becomes forbidden, and corrections go forward as new entries.

A table may carry both marks, or neither, and it is still exactly one kind. That is the tell that they are a different sort of thing altogether.

If you find yourself wanting a third, check first that it is not a kind you have mislabelled. Two marks each changing exactly one rule is a pattern with two data points, and it is thin. A candidate has already turned up — a counter that hands out order numbers, where the value being claimed is the cargo the lock rule forbids — and it is recorded rather than adopted, for that reason.

04c

When the rule is right and the table is still wrong

Every rule so far applies to a table once somebody has decided what it is. The damage lives in the stretches nobody decided about — and a method that only rules on what it is shown will keep walking past them.

This is not hypothetical. A production table was found incrementing counters from six separate places, drifting eightfold, unnoticed for months. The rule that forbids exactly that — no front door, never edited, never incremented — was already written, word for word. It never fired, because nobody had ever classified the table. It looks like a noun: one row per person per webinar, its own id, proper foreign keys.

The rules were never the missing part. A way to go looking was. Three searches, in order.

1 — columns that are answers, not facts

Names ending _count, _score, _seconds, _progress, or beginning total_. Those are answers somebody worked out. A table holding them is a summary however it is shaped — own id, foreign keys and all. Shape is not evidence. The columns are.

2 — count the writers

Search the application for every update, upsert and increment against that table. More than one writer is the finding. A summary has exactly one writer, and it is the rebuild. Six writers means six places that can drift apart, none of which ever recounts.

3 — measure the drift, and this is the step that convinces people

Count the same fact twice — once from the counter, once from the river — and put the two numbers next to each other. Do not argue the principle. Show the gap.

A summary wearing a noun's clothes

Watch records held poll answers, CTA clicks, chat messages and an engagement score as running totals, incremented by hand from six places and never recomputed. Counted against the river that holds the same events, the totals had drifted — silently, for months. Every filter asking "did they answer a poll" was returning the wrong people.

8,617 by the counter
1,046 in the river

Nothing errored. No test failed. That is the ordinary way this fails, and it is why the search matters more than the rule.

04d

The razor — if it is not mandatory, it should not exist

A table, a column, an index, a cache. The simplest thing that works is the right thing, and anything extra carries the burden of proof.

A derived table is never built in advance. It is built when something measurably became too slow, and its existence is the record of that measurement. "We will need this later" is how a schema fills up with second originals — each one true on the day it was written and drifting quietly ever after.

The testWhat breaks if this does not exist? If the answer is only "it gets slower", that is a real answer — but it has to be measured, not predicted.

Three consequences worth stating outright.

speed is the only reason a derived table exists

If it holds a fact that lives nowhere else, it is not derived and the razor does not apply to it. What you have is a river or a noun that has been mislabelled — go back to the kinds.

reach for the cheapest instrument first

An index, then a view, then a materialised view, then a table. A materialised view is worth reaching for before a table, because nobody can write to one: "derived, never written directly" stops being a promise anybody can break and becomes a property of the thing.

the rebuild is part of the table

A derived table with no working rebuild is a second original wearing a faster coat. Ship them together or ship neither — and "the rebuild could be written" is not a rebuild.

It says “justify yourself”, not “delete”

The commonest way to misapply the razor is to read it as fewest tables win and start removing things. It says something narrower: every table must earn its place — and being measurably needed for speed is a real reason.

So the usual outcome of applying it to a working system is not deletion. It is demotion.

demotion — same table, different status

Before: an original — the only place the fact lives.
After: derived — rebuilt from the river, never written directly, kept for speed.

A worked example. An access table answers “can this person watch this?” on every page load. It can be folded from the river — grants minus revokes — so it is not an original. But it sits on the hot path, and replaying somebody's history thousands of times a day is exactly the case a derived table exists for. Delete it and it returns within a week wearing a cache's name.

The test that separates themIs anything waiting on this answer? A dashboard count, a funnel map, a nightly report — nobody waits, so it can go. An authorisation check — somebody waits, every time.

Rewire before you delete, always. Rewiring is reversible; deleting a table that turned out to be the only copy is not — and “it looked derived” is precisely how a second original gets destroyed. Demotion is most of the work and none of the drama: stop the direct writes, add the rebuild, prove the rebuild agrees with what is there, and only then argue about whether it should exist at all.

05

How a lock actually works

Locks are the least intuitive archetype, and obvious once you see the problem they solve.

You send a campaign to 10,000 people. Halfway through the server restarts — a deploy, a crash, anything. The job wakes up and starts again, not knowing it already emailed 5,000 people. Those 5,000 get it twice.

The obvious fix is to check first: before emailing someone, look whether we already did. That works with one worker. You have several, for speed.

CHECKING FIRST — FAILS worker A worker B “emailed yet?” “emailed yet?” NO NO sent twice Both checked. Both were told the truth. There is always a gap between asking and acting. CLAIMING THE SLOT — WORKS worker A worker B writes the row writes the row ONE SLOT ONLY campaign X · Levan A — accepted B — refused There is no gap, because asking and claiming are now the same single action.
Instead of asking the database a question, you make it enforce a rule: for this campaign, each person may appear exactly once. B's refusal is not a failure to fix — it is the answer, delivered in the only way that cannot be raced.
05b

Two things a river cannot do alone

Push the idea to its end — everything is derived from the river, including the nouns — and it very nearly holds. It fails in exactly two places, and both are worth knowing, because the argument is good enough to be tempting.

Something must be able to refuse

A river narrates what happened. By the time anything is derived from it, the thing has already happened. But refusing has to occur at the moment of writing, and the obvious approach — look first, then write — has a gap in it.

Two people register with the same address in the same instant. Request A looks: nothing there. Request B looks: nothing there. Both write. Both checks were correct; they simply looked before either had written. Rebuild from that river afterwards and you will faithfully rebuild two of them, because two is what happened.

The testCan this be checked and claimed in one indivisible actreserve it, or fail — rather than looked up and then written?

This does not mean the river cannot be the single source. Put the constraint on the river and it refuses the second line itself. But notice what just happened: whatever holds the rule that refuses is the noun, whatever it is called.

So river and noun are two jobs, not necessarily two tables. One narrates; one refuses. You may put both jobs in one table. You may not have zero things doing the second one.

A minted value has to ride on the event

Some columns are not facts that happened — they are values the system invented. An access token. A generated identifier. Anything already printed inside a link that was emailed to somebody.

Rebuild that row and a different token is minted. Every link already sent stops working. The rebuild was faithful to the events and still broke the world, because the value was never in the events.

The fix belongs inside the principle rather than beside it: the event carries the minted value at the moment it is minted. Then the river holds it too, and the row is rebuildable after all. This is a requirement of deriving everything, not an exception to it.

And check the side doors before believing either side

"Everything is in the river" is a claim to measure, not to assume. Count the rows in the noun with no matching line in the river.

Done on a real system, the answer was 24,710 imports and 1,181 migrations — and exactly three ordinary registrations. Bulk history poured in from outside, whose events had happened in somebody else's system or never happened at all. In normal running the river was complete; the hole was the importer.

That number cuts both ways, which is why it is worth having. It refutes the lazy objection ("the river is full of holes") and it names the real work ("close the side door") in the same measurement.

06

One case, start to finish

Everything above, used once on a real question — including the two places it nearly went wrong.

The situation

Automations put people through a sequence of emails. Two lines go into the contact diary: entered the welcome sequence, and exited the welcome sequence. There are 41,765 of them.

Separately, an enrolment table records who is currently in which automation.

Same fact in two places — so by rule 1, one has to go. All 41,765 diary lines were marked for deletion.

What was actually true

35,541 of the 41,765 had nothing left to be a copy of. The automations had been deleted at some point, and the enrolment records went with them. So for 85% of these lines, the “original” we were deferring to no longer existed.

The arrow had been drawn backwards. And once you ask which one is the original? — section 02 — it was never a judgement call: “entered on Tuesday” is an event, so it's a river line. “Is currently enrolled” is a state. The events are the original, always.

Then the enrolment table turned out to be three things

This is the part worth generalising. What looks like one simple table — a list of who's in which automation — is doing three separate jobs:

JOB ONE
Derived state

Who is currently enrolled. Entirely derivable — it's just everyone who entered and hasn't exited.

JOB TWO
A gap

Which step of the sequence they're on. Not derivable — nothing writes step-moves into the diary.

JOB THREE
A lock

Stopping one person being enrolled twice by the same trigger. Enforced in the database, and invisible in the code.

Notice only job one is a mirror. Someone seeing that and saying “this is derivable, we can rebuild it” would be right about job one and would silently destroy the other two. People would get enrolled twice, and nothing would explain why.

Worth keeping

Before you simplify a table, count its jobs. A table that looks like one thing is the normal case, not the exception.

And there is a procedure for it — it is not a matter of noticing. List three things about the table in front of you. Every write site: two write paths that never overlap are two jobs until proven otherwise. Every read site, and what it filters on: a query filtering on a column most rows leave empty is reading a different job from the one that fills it. Every constraint, unique indexes especially — a uniqueness rule is always a job, and it is the job most often invisible in the code. The enrolment table gave three writers, two readers filtering on different columns, and one unique index. That is exactly the three jobs it turned out to have.

Job two is where rule 3 earns its place

“Which step are they on” genuinely cannot be worked out from the diary. The tempting conclusion is so we have to keep this table as an original.

Rule 3 says that's the wrong conclusion. It isn't a verdict, it's a symptom — and the question is why is the diary missing this? The answer is unremarkable: nothing writes a line when someone moves from step 3 to step 4. Not impossible, just never done.

So the fix is to record step-moves, after which job two becomes derivable like job one, and the whole table can be a mirror.

And it had already been solved once

The goals side of the same system used to have the identical column — a box holding “which step is this person on.” It's gone. Both it and its companion table survive only as comments explaining why they were removed. Position is now worked out by reading the diary.

Same shape, same fix, already shipped — just never applied to automations. Worth checking, when you find one of these: has some other part of the system already solved it?

Job three is what actually blocks the work

The lock and the state live on the same rows. By the summary's rule, that means the table cannot be rebuilt — emptying it to recompute it would leave the double-enrolment guard guarding nothing.

So moving the lock out is not a detail of the tidy-up. It's the first step of it, and nothing else can happen until it's done.

Where it landed

Keep all 41,765 lines — they were miscategorised, not wrongly judged. Everything else is a project, and nothing is broken while it waits.

07

What this found, in one real system

None of these are illustrations. Each was found by asking “which kind is this?” and measured against a live database on 5 September 2026.

A river filled with heartbeats

The record of who did what to the business is a river. It was being filled automatically with every web request — including typing indicators and background token refreshes, which are heartbeats. Two kinds in one table, and the real entries inherited the heartbeats' 90-day deletion.

149real entries
out of 85,984 rows

A lock carrying cargo

Webinar registrations are locks. Six columns were hung on them, duplicating a record they already held a pointer to — including the access token people log in with. Measured where the copies must agree, they don't.

149people holding two
different access tokens

Derived tables with no rebuild

Fourteen tables store counts a river could produce. They are accurate today — but only ever nudged up by one, and not one has a rebuild. So the handful already wrong will stay wrong permanently.

0rebuilds, against
15+ places that increment

A river with no vocabulary

The audit log's name column takes free text, with the identity of the thing glued inside the sentence. So every line is unique — nothing can be counted, and “everything that happened to this ticket” has no answer.

0.17%of rows have the
“which thing” column filled

A diary crushed into a noun

The life of a sending domain lives as fourteen date columns across five nouns. Each column holds one occurrence.

14date columns where
a river should be

One category, two opposite correct answers

A block of 169,425 alarm rows was going to be moved as one group. 197 of them are still pending — which puts them inside a uniqueness rule, making them locks. The other 169,228 are finished and hold nothing. Same table, same category, and no single operation is right for both.

The finished ones also turned out to carry cargo: 71,783 notifications point back at them through a link the database doesn't know about — so nothing would have refused the delete.

71,783rows that would have
silently lost their origin

Six subjects have a river. None of them is a business object.

Counted properly, six subjects keep a log: the person (1.5M lines), the message, the conversation, the workflow run, who did what, and the media file — that last one only for its deletion.

Every one of those is about people, or about talking to people. Not one is about the business's own objects — an account, a sending domain, a campaign, a page, a course. Twenty-two tables hold that kind of history in columns instead, each slot keeping only the most recent occurrence.

Sorting all twenty-two by whose history it was produced a total split, not an approximate one. Person, or a person paired with something: safe. The thing itself: overwritten.

0rivers about the
business's own objects

Two logs of the same thing, in two places

Email deliveries and SMS deliveries are recorded in two separate tables. Both hold the same columns — which message, which person, what happened, when — and both point at rows in the same notifications table. Same subject, recorded twice, with two vocabularies.

A column-counting scan missed this entirely: the tables are small, so they share only six meaningful columns — under the threshold. Asking what they point at found it immediately.

2rivers where the system
says there is one subject

And most of it was already right

Thirteen rivers carry facts about a person. Twelve reach the contact diary. Two more are correctly kept out of it, because they're heartbeats. The funnel map is textbook derived state — deleted and rebuilt on demand.

12/13rivers arriving
where they should
08

Rivering a table you have never seen

This is what rivering actually consists of, table by table. Six questions, in order — the first “yes” tells you the kind and archetype, and that tells you the rules.

These are deliberately not in the same order as the kinds and archetypes above. That order teaches — each defined using only what came before. This one identifies — the sharpest question first, so most tables are settled in one or two steps. Same kinds and archetypes, different job.

Before question one — name the database you are rivering

A kind is not a property of a thing in the world. It is a property of a table in a system. A copy of somebody else's original is derived and external, whatever shape it has.

The same customer is a noun in the operational database and derived state in the warehouse that copies it. So a warehouse dimension carrying valid_from and valid_to is not a noun breaking the no-history rule — it was never a noun here. Same for a read replica, a search index, a cache. Settle the scope first, or you will apply a noun's rules to something that is really a mirror.

The whole ladder, on one screen

0. Name the database you are rivering. A copy of somebody else's original is derived and external.

1. Did something happen? → river — then keep going to 2.
2. Same situation repeated; one row meaningless alone? → heartbeat. No — every row is a fact, but there are far too many per subject for a timeline?branch (a person's call).
3. Would deleting a row cause something to happen twice? → lock.
4. Can you name the line that would have to be written? → derived (no rebuild yet = derived with a gap). Can't name one? Ask why not — outside this database means external.
5. Has the moment finished? → summary, else state.
6. A thing in its own right? → noun.

None of them? It is unrivered — not a kind, just work nobody has done. Put it on the list.

The full version, with what each answer commits you to:

01Does a row describe something that happened?→ A river. Now decide its subject, unit, vocabulary and retention. Do not stop here — go on to question two. A heartbeat answers yes here correctly, because it genuinely is a river; question one is not wrong about it, only incomplete.
02Is it the same situation repeated — does one row mean nothing on its own?→ A heartbeat. The same river, extracted because its unit is finer than a timeline wants. Decide its retention; make sure the river it was pulled from carries the conclusion. If the answer is no — every row is a fact standing alone, but there are far too many per subject for a timeline — it is a branch: the same river in a separate channel. That one is a person's call about the timeline, not something the schema can settle.
03Would deleting a row cause something to happen twice?→ A lock. Give it nothing but the slot it claims. A second thing is identity and belongs on the noun; a second action is a lock and lives alone.
04Can you name the line that would have to be written?Not “should this be derivable” — that is a question about ambition, and almost anything could be derived if you were willing to write a river for it. Name the actual line. For a table holding a user and a team you can: “added to team”. For a person's surname you cannot, because no event produces it — it simply is. → You can name it: derived. Then ask whether that line gets written, and the rebuild exists, today. If not, this is derived with a gap, and you have already named the repair. → You cannot name any line: ask why not. If the source of truth is outside this database — another company's system, your own repository's seed data, or the physical world — mark it external and carry on to question five.
05Has the moment finished?→ Finished is a summary; still moving is state. State is written as a consequence of the river, never beside it. One value per thing → a column on the noun. Many → its own table.
06Is it a thing in its own right, with an identity of its own?→ A noun. It may hold what it is. It may not hold what happened to it. A pairing of two other things is not a thing in its own right.
None of the above → it is unrivered.Not a third kind, and not a category of table at all — a fact about the work. Nobody has rivered it yet, so it goes on the to-do list and never into the register as an answer. A kind is what a table is, and every table has one; a table is either rivered or waiting for someone. Giving it a name like “undecided” would invent a resting place. Unrivered says out loud that work is owed.
Question four used to lie, and its first repair lied differently

It asked could you rebuild this. When the river was missing it answered no and walked on — precisely what rule 3 forbids. The first repair asked should you be able to, which fixed that and introduced a worse fault: a question you can steer by how ambitious you feel is not a question. Almost anything could be derived if you were willing to write a river for it — the same flaw this page identifies in “one subject, one river”, arriving by a different door.

So it points instead of asking. Name the line. team_memberships — two columns, a user and a team, nothing recording who joined when — has an obvious line (added to team), so it is state with a missing river, not a mystery. A surname has none, so it is simply what a noun holds. And knowledge_chunks, a document's text chunked and embedded, is state built from a noun: delete every row, run the chunker again, get it back.

Before adding a river — three questions

Reached for a new log? These three, in order, usually stop you.

iIs this a different subject, or just more detail?More detail goes into the log you already have, as more kinds of line. Wanting to know more about someone is never grounds for a second log about them.
iiWhat noun would its rows point at — and does a log already point there?If one does, you are about to record the same thing in two places. Add to that one instead.
iiiIs the subject a business object nobody logs yet?An account, a domain, a campaign. These usually belong together in one “what was done to this account” log, not one log each — and that log often already exists, unloved, as the audit trail.

The counter-intuitive result of asking these on a real system: the answer was fewer logs, not more. Five subjects looked like they each wanted one — the account, the sending domain, the email stream, the campaign, the plan. They are all the same subject wearing different hats: things inside the business that a person on the team acts upon. One log covers all five, and its shape is already sitting there in the audit trail — who, what, which thing, when.

Before you rebuild or delete anything — two more

These are separate from identifying the kind, and they are the two that cost real damage when skipped. Both were learned the hard way, in that order.

ADoes anything rely on this row simply existing?A rule in the database can make a row unrepeatable — that's a lock, whatever else the row is doing. If one is riding on a table you were about to rebuild, it has to be moved to its own table first. And check the database itself: these rules can be real, enforced, and completely invisible in the code that describes the table.
BWhat points AT this row?Not what it points at — what points back. Some links are known to the database, which will refuse a destructive delete. Others are just an identifier copied into a column, with nothing enforcing it. Those delete cleanly and silently, and you find out much later.

Two questions that catch almost everything else

Question one

If this row stopped existing, what would break — not who reads it, but what stops working?

“Nobody reads it” is a weaker answer than it sounds. The right question is what depends on it. In the audit log, 85,835 rows turned out to be read by nothing at all — every feature that queries that table filters on a column those rows leave empty.

Question two

If this can't be worked out from a river — why not?

“It can't be derived” is a symptom, not a verdict. Usually the river is missing something and the fix is to start recording it — after which the copy becomes ordinary derived data.

Where this came from. The general principle — store what happened, compute the rest — is well established and goes by event sourcing. The formal name for state and summary is a materialized view; the formal name for a river's subject is an aggregate root. Your bank balance works this way: it isn't stored anywhere, it's the sum of your transactions — which is exactly why a statement can always show you why the balance is what it is.

Three things here are not in the textbook. Heartbeats — the rule that repeated measurement is not history — are usually ignored, and standard event sourcing happily mixes metrics into the stream. “It can't be derived is a symptom” is an auditing rule; event sourcing assumes the log is complete and offers nothing for the moment you find it isn't. And the unit rule is missing entirely — the literature has almost nothing to say about how big one line should be, which is the first question anyone actually hits.

How these rules were arrived at, and why that matters for trusting them. None was designed. Each came out of a real audit of a real system, usually from something going wrong — and several replaced an earlier rule of mine that turned out to be worse. “Same subject, one river” started as a column-counting scan, which found two merges that were both already deliberately decided and missed the one that was real. “A lock lives alone” came from noticing that two of these archetypes give opposite instructions about deletion. “State is a column, not a table, unless a thing can have many” replaced a category called “settings” that never made sense. A rule that has not yet been wrong about something has not been tested.

On the name. Rivering is coined, and it is coined for one word rather than for itself: unrivered. Most databases have large stretches nobody has decided about, and there was no word for that state — so it never got pointed at, argued about, or scheduled. A problem with no name stays invisible. River · rivered · unrivered; the log itself is a river, and the work of deciding is rivering.

The case that was open, now closed: “external” is a mark, not a kind. Some data is copied from systems the business doesn't own — counts pulled back from Facebook — and equally, seed defaults that ship in its own repository. None of it can be worked out from a river in this database, and question two's answer is “because the source belongs to someone else.” That does not earn a third kind. The source of truth is outside this database attaches to kinds and archetypes that already exist: a borrowed thing is a noun marked external; a borrowed count for a closed month is a summary marked external; a borrowed current number is state marked external. The mark changes exactly one rule — rebuild becomes refetch, and a refetch is not guaranteed to agree with what you had. Everything else about the kind still applies.

And a second mark: “sealed”. A summary can be frozen by an act outside the database — a signature, an audit, a filing, a legal deadline. Once an accounting period is closed its figures may not be recomputed even though they still could be: a correction is posted as an adjusting entry in the next period, never as a rebuild of the closed one. A sealed summary has stopped being derived and become an original. The mark changes one rule — rebuild becomes forbidden, and corrections go forward as new entries — and everything else stays true.

Two marks, both orthogonal to kind, both changing exactly one rule. If you find yourself wanting a third, check first that it isn't a kind you have mislabelled.

When reality disagrees with the derivation, write a line — don't edit the copy. The shelf says 1,150 and the computed stock says 1,200. The shelf wins, and that is not the arrow running backwards. The fix is a new river line — counted 1,150 on the 31st, shrinkage −50 — after which the derivation produces the right answer on its own. An observation of the world is an event like any other, and it belongs where events belong.

For the reader who wants the arguments checked. A third companion, What Rivering Proves, states five of this page's claims as propositions with proofs, and labels the rest — empirical, heuristic, definition, policy — with what would falsify each. It is deliberately not the place to start. It exists because writing it changed four things on this page, including downgrading one rule from a theorem to a robustness argument.

How this stands against the established methods. Five of them do the same move — a small fixed vocabulary every table must fit. Kimball (fact, dimension), Data Vault (hub, link, satellite), Anchor Modeling (anchor, attribute, tie, knot), event sourcing (event store, read model), and the ERP split of master / transaction / reference data. This is the sixth attempt, not the first.

The convergence is the evidence. Data Vault's satellite and Anchor's attribute table both exist to keep a thing's history out of the thing itself — which is the noun's rule 1, reached independently by people solving auditability and schema evolution rather than this. A rule that three unrelated communities land on is load-bearing, not taste.

What differs is the job, not the ideas. All five are prescriptive — build a new warehouse this way. This is diagnostic: point it at a database somebody already grew by accident. Four of the five are warehouse-layer; this targets the operational database, which almost nothing classifies. And none of them has a heartbeat, a lock as a named shape rather than a constraint, a unit rule, or the auditing posture that “it can't be derived” is a symptom.

Where they are better, and the position on each. Formality — Anchor is 6NF with proofs; the answer is not to chase proofs but falsifiability, every rule checkable by someone who is not an expert. (The unit rule became a number; question four became “name the line”.) Temporality — Anchor handles it by construction, so borrow it: time is structural, not a rule you remember, and two times belongs in decision one. Tooling — theirs generate schemas, ours diagnoses one; that is a different fight and the lead worth extending. Performanceout of scope, said out loud. This method has nothing to say about query speed; use a dimensional model for that. The refusal is what makes the other three credible.

And the real competition is not those five. The one thing aimed at the same reader is a folk taxonomy from beginner tutorials — entity, lookup, junction, transaction, audit/history. It maps cleanly onto these kinds and archetypes (entity → noun; transaction and audit → river; lookup → a noun marked external; junction → state, usually plus a lock) and has no heartbeat, summary or lock at all. More to the point, it sorts by shape — “a junction table has two foreign keys” — where these sort by job. A shape carries no rules. That is why nothing hangs off their list and everything hangs off this one.

How far this has been tested — the denominator. Say this plainly whenever the kinds are called complete, because the sample is small. The list was derived from, and stress-tested against, one relational database of 270 tables, in one company, built by one team's habits. It has since been run against seven domains nobody here designed: double-entry accounting (which produced the sealed mark), perpetual inventory (which confirmed it — the physical count is an event, so you write a line), bitemporal records in banking, insurance and health (which produced the two-times rule), time-series rollups (which confirmed that a summary outliving its river must be a decision), graph databases (independent confirmation — Neo4j's own guidance splits “a thing that exists independently” from “an event between two things”, which is noun and river in other words), warehouse star schemas (which produced the scoping rule above), and document stores (where the concept holds but the mechanism degrades — nothing enforces a reference, and events can be embedded inside their parent). No domain has yet required a third kind. The archetypes are open and have grown once — the branch, after a production river was found carrying 317,000 rows of page views that fitted no existing shape.

What that does not license: it has not been run against anything shaped for a regulator rather than an application, nor against a system where the same fact lives in two stores at once — Postgres and Redis, say. The “one original” rule is silent about which store holds it, and the scoping rule is only half an answer. Anyone calling the list universal is claiming more than has been measured. The honest sentence is: two kinds held across one relational system of 270 tables and seven outside domains; the archetypes are open and grew once, with the branch; everything else that did not fit became a mark, a rule, or a scoping question — never a kind.