What a Frontier LLM Actually Is
Strip away the hype. A large language model is four numbers in a trench coat.
Tokens. The raw material. One token ≈ 3–4 letters. “Tokenizer” becomes two tokens: token + izer. LLaMA 3 was trained on 15.6 trillion of them — the rough equivalent of every book, webpage, and article a human could consume across 150,000 lifetimes.
Parameters. The machinery. 405 billion dials inside the model, each tuned during training so the right answer emerges from the right input. More dials = more nuance. More cost. More memory. More everything.
FLOPs. The labour bill. Every token processed against every parameter requires roughly 6 mathematical operations. Multiply it out: 6 × 405B × 15.6T = 3.8×10²⁵ operations. A number that has no human intuition attached to it.
TFLOPS. The machine speed. Each H100 GPU can execute 400 trillion operations per second — in theory. In practice, data transfer between chips, memory bottlenecks, and coordination overhead drag real utilisation down to ~45%. That gap between theoretical and actual is called Model FLOP Utilisation (MFU). Even Meta, with 16,000 H100s running in parallel, couldn’t push past it.
The result: 70 days. 26 million GPU hours. ~$75 million. One model. No memory. No identity. No values.
The Number That Changes Everything
Buried in the Stanford CS229 lecture on LLMs is a metric most people scroll past.
Context window: 200,000 tokens.
This is the model’s working memory — everything it can see at once. The conversation you’re having, the documents you’ve loaded, the instructions it’s been given. All of it competes for that fixed space.
Here’s the collision that nobody talks about:
Context window (fixed ceiling) = ~200,000 tokens
One memory entry (md file) = ~500–2,000 tokens
Corpus at 100 entries = ~100,000 tokens ✓ fits
Corpus at 200 entries = ~200,000 tokens ✓ at the limit
Corpus at 500 entries = ~500,000 tokens ✗ impossibleThe moment your memory corpus exceeds the context window, something has to be forgotten. Not chosen. Not managed. Just — dropped.
This is the statelessness problem. Close the tab. Thirty thousand tokens of conversation history vanish. The model retains nothing. The $75M machine has amnesia by design.
Where DIE Enters
The DIE framework — Dimensional Intelligence Expansion — is built on four empirical conditions that distinguish genuine distributed intelligence from statistical autocomplete at scale.
Two of them are directly implicated by everything above.
C1 — Memory Accumulation
The condition: Can the agent demonstrably accumulate experience across interactions?
In the centralised LLM paradigm, the answer is no. Each inference call is stateless. The model was trained once, frozen, and deployed. It cannot learn from what happens to it after deployment. Fifteen trillion tokens of pretraining, and then — nothing. Every conversation starts from zero.
The DIE approach treats this differently. Memory accumulation happens through externalised episodic storage — markdown files written to disk after each session, forming a growing corpus of structured experience. Each entry is timestamped, attributed, and versioned. The corpus is the memory. The act of writing to disk is C1.
“Files are memory. If it’s not written to disk, it doesn’t exist.”
C1 is not a philosophical claim. It’s a measurable condition: does the agent’s knowledge base grow monotonically across interactions? Can that growth be verified? The corpus architecture makes it auditable.
C2 — Memory Loss (Controlled Forgetting)
The condition: Does the agent lose memory in a controlled, principled way — or chaotically?
Here is the brutal irony of C1 success: the better an agent gets at accumulation, the sooner it hits the context window ceiling. C1 success causes C2 onset. An unbounded corpus colliding with a bounded context window produces forced, random forgetting — the worst possible outcome.
This is not a hypothetical. It is a mathematical inevitability.
The DIE framework insists that C2 must be designed, not merely suffered. Controlled forgetting requires:
Retrieval over loading. Don’t load the entire corpus into context. Embed all entries as vectors. At query time, retrieve only semantically relevant entries. The corpus can grow arbitrarily; working memory stays bounded and purposeful.
Hierarchical compression. Older entries get summarised. Raw detail is archived; distilled essence stays hot. The agent retains the shape of old experience without the full weight of it.
Memory tiering.
Hot → last 7 days (always in context)
Warm → last 90 days (loaded on relevance signal)
Cold → archive (queried explicitly)The key distinction: in the centralised LLM paradigm, forgetting is a hardware constraint. In DIE, forgetting is a design decision. C2 is not about whether the agent forgets — it will. C2 is about whether the forgetting is principled.
The Bitter Lesson, Reframed
Richard Sutton’s Bitter Lesson (2019) observed that as compute scales, the things that seem architecturally clever become irrelevant. Raw scale wins. The Stanford lecture confirms it: scaling laws show that doubling compute reliably decreases loss, regardless of architectural novelty.
The DIE corollary is this:
If compute is the only axis that matters, and compute has physical ceilings — then the winning strategy is to change the axis entirely.
Not more parameters per model. More agents per mesh. Not deeper context windows. Better memory management across a distributed intelligence layer.
The $75M machine is impressive. But it is impressive the way a single very large engine is impressive — before someone points out that a thousand smaller, coordinated engines can move more with less friction, less capital, and no single point of failure.
What This Means Practically
The ThinkMasters corpus — the growing archive of structured entries on VM2210 — is not just a research database. It is a live implementation of C1 and C2 in the smallest possible form.
Every entry added = C1 in action.
Every retrieval decision = C2 being designed.
Every tiering and summarisation choice = controlled forgetting being practised.
The infrastructure is simple. The governance is what matters.
The frontier labs spent $75M to build a model that forgets everything the moment you close the tab. DIE asks a different question: what if forgetting was a feature, not a flaw — provided it was principled, auditable, and under the agent’s own governance?
That question is C2. And it doesn’t cost $75M to test.
DIE Preprint v1 — April 2026. Empirical validation ongoing via OpenClaw/agenti2 on Base. PI: atg.eth.