# The web as ground truth

## What a model remembers

A language model has two kinds of memory.

Parametric memory lives in the weights, and is created during training: broad knowledge, patterns, associations, a rough impression of everything the model has seen.

Contextual memory is part of the model prompt: the specific documents, facts, and history the model is given as evidence to produce a specific answer.

The two systems have complementary properties.

Parametric memory is broad, fast, and useful, but lossy.

Weights are a form of compression. Training preserves patterns and loses detail, often the details that become valuable later: a specific legal precedent, a precise figure in a chart, or anything too obscure to be well represented in the training run.

A frontier model can explain how a new class of cancer drugs works, but ask it for the overall survival figure from a Phase 3 readout reported last month, and it cannot. It also cannot reliably tell you whether a number came from the trial's primary endpoint or from a secondary analysis someone summarized on X.

Parametric memory is also stale. Retraining a frontier model costs tens of millions of dollars and takes many months. A model's weights cannot keep up with a world that changes daily. The richest training data can't help a model tell what affected the price of oil yesterday or what amendment a European regulator certified an hour ago.

Parametric memory alone falls short for work that demands recency and precision, which is most work that drives action in the real world.

Contextual memory can invert these properties. Given the right content, it can be exact, current, and traceable to its source.

But it has a challenge of its own: finding the right content, extracting the relevant parts, and presenting them in a form the model can use without overwhelming its limited context window. Retrieval is becoming the central bottleneck in AI.

## Retrieval by default: Harnessing the web for intelligence

The open web is the largest, freshest, broadest record of human knowledge in existence. Scientists, regulators, journalists, and companies all publish to the web before any other medium. Without it, a model is capped at whatever it memorized before its training cutoff.

All knowledge work depends on it. The best employees look things up: they check current prices, read recent filings, compare sources, and stay current in their field. Their raw intellect matters, but their impact depends on access to the world's information.

The same logic applies to AI. A model cut off from the web is a knowledge worker with vast intellect but no internet access. The model cannot learn new things, verify a claim, or look up the veracity of a source. You wouldn't impose that constraint on an employee. The same standard should apply to the AI systems used throughout your company.

Retrieval belongs in the default architecture of any AI system that handles real-world decisions.

## Agents don't think in web pages

Retrieval augmentation for AI systems began as little more than the human web search that preceded it: ten blue links, fetched by a thin wrapper around a search engine and dropped into the model's context for it to sort out.

Decades of the web shaped search into a form optimized for human consumption. For every bit of pure information, there are thousands of bits of scaffolding designed around how humans find and view content. But for models, anything that isn't information is overhead: tokens to sift through, pay for, and discard before arriving at the kernel of knowledge they need.

Even a trivial query like "who won the French Open yesterday" surfaces 99% discardable content: HTML markup, SEO filler, navigational headers, and breathless tournament coverage. Retrieve full pages of this and you quickly run out of the model's precious context budget.

A phenomenon called "context rot" makes this worse: the performance and accuracy of models degrade as their context fills up. The larger the haystack, the more likely you are to find something that resembles, but isn't, the needle you're looking for.

Across these workflows, the atomic unit of retrieval is the snippet of pertinent evidence: the sentence, paragraph, table row, chart in a PDF, or code block that nudges the model closer to a correct answer. The webpage is too coarse a unit for the job.

Because the atomic unit can be as small as a single figure traced back to its source, retrieving snippets instead of full pages unlocks a tremendous leap in the amount of reasoning that can be performed for any question, freeing up context space for more snippets and additional reasoning. Producing those snippets at web scale is a prerequisite for a context layer that can ground agents at scale.

## The context tax: paid in cost, latency, and accuracy

A page full of SEO is a minor annoyance for a single user. But at the scale we expect AIs to operate, that overhead becomes an existential limit on how capable agents can become. One employee may run hundreds of coding agents; a single CRM may enrich millions of records overnight. An agent doing deep analysis can easily encounter over a thousand pages as it works. Every "junk" token in that loop becomes a tax.

It is a cost tax, because the model has to process tokens that don't help the answer. It is a latency tax, because longer context slows inference. And it is an accuracy tax, because of finite context windows and context rot.

Attention, the mechanism LLMs use to process contextual information, creates a steep cost curve as context grows. When engineers say attention has a "quadratic" cost, they mean the compute and memory required rise far faster than the number of context tokens themselves. It is 100 times more expensive to process a 10,000-token document than a 1,000-token one.

Reasoning-heavy models already spend tokens planning, checking, and synthesizing before they answer. Make them grind through entire web pages on top of that and they slow to a trickle.

Ask a frontier model to play to its strengths instead: help it find the right pages, strip them to the relevant passages, and let it operate on information-dense snippets. That density becomes an order-of-magnitude multiplier on the efficiency of every LLM call.

With a capable retrieval layer, agents can spend their reasoning budget on the work itself rather than on cleaning up the input. They complete tasks faster, more accurately, and at a fraction of the cost, cheap enough that reaching for the web becomes the default for every agent.

## Built for the median question

If grounding matters this much, why aren't the web search capabilities offered by model labs sufficient?

The web search the labs offer is a great form factor for plenty of use cases. But it's tuned for the median question: speed, common queries, and surface-level insight, the kind that comes up in everyday chat use. Labs hill-climb on user feedback, which rewards a fast response over a thorough one.

There's also no financial incentive for the labs to make that search token-efficient. A frontier model can extract the answer from a raw web page, but that's a staggeringly expensive use of frontier intelligence. And for a lab deciding whether to improve compression or intelligence, a token saved is revenue given up. As enterprises wake up to the cost of inference, deciding when those tokens are worth it falls to them, not the labs.

A one-size-fits-most product can only offer so much control. Standardization trades away the ability to preserve provenance from index to snippet, or to decide which parts of the web to trust more or less.

Serious enterprise and specialized agentic workloads demand more: multi-step research where missing a single source can invert the conclusion of a drug trial, and agents that inspect hundreds or thousands of pages without drowning the model in tokens.

## The past, present, and future of retrieval

With each generation, models and grounding grow more interconnected. ChatGPT launched without web search and with its training data behind by 14 months. Soon entire product categories appeared specializing in combining chat with search, often at the cost of latency or a weaker underlying model. Labs now train models on knowing how and when to search. And retrieval itself is evolving, from dense snippets to agentic task execution: subagents specialized in long-horizon retrieval that run dozens of lookups to finish a single task.

The step after agentic retrieval is still on the research frontier, and no architecture has won yet. But the retrieval layer's properties are already clear: its latency, density, fidelity, and provenance. These will set the ceiling on every AI system built on top of them.