The retrieval problem nobody's talking about

The new model has a one-million-token context window. The marketing page implies you can stop thinking about retrieval. Just dump everything in. Let the model figure it out.

Try it.

Getting 1M tokens into the model was never the hard part. The hard part is that 1M tokens is a sliver of what an organization actually knows. A real enterprise corpus (emails, tickets, documents, code, meeting notes, billing records, customer history) runs into the trillions of tokens. And the relevant 2,000 tokens for any specific question are scattered across causal chains that span months. An email from January references a decision recorded in a Jira ticket from March that became a contract clause in October that’s now contradicted by a billing record from last week.

You cannot solve this by making the window larger. There is no window big enough to hold the trillion-token corpus, and even if there were, the model could not reason effectively across it. The needle isn’t in the haystack. The needle is a chain of needles through the haystack, and most of the needles can only be identified as needles after you’ve found the others on the chain.

This is why every “we have a long context window” story is followed, six months later, by “we built a retrieval system.” Retrieval was never the hack you bolt on to compensate for short windows. Retrieval is the actual unsolved problem. Long windows postponed the conversation. They did not end it.

The teams that figure this out first will have built something underneath the model: hybrid indexes that combine vector, keyword, and graph; causal-chain tracking across time; temporal state awareness; a way to resolve contradictions when two true-on-their-day facts disagree. A real synthesis layer. The teams still betting on the window will keep wondering why their demos work and their production systems do not.

The window was never the answer. The retrieval system is. Always was.