The uncomfortable answer is that the prompt stopped being the bottleneck a while ago, and almost nobody told the people still building prompt libraries. This is the piece that tells you.
I built a prompt tool once. I’ve told that story before. A clean little web app where entrepreneurs could store their prompts, run them, and stop losing good work in a graveyard of Google Docs. A few hundred signups the first week. Glowing messages. Six weeks later I could count the weekly active users on my hands.
I blamed the product. I blamed onboarding. I blamed myself, mostly, and for a long time.
I was wrong about all of it. The tool didn’t fail because it was badly built. It failed because it was solving the wrong layer. People weren’t losing their prompts. They were losing their context. And a shelf full of perfect prompts does nothing for a person whose real problem is that the model has no idea who they are, what they sell, what they already tried, or what happened in the last conversation.
So here is the direct answer to the question in the headline. Your AI is not getting worse because your prompt got worse. It is getting worse because the pile of stuff in front of the model got bigger, and every model on the market degrades as that pile grows. The constraint moved. It is no longer what the model can do. It is what you choose to put in front of it.
That shift has a name now. Context engineering. Anthropic calls prompt engineering a subset of it. And I want to be honest with you, because I am the guy known for the Perfect Prompt Framework: that framework is not dead, but it is no longer the whole game. It is one component inside a much bigger problem, and I would rather tell you that myself than let you find out six weeks after the applause.
Key Takeaways
- Context engineering has replaced prompt engineering as the highest-leverage skill, and Anthropic now formally defines prompt engineering as a subset of it.
- Chroma’s Context Rot research tested 18 leading models and found every single one degrades as input length grows, even on trivial tasks.
- On a real conversational benchmark, every model tested did better with roughly 300 tokens of relevant context than with the full 113,000-token history.
- Anthropic’s own internal evals showed that trimming stale context improved agent performance by 29 percent, and pairing it with a memory tool improved it by 39 percent.
- The people winning right now are not collecting prompts. They are curating what enters the window, and deleting the rest.
The Problem
Here is what nobody wants to admit about the prompt library era.
We were all optimizing the one part of the system that was easy to see. A prompt is a thing you can copy. It fits in a screenshot. It sells well. You can build a course around it, a Notion template around it, a whole personal brand around it. I know, because I did.
Meanwhile the actual failure mode was happening somewhere you couldn’t screenshot. It was happening in the accumulated sludge of a long conversation. In the eleven documents you pasted in “just so it has everything.” In the tool output from forty turns ago that nobody will ever read again but that is still sitting there, quietly eating the model’s attention.
You have felt this even if you never named it. The chat that was brilliant for the first twenty minutes and then started forgetting your instructions. The agent that nailed the first three files and hallucinated the fourth. The research thread that got vaguer the more you fed it. You probably assumed you needed a better prompt. So you wrote a better prompt. And it worked for a bit, because a better prompt buys you a little runway, and then the same decay came back.
I want to acknowledge how genuinely hard this is to see. Every incentive in the AI content world points you at prompts. Every model release points you at intelligence benchmarks. Nobody sells you a course on “the art of deciding what to leave out,” because leaving things out does not demo well.
And there is a second reason this hides. The intuition is backwards. More information feels like more help. Bigger context window feels like more room. Every instinct you have from being a human working with other humans says that giving someone the full picture makes them better at the job.
For language models, that instinct is wrong. And it is not wrong at the margins. It is wrong in a way that has now been measured across every major model family.
So here is the reframe. Stop thinking of the context window as storage. Start thinking of it as a budget you are spending. Everything you put in front of the model costs you something, and the currency is the model’s attention.
The Evidence
This is not vibes. Let me give you the receipts.
1. Chroma measured it across 18 models, and every one of them rotted.
In July 2025, Kelly Hong, Anton Troynikov, and Jeff Huber at Chroma published a technical report called “Context Rot: How Increasing Input Tokens Impacts LLM Performance.” They evaluated 18 large language models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3. They deliberately held task difficulty constant and varied only the input length, so any drop in performance could be blamed on length alone and nothing else.
Every model degraded. Not some. Every one. On tasks as simple as retrieving one sentence, and on a task as brainless as copying a list of repeated words back out.
Their closing line is the one I keep coming back to: “Whether relevant information is present in a model’s context is not all that matters; what matters more is how that information is presented.”
2. Three hundred tokens beat one hundred and thirteen thousand.
This is the finding that should reorganize how you work. Chroma ran the models against LongMemEval, a benchmark built on long chat histories. Two conditions. In one, the model got the full conversation history, averaging about 113,000 tokens. In the other, it got only the relevant excerpts, averaging about 300 tokens.
Across all models, performance on the focused prompts was significantly higher. Same question. Same required answer. The only difference was how much irrelevant material was sitting alongside the thing that mattered.
Read that again. Roughly 300 well-chosen tokens outperformed 113,000 tokens containing the same answer.
3. Coherent, well-organized context made things worse.
Chroma also compared logically structured haystacks against haystacks where the sentences had been randomly shuffled. Common sense says structure helps. Across all 18 models and every configuration, models performed better on the shuffled version. Structural coherence consistently hurt.
I am not telling you to shuffle your documents. I am telling you that our intuitions about what helps a model are not reliable, which is exactly why this has to be measured rather than guessed.
4. This is not new, and the labs have known for years.
Back in 2023, Nelson Liu, Percy Liang, and colleagues at Stanford published “Lost in the Middle,” which found a U-shaped curve: models do best when the key information sits at the very start or the very end of the input, and noticeably worse when it is buried in the middle. In their multi-document test, GPT-3.5 Turbo did worse with the relevant document placed mid-context than it did with no documents at all.
Adding the right answer made it worse than adding nothing. That is how strange this failure mode is.
5. Fixing context is worth more than upgrading the model.
In September 2025, Anthropic published internal eval results for its context management tooling. Automatically clearing stale tool calls and results improved agent performance by 29 percent over baseline. Adding a memory tool on top took it to 39 percent. On a 100-turn web search evaluation, the same trimming reduced token consumption by 84 percent while letting agents finish work that otherwise failed outright.
A 39 percent lift, from housekeeping. No model change.
6. The vendors are now selling efficiency, not IQ.
Look at how OpenAI launched GPT-5.6 on July 9, 2026. The headline on the announcement page was not about intelligence. It read: “More intelligence from every token, stronger performance per dollar.”
The specifics back it up. On the Artificial Analysis Coding Agent Index, GPT-5.6 Sol posted 80, about 2.8 points above Claude Fable 5, while using less than half the output tokens. On OSWorld 2.0 it hit 62.6 percent and beat Opus 4.8 while using 85 percent fewer output tokens. Clio reported that OpenAI’s Programmatic Tool Calling cut prompt tokens by 38 percent with no quality loss.
You may have seen the “54 percent more token efficient” number going around. Be careful with it. Sam Altman said it to CNBC at the Allen and Company conference in Sun Valley, not on the launch page, and OpenAI has not published the baseline it was measured against. Treat it as a company claim, not a verified benchmark. The direction of travel is real even if that particular figure is unaudited: the frontier labs are now competing on what you can leave out.
What Actually Works: Curation Over Incantation
Anthropic’s engineering team wrote the sentence I would put on a wall if I had wall space left. Good context engineering means finding “the smallest set of high-signal tokens that maximize the likelihood of your desired outcome.”
Smallest. Not most. Smallest.
They frame it as an attention budget. The model has a finite pool of attention, and every token you add draws it down. This is not a metaphor they invented for a blog post. It falls out of the architecture. Transformers let every token attend to every other token, which means n tokens create n-squared pairwise relationships. Stretch that far enough and focus thins out. Models also see far more short sequences than long ones during training, so they simply have less practice at holding a thread across a hundred thousand tokens.
So what do the people who actually do this well spend their time on? Not phrasing. Go look at where the highest-signal AI communities have drifted, places like r/ClaudeAI and r/LocalLLaMA. The conversation is not about clever wording anymore. It is about context files, subagent architecture, compaction strategy, and cost management. That is not a coincidence. That is a community that has run into the real wall.
Here is the shape of what works, and it is four moves.
Curate the front door. A small, dense, current context file beats a giant one. Claude Code’s CLAUDE.md pattern exists for exactly this reason: a compact file of who you are and how you work, loaded up front, so the model never has to guess.
Compact deliberately. When a conversation gets long, summarize and restart rather than dragging the corpse forward. Anthropic describes compaction as distilling a context window down, keeping architectural decisions and unresolved problems, discarding redundant tool output. Their guidance is to maximize recall first, then trim for precision.
Retrieve just in time. Do not pre-load everything. Keep lightweight pointers, file paths, links, saved queries, and pull the actual content only when it is needed. This is how Claude Code analyzes databases it could never fit in a window.
Split into subagents. Give a focused task its own clean window. Let it burn tens of thousands of tokens exploring, and let it hand back one tight summary. Anthropic reports subagent summaries commonly land in the 1,000 to 2,000 token range. The mess stays quarantined. The main thread stays sharp.
And my personal proof? The prompt tool. I spent months making the incantation layer beautiful. Nobody came back. The thing my people actually needed was a way to make the model already know them before they typed a word. I built the wrong layer, gorgeously.
Practical Steps
Write one context file and keep it under a page. Who you are, what you sell, who you serve, how you talk, what you have already decided. One page. If it grows past that, you are journaling, not engineering. Load it at the start of serious work instead of re-explaining yourself every session.
Start a fresh chat far sooner than feels natural. The moment a thread starts drifting, do not push through it with a better prompt. Summarize the state in a short paragraph, open a new window, paste the summary. You are manually doing what compaction does automatically, and it works.
Delete before you add. Before pasting the fourth document, ask which of the first three the model still needs. Remember: 300 relevant tokens beat 113,000 in Chroma’s test. When output goes sideways, your first move should be removing context, not adding it.
Put the important thing first or last, never in the middle. “Lost in the Middle” found a U-shaped curve for a reason. The key instruction, the key constraint, the key fact goes at the top of your input or the very bottom. Not buried at token 40,000.
Cut the distractors, not just the noise. Chroma drew a sharp line between irrelevant content and distractors, material that is topically related but does not answer the question. Distractors hurt more, and they compound as input grows. That near-miss document about your old pricing is more dangerous than a random one.
Give big side-quests their own window. Research, log-digging, competitor scans, anything that will generate a mountain of material you will reference once. Do it somewhere else. Bring back a paragraph, not the mountain.
Track cost per finished task, not cost per token. A cheap model that fails twice is expensive. This is the metric the labs are now competing on, and it is the only one that tells you whether your context strategy is actually working.
Frequently Asked Questions
Is prompt engineering dead?
No, but it has been demoted. Anthropic explicitly defines prompt engineering as a subset of context engineering. Writing clear instructions still matters and always will. It is just a small fraction of what determines output quality in real multi-turn work, where system instructions, tools, files, and message history all compete for the same finite attention budget.
What is context engineering in plain English?
It is deciding what goes in front of the model, and what does not. Anthropic defines it as curating and maintaining the optimal set of tokens during inference, including everything that lands there outside your prompt. Prompt engineering asks how to word the request. Context engineering asks what configuration of information makes the right answer likely.
If models have million-token windows, why not paste in everything?
Because window size and effective performance are different things. Chroma tested 18 models and found all of them degrade as input grows, even on trivial tasks. In their LongMemEval test, models given roughly 300 relevant tokens outperformed the same models given the full 113,000-token history containing that same answer. Capacity is not competence.
Does this apply to me if I just use ChatGPT and never touch code?
Yes, and it is probably costing you daily. Context rot is what is happening when a long chat starts forgetting your instructions or contradicting itself. The fixes are non-technical: keep a short context file, start fresh threads early, put key instructions at the beginning or end, and remove documents the model no longer needs.
Will better context actually save me money?
Anthropic’s internal evals say yes. Automatically clearing stale tool results cut token consumption by 84 percent on a 100-turn web search evaluation while improving performance. Context editing alone lifted agent performance 29 percent over baseline; paired with a memory tool it reached 39 percent. Fewer tokens and better output, from the same model.
The Close
I built something good once and watched it die anyway. I have made my peace with that. What I have not made peace with is watching other people spend 2026 polishing the same layer I polished, for the same reasons, with the same certainty, heading for the same silence after the applause.
You are not behind because you haven’t found the right prompt. There is no right prompt waiting for you in someone’s swipe file. The prompt was never the bottleneck. It was just the only part of the machine that was visible from the outside, so that’s where all of us went, and we went there with real energy and real craft and it did not matter as much as we thought.
The constraint is what you put in front of the model. That is the whole thing. And it is a strange kind of good news, because it means the lever is entirely in your hands. You do not need a better model. You do not need a bigger window. Anthropic got 39 percent out of housekeeping. You need the discipline to decide what matters and the nerve to delete the rest, which is harder than writing a clever prompt and worth infinitely more.
I’m still the guy who built the Perfect Prompt Framework, and I’ll defend every piece of it as a component. But I’d rather revise myself in public than be right in a room nobody’s in.
Stop collecting incantations. Start curating.
Curation beats incantation, every time.
If this landed, come find me. I write about this stuff daily, I get it wrong in public on a regular basis, and I would rather build this understanding alongside you than perform it at you. Follow along, connect with me, and join the conversation with the entrepreneurs already doing this work. The people figuring out context are going to look like magicians in about a year. Be one of them.
About Jonathan Mast
Jonathan Mast is the founder of White Beard Strategies, where he provides AI coaching and mentorship to entrepreneurs who want to actually use this technology rather than just read about it. He is a speaker, a writer, and the creator of the Perfect Prompt Framework. He has built things that worked, built things that failed, and believes the second category makes for better teaching. He writes daily for entrepreneurs who are tired of hype and want the part that is true.
