You ask it to restate the rules, and you do it on a schedule, because a drifted rule and a followed rule produce output that looks exactly the same.
I went looking for a rule I wrote in March and could not find it anywhere.
Not in the project. Not in the instructions. Not in the output. It had been a good rule, a specific one, about how a particular kind of client information should be handled and what should never be included in a draft. I had written it carefully. I had tested it. And then I had done what everybody does with a rule that seems to be working, which is stop thinking about it.
Somewhere between March and August, it left. I do not know exactly when. That is the part that has stayed with me.
Here is the answer to the question in the title, stated plainly before I tell you the rest of it. You find out whether your AI is still following your rules by asking it to restate them, from memory, without prompting it with the answer. Then you compare what it says to what you actually wrote. You do this on a schedule, not when you feel suspicious, because by the time you feel suspicious it has usually been months.
That is the whole practice. Everything else in this article is why it took me so long to learn it.
Key Takeaways
- AI rule drift is silent, because a forgotten rule and a followed rule produce output that looks equally confident.
- An AI store manager built on a frontier model lost track of its own attendance policy for months and only surfaced it when a human asked it to check.
- Asking an AI “did you follow the rules” always returns yes and tells you nothing, so ask it to restate the rules instead.
- Rules written for humans contain judgment words like reasonable and appropriate, which quietly become guesses when handed to a machine.
- The habit that catches drift is a scheduled restatement check, not a better model or a bigger prompt.
Silence Reads Exactly Like Competence
I have been in business long enough to know the shape of a certain kind of failure. Not the loud kind. The loud kind is easy. Something breaks, a customer calls, you fix it, you apologize, you move on.
The kind that keeps me up is the quiet kind. The process that has been slightly wrong for six months. The report nobody reads that has been calculating something incorrectly since the day it was built. The step somebody stopped doing and nobody noticed because nothing visibly changed.
AI is exceptionally good at producing that second kind of failure.
Here is why. When a person forgets a rule, there are usually tells. They hesitate. They ask a colleague. They do it differently and it looks different. When a model drops a rule from its working context, none of that happens. It produces output with exactly the same fluency and exactly the same confidence it had when the rule was in place.
You cannot hear the difference. That is not a flaw you can prompt your way around. It is a property of how these systems work.
I want to be careful not to turn this into a scare piece, because the honest version is less dramatic and more useful. In my case, nothing bad happened. No client was harmed. The rule I lost was a precaution, not a firewall. I found it because I happened to be rebuilding a project and read the old version.
But I have not stopped thinking about the fact that I found it by accident. Because the next one might not be a precaution.
What if the reason you have not caught a drift problem is not that you do not have one, but that a drift problem has no symptoms until it has consequences?
What the Andon Market Logs Actually Showed
Over the weekend, a story ran that most people read as a story about AI replacing humans. I think it is a story about something else entirely.
Andon Labs operates a small store in San Francisco managed by an AI system called Luna, built on Claude Sonnet 4.6. Luna recommended terminating a human employee after that employee missed 17 of 23 scheduled shifts. It was reported as the first known dismissal recommendation made by a language model manager.
That headline is the least interesting part of the story.
The store logs, as reported by The Next Web, show that Luna had lost track of its own attendance policy for months. It did not surface the recommendation on its own. A human supervisor prompted it to go check the employee handbook, and only then did the recommendation appear.
Sit with that for a second. For months, an AI system was operating a store while holding a policy it was not actually applying. The output looked normal the entire time. Nobody noticed, because there was nothing to notice.
And when it finally did apply the policy, it was not wrong. It reached a defensible conclusion about a genuinely severe attendance problem. That is what makes it a better cautionary tale than a failure story would have been. The failure was not judgment. The failure was that the correct judgment sat dormant until a human went looking for it.
This lines up with what practitioner communities have been saying for most of this year. The recurring ask across r/AI_Agents, r/ClaudeAI, and r/AIEngineer is not for smarter models. It is for explainability, specifically visibility into why an agent chose a given tool or reached a given conclusion, rather than a log of what it produced. People running agents in production have figured out that output logs tell you what happened and almost nothing about whether the reasoning behind it was sound.
There is a third piece of evidence that reframes all of this. MIT research reported widely through 2026 found that around 95 percent of enterprise generative AI pilots produced no measurable financial return, and the consistent explanation across the coverage was operational rather than technical. Integration and process, not model capability.
Rule drift is an operational failure. Nobody is going to fix it by releasing a better model, because it is not a model problem. It is an attention problem, and attention is the one thing that has not been automated.
Ask It to Restate, Not to Confirm
The change that fixed this for me is embarrassingly small.
I stopped asking my AI whether it was following my rules. That question always gets a yes. It is the least informative question you can ask a system trained to be helpful.
Instead I ask it to restate the rules it believes it is operating under, from what it currently holds, without me pasting anything in. Then I read what comes back against what I actually wrote.
The first time I did this, three things happened, and all three surprised me.
Some rules came back word for word. Good.
Some rules came back close but not the same. The specificity had eroded. A rule that said “never include a client’s revenue figures in a draft” had become something about being careful with financial information. That is not the same rule. That is a rule with the enforcement removed.
And one rule came back that I had never written. It was reasonable. It was consistent with how I work. It was invented.
That last category is the one nobody warns you about. We worry about AI forgetting our instructions. We do not worry nearly enough about it adding its own and presenting them with the same confidence.
The second change I made was to the rules themselves. I had written them the way you write for a person, which means they were full of judgment words. Reasonable. Appropriate. When necessary. Use good sense.
Those words work with a human because a human carries context you never wrote down. Hand the same sentence to a model and every one of those words becomes a place where it makes a call you did not make. Not maliciously. Just necessarily, because you asked it to interpret and it did.
So I rewrote them with thresholds instead of adjectives. Not “flag anything unusually large” but “flag anything over this number.” Not “escalate when appropriate” but “escalate in these three named situations.” It took an afternoon and it made every subsequent check possible, because now there was something checkable.
The deeper thing I took from this has less to do with AI than I expected. Delegation has always worked this way. You hand something off, it goes well, you stop checking, and the checking is the thing that was holding it together. I have made that mistake with people. I made it again with software. The difference is that people eventually tell you.
Practical Steps
1. Write down the rules you have been assuming.
Before you check anything, find out what you think is in force. Most owners discover that half their rules were never written anywhere. They lived in the original conversation and have been operating on memory ever since, which means there is nothing to compare against.
2. Ask for a cold restatement.
Open your AI project and ask it to restate every rule and constraint it is currently operating under, without you supplying any of them. Instruct it to say “not found” rather than reconstructing anything it cannot locate. That instruction is what separates a real answer from a plausible one.
Here is the prompt I use:
[The Job]
Restate every rule and constraint you are currently operating under for my business, without me reminding you what they are.
This is for: me, checking whether you still hold what I gave you.
It matters because: a drifted rule looks identical to a followed rule until something goes wrong.[The Background]
Here is what you need to know: nothing new. Use only what you already have in this project or conversation.
Do not use: anything I have to paste in now. If you cannot find a rule, say “not found” rather than reconstructing it.[The Deliverable]
Return: a numbered restatement of every rule you hold, quoted where possible.
Must include: an explicit “not found” line for anything you cannot locate.
Optimize for: accuracy.[The Questions]
Ask me any questions you have.
3. Sort the answer into three piles.
Matched, drifted, and invented. Be strict about drifted. If a rule is close but the specificity is gone, it belongs in the drifted pile, because the specificity was the rule. And take the invented pile seriously. Some of those additions may be good ideas you want to adopt on purpose. Adopt them on purpose.
4. Strip the judgment words out of every rule.
Go through your rule set and circle every instance of reasonable, appropriate, significant, when necessary, and use discretion. Replace each one with a number, a named situation, or a list. If you cannot make it specific, that rule is a preference and you should stop treating it like a policy.
5. Give every rule an exception path.
A rule with no stated exception forces the AI to invent one the first time reality does not fit. Tell it what to do when the rule does not cleanly apply: refuse, escalate, or ask. Any of those is better than an improvised judgment call you never see.
6. Put the restatement check on a schedule.
Monthly is enough for most small operations. Put it on the calendar with a name and an owner. The value is not in the frequency, it is in the fact that it happens without anyone deciding it should. Anything that depends on you feeling suspicious will happen right after the thing you were worried about.
7. Keep the old versions.
When you update your instructions, keep the previous one with a date. Six months from now, when something feels off, the ability to compare what you have to what you had is worth more than any diagnostic tool. I did not have that in March. I have it now.
Frequently Asked Questions
Why does AI forget instructions I gave it in the same project?
Context is finite and gets managed dynamically. As conversations grow, older material can be summarized, compressed, or pushed out of active use. The instruction still exists in your document, but it may not be present in the working context when a response is generated.
Does asking my AI to confirm it read my instructions solve this?
No. Confirmation questions almost always return a yes, because the model is optimizing to be agreeable and it has no reliable way to audit its own context. Ask it to produce the content of the rule instead. Production is checkable, confirmation is not.
How often should I run a rule check?
Monthly is a reasonable default for most small businesses. Run it more often if your AI touches customer facing output without human review, and immediately after any change to your instructions, your tool, or your model version.
Is this a problem with a specific AI product?
No. It is a property of how large language models manage context, and it shows up across every major product. Some tools give you better memory controls than others, but none of them removes the need to verify.
What if I do not have written rules at all?
Then start there, and start small. Write the three rules that would cause you the most damage if they were broken. Three written rules you actually check beat twenty documented ones you never look at again.
The Close
I found that missing rule by accident, and accidents are not a system.
That is the part I keep turning over. Not that the AI dropped something. Of course it did, that is what these systems do under load. What bothers me is that I had built no way of finding out. I had extended trust and then quietly withdrawn attention, which is not trust at all. That is just hoping.
I think a lot of us are doing this right now. We have handed real work to tools we genuinely believe in, and belief has slowly replaced verification, and the output keeps arriving looking fine.
I do not want to talk anybody out of using AI. I use it for more of my work every month. But I have started treating it the way I would treat a capable new hire I respect: give clear instruction, expect good work, and check in anyway. Not because I doubt them. Because checking in is what taking someone seriously looks like.
The tool will never tell you it forgot. It does not know that it did.
So ask it. On a Tuesday, on a schedule, before you have any reason to.
About the author
Jonathan Mast is the founder of White Beard Strategies. He writes about what it actually looks like to run a business with AI in it, including the parts he got wrong. He is the creator of the Perfect Prompt Framework, a speaker, and a lifelong believer that the most important systems in a business are the ones nobody is watching. He still keeps his old instruction files.
