Everyone teaches the expert role line as step one of a good prompt, but the published research says it does not reliably improve accuracy, so here is what I put in its place after cutting it from my own framework.
Suggested SEO meta title: Does "Act As An Expert" Prompting Actually Work? What Research Shows
I Cut The Expert Role From My Own Framework
Version 1.0 of my Perfect Prompt Framework opened with an expert role. Step one was telling the AI who to be.
Version 2.0 does not. Step one is now [The Job].
I removed it for an unglamorous reason. The role line was the part everyone already got right. "You are a world-class copywriter with twenty years of experience." Perfect. And the output still came back vague.
So it was the easiest thing to cut. I cut it. Nothing got worse.
Here is the direct answer to the question in the headline: no, telling an AI to act as an expert does not reliably make its answers better. The largest study on this tested 162 different personas across four families of models and 2,410 factual questions and found adding a persona did not improve performance compared to adding no persona at all.
What it can change is tone. A confident voice on a wrong answer is worse than a hesitant one, because you stop checking.
Candidly, that is the part I did not see coming. I thought the role line was neutral. Harmless filler. It turns out that in some settings it is not filler at all. It is a confidence dial with no accuracy dial attached.
Two pieces of prompting orthodoxy broke this week, and they broke in the same direction. The expert persona. The bigger agent swarm. Both are additions that cost more than they return.
Less is the upgrade. That is the thesis. The rest is evidence and a replacement system.
Key Takeaways
- Adding an expert persona does not reliably improve factual accuracy, and the best persona for a given question is close to impossible to predict in advance.
- A confident-sounding role can raise your trust in an answer without raising its quality, which is the dangerous combination.
- Long chats degrade measurably, so starting fresh with a short summary beats pushing one thread further.
- More AI agents in parallel buys speed, not efficiency, and the coordination penalty compounds as you add agents.
- Specifying what to leave out moves output quality more than naming a role ever did.
Why "Act As An Expert" Became Gospel
It became gospel because it feels like it works.
You type "you are a senior financial analyst" and the answer comes back wearing a suit. Better vocabulary. More structure.
That feels like intelligence. It is actually just register. The model shifted which words it reaches for, not which facts it knows.
I understand why it spread. It is teachable in four seconds, and it gives a non-technical business owner something concrete to do on their first prompt. That first small win matters.
I taught it for exactly that reason. It was a good on-ramp.
Here is the thing about on-ramps, though. They are supposed to merge you onto the highway. Mine kept people parked on the shoulder, satisfied, because they had done the one step everybody called important.
I watched this pattern over and over. Someone would show me a prompt and the role line would be beautiful. Three adjectives, a seniority level, maybe an industry. Then the actual request would be nine words long with no context, no audience, and no mention of what the thing was for.
The role was doing all the work in their head and none of it in the prompt.
I want to be fair to the difficulty. This is confusing territory, because the AI companies use personas in their own default system prompts. ChatGPT ships "you are a helpful assistant." So when a researcher tells you personas do not help, you are entitled to ask why the labs keep using them.
The honest answer is that a persona does other jobs. It sets tone. It constrains vocabulary. It makes a product feel consistent. Those are real jobs. They are just not accuracy jobs, and accuracy is what most entrepreneurs think they are buying when they type the role line.
That gap is the whole problem. Here is the research.
What The Research Actually Found
One. Personas did not improve accuracy across 2,410 questions.
The paper is "When 'A Helpful Assistant' Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models," by Mingqian Zheng, Jiaxin Pei, Lajanugen Logeswaran, Moontae Lee, and David Jurgens. It was accepted into Findings of EMNLP 2024.
They built 162 roles spanning six types of interpersonal relationship and eight domains of expertise, then ran them against four popular families of LLMs on 2,410 factual questions. Adding a persona did not improve performance compared to the control with no persona.
The part that should end the debate: they also tried to find a way to pick the best persona for each question in advance. Cherry-picking the winner after the fact did improve accuracy. Predicting it beforehand performed, in their words, often no better than random selection.
So the upside exists but you cannot aim at it. That is not a technique. That is a slot machine.
Two. Role prompts change the surface, not the reasoning.
A separate 2025 study, "Dissecting Role Cognition in Medical LLMs via Neuronal Ablation," looked inside the model. The researchers tested clinical role prompts, medical student versus resident versus attending physician, and found role prompts did not significantly enhance medical reasoning. They primarily affected surface-level linguistic features, with no evidence of distinct reasoning pathways for different roles.
Same brain. Different costume.
Three. The confidence problem is documented separately.
UNPOPULAR OPINION: the confidence of an AI answer tells you almost nothing about its correctness, and prompting choices can widen that gap.
Research presented at ICLR 2025, "Taming Overconfidence in LLMs: Reward Calibration in RLHF," found the training process used to make models helpful pushes them toward verbalized overconfidence, with the reward models involved biased toward high-confidence scores regardless of actual response quality.
That is why the r/PromptEngineering post that topped the subreddit on October 4, "Has anyone noticed that asking llms to act like an 'expert' actually leads to them being more confidently incorrect?", landed at 47 points and 35 comments. Practitioners described from the outside what calibration researchers describe from the inside.
Four. More agents buys speed, not efficiency.
Toby Ord published a piece called "Swarm Scaling" that Jack Clark covered in Import AI 475 on October 5. Credit to Ord for the work and to Clark for surfacing it.
Ord's framing is that AI swarms are a new form of inference scaling, and the main reason to use one is speed rather than efficiency. As printed in the newsletter: "The 4-agent swarm needed about twice the total number of tokens to get the same performance, but in terms of tokens per agent, it only needed half as many. Since the agents are run in parallel, this means it can theoretically achieve the same task in half the time."
Then the cost. Ord identifies a "stepping on toes" parameter borrowed from economists who study large human teams: "Scaling up the number of agents in the swarm by 10x doesn't get as much performance as using 10x as many tokens with one agent. Instead it gets 10^lambda x as much, which is 3x to 5x. And this shortfall accumulates quickly for larger scaleups, with the swarm falling further and further behind."
Ten times the agents. Three to five times the result.
Which is why "The biggest improvement to my multi-agent system was making the agents less important" topped r/AI_Agents on October 4 with 30 points and 39 comments. The builders found the curve before the rest of us read the math.
Five. Long chats actually do get worse.
Chroma Research published a report called "Context Rot" evaluating 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, and found models do not use their context uniformly. Performance grows increasingly unreliable as input length grows, even on simple tasks. That sits on top of the "Lost in the Middle" work by Nelson Liu and colleagues, which showed models retrieve best from the start and end of a context and worst from the middle.
On October 5 and 6, "your long chat getting dumber the longer it goes isn't in your head either. starting fresh with a summary fixes it" ran in r/PromptEngineering, and "Your AI chat got too long. Now what?" ran in r/ClaudeAI. The same observation surfaced in six different subreddits inside 48 hours.
All five findings point the same way. The wins are in subtraction.
The Perfect Prompt Framework 2.0
Here is the architecture I use now. Four steps, in order, each one earning its place by eliminating a specific failure.
[The Job] What you want done, who it is for, and why it matters.
This is step one because it is the step that actually changes the output. The AI is not guessing at competence, it is guessing at purpose. "Write an email" and "write an email to a customer who just missed their second payment and I want to keep the relationship" produce different work because only one of them has a job.
[The Background] What the AI needs to know. Files, facts, constraints, and what to leave out.
The "what to leave out" clause is new in 2.0 and it is the one people underestimate. Given the context rot findings, every irrelevant document you paste in is a tax on the answer. One more thing in the middle of the pile for the model to lose.
An r/PromptEngineering post on October 5 put it better than I could: "telling the model what to leave out fixed more than any instruction about what to include."
One caution, because I would rather be accurate than clever. Scoping and forbidding are not the same thing. "Use only the 2026 figures, ignore the 2024 deck" is scoping, and it works. "Do not use the word synergy" is forbidding, and a January 2026 paper, "Semantic Gravity Wells: Why Negative Constraints Backfire," found that naming a forbidden word primes the model to produce it. In 87.5 percent of measured violations, that priming effect was the cause.
So define the boundary of the material. Do not play whack-a-mole with words.
[The Deliverable] Format, length, required elements, and optionally the perspective it is written from.
Here is where a role goes if you still want one. Not as an identity claim. As the point of view the answer is written from.
"Written from the perspective of the operations lead who has to execute this" is useful. "You are a world-class operations expert" tells the model nothing about the output.
That sounds like splitting hairs. It is not. One shapes the deliverable. The other decorates the preamble.
[The Questions] The exact phrase "Ask me any questions you have."
This is the step that saved my prompts, and it is the opposite of a persona. A persona tells the model to pretend it has what it needs. This line tells it to admit what it does not.
I would rather answer three questions up front than rewrite a 900-word draft built on a wrong assumption.
The system logic: purpose, then material, then shape, then a check for gaps. Role gets demoted from an identity to a camera angle, which is all it was ever doing.
How To Rebuild Your Prompts This Week
1. Delete the role line from your three most-used prompts. Run them again with nothing else changed and compare side by side. On most business writing tasks you will not be able to tell which is which, and that is the point.
2. Replace it with a job statement. One sentence: what gets produced, who reads it, what happens next because of it. People skip that third part, and it is the one that disciplines the whole answer.
3. Write your exclusions explicitly. Name the material that is out of scope. "Use the attached Q3 report only, ignore the older forecast" beats three adjectives about expertise.
4. Move any role you still want into the deliverable. Phrase it as perspective, not identity. "Written for a reader who has never used this software" beats "you are an expert technical writer."
5. Always end with "Ask me any questions you have." Then actually answer them. Do not type "just proceed." You skipped the valuable part.
6. Start a new chat when the thread gets long. Paste a four-line summary of the decisions you already made. The degradation is gradual, so you get no warning. Treat 20 exchanges as your reset signal.
7. Add an agent only when you need the answer faster. Not when you want it better. Ord's numbers say more agents buy time, not quality, and you pay for them either way.
Here is a full worked example.
[The Job]
Write a one-page client update explaining why the [PROJECT NAME]
delivery date moved from [ORIGINAL DATE] to [NEW DATE]. It is for
[CLIENT NAME], who is a [CLIENT ROLE] and is not technical. It matters
because this is the second schedule change and I need them to stay
confident enough to renew in [RENEWAL MONTH].
[The Background]
Use only the two attached files: the project timeline and my notes from
the [DATE] call. The delay cause is [ROOT CAUSE IN PLAIN LANGUAGE].
Leave out internal vendor names, our staffing changes, dollar amounts,
and anything from [SUPERSEDED DOCUMENT NAME], which is out of date.
[The Deliverable]
One page, under 400 words. Four short sections: what changed, why, what
we are doing about it, what we need from you. Written from the
perspective of someone reading this between meetings who has to make
one decision. Plain language, no jargon, apologize once at most.
[The Questions]
Ask me any questions you have.
No role line anywhere in it. It is harder to write than "act as an expert project manager," and that is exactly why it works better. The effort moved from the costume to the brief.
Frequently Asked Questions
Does "act as an expert" ever help?
Sometimes, but you cannot predict when. The Zheng and Pei team found that cherry-picking the best persona per question after the fact did raise accuracy, while picking it in advance performed often no better than random. If a gain you cannot aim at is your strategy, you do not have a strategy.
Is there research that says role prompting does work?
Yes, and I will not hide it. A 2023 paper, "Better Zero-Shot Reasoning with Role-Play Prompting," reported ChatGPT accuracy on the AQuA math set rising from 53.5 percent to 63.8 percent with role-play prompting. The evidence is genuinely mixed and task-dependent. That mixed record is why I stopped teaching it as step one.
Should I stop using personas for brand voice?
No. Voice and tone are exactly what personas are good at. The research found personas shift surface-level language, which is what you want when you are matching a brand. Just do not expect a voice instruction to make the facts more reliable. Those are two different jobs.
How long is too long for one AI chat?
There is no hard number, because Chroma found the decline is gradual rather than a cliff and shows up well before the window is full. A practical rule: when you notice yourself re-explaining something you already explained, start a fresh chat and paste in a short summary of your decisions.
Do more AI agents produce better work than one?
Mostly they produce the same work faster. Toby Ord's swarm scaling analysis found 10 times the agents returns roughly 3 to 5 times the performance of 10 times the tokens on a single agent, and the shortfall grows as you scale up. Add agents for speed, not for quality.
The Easiest Thing To Cut Is Usually The Thing You Were Proudest Of
I taught the expert role line. I put it in step one. People repeated it back to me, and I felt good about that.
Then I cut it, and nothing got worse.
I am telling you because the honest version of expertise is admitting which parts of your advice were load-bearing and which were decoration. The role line was decoration. A nice hat on a prompt that still had no job.
You are not behind because you have been typing "act as an expert." You were following the best advice available, and most of us were giving it. We learn in public and we correct in public.
So correct it this week. Delete the role line from one prompt. Write a real job statement instead. Tell the AI what to leave out. End with "Ask me any questions you have."
Then watch what happens when you stop dressing up the prompt and start briefing it.
The AI never needed to believe it was an expert. It needed you to tell it what you actually wanted.
About the author
Jonathan Mast is the founder of White Beard Strategies, where he teaches non-technical entrepreneurs to use AI to amplify the skill and experience they already have rather than replace it. He runs a large community of business owners learning AI together, created the Perfect Prompt Framework, and speaks on practical AI adoption for small businesses.
P.S. Version 2.0 of the framework is one step shorter than version 1.0 and gets better results. I am told that is a bad way to sell a framework. I think it is the only honest way to update one.
Sources
- When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models (Zheng, Pei, Logeswaran, Lee, Jurgens; Findings of EMNLP 2024)
- Prompting with Social Roles, code and data repository
- Dissecting Role Cognition in Medical LLMs via Neuronal Ablation
- Better Zero-Shot Reasoning with Role-Play Prompting
- Taming Overconfidence in LLMs: Reward Calibration in RLHF (ICLR 2025)
- Swarm Scaling, by Toby Ord
- Import AI 475: "Who chooses what AI gets to do?" by Jack Clark
- Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma Research)
- Lost in the Middle: How Language Models Use Long Contexts (Liu et al.)
- Semantic Gravity Wells: Why Negative Constraints Backfire (Rana, January 2026)