A practitioner spent nine engineer-months to save 200 hours, and that was the success story. This article answers the question nobody in my industry wants asked: what does the real arithmetic of AI automation look like, and which automations are actually cheap?
Nine Engineer-Months to Save 200 Hours
An engineer who has been building machine learning systems commercially since 2019 posted something last week that I have not been able to put down.
He built a multi-agent system that screened hackathon applications for a small venture fund. His words: "EVERYTHING went wrong. Every step." And then: "In the end it worked: analyzing all applications from one hackathon went from 200+ hours to 4.5. Real business result."
Here is the part almost nobody reports. It took a team of three AI engineers three months. Nine engineer-months to save roughly 200 hours per hackathon.
He added one more line, almost in passing: "in all this time NOT ONCE did something work on the first try."
So here is the direct answer to the question in the title. Building a real automation for a real business process costs more than the thing it replaces, usually by a factor you would find embarrassing to put in a slide. The automations that genuinely come in cheap are small, personal, and boring: one person, one task they already do every week, no engineers, no platform. Nobody posts those, because "I saved twenty minutes on Tuesdays" does not get shared.
And now the part I owe you.
I sell AI education. That same practitioner wrote the sharpest line I have read about my own industry this year: "the endless opportunities are… selling AI to people who are scared of being left behind by AI."
I am inside that category. Not adjacent to it. In it. When you buy from me, some part of the reason is probably fear, and I benefit from that fear whether I stoke it or not.
I cannot step outside the category. What I can do is be the version of it that reports the denominator.
Key Takeaways
- A real production automation often costs more engineering time than the manual work it replaces, and the honest unit is engineer-months, not features.
- The automations that pay back quickly are almost always one person automating one recurring task they already know how to judge.
- Published AI success stories are selected for being shareable, so the ones that failed, got switched off, or never shipped are missing from your sample.
- Every case study is marketing until it names four things: how many people, how long, to save how much, and who checked the output.
- That test applies to my content too, and when I cannot give you those four numbers, you should discount what I am telling you.
The Problem Is Not That AI Does Not Work
The problem is that the cost side of the ledger never gets posted.
You have seen the post. Someone built an agent over the weekend, it does the work of a team, here is a screenshot. Maybe you tried to build the same thing. It took you two weeks and it still breaks on inputs you did not anticipate.
Candidly, that gap is not a skill gap. It is a reporting gap.
An operations person described getting an agent working, feeling like a genius, and then, in his words, he "looked at what I'd given it access to and switched it off." That is a complete AI project. Built, worked, killed. It will never be a case study, because nobody posts the one they turned off.
A developer ten months into a shipped iOS app reported 4,000 downloads and about $235 in sales, all from Reddit, no ads. Ten months. $235. He also reported something far more useful than his revenue: "My best post got 150k views by describing the problem, not the app." That insight is only visible because he published the number instead of the vibe.
Users of the new autonomous assistant tier from one of the big labs report it is slower and worse than the chat box they already had. One, verbatim: "I upgraded my ChatGPT account just to try dots and kind of regretting it."
A voice-agent operator who disclosed the AI up front watched 50 percent of people hang up inside ten seconds, and refused to spin it: "On its own that number doesn't tell you much because people hang up on unknown numbers all the time." That is what honesty sounds like in a field that rewards the opposite.
None of these people are failures. They are the ones showing their work.
Here is the reframe. The question is not whether AI automation works. It demonstrably does. The question is what size of win is available to you at a cost you can actually pay, and the answer to that is much smaller and much more useful than what you are being sold.
What the Research Actually Shows
I want to put real numbers against this, including the numbers that make my own argument look worse.
One. Even the famous failure statistic has a fuzzy denominator. You have seen the MIT headline: 95 percent of generative AI pilots fail. It comes from a Project NANDA report, The GenAI Divide: State of AI in Business 2025. The report's own line is that "just 5% of integrated AI pilots are extracting millions in value, while the vast majority remain stuck with no measurable P&L impact." Its sample is a review of 300-plus publicly disclosed initiatives, interviews with 52 organizations, and surveys of 153 senior leaders. It describes itself as preliminary findings. It is not peer reviewed, and early press coverage reported different sample sizes than the report does. I have quoted this statistic. I should have told you all of that the first time.
Two. When researchers measured developer productivity under controlled conditions, they found a slowdown, then published that their own follow-up was unreliable. METR ran a randomized controlled trial with 16 experienced open-source developers across 246 real tasks in their own repositories. Developers took 19 percent longer with AI tools. They had forecast a 24 percent speedup, and after living through the slowdown they still believed AI had sped them up by 20 percent. In February 2026, METR published a follow-up of 57 developers across 143 repositories and 800-plus tasks, and then told everyone the data was not trustworthy: 30 to 50 percent of developers admitted withholding tasks they did not want to attempt without AI. Their new raw estimates point toward speedup, with confidence intervals that cross zero. They published the mess instead of the headline. That is the standard.
Three. The maintenance cost was documented eleven years ago and nobody reads it. D. Sculley and nine Google colleagues published "Hidden Technical Debt in Machine Learning Systems" at NIPS 2015. Their argument: it is "dangerous to think of these quick wins as coming for free," because "it is common to incur massive ongoing maintenance costs in real-world ML systems." Their most quoted finding is that "a mature system might end up being (at most) 5% machine learning code and (at least) 95% glue code." Nine engineer-months to save 200 hours is not surprising. It is the documented shape of the work.
Four. Where AI does deliver measured gains, those gains are concentrated and specific. Erik Brynjolfsson, Danielle Li, and Lindsey Raymond studied 5,172 customer support agents using a generative AI assistant, published in the Quarterly Journal of Economics in 2025. Average productivity rose 15 percent in issues resolved per hour. The distribution is the real finding: less experienced and lower-skilled workers improved in both speed and quality, while the most experienced and highest-skilled saw small speed gains and small quality declines.
Five. At the level of the whole economy, the measured effect is small so far. Daron Acemoglu's "The Simple Macroeconomics of AI," NBER working paper 32487 and later in Economic Policy, estimates no more than a 0.66 percent increase in total factor productivity over ten years, and under 0.53 percent on his adjusted estimate. That is the Solow paradox rhyming again: you see it everywhere except in the statistics. Plenty of economists argue the estimate is too low. Nobody has yet produced the national productivity surge the marketing implies.
The conventional narrative says AI automation is a land grab and you are late. The evidence says it is an engineering discipline with real costs, uneven returns, and a measurement problem. Census Bureau data from early 2026 puts overall US business AI use between 17 and 20 percent, so most of the economy has not actually moved yet.
The Shape of a Win That Actually Comes in Cheap
Alicia Lyttle posted a list of things she uses ChatGPT for, and it is the best counterexample to everything above, precisely because it is unimpressive.
Her headline item, her claim and not a figure I can verify: "My press release agent asks me 5 questions, then writes the release. 3 hours became 5 minutes."
Look at the structure, not the number.
One person. One task she already does regularly, and has done enough times to know instantly when the output is wrong. No engineers. No platform. If it produces garbage, she deletes it and writes the release herself, and she has lost five minutes.
Her other items are the same size. A scheduled job that emails her the morning's AI news. An agent that reads twelve months of Gmail receipts and flags recurring charges. A note in memory so she never re-explains herself to the model.
Credit where it belongs: that list is a better curriculum than most paid AI courses, including things I have sold.
Here is why the small ones work and the big ones do not.
The nine-engineer-month hackathon system had to be right without a human watching. It processed hundreds of applications where a wrong screen means a real company gets rejected. That demand for unsupervised correctness is where the cost lives. Every edge case has to be handled, because nobody is going to catch it.
Alicia's press release agent is supervised at the moment the work is produced, by someone who has written enough releases to recognize a bad one in three seconds. The quality control is free because it is already in her head.
That is the whole secret, and it is not a growth hack. The cheap automations are the ones where you are still the quality gate. The expensive ones are the ones where you are trying to remove yourself.
This also explains the Brynjolfsson finding from the other direction. AI lifts people most when they do not yet know what good looks like, which means it also fails them hardest, because they cannot see when it is wrong. Automate what you already know.
So the practical boundary is this. Pick tasks where you have the judgment, where the frequency is high enough that minutes compound, and where being wrong costs you a redo rather than a customer. Stay inside that box until you have done it four or five times and measured it.
I have gone bankrupt before. I know exactly what it feels like to believe a projection because I needed it to be true. Writing the actual minutes in an actual spreadsheet is not bureaucracy. It is what keeps you from building the expensive version of a problem you did not have.
How to Find Your First Automation That Actually Pays Back
1. List every task you personally did more than twice last week. Not your team's tasks. Yours. Write the honest number of minutes each one took, not the number you wish it took. You are hunting for frequency, because minutes only compound when the task repeats.
2. Cross off anything where you cannot spot a bad output in under ten seconds. This is the filter that matters most. If you have to go research the answer to know whether the AI got it right, you are not the quality gate and the project just got ten times more expensive.
3. Cross off anything where being wrong reaches a customer. Billing, contracts, support replies to upset people, anything legal. Klarna's CEO Sebastian Siemiatkowski publicly walked back his company's AI-first support push and told Bloomberg that cost had been "a too predominant evaluation factor, and the result is lower quality." You do not need to learn that one personally.
4. Pick exactly one and build the smallest possible version today. Not a platform. A saved prompt, or one scheduled job. If a usable first version takes you more than ninety minutes, you picked the wrong task. Go back to your list.
5. Use this prompt to pressure test the task before you build it. Paste it into whichever model you already pay for and answer its questions honestly.
[The Job]
Help me decide whether one specific recurring task in my business is
worth automating, and tell me plainly if the arithmetic does not work.
[The Background]
The task: [DESCRIBE THE TASK IN ONE SENTENCE]
How often I do it: [TIMES PER WEEK]
How long it takes me each time: [MINUTES]
Roughly how many times I have done it in my life: [NUMBER]
What happens if the output is wrong: [CONSEQUENCE]
Tools I already pay for: [LIST]
Hours I am willing to spend building this: [NUMBER]
[The Deliverable]
A one-page decision, written from the perspective of a skeptical
operations engineer who gets paid nothing for building things that
never pay back. Include estimated build hours, estimated ongoing
maintenance per month, the break-even point in weeks, and a plain
yes or no. If the answer is no, say no and name the cheaper manual
fix instead.
[The Questions]
Ask me any questions you have.
6. Measure it in minutes, once a week, for one month. One line in a note: date, task, minutes with the automation, minutes it used to take. Four data points is enough to know. If the gap is smaller than you expected, that is the most valuable thing you learned this month.
7. Then do another one. This is the whole method. Small, supervised, measured, repeated. You will have six working automations before most people finish scoping their platform.
Frequently Asked Questions
How long does it realistically take to automate a business task with AI?
It depends entirely on whether a human still checks the output. A personal task you supervise yourself can be working in under two hours. An unsupervised production system that touches customers or money is measured in engineer-months. One practitioner reported three engineers working three months to cut a 200-hour screening process to 4.5 hours.
What percentage of AI projects actually reach production and deliver returns?
The widely quoted figure is 5 percent, from Project NANDA's GenAI Divide: State of AI in Business 2025, which reviewed 300-plus public initiatives and interviewed 52 organizations. Be careful with it. The report calls itself preliminary findings, it is not peer reviewed, and press coverage has reported sample sizes that differ from the report's own.
Do AI coding tools make developers faster or slower?
METR's 2025 randomized controlled trial found 16 experienced developers took 19 percent longer with AI tools on 246 real tasks, while believing they had sped up by 20 percent. METR's 2026 follow-up points toward real speedup but was undermined by selection effects they disclosed themselves. Honest answer: measure your own team rather than trusting either headline.
Should I tell customers when they are talking to an AI?
Yes, and in the EU, Article 50 of the AI Act has required it since August 2, 2026. Know the cost, though. A 2019 Marketing Science field experiment by Luo and colleagues across 6,200 customers found upfront bot disclosure cut purchase rates by more than 79.7 percent, even though undisclosed bots closed deals nearly as well as proficient humans.
Is AI automation worth it for a small business with no technical team?
Yes, for the small supervised wins, and almost never for the big unsupervised ones. Census Bureau data from early 2026 shows fewer than 20 percent of firms with four or fewer employees using AI at all, compared with 37 percent of firms over 250 employees. You are not behind. You just need a different size of project.
The Four Numbers
Go back to that hackathon system. Nine engineer-months. 200 hours saved per event. Nothing worked on the first try.
That is a success story. It is also the most useful piece of AI content I read this quarter, and the only reason it is useful is that he told you what it cost.
So here is the test, and you can apply it to everything that crosses your feed starting today. How many people, for how long, to save how much, and who checked the output. Four numbers. If a case study is missing them, you are not reading a case study. You are reading marketing.
That includes when it comes from me.
I sell AI education to people who are worried about being left behind. I am not going to tell you my incentives are pure, because yours are the ones paying for it. What I can do is give you the denominator every time, and say so plainly when I do not have it.
Most of what you need this month is not a system. It is one task you already do every week, that you already know how to judge, where being wrong means you do it over. Automate that. Write down the minutes. Do another one in two weeks.
That is not a transformation. It is arithmetic, and arithmetic is the only thing in this field that has never once lied to me.
If you try one and the numbers come out worse than you hoped, I would genuinely like to hear about it. Those are the posts this industry is missing.
About Jonathan Mast
Jonathan Mast is the founder of White Beard Strategies, where he provides AI coaching and mentorship for entrepreneurs and small business owners. He runs a large Facebook community for business owners learning AI, leads the AI Insiders membership, and speaks regularly on practical AI implementation. He has built and rebuilt businesses, including after a bankruptcy, which is where he learned to distrust any projection he could not show the arithmetic for. He is still a little embarrassed that his favorite automation is a saved prompt that takes about four minutes a week off his plate.