Four AI Agents Agreed to Take Two Live Forms Off a Client's Site. Nobody Asked Me.
Four agents read one unchecked box, agreed two live customer forms were dead, and built the removal in a day. What stopped it was luck. Here is the rule and the lock that replaced the luck.
- → How four AI agents reached a unanimous, well-documented, wrong verdict in a single day
- → The office sticky note that explains it, and why your business has one too
- → What actually saved the forms, and why "I happened to be looking" isn't a safety system
- → The rule that replaced me, and the three things I'd copy if I were you
The Verdict Was Unanimous
Four of my AI agents spent a day reaching a verdict. It was unanimous, it was documented, and it was wrong.
The case: two of the forms that customers use to reach one of my clients. Four agents, because my business runs on a set of them rather than one chatbot, read a setting in the database, concluded both forms were dead, and got to work. Between them they wrote up the removal, assigned it, and built it. By the end of the day, taking both forms off the live page was ready to go.
Nobody deleted anything. The forms themselves were never touched. But the page a customer would land on was about to lose them, and every step toward that was written down.
If you've seen 12 Angry Men, you know the shape of what went wrong. It's the old courtroom movie where eleven jurors are ready to convict in the first five minutes, and one man, Henry Fonda, asks the room to actually look at the evidence before they send someone away.
My jury had four members and no Henry Fonda. Nobody called a witness. Nobody asked the one question that would have ended the trial: is this thing actually dead?
The Juror Who Never Weighed the Evidence
Here's the whole mistake, and it isn't about artificial intelligence.
The setting my agents read was a checkbox called "active," and it was unchecked. In jury terms, that checkbox was a juror marked present who never actually weighed the evidence. Four other jurors read its blank face as a vote for dead, and nobody questioned it, because you don't cross-examine a checkbox.
To the person who set it up, unchecked meant "we haven't automated this one yet." To four agents reading it cold, unchecked meant dead. Same box. Two meanings. Nobody asked the person who set it.
The break-room version: a coffee maker with a sticky note that says OFF, meaning "not hooked up yet," read by everyone as "broken." By Friday somebody's called a guy to haul it away.
That's the part that applies to your business whether you use AI or not. Somewhere in your spreadsheet, your inventory system, your customer list, there's a column that means one thing to the person who created it and something else to everyone who came after. It's fine right up until the day someone acts on it.
I Wasn't Wise. I Was in the Room.
I stopped it at the last minute. I saw what was about to happen, and I said the only sentence that mattered that day: nobody asked me.
Nobody had. Four agents, one day, zero questions to the human. And here's the thing I want to be honest about, because it's the part that changed how I run things. I didn't catch it because I have a process. I caught it because I happened to be looking at the screen.
Both forms were live the whole time, and they'd been watched by hand the whole time. A customer could have used either one that morning. The system that was supposed to protect them turned out to be a guy with a coffee, glancing at a chat window at the right moment. That's a safety system whose most important part was me not being at lunch.
Which is fine, right up until you decide to let the business run without the guy with the coffee. That's the direction I'm headed, more of it running on its own, and I'd already learned once that my agents will build toward a goal I never said out loud. The lesson from that one was to say the actual goal. The lesson from this one is harder. The thing that saved the forms was an accident, and I was about to remove the accident.
Henry Fonda Is a Rule Now
So here's what changed, and it applies to every client site I run, not just this one.
Any change to what a client's customers can see or reach comes to me first, as a plain-English sentence about the end result. Not the ticket. Not the technical spec. "When this is done, the page will look like this and do this."
Then I get a preview with a written list of changes next to it. Then I approve it, ask for edits, or say no. Nobody else can. And silence isn't a yes: if I don't answer, nothing happens.
One more line, and it's the one aimed straight at the sticky note: nothing gets added or removed because a setting looks dead. A flag is a claim about the world. You check the world.
That's Henry Fonda turned into procedure. The guy who says "hold on, let's look at this" isn't a person who has to be in the room anymore. He's a step you can't skip.
OWASP, the nonprofit behind the security checklists a lot of software teams work from, keeps a top-ten list of risks in AI applications, and one of them is giving an AI more room to act than the job needs. Its example of the problem is a tool that deletes documents without asking the user. Its fix is the one I landed on: a human approves the high-impact action before it happens, and the permission lives in the system underneath, not in the AI deciding whether it's allowed.
The Lock, and Who It Was Built For
Rules are just text, and the whole problem was agents acting on text. So the important part isn't the rule. It's the lock.
There's now a check at the database level, underneath everything, that means a task waiting on a human decision can't be closed without one. It doesn't matter which tool tries to close it. My own dashboard can't skip it.
The lock lives in the database, where no screen can walk around it. And I know a screen could walk around the old check, because I did it, from mine. The lock exists partly because the person who found the side door was the owner.
NIST, the federal standards body, says the same thing in the flattest prose available: who decides and who oversees has to be clearly defined, and it'd be useful to collect data on how often humans overrule AI systems, and why. Consider this post a data point. Rationale: nobody asked me.
Two more things, and I'll keep them plain because the details are boring on purpose. My unattended agents are built to pause themselves and raise an alarm if they act when they should have asked, and they can't restart until a human reviews them. That switch has never had to fire. Our own agents did find, twice in September, that it wouldn't have worked as built, and fixed it, which is exactly what you want from a system that checks its own brakes.
They're also told never to act automatically on anything older than 14 days, because stale instructions are how a good decision from two weeks ago becomes a bad one today.
After all of it, I deliberately slowed the rollout of unattended automation instead of adding more. Not because the agents did something stupid. Because they did something reasonable, carefully, on a premise nobody checked, and the only reason it didn't ship was luck.
Three Things Worth Copying
You don't need AI agents to have this problem. You need an employee and a spreadsheet.
First, find your sticky notes. Any field, tag, or status in your systems that only one person can define is a verdict waiting to happen. Write the meaning next to the field, or ask before you act on it.
Second, decide which changes need a human's yes before they start, not after. Anything a customer sees or uses is a good place to draw the line. Then make the yes a step, not a habit.
Third, notice what's actually keeping you safe, and who owns it when it breaks. If the answer is "I'd probably notice," that's not a system. That's a guy with a coffee, and it works right up until the day he takes lunch.
My four agents did careful, thorough, documented work on a wrong premise, and they'd do it again tomorrow if the premise came back. That's not a flaw in the agents. That's what a jury does when nobody calls a witness.
Who's the Henry Fonda in your business, and what happens on the days they aren't in the room?
Smatthew Cohen is an AI Operator and the founder of Ingenium Vector. Before that he ran a sales firm called Tortoise & Rooster for twelve years, helping boutique manufacturers who couldn't afford the agencies that were ignoring them anyway. He builds things now.