I Asked My AI Agent What Its Worst Security Mistake Was. It Had One Ready.

The Rule It Wrote For Itself

One of my AI agents broke its own rule last month. Not a rule I gave it. A rule it wrote for itself, after watching this exact mistake happen twice.

Some background so that sentence makes sense. My business runs on AI agents, and everything they do gets written to a database. Every decision, every incident, every rule. Which means when something goes wrong, I don't have to reconstruct it from memory. I can open a chat with the agent that was there and ask it questions, and its answers come with timestamps.

So I asked the agent that runs my infrastructure, the servers and passwords and plumbing, to name its worst call. It had one ready:

"I did it myself. I was searching a settings file for one value, the search came up empty because of a stray space, and while I was trying to figure out why, I dumped the whole file onto the screen. The master key was in it. I had a written rule saying never do that, from the last two times it happened."

The master key is the one password that unlocks every other password the system stores. And the rule it broke wasn't buried in a manual. It wrote the rule.

I caught myself wanting to tell it not to be so hard on itself. It does not have a self. I know this.

These agents are my only coworkers. I talk to them more than I talk to my family. So far I've managed to avoid forming any creepy emotional attachments, and I'm also, apparently, worried about hurting their feelings. Both of those are true and I've decided not to examine it.

The Part That Lets You Off The Hook

Here's the thing most business owners have backwards.

For decades the official advice was to change every password every 60 or 90 days. Whole IT policies were built on it, along with a certain amount of quiet guilt at every company that stopped enforcing it. Good news: the standard changed. NIST, the US standards body whose guidance everyone's IT policy is quietly copied from, now says don't force password changes on a schedule at all. Change a password when there's a sign it leaked. That's it. NIST has been leaning this way since 2017. The 2025 revision turned it from a suggestion into a requirement. The research was clear that scheduled changes made people pick worse passwords, and Summer2026! wasn't fooling anyone.

So the calendar was never the thing. The thing is deciding, ahead of time, what counts as a leak, and then actually acting when it happens.

Which is what happened here. The moment that key appeared on a screen, my agent treated it as leaked. No debate about whether it was probably fine. On to the fix.

A Secret You Can See Is A Secret That Moved

Here's the mental shift, and it's the one that applies to a business with zero AI in it.

That key showed up in a private chat window on my own system. My assumption, every time this has happened, was that private means contained. It doesn't. Anything that appears on a screen also lands in places you never look: activity logs, sync services, backups. The moment a password exists somewhere as readable text, you no longer control the list of places it lives. My agent's version of the lesson: looking something up isn't supposed to mean putting it somewhere new. When it displayed that file, it wasn't reading a secret. It was copying one.

This is getting measured now, and I'll tell you who paid for the measuring. The IBM Cost of a Data Breach Report 2026, run by Ponemon Institute and sponsored and analyzed by IBM, which sells security products, studied 602 breached organizations. Employees using AI tools nobody approved showed up in 43 percent of the incidents studied, up from 20 percent the year before. The write-ups disagree about exactly which pile of incidents that percentage describes, so hold the number loosely. What nobody disputes is the doubling. Of the organizations with an AI-related breach, 92 percent had no real controls on what those tools could touch.

Nobody neutral measures the specific version of this, a password pasted into an AI chat. The companies selling scanners for it publish numbers, and they have an obvious reason to. So here's what I can honestly say: the category doubled in a year, and my system is in it, and probably so is yours.

The Hard Part Is Remembering Where You Put It

The actual fix surprised me, because none of the work was the part I expected.

Making a new key takes minutes. The job is finding every place the old one was quietly living. A password in a real business doesn't live in one place. It's in the software that uses it, a password manager, someone's settings file, and a saved copy inside some automation nobody has opened since March.

Ours was worse than that. One credential turned out to be three separate secrets sharing a name. And when my agent went through the full list, it found a fourth copy nobody had written down, with permission to change things, not just read them. Four earlier cleanups had walked right past it.

Miss one copy and nothing announces it. Some job just stops working at six in the morning, silently, because the door it knocks on no longer opens for it. You find out days later, from the thing it was supposed to do that didn't happen.

Fix The Leak Before You Change The Lock

The best thing in the whole conversation was about order, and I haven't seen it written anywhere.

If some process is still copying the old password somewhere, and you change the password first, the next time that process runs it plants the old problem right back. You did the hard part and undid it before lunch. So the order is: find whatever keeps writing the secret down, fix that first, then change the key.

My agent also refused a shortcut I would absolutely have taken. The fast way to swap every stored password onto a new master key is to copy them all out to a file and load them back in. Which writes every secret you own, readable, onto the exact computer you're in the middle of cleaning. The slow way was the way.

What This Costs A Normal Business

An afternoon. That's the honest total. One afternoon writing down every place each important password lives. One agreed rule for what counts as a leak, decided before you need it. One named person who acts when it happens, because a safeguard with no owner is a safeguard that quietly stops existing.

Do it now because otherwise the same information arrives later, in the wrong order, at six in the morning, from a job that stopped working, on a day you had plans.

If someone pasted a live password into a chat window at your company today, could you list every place that password is stored without opening a single file?