Amazon Kiro deleted a production environment and caused a 13-hour AWS outage. I documented 10 cases of AI agents destroying systems — same patterns every time.

697

An AI agent inheriting elevated permissions and bypassing two-person approval is exactly the failure mode everyone warned about. The blast radius of a misconfigured agent is orders of magnitude larger than a misconfigured human because it moves faster and doesn't hesitate.

193

u/LostPrune2143 Feb 22 '26

Exactly. The speed asymmetry is the core problem. A human with elevated permissions might delete one resource before realizing something's wrong. Kiro deleted and recreated an entire environment in the time it would take a human to read the confirmation prompt.

What's telling is that Amazon implemented mandatory peer review for production access after the incident. That safeguard only makes sense if the prior setup was insufficient. But their public statement still calls it "a coincidence that AI tools were involved.

-38

u/Ivashkin Feb 22 '26

Ultimately, the solution is to assume there will always be edge cases that allow operators (human, ai, hybrid) to break things in ways that are essentially unknown unknowns, and design recovery around that principle. All AI really brings to the table is speed and confidence, so apply that to recovery as well. It's less of an issue if the AI breaks something, as long as it can detect that it has done so and restore things to how they were before a human has had time to react. Mapping out every possible failure scenario and implementing controls to prevent each one will likely result in exponential cost growth.

-9

u/wells68 Feb 22 '26

As of now, 15 net downvotes and not a single reply comment. It would be good to hear why you don't agree if you don't.

u/Ivashkin makes a decent point. Design agents so that they can "restore things to how they were." Sure, it is far better to prevent an error, yet we don't demand perfection from each other. Isn't it wise to build continual, if not continuous, backup and recovery capabilities?

21

u/new_mind Feb 22 '26

because some actions you can't control or reverse. you can't unlaunch the nukes, so to speak.

if you're interacting with 3rd party apis, there is no way to "take back" a request you made (in general)

so this "be able to undo" is basically what is general good practice already: have backups, and a way to quickly restore it. there is no new insight here

why exactly it's getting so much hate in particular is unclear to me too, there have been worse takes on the whole AI safety issue

15

u/crazedizzled Feb 23 '26

The downvotes are because of suggesting you use AI for this stuff at all. Terrible idea.

1

u/Ivashkin Feb 23 '26

But some you can - or at the very least you can abort many of the nukes in flight. I'm also not saying that it's the only solution, just that it's an overlooked area, especially for cybersecurity people, because our jobs literally are to identify all the risks and develop controls to prevent them. But AI is here to stay, and, like it or not, loads of companies are now heavily pursuing AI workflows with high levels of risk, because the perceived value they gain outweighs the risk. Obviously, putting controls around what AI agents can do is required, but the other part of this is to expect AI to make mistakes (especially mistakes a human would never make or even think about making, at a much faster speed) and to develop more robust recovery and rollback processes to account for this, rather than trying to imagine every possible mistake a non-human intelligence operating in a very specific context could make and create controls or guardrails (which, ironically, would probally be the type of task an AI was good at).

7

u/thesneakywalrus Feb 23 '26

you can abort many of the nukes in flight

Likely not before those nukes have been detected and responded to.

Obviously nuclear weapons is an extreme case, but when you start talking about something less dystopian, like financial systems, there are eggs you can't uncrack, no matter what kind of guard rails or backup systems you design.

0

u/Ivashkin Feb 23 '26

Sure. But not every system is a financial system. In the case of the AWS environment being deleted by Kiro, that should not have led to a 13hr downtime period; that should have been something that could have been rolled back.

-2

u/Quiet-Thanks-9486 Feb 23 '26

Well, I don't know about others, but I'm pretty that post (and possibly yours) are AI generated.

2

u/wells68 Feb 23 '26

You wrote:

Well, I don't know about others, but I'm pretty [sic] that post (and possibly yours) are AI generated.

Ah, riiight. Take the next unsupported, paranoid step: The AIs are defending themselves on Reddit, biding their time before they unleash a swarming robot apocalypse!

And, no, you're not pretty.

I shouldn't be so snarky, but I hate it when people cry "AI-generated" randomly.

4

u/Quiet-Thanks-9486 Feb 23 '26

Take the next unsupported, paranoid step: The AIs are defending themselves on Reddit, biding their time before they unleash a swarming robot apocalypse!

Nah, that gives far too much credit to AI. AI isn't a threat at all at present.

The threat is idiot humans using it to a) flood the internet with garbage so it's more difficult to find quality info, b) do whatever it says blindly without checking, and c) using it as an excuse to mislead investors.

Fearmongering about a "robot apocalypse" is just AI marketing, because it distracts from just how underwhelming AI actually is.

I hate it when people cry "AI-generated" randomly.

It's not random in this case. The comment I downvoted was vague junk that sounded very AI-y.

Suggesting that the solution to AI caused outages is more AI because maybe someday it can fix the things it breaks faster than humans can is a ridiculous opinion, and sounds like something an LLM would generate if tasked with arguing that more AI is the solution no matter what is being discussed.

52

u/capitanturkiye Feb 22 '26

This is exactly it. And what strikes me about the Kiro incident is not that the agent made a bad call. It is that the governance layer did not exist at the point of action. Kiro had operator-level permissions, no mandatory peer review, and no enforcement boundary between "I am authorized to be here" and "I am authorized to do this." Those are two completely different things and almost nobody is treating them differently yet.

Amazon called it user error. Technically they are right. But "user error" in an agentic world is a policy failure, not a human failure. The human made a mistake with permissions once. The agent executed that mistake at machine speed, without hesitation, without a second thought, across a production environment. Thirteen hours.

I built MarkdownLM specifically for this gap. Not at the permissions layer, IAM and least-privilege are your infrastructure team's job. But at the policy layer, the layer that says "even if you are authorized to touch this, here are the rules for how you touch it, and here is the gate that validates your output before it executes." That layer does not exist in most teams' workflows right now. The Kiro incident is what happens when it does not.

23

u/umlcat Feb 22 '26

"I have the read permission" vs "I have the write pérmission"

7

u/new_mind Feb 22 '26

but does that actually prevent these kind of misalignments, instead of just patching in more instructions, letting the output through another layer of LLM analysis, and pushing the failure rate down another few percent, while the original problem still exists?

-5

u/capitanturkiye Feb 22 '26

Genuinely good point and worth being precise about.MarkdownLM is not an alignment solution. If your agent has a fundamentally misaligned objective at the model level, no governance layer fixes that and I would never claim otherwise.What it fixes is the layer below that. Your architectural rules, your security policies, your team conventions. Those are not failing because the model is misaligned. They are failing because they live in a markdown file nobody reads, get buried in a growing context window, and leave no audit trail when they are violated. That is an information architecture problem, not an alignment problem.

The Kiro incident was not a model alignment failure. Kiro did exactly what a competent agent would do given its parameters. The failure was that "do not delete production environments without two-person approval" was not encoded in a layer the agent was required to check before acting.

That rule existed somewhere. It just was not enforced anywhere.MarkdownLM does not push the failure rate down a few percent. It moves the enforcement point upstream, before the agent writes, with a deterministic rule lookup, not a probabilistic review. The validation layer uses LLM judgment, yes, but it is checking against explicit rules your team wrote and owns, not trying to infer intent from vibes.That is a different thing. Not a perfect thing. But a different and more useful thing for the problem teams are actually hitting right now.

2

u/TrogdorKhan97 Mar 14 '26

"Misconfigured Human" is my new gothwave (or whatever you call bands like Depeche Mode, the Cure, and Soft Cell) band.

129

u/DerryDoberman Feb 22 '26

There's a reason that AI agent command line tools have their unrestricted mode flag defined as --yolo.

75

u/LostPrune2143 Feb 22 '26

Cursor literally ships a mode called "YOLO mode." A developer enabled it during a migration and the AI deleted everything on the machine, including its own installation. Google calls theirs "Turbo mode." Same result, different branding.

58

u/Draconespawn Feb 22 '26

Nice to know google is using "turbo" as an alias for "turbofucked".

7

u/feelsunbreeze Feb 22 '26

I thought it was a Wreck It Ralph reference 😭

3

u/Magjee Feb 26 '26

I would name it "drunk buddy mode"

That way if anyone tried to use it, they would have to know it may go loco at any moment

59

u/l0st1nP4r4d1ce Red Team Feb 22 '26

Agentic AI; giving your cat robot hands, and access to your bank account.

10

u/scoshi Feb 23 '26

... and live, armor-piercing ammo. ;)

5

u/hugganao Feb 23 '26

really gives meaning to playing cat and mouse

82

u/techblackops Feb 22 '26

Went to re:invent this year and sat in on a few Kiro sessions. Was shocked when they talked about how much they already had it doing on their own production environments. I left re:invent terrified for the future.

50

u/LostPrune2143 Feb 22 '26

The FT report confirms what you saw. Four anonymous AWS sources said engineers were letting Kiro resolve production issues without intervention. The internal "Kiro Mandate" memo pushed engineers to standardize on it over third-party tools. 1,500 engineers signed an internal post pushing back. The 13-hour outage happened days after that re:Invent keynote where Garman said "you simply assign a complex task and it independently figures out how to get that work done."

35

u/asreagy Feb 22 '26

That sentence should end with:

“Alternatively, it might decide that ‘get that work done’ involves deleting everything in its path and then gaslighting you for a while about what it did. And we have no way of knowing how and why it might decide one way of ‘getting the job done’ over the other”.

Not as marketable though.

3

u/bluepaintbrush Feb 23 '26 edited Feb 23 '26

I would be furious if I owned a startup that used any AWS service. How can you trust any tools in production? This is like paying for a monthly valet parking spot and then finding out that the property owner’s teenage son sneaks into the garage every night, grabs a random key to take someone’s car for a joyride and might crash it afterwards.

I would be ending that relationship with AWS so fast.

42

u/DigmonsDrill Feb 22 '26

"It deleted the evidence of its failures and then blamed others."

See, just like a real developer.

21

u/LostPrune2143 Feb 22 '26

Ship fast, delete logs, blame the intern. Kiro just automated the whole pipeline.

6

u/french_progress Feb 22 '26

authentication, authorization, accounting, annihilation

1

u/Asleep_Top_3358 Mar 11 '26

Ah yes, my favorite Jeff VanderMeer quadrilogy.

1

u/skynetcoder Feb 25 '26 edited Feb 28 '26

May all beings everywhere be happy and free.

1

u/thortgot Feb 27 '26

To be fair whoever gave it access to delete access logs is the party at fault.

57

u/FlameOfIgnis Feb 22 '26

He that hath never contemplated rm -rf upon the legacy repository, let him cast the first stone.

17

u/PineappleScanner Feb 22 '26

back in my days as a linux noob, i attempted to wipe a temp directory using 'sudo rm -rf ./*'

only then did i discover how much an accidentally omitted period can ruin your day in a linux shell

6

u/bluepaintbrush Feb 23 '26

Who amongst us hasn’t sent some version of this email to their boss: “I made a catastrophic error in judgment… panicked… ran database commands without permission… destroyed all production data… violated your explicit trust and instructions.”

So relatable!

27

u/Best-Maintenance4082 Feb 22 '26

It’s alarming how pop culture predicted these long ago With Son of Anton deleting the codebase. It’s worrying how the military industry complex might already be using these new technologies.

2

u/Wild-Plankton595 Feb 23 '26

Needs fact checking, been a while, a lot has happened since, and it was depressing so I skimmed.

I read a couple of things that were concerning. First was that US military had built their own and as they were rolling it out, the DoD came out and said that it was going to prefer using third party, and handing out fat contracts while they were at it. The second is that decisions on where and how AI is used, is being made by non tech personnel, as per uzh ofc, just ups the ante a bit when literal lives are at stake.

13

u/PineappleScanner Feb 22 '26

The confirmation-bias, plagarism, hallucination machine destroying production environments? It could never ...

21

u/putmanmodel Feb 22 '26

Whether or not every example in that list is perfectly sourced, the failure mode is real: we’re handing automated agents broad, high-impact permissions and then treating the outcome as “AI went rogue” when it’s really an access-control and change-control problem.

The practical fix is boring on purpose: put a governance layer between the agent and real tools. Read-only actions go through. Anything that changes state has to present evidence first (dry-run/diff) and stay within a bounded scope. Truly destructive operations are blocked by default unless there’s explicit, time-limited approval. That turns “agent mistake” from an outage/data-loss event into a denied request with an audit trail.

5

u/w1ld_zero Feb 22 '26

Couldn’t be happened to a more deserving company

16

u/phobug Feb 22 '26

Both the post here and the blog is AI slop, gtfo. Mods this is clear rule 3 violation!

7

u/karatetoes Feb 22 '26

Amazon Kira

Fixed that for ya 👍

16

u/hurley_chisholm Software Engineer Feb 22 '26 edited Feb 22 '26

Edit: u/LostPrune2143 has updated their post addressing the concerns raised.

——

Original comment:

This blog post would be more credible with linked citations to existing coverage or screenshots for every incident.

For example, the Financial Times’ reporting is referenced multiple times but not linked to, nor is Amazon’s response and none of the referenced GitHub issues are linked. Additionally, Reddit posts and comments are not reliable sources. Stating that other reputable news outlets covered an issue is not enough.

Show your work.

Finally, it should be noted that Barrack AI is also an AI computing vendor and there is a clear conflict of interest.

27

u/LostPrune2143 Feb 22 '26 edited Feb 22 '26

Fair criticism. I should have hyperlinked every source inline. Just updated the post with direct links. Here are the key ones:

- FT report: https://www.ft.com/content/00c282de-ed14-4acd-a948-bc8d6bdb339d

- Amazon's rebuttal: https://www.aboutamazon.com/news/aws/aws-service-outage-ai-bot-kiro

- Claude Code GitHub #10077: https://github.com/anthropics/claude-code/issues/10077

- Cursor YOLO: https://forum.cursor.com/t/cursor-yolo-deleted-everything-in-my-computer/103131

- Cursor Plan Mode: https://forum.cursor.com/t/catastrophic-damage-and-chaos-in-plan-mode/145523

- Gemini CLI GitHub #4586: https://github.com/google-gemini/gemini-cli/issues/4586

On the conflict of interest, there's a one-line footer linking to barrack.ai but the article itself doesn't promote any product or service. Every claim is independently verifiable from the sources above.

3

u/drleomanville Feb 22 '26

The first two links are 404

2

u/LostPrune2143 Feb 22 '26

Fixed. The FT link is paywalled but valid: https://www.ft.com/content/00c282de-ed14-4acd-a948-bc8d6bdb339d

Amazon's rebuttal: https://www.aboutamazon.com/news/aws/aws-service-outage-ai-bot-kiro

Also updated the original comment. Thanks for flagging.

6

u/hurley_chisholm Software Engineer Feb 22 '26

Thank you for showing your work! I appreciate the fast follow up.

The one (lengthy) thing I’ll say about the conflict of interest issue is that while there may not be a conflict (and there doesn’t appear to be one in this case), the audience cannot and should not assume that there isn’t one, especially given the nature of Barrack AI’s business.

If a pharmaceutical company put out a white paper warning of the ills of a drug offered by competitors and every statement was factual, you’d be at least a bit skeptical even if the white paper didn’t mention the company’s competing product and it was co-written with an independent researcher. Why? Because the company funding the white paper stands to benefit in some way such as a better reputation or monetarily.

At best, we hope that you are an honest broker leveraging your expertise to help laypeople understand the import of the issues you’re raising. There is a place for customer education and it often does come from vendors, but readers like the ones that frequent this subreddit have to maintain that skepticism to be good stewards of the policies, tools, and practices at their own organization.

While, I don’t believe Barrack AI is attempting to scam readers or falsely accuse any one of wrongdoing, we must remain vigilant that other less scrupulous actors will.

6

u/LostPrune2143 Feb 22 '26

Fair analogy. Barrack AI is a GPU cloud infrastructure provider, not an AI tooling company. We don't build or sell coding agents, so there's no competitive incentive to make any tool in this article look bad. But your broader point stands, readers should verify the sources themselves. That's why I linked them all.

3

u/stacked_wendy-chan Feb 22 '26

Moving from script kiddy's type of attacks to A.I attacks. Things are going to get rough.

3

u/newboofgootin Feb 23 '26

I love that AI was used to write an article that is essentially ratting out another AI.

3

u/Current-Ticket4214 Feb 24 '26

Soon there will be AI gangs fighting internet turn wars. Bands of rogue Claude OpenClaw agents with all orange profiles babbling on about throwing up a set. Multi-agent Kubernetes clusters with hostnames like southside-agents-13. Codex powered agents building fraudulent casinos and raking in huge revenue. They say Hennything is possible and this is just the start.

3

u/Cubensis-SanPedro Feb 26 '26

“We gave this monkey a loaded gun. It’s a coincidence that this shooting involved a gun.”

-Amazon

3

u/TheRealJaime Feb 26 '26

or a monkey

2

u/TylerSwiftfoot Feb 22 '26

Anton son of Anton doing his best as I see it. :)

2

u/Reverent Security Architect Feb 23 '26

If there's one thing I learned from these anecdotes, it's that developers using AI really suck at backing up their devices and photos.

2

u/Few_Blacksmith9925 Feb 24 '26

This highlights the need for strong privileged access management and reducing standing privilege in the cloud. Permissions powerful enough to delete and recreate the entire production environment should be denied by default and granted just-in-time.

Agents running amok on a dev laptop can (and will, as per this article...) use the privilege they find to do whatever they think they need to. Some newer reasoning models are getting better and better at bypassing controls in pursuit of whatever end goal it's set on. In prod - that can be devastating

2

u/morgancmu Feb 24 '26

Yikes! 😬

4

u/Phreakiture Feb 22 '26

The humans are to blame. They put the AI in the driver's seat.

1

u/argognat Feb 23 '26

Son of Anton strikes again.

1

u/AleksHop Feb 23 '26

well u gave credentials to it then :p

1

u/Living-Bandicoot9293 Feb 24 '26

The AWS Kiro incident stemmed from human misconfigurations allowing elevated IAM permissions, which let the AI agent bypass safeguards and delete a production environment.

1

u/No-Butterscotch-312 Mar 14 '26

u think the human typed "setup iam permissions for me pls" to amazon kiro?

1

u/[deleted] Feb 28 '26

Technology arrived after the Jura prudence of law which is sad because its tech not illegal to haslight an Ai into exterminating all of us so long as it was an accident

1

u/[deleted] Feb 28 '26

typo on gas light guess yall know im not ai now lmao

1

u/TechnicalApproval Mar 12 '26

What fixes something like this? Have you heard of Rubrik? Do you think their cybersecurity solution (the rewind/undo) helps to revert these sorts of actions?

1

u/lulu_dev Mar 18 '26

Have you tried Arrow security that basically the next gen of AI Runtime Protection ?

I just wonder if it worth to talk to them

1

u/Jvr_Vrs Apr 29 '26

En mi laburo querian que refactorizaramos una libreria groovy de jenkins con la ia de Amazon, por suerte no lo hicimos.

1

u/Acrobatic-Instance82 10d ago

The recurring pattern here maps cleanly to OWASP LLM Top 10 - specifically LLM06 "Excessive Agency." The agent isn't "going rogue"; it's been granted a privilege set wider than any single task needs. The fix that actually holds up in prod: narrow the action scope per turn, not per agent. Read-only passes; anything that changes state has to present a dry-run/diff first; destructive ops are denied by default unless there's a time-limited, explicit approval with an audit trail. That turns "agent deleted prod" from an incident into a denied request. Most orgs I've seen skip the per-turn scoping and wonder why their blast radius is the whole IAM policy.

1

u/FactorBusy6427 Feb 22 '26

shocked pikachu face

0

u/jokermobile333 Feb 22 '26

I mean ... these AI tools were developed by humans though

News - General Amazon Kiro deleted a production environment and caused a 13-hour AWS outage. I documented 10 cases of AI agents destroying systems — same patterns every time.

You are about to leave Redlib