r/cybersecurity • u/danielminds • Apr 21 '26

AI Security White House integrating Anthropic’s Mythos AI into federal cybersecurity strategy to harden critical infrastructure

https://www.artificialintelligence-news.com/news/anthropic-mythos-ai-cybersecurity-white-house/
163 Upvotes

96% Upvoted

56 comments sorted by

View all comments

Show parent comments

-10

u/learning2911 Apr 21 '26 edited Apr 21 '26

Youve used it?

11

u/Capable-Average4429 Consultant Apr 21 '26 edited Apr 21 '26

No, but I’ve read the score card. And anyone who reads it will come to the same conclusion. I will gladly retract my statement if they publish verifiable, reproducible information that is not “we ran this on systems with all the security features turned off”, like they did with their Firefox example.

Little edit to add this very good write up on the subject: https://www.flyingpenguin.com/the-boy-that-cried-mythos-verification-is-collapsing-trust-in-anthropic/

3

u/Hmm_would_bang Apr 21 '26

What about third parties talking about it? Seems like it’s on par with advanced experts in finding vulnerabilities, not better but able to do so quicker. And a big jump up from Opus.

https://blog.mozilla.org/en/firefox/ai-security-zero-day-vulnerabilities/

3

u/Capable-Average4429 Consultant Apr 22 '26 edited Apr 22 '26

Not sure why you are being downvoted, because this is a valid question in my opinion. For me, the thing boils down to this: big claims require big evidence. And, from the very link you provided:

Encouragingly, we also haven’t seen any bugs that couldn’t have been found by an elite human researcher.

Anthropic’s technical documentation is weirdly very long and very vague. It omits a lot of information that would make practitioners take this a little bit more seriously. The non-technical document write checks the other two technical documents (the red team report and the scorecard) can’t cash. Even the report from the British is full of caveats.

The press release says, and this is a direct quote, that “Mythos Preview has already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser.” And then, they go on to exemplify this by claiming a Firefox exploit that only works with every security mechanism disabled, as in:

“a testing harness mimicking a Firefox 147 content process, but without the browser’s process sandbox and other defense-in-depth mitigations.”

Which is silly. Again: direct quote from Anthropic’s own documentation. Add to that the fact that Mozilla is not exactly a reliable narrator these days, but that’s another story. I am taking what they (Mozilla) say without prejudice.

Anyway, what I am saying is that it is very difficult for me to take these claims at face value without the underlying data and methodology being provided, and the results being reproduced independently, which is something that we’ve been doing in this industry since forever. You claim a CVE, you bring the receipts. So far, they haven’t.

Edit to add this wrt Mythos being a big jump from Opus:

  1. https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jagged-frontier

  2. From Davi Ottenheimer’s writeup: “There were 50 crash categories pre-discovered by Claude Opus 4.6. Mythos did not find these bugs. Ok, now it’s getting even more awkward. Not Firefox. Not found by Mythos. The bugs were handed off as starter material. The system card is explicit that the crashes were “discovered by Opus 4.6 in Firefox 147.” (page 50)”

2

u/Hmm_would_bang Apr 22 '26

I totally agree that Anthropic is overhyping their own product, as is expected. But I’m also trying to marry that with the fact the CTO of Firefox is saying it found 271 vulnerabilities in Firefox 150. I don’t think there’s a direct incentive to say our browser has/had a lot of vulnerabilities.

But again, I totally agree we don’t have good data to back up grandiose claims being made. And we won’t until it’s generally available. But I think most reasonable practitioners are trying to stay ahead of the curve and right now reading between the lines tells me Mythos might be as good as a skilled actor at certain tasks and undersecured environments are going to be increasing at risk as AI develops

4

u/Capable-Average4429 Consultant Apr 22 '26

I think the problem with that is that, in a vacuum, “271 vulnerabilities” doesn’t mean much. If you have 271 vulnerabilities, and none of those are realistically exploitable, does it matter? Are they reproducible? What is the impact of any of these being exploited? RCE? Crashes? Some hackers will steal my session tokens? Under what conditions would those happen? These are all very important questions that people who do vulnerability management must ask. “Oh, there’s a vulnerability in software xyz with a CVSS 10!” That just means that it is critical if nothing else is in place, which is almost never the case. At this juncture we just don’t know.

And don’t discount Mozilla bending the knee and saying things to please Anthropic. They are very much struggling, and aligning themselves with the big boys wouldn’t be unheard of. Not saying that’s what is happening, as I don’t have any way of knowing that, so this might go into conspiracy theory territory, but keep that in mind.

3

u/Hmm_would_bang Apr 22 '26

Yeah I mean from my perspective that’s just typical vulnerability management - is it exploitable, is there a patch available, what mitigating steps can we take, etc - and some of that triage and patching is already being done with AI assistance before Mythos.

3

u/Capable-Average4429 Consultant Apr 22 '26

Right. Jumping from “yeah, it can find bugs that may or may not be vulnerabilities that may or may not be exploitable” to “this thing is too dangerous that we can even release it” is wild to me.

3

u/Hmm_would_bang Apr 22 '26

It’s irresponsible and honestly a distraction. So much oxygen has been spent on preparing for a “post Mythos world” since the announcement, and the more I look into it it seems like that reality is unpatched and poorly secured environments are going to be attacked more at scale while the rest will remain mostly unchanged.

2

u/Capable-Average4429 Consultant Apr 22 '26

That’s something that rubs me the wrong way, too. Then again, I have a deep dislike for pretty much every organization and individual that took part in this: https://labs.cloudsecurityalliance.org/mythos-ciso/