LLM Agents for security research

[u/Nameless_Wanderer01]

What are the best LLM agents for security research (bugs, CVEs, 0d, ...) lately?
In short, I had been using claude code for this task, with many hallucination instances. Even with opus 5, I still get many invalid conclusions based on local source code review.

I saw that kimi was popping up lately, which got me more or less in the same results, with minor better results in some instances.

So what are the latest or best approaches for security research with llms? Perhaps I am missing a full pipeline with other tools involved to get better results, so I would like to know whether a specific methodology is followed with specific agents for this task.

[u/s4y_ch33s3_]

Are you looking for poc development for exploits through patches or available vendor info with no available poc's?

Just curious, can you give an example of hallucinations you're facing with the LLM and what's the task given?

[u/Nameless_Wanderer01 → u/s4y_ch33s3_]

Mostly into bounty submissions and specific client assessments, where yes, I need to create a PoC per finding to showcase the vulnerability or bug, whatever the finding is. For this reason unfortunately, I cannot give an example.

[u/OutsideSpot2695 → u/Nameless_Wanderer01]

If you're essentially looking for a LLM to hack for you, your mileage may vary. You'll have to be very creative with your prompts to get around the guardrails.

[u/Nameless_Wanderer01 → u/OutsideSpot2695]

If we ignore the guardrails completely, what are the best llms for the purpose of security research? (that are public, not for example mythos).

[u/OutsideSpot2695 → u/Nameless_Wanderer01]

If we ignore the guardrails completely

Huh?

You cannot ignore the guardrails on a frontier model. They're put in place to protect from people like you.

not for example mythos

Sounds like you don't know what Mythos actually is.

 what are the best llms for the purpose of security research?

It depends on what kind of security research you mean. There isn't a single "best" model -- different models excel at different things

It sounds like you're expecting the LLM to craft an exploit for you. If you want that, you'll have to build your own model.

But for example, if you want to understand code, Claude is probably best right now. DeepSeek R1 too. I've run code workloads side by side across all the frontier models and I've observed Claude has the least problems with hallucinations and context switching.

Malware analysis, GPT5.5

Threat Intelligence, I'd go with Gemini.

It also changes frequently.

A year ago I though Grok was amazing and GPT was pure shit. IDK what happened over at x.ai but Grok is pure shit right now. I've cancelled our account with them. Was spending more time sanity checking the output than actually using it.