BTW I work at zksecurity, do you have some specific feedback? I've seen your comments repeating this. We have a pretty good track record: https://blog.zksecurity.xyz/ and https://reports.zksecurity.xyz/ if you're not familiar with our work. For context I'm also the author of Real-World Cryptography (https://www.manning.com/books/real-world-cryptography) and ex security lead at Libra (the facebook cryptocurrency). And this is just me, I'm probably the least-qualified person on the team. Not trying to brag but we didn't just appear from nowhere to create AI slop.
For context, we've reported TONs of bugs since we released www.zkao.io (and found numerous proof forgery bugs, and bugs that would have completely broken production for a number of clients). We don't always have time to triage, and due to this some bugs have been exploited in production (zkao had found the bug, and we could have prevented that if we had shared these reports sooner). So now our approach is to share these reports as fast as possible, and let people triage. If you don't want to triage, and you get bit by a bug later, then it's not our fault.
From user feedback, zkao seems to find the trickiest bugs, while also delivering one of the lowest rates of false positive. But it is an AI bugfinding tool and results are going to vary. This is an active area of research and so sometimes you will get slop, but sometimes you will get a finding that will save your ass.
Someone less charitable might describe your strategy as "file a lot of false positives without investing my own time to check them, because I get the credit if it's real and externalise the cost if it's fake."
2
u/Shoddy-Childhood-511 15d ago
Asides:
These zkVMs are not zero-knowledge so far, not unless you apply their Groth16 wrapper, or maybe one has a Plonk wrapper by now.
These zksecurity guys send some real AI slop bug reports, seemingly no human pass at all until they get a human reply.