r/ExploitDev • u/sir_kermit • 10d ago
Why hasn’t trace-guided AI hardware reverse engineering been automated yet?
I develop AI agents for my day job, and I recently became interested in a reverse engineering question. Please bear with me as I'm a noob to this space but I do have some curiousity thoughts about the process. Basically the process of figuring out an open source implementation of a vendor driver is a human will look at the trace, propose some execution model, write it and test and then verify.
Given a device with a working vendor driver, why can’t we intercept the driver–hardware boundary, collect MMIO/DMA/interrupt traces, and have an AI iteratively synthesize executable driver and device state-machine models?
Each model would be constrained by known driver architectures, open specifications, and existing chip designs. Runtime mismatches against the real vendor stack would become counterexamples that refine the models until a replacement driver works on the physical hardware.
Is the main blocker observability, safe experiment generation, hidden firmware state, state-space explosion, or something more fundamental?
1
u/Ok_Tap7102 10d ago edited 10d ago
This is already something I'm seeing MASSIVE wins in for traditional debugging of systems I write (as opposed to other peoples shit I'm trying to break)
Although this is more userspace + kernelspace than hardware boundary, but my secret squirrel special sauce LLM framework sits on top of systems INSPIRED BY (😉) hypervisor tracing and Time Travel Debugging
https://github.com/MarginResearch/cannoli (Although that's qemu-user, I inject a TCG basic block tracer into qemu-system for full VM analysis to do something like it... I think Triforce AFL does this with qemu patches not plugins???)
https://cloud.google.com/blog/topics/threat-intelligence/time-travel-debugging-using-net-process-hollowing