Lessons Learned Running Two Agentic RE Workshops at DEF CON 34
A small model on a laptop found the bug. The system around it is what told the real one from the thirty that looked just as convincing.
I taught two four-hour workshops at DEF CON 34, both built around one question: how close can local AI models get to frontier-level reverse engineering, and what do I have to build around them to close the gap?
Both workshops came at that question from opposite ends and landed in the same place. The model is rarely what decides whether you find the bug. What decides it is everything around the model: how you feed it evidence, which tools it can reach, where you make it stop, and whether or not you can validate the result.
Reaching Mythos: Improve the System
Friday afternoon, LVCC W228. This one treated reverse engineering as a vulnerability-research pipeline and pointed local, open-weight models at real targets.
What we did:
- Ran local models against a pre-patch FreeBSD checkout and read the noise, not just the best-looking finding.
- Added reachability as its own pipeline stage and watched which findings survived.
- Switched from source to D-Link firmware, driving Ghidra through MCP to validate a known command-injection path.
- Pushed toward the exploitability frontier and stayed honest about what static evidence could not prove.
What students took away:
- A smaller local model in a good harness beats a bigger model handed a weak one-shot prompt.
- On a laptop, the local models reached the same pre-patch FreeBSD bug that put the frontier Mythos preview in the headlines. The raw capability was already there.
- What was missing was scaffolding. A broad scan did surface the real stack overflow, but it sat in a pile of roughly thirty plausible false positives. Adding a reachability stage thinned that pile and left the genuine finding standing, and that came from the harness, not a bigger model.
How it went: The room filled and stayed hands-on. People ran the pipeline, hit the same false-positive pile I did, then wrote the filter stage that cut it back down. Running the agentic pipeline themselves, on real local models, is what let students see the real work still to do.
Building Agentic RE Skills: Capture the System
The second workshop came at it from the other end: take a common RE workflow and capture it as a Skill.
What we did:
- Reversed a Windows driver by hand in Ghidra, then handed the same bounded job to a coding agent over PyGhidra MCP.
- Turned that workflow into a portable
SKILL.mdwith an explicit output contract and a vulnerability-review step. - Cold-ran the Skill against a different driver with the opposite dispatch shape.
- Built the same way for patch diffing with
ghidriffand for triaging a malware sample against a public write-up.
What students took away:
- Doing the driver by hand first is what made the agent's run legible afterward. Once you have walked the dispatch path yourself, you can tell whether the agent is doing the real work or just handing back a tidy summary.
- Capturing a workflow only counts if someone else can rerun it. A stack of prompts that works once, in the session where you wrote it, is not a reusable Skill.
- The real test was the cold run. Hand the
SKILL.mdto a fresh agent, point it at a driver it had never seen, and watch whether it held up, kept its evidence, and stopped when the proof ran out.
How it went: Watching people cold-run a Skill they had written minutes earlier, against a binary they had never seen, and get a usable result was the payoff of the whole session.
What I Took From Both Rooms
More than eighty people came through the two workshops, and the range of who showed up is the part I keep thinking about.
Some walked in with almost no reverse engineering background and still got real work done, because the local models carried them past the parts that usually stop a beginner cold. Others have spent years in RE and vulnerability research and left surprised that local, open-weight models were a serious option at all, and not just a toy next to the frontier APIs. Plenty of people in both rooms just wanted to see how fast the loop could go, and were genuinely impressed once they ran it themselves.
That mix is what stuck with me. This is not a corner a few specialists are poking at. Researchers at every level are trying to work out where AI fits into what they do, and a couple of full rooms will teach you more about that than any benchmark.
The Workshops Are Over, but We Kept Some of the Lab Open
Four hours was too short really, but we kept some of the exercises live for you to try. They live in the DEF CON 34 Agentic RE Challenge Lab. It runs in the browser, combines hosted local models with Ghidra and PyGhidra MCP, and pushes on the part that actually matters: can you back what the model says with evidence that holds up when you check it?
The full Agentic RE course connects the topics from both workshops and covers much more. The next live virtual cohort runs September 28 to October 2, 2026, four hours a day, 32 CPE hours, with course inference provided so you do not need a GPU or a paid API account.
Keep Building the System
The workshops are over. The challenge lab is open. Bring your skepticism, your curiosity, and give it a try. See what local AI can do.
New here? Join the newsletter for the next round of workshops and cohorts.