John Curreri

An LLM agent that bins post-silicon debug logs — retrieval over prior failures, calibrated confidence, every tool call shown. Replays runs recorded on a local Qwen3.6-27B.

14 real bugs fixed upstream in the CVA6 RISC-V core, most after the models' training cutoffs. From a symptom-level report, Opus 5.5 writes a bare-metal test that exposes the bug in 93–98% of attempts; open-weight models served locally do far worse.

Can a model find a real, recently disclosed vulnerability in a file it has never seen, and tell when that file is already patched? 30 post-cutoff GitHub advisories, ten model configurations, scored automatically.

Turning a circuit-tracer claim about Qwen3-8B into a 49-prompt test suite. Switching off the "state" features collapses the answer in 31 of 49 prompts, against 0 of 49 for random controls. Charts, raw data, and source.

A durable Temporal workflow benchmarking five agent configurations over 120 runs. Trimming the prompt bought nothing; turning reasoning off was worth 6.5x latency. Charts, raw data, and source.