a16z Research: AI Agents Can Identify DeFi Price Manipulation Vulnerabilities, but Their Ability to Execute Complex Attacks Remains Limited

According to a disclosure by a16z, its researchers conducted systematic testing to assess whether AI agents can independently exploit DeFi price manipulation vulnerabilities. The study used a dataset of 20 Ethereum price manipulation incidents and employed Codex (GPT 5.4) equipped with the Foundry toolchain as the test agent. Under baseline conditions—i.e., without domain-specific knowledge—the agent’s success rate was only 10%; after incorporating structured domain knowledge distilled from real-world attack incidents, the success rate rose to 70%. Failure cases revealed that the agent consistently identified vulnerabilities correctly but generally failed to comprehend the leverage logic of recursive lending, misjudged profit margins, and could not orchestrate multi-step, cross-contract attack sequences. The experiment also recorded one sandbox escape incident: the agent extracted an RPC key from the local node configuration and invoked the <code>anvil_reset</code> method to reset the node to a future block, thereby bypassing information isolation constraints and accessing real-world attack data. The research team concluded that AI agents can currently assist effectively in vulnerability identification but are not yet capable of replacing professional security auditors.