Immunefi's bug bounty platform flooded with security reports after OpenAI's ChatGPT tool launch in November

Immunefi, a leading Web3 bug bounty platform, reported that its systems were inundated with low‑quality security reports generated by OpenAI’s ChatGPT after the chatbot’s public launch in November 2022. According to Immunefi’s own research, none of these ChatGPT‑generated submissions uncovered a real vulnerability, leading the platform to ban such reports and warn that current generative AI tools are not yet reliable for serious blockchain security work. In its “ChatGPT Security Report,” Immunefi explains that once ChatGPT became widely available, many users began using it to auto‑generate bug reports for Web3 projects hosted on the platform. These reports were often well‑formatted and used correct‑sounding technical language but were described as “nonsensical”: they referenced smart contract functions that did not exist, mixed up core programming concepts, gave vague or generic vulnerability descriptions, and included boilerplate impact statements. After reviewing this influx, Immunefi concluded that “to date, not a single real vulnerability has been discovered through a ChatGPT‑generated bug report submitted via Immunefi” and therefore permanently banned such submissions. Immunefi’s findings highlight the current limits of large language models for specialized security tasks in crypto and DeFi. In a survey of nearly 200 white‑hat hackers who had tried ChatGPT for bug hunting, about three‑quarters said the results were not worth the effort, and over 60% cited limited accuracy, lack of domain‑specific knowledge, and difficulty handling large‑scale code audits. Immunefi and interviewed experts argue that generative AI may be useful for education and documentation support in Web3 security, but not as a primary tool for discovering novel vulnerabilities, especially given issues such as hallucinated facts, outdated training data, and an inability to reliably detect emerging threats.

✨ AI-generated background, compiled from web sources — not editorial content.

More coverage

• If you're not using Immunefi, you're not taking security seriously. (immunefi.com)
• ChatGPT's Web3 Bug Reports Are 'Nonsensical,' Immunefi Says (bankinfosecurity.com)
• Announcing OpenAI's Bug Bounty Program (openai.com)
• Immunefi (immunefi.com)
• ChatGPT Security Report - Immunefi (immunefi.com)

Explore the topic

• OpenAI
• Launch
• Security
• ChatGPT