Mar 19, 2025
Open source software underpins the nation’s critical infrastructure systems, such as bridges, highways, hospitals, power plants, and utilities, making it a top target for cyberattacks by actors around the world. Through DARPA’s AI Cyber Challenge (AIxCC), in collaboration with the Advanced Research Projects Agency for Health (ARPA-H), seven finalist teams composed of experts across academia and industry are designing novel cyber reasoning systems (CRSs) incorporating cutting-edge large language models to automatically find and patch vulnerabilities in open source software.
The AI Cyber Challenge final competition will kick off this spring and culminate at DEF CON 33 in August 2025. Guided by the newly-released AIxCC Final Competition Procedures and Scoring Guide, the Final Competition will take place over a series of four rounds in 2025. Three rounds will be unscored exhibition rounds and one round – the final round – will be scored. In each round, each team’s CRS will have limited time to find and patch vulnerabilities in software challenges based on real-world software that is critical to industry, national security, and the public.
“Cyber threats to critical infrastructure are broad and unrelenting,” said Andrew Carney, program manager for the AI Cyber Challenge. “We’re looking for breakthrough systems that can give software defenders an edge when it comes to outpacing adversaries. We saw from the AIxCC Semifinal Competition last year that AI systems can develop and deploy patches for software vulnerabilities in minutes and at a fraction of the cost of existing methods. Now, we’re raising the bar and putting the top systems to the test against a broader and more complex range of challenges, with the goal of developing systems that the public and private sectors can use immediately to secure critical code.”
Competitor systems will be scored according to a scoring algorithm based on the ability to find and fix vulnerabilities, as well as analyze bug reports. The vulnerabilities will fall into two categories:
- Introduced synthetic: The competition will intentionally introduce synthetic vulnerabilities into challenge project software, which will only be used during the competition and not pushed to the public code base.
- Real-world: Vulnerabilities could exist in the challenge project software that were not intentionally introduced as part of AIxCC. Such vulnerabilities will be scoreable. If any zero-day vulnerabilities are found throughout the course of the competition, they will be responsibly disclosed according to the Linux Foundation’s vulnerability disclosure best practices.
While CRSs are tasked with finding vulnerabilities, the AIxCC Final Competition scoring algorithm prioritizes a CRS’s ability to patch vulnerabilities while maintaining functionality, assigning three times the weight to patching vulnerabilities than identifying vulnerabilities alone. To incentivize CRSs to identify and patch vulnerabilities quickly, certain scoring elements decrease over time. In addition to the code that will be scored as part of the final competition, competitors may also submit vulnerabilities and patches for unscored regions of code for additional research.
When a round is complete, AIxCC organizers will provide each team with access to CRS data collected during each round for the respective team’s CRS. Teams may use this feedback to improve their CRS for the next round, as applicable.
AIxCC represents a first-of-its-kind collaboration between the public sector and leading AI companies. Anthropic, Google, and OpenAI are providing technical support and have each donated $350,000 in credits – $50,000 to each team – to support CRS development for the Final Competition, in addition to the $5,000 in credits Anthropic, Google, Microsoft, and OpenAI provided to each team for the Semifinal Competition. Microsoft and the Linux Foundation’s Open Source Security Foundation continue to provide subject matter expertise to challenge organizers and participants. DARPA’s intent is for the CRS designs to be capable post-competition of analyzing any open source or private code base.
The Final Competition comes on the heels of a successful Semifinal Competition, during which competitors’ systems discovered 22 unique synthetic vulnerabilities in challenge projects and patched 15 vulnerabilities. Competitors’ systems also found one real-world bug in SQLite3, an open source database engine used in billions of devices and applications around the world. The team responsibly disclosed the bug.
AIxCC follows the DARPA Challenge model, which applies the power of competition to drive innovation. AIxCC will award $8.5 million in prizes in the Final Competition, including $4 million for first place, $3 million for second place, and $1.5 million for third place. These awards will bring the cumulative total of AIxCC prizes to more than $29 million awarded to help usher in a future in which software vulnerabilities are patched nearly as quickly as they’re found.
AIxCC at DEF CON 33 and beyond
The AI Cyber Challenge will return to DEF CON at the Las Vegas Convention Center from Aug. 7-10, 2025, where challenge organizers will announce the results of the Final Competition as well as host an immersive and interactive experience to bring the challenges of securing critical infrastructure to life.
Competing teams have agreed to release their competition CRSs as open source software following the Final Competition. Releasing teams’ CRSs as open source software aims to accelerate and facilitate the availability of AIxCC-developed technology for the benefit of the cybersecurity and software development communities.
For more about the AI Cyber Challenge, please visit the AI Cyber Challenge website.
###
