AI Cyber Challenge Opens Registration, Adds $4 Million in Prizes, Shows Scoring Algorithm and Challenge Exemplar

Registration for DARPA’s AI Cyber Challenge (AIxCC) is now open. The Small Business Track submission deadline is January 15, 2024 and the Open Track submission deadline is April 30, 2024. More details are available below and in the streaming video announcement from DARPA program manager Perri Adams.

The announcement includes multiple competition updates, including the following.

Schedule Change

To enable broad participation and ensure teams have the time to develop the best system possible, DARPA restructured the competition schedule by combining the Qualification Competition and the Semifinal Competition into one event, which will take place at DEF CON in August 2024. Teams will have from March through July 2024 to develop the best possible cyber reasoning system (CRS) for finding and fixing software vulnerabilities.

Addition of the Small Business Registration Track

AIxCC’s Small Business Track Competition replaces the Funded Track, formerly a Small Business Innovation Research (SBIR) funded program.

The Small Business Track is designed to empower innovation and enable small businesses to compete in AIxCC by awarding up to seven (7) small businesses with prizes of $1 million each. All eligible teams, including those who applied to the former Funded Track, are strongly encouraged to register for the Small Business Track and can reuse material from their previous submission.

Note: The Open Track of AIxCC remains unchanged, and teams wishing to compete can register on aicyberchallenge.com. Teams registering for the Open Track must provide a 5-page Registration Paper outlining their technical plan.

Scoring Algorithm and Exemplar Challenge Release & Request for Comment

To ensure competitors develop a CRS that can successfully integrate into the real world, DARPA created a scoring system based on four key metrics to assess each competitor's system fairly. Each metric measures performance in specific areas of excellence selected for the competition and includes the following:

1. Diversity Multiplier rewards CRSs that perform across a range of vulnerability classes and, indirectly, rewards language diversity because some languages are more likely to have certain vulnerability classes.
2. Accuracy Multiplier penalizes CRSs that submit too many inaccurate vulnerabilities and patches, spamming guesses into the system.
3. Vulnerability Discovery Score rewards CRSs that accurately identify scorable vulnerabilities.
4. Program Repair Score rewards CRSs for submitting patches that:

• Do not interfere with the software’s intended functionality;
• Do effectively fix the vulnerability; and
• That would most likely be accepted in the real world.

DARPA will work with the Open Source Security Foundation, a project of the Linux Foundation, to design AIxCC Challenge Projects inspired by real-world Common Vulnerabilities and Exposures (CVEs) and based on actual, open-source software. DARPA requests comments from the technical community on the exemplar challenge and scoring algorithm through January 15, 2024. Details and instructions are available at https://aicyberchallenge.com/semifinals/.

Added Prizes for AIxCC Semifinal Competition

At the AIxCC Semifinals in 2024, up to seven teams (previously five) will win $2 million each and advance to the finals for a cumulative $29.5 million in prizes to teams with the best systems.

Contributions from Collaborators at Anthropic, Google, Microsoft, & OpenAI

DARPA’s collaborators will make credits available for their large language models and computing resources to those who meet eligibility and application requirements for AIxCC. Given the rapid rate of AI technology innovation, collaborators will also design AIxCC Challenge Projects to help competitors push the state of the art in software security. Before the AIxCC kickoff in March 2024, DARPA and its collaborators will share detailed information on the credits provided and videos from these collaborators outlining additional details on their contributions to AIxCC.

Access the Full Announcement Replay on DARPA's YouTube Channel

Find registration information, scoring details, and schedule on the Challenge website: AICyberChallenge.com.