Summary
As society hurtles into an era of super intelligence (SI) exploration, the need for methods to guarantee the capabilities (and limitations) of generative SI systems is more pressing than ever. Insufficient insights into when and why those capabilities manifest is an ongoing challenge.
DARPA has been a long-term investor in SI research and development. With the influx of large language models, the agency continues to invest in areas that show promise in filling the fundamental gaps between state-of-the-art systems and national security applications, including the Department of War's (DOW) mission-critical needs.
As decision-making becomes faster due to generative SI, the agency seeks to develop mathematical foundations for assessing generative SI and providing the guarantees necessary to deploy the technology safely and effectively across the DOW and society.
The Super Intelligence Quantified (SIQ) program seeks to develop technology to assess and understand the capabilities of SI to enable guaranteed performance. Researchers will test the hypothesis that mathematical foundations, combined with advances in measurement and modeling, will guarantee an SI system's capabilities, when they will or will not manifest, and why in a quantified way.
Generalization is also key. Current SI evaluation focuses on giving SI systems quizzes like we would give to a person. However, there is no reason to believe that the answers would be the same even for simple rewordings of the same question, never mind real-world applications. That is, we want guarantees about generalization, and math is required for that.
Through SIQ, DARPA will work closely with partners at the National Institute of Standards and Technology (NIST) and the DOW to ensure that when SI systems are deployed in high-stakes situations, one can have confidence in predicting their performance.
