SABER

There is a growing desire to integrate rapidly advancing machine learning (ML) technologies into Department of War (DOW) autonomous systems for operational deployment of the battlefield. ML may give battlefield advantage by helping improve the speed, quality, and accuracy of decision-making while enabling autonomy and assistive automation. 

Due to the statistical nature of ML, a significant amount of work has focused on ensuring the robustness of ML-enabled autonomous systems at deployment time to natural degradations in performance caused by data distribution shifts (for example, from a highly dynamic deployment environment) and ML model drift.

However, as early as 2014, researchers demonstrated the ability to manipulate ML given adversary control of the input. Additional work has confirmed the theoretical risks of data poisoning, physically constrained adversarial patches for evasion, and model stealing attacks. These attacks are typically tested in simulated or physical environments with relatively pristine control compared to what might be expected on a battlefield.

Today, there is still a limited ability to operationally assess deployed military ML-enabled autonomous systems for adversarial vulnerabilities, and the “theoretical” counter-autonomy tools have not been practically demonstrated in operational settings. As a result, the operational security risks of ML-enabled autonomous systems remain largely unknown.

Securing Autonomy for Battlefield Effective Robustness (SABER) aims to build an exemplar red team equipped with the necessary counter-autonomy techniques, tools, and technical competency to operationally assess ML-enabled autonomous systems. SABER seeks to establish a sustainable model for an operational red-teaming process for the DOW. Our red team will target operationally assessing ML-enabled autonomous ground and aerial autonomous systems that could be deployed within the next 1-3 years.