New program seeks to develop simulated social systems of varying complexity against which to test the accuracy of social science modeling methods
Apr 7, 2017
The social sciences can play important roles in assisting military planners and decision-makers who are trying to understand complex human social behaviors and systems, potentially facilitating a wide range of missions including humanitarian, stability, and counter-insurgency operations. Current social science approaches to studying behavior rely on a variety of modeling methods—both qualitative and quantitative—which seek to make inferences about the causes of social phenomena on the basis of observations in the real-world. Yet little is known about how accurate these methods and models really are, let alone whether the connections they observe and predict are truly matters of cause and effect or mere correlations.
To improve knowledge of social science modeling’s capabilities and limitations, DARPA today announced its Ground Truth program. The program aims to use artificial, yet plausible, computer-based social-system simulations with built-in “ground truth” causal rules as testbeds to validate the accuracy of various social science modeling methods.
“The real-world operates according to dynamic, interactive, non-linear, and sometimes adaptive and changing rules that we don’t understand very well, all of which limit our efforts to determine causality in social systems,” said Adam Russell, program manager in DARPA’s Defense Sciences Office. “We want to develop computationally simulated worlds where we create and therefore understand all the causal processes and rules. Then we can test a variety of social science modeling methods to see how well they identify the known causal processes built into the simulation.”
The plausible simulations developed in Ground Truth will serve as objective testbeds for calibrating social science modeling methods. “By creating a testbed environment, we’re essentially seeing if social scientists can develop models that accurately ‘reverse engineer’ an artificial social system by correctly identifying the causal rules designed into the simulation,” Russell said. “We call it forensic social science. Researchers are like detectives trying to unpack why and how agents in a system act in a certain way under different circumstances. It’s much easier to test how good any given method is if you have an objective set of known rules by which we can measure success.”
Ground Truth will solicit one group of researchers to create social simulations with associated ground truth rules, known only to them, while challenging another group of researchers to create innovative teaming approaches to “discover” the rules in those simulations. DARPA and its independent test and evaluation team will score the modeling teams’ abilities to identify and predict causal ground truth in different simulations with different degrees of social complexity.
If successful, Ground Truth will demonstrate a principled approach for testing the power and limitations of various social science modeling methods; explore new modeling approaches for describing and predicting different kinds of complex social systems; and inform future modeling investments for research and operations.
A Ground Truth webcast Proposers Day for interested proposers is scheduled for April 20, 2017. DARPA seeks expertise in social sciences (economics, political science, anthropology, sociology, social psychology); computational social science; modeling and simulation (including agent-based modeling, system dynamics modeling, Bayesian generative modeling, etc.); quantitative and qualitative social methodologies; complexity sciences; mathematics; and forecasting.
For information on webcast registration and other details, see the Special Notice on FedBizOpps here: https://go.usa.gov/xXU5a.
###
Media with inquiries should contact DARPA Public Affairs at outreach@darpa.mil