Racing at the edge | Ep 87

Intro Voices
Coming to DARPA is like grabbing the nose cone of a rocket and holding on for dear life.

DARPA is a place where if you don't invent the internet, you only get a B.

A DARPA program manager quite literally invents tomorrow.
Coming to work every day and being humbled by that.

DARPA is not one person or one place. It's a collection of people that are excited about moving technology forward.

Tim Haynes
Hello and welcome to voices from DARPA. I'm your host, Tim Haynes.

Announcer
If you don't recognize it, that is a Volkswagen tour coming in through the finish line configuration.

Ladies and gentlemen, boys and girls. It's been done.

How about that?

Tim Haynes
We just heard the sounds of an autonomous vehicle named Stanley crossing the finish line to win DARPA's second off-road Grand Challenge back in 2005. It was one of five vehicles to successfully complete the 132 mile desert course without any human intervention. In the 20 years since, competitors from that challenge have helped establish the rapidly advancing self-driving car industry.

Our guest today, Doctor Stuart Young, program manager in the Tactical Technology Office, was at the first Grand Challenge event in 2004 where there wasn't quite as much to cheer about.    

Stuart Young
I was actually in the desert as an engineer from the Army Research Lab for the first race, with all the hype and hoopla and excitement and, you know, then things, as we know, didn't go so well. On the first experiment, the farthest vehicle went about seven miles. And there was a little bit of, you know, melancholy about, like, why did this - you know, should have done better.

But the thing about the environment is you can't predict the edge cases, kind of by definition. And I think it’s a really great foreshadowing of the problem that I was interested in.

Tim Haynes
Stuart was at the Army Research Laboratory for 29 years working on various programs, including several from DARPA.

Stuart Young
Ran a division where we, you know, started focusing on developing an internal research team focusing on field autonomy, off road autonomy. Then I was invited to come to DARPA and try to work on that problem at the DARPA level. So I jumped at the chance. My career goal was always to be a DARPA program manager, so I'm super excited to be here.

Tim Haynes
Stuart joined the agency in early 2020, quickly developing a portfolio focused on advancing autonomy in multiple domains.

Stuart Young
Once I got here, I was handed the ALIAS program to complete that program, which is Aircrew Labor In Cockpit Automation system, focusing on developing autonomy aids to improve safety and reduce cognitive burden of pilots on enduring aircraft – some people call them legacy or older aircraft. And then the main thing I came to DARPA to do is create RACER.

Tim Haynes
RACER stands for Robotic Autonomy in Complex Environments with Resiliency.

Stuart Young
We need to be able to travel fast on cross-country, off-road terrain. We need to do that at a speed that is operationally relevant. So the bottom line is we have to do it fast, and we have to do it reliably, so that the warfighters can have it as a tool in their toolbox.

Tim Haynes
The program kicked off in late 2021, and at the time of this recording in spring of 2025, RACER is in its fifth big experiment, rigorously testing its algorithms on platforms in the real world

In case you haven't seen videos from RACER yet…

Radio Voice
Three…

Tim Haynes
… we want you to be able to picture what Stuart is talking about.

Radio Voice
Two…

Tim Haynes
RACER is using two very different kinds of vehicles to test its autonomy algorithms.

Radio Voice
One… Mark. Go for auto.

Radio Driver Voice
Copy – switching to autonomy.

Tim Haynes
The first is a two ton, four wheeled, all-terrain vehicle, a little bigger than a golf cart. The other is a 12 ton, 20ft long vehicle with two giant treads that looks like a tank. So imagine both of these vehicles tearing across deserts, up and down scrubby hills, swerving through forests, and tackling all kinds of other terrain at up to 30 miles an hour with nobody aboard.

That's RACER in the field. We've been posting RACER video highlights since the first experiment. You can check out links in the show notes or visit the DARPAtv YouTube channel to see more. Now back to Stuart and RACER.

Stuart Young
We have no expectation that the trafficability of the terrain directly in front of us is navigable by the vehicle or not. And so constantly we have to evaluate that. And so that's computationally very expensive. We're doing it very close to the vehicle as well as as far out as possible, which leads to the speeds that we can attain.

That is not something that is static either. So you can imagine driving across a dry lakebed looks perfectly tropical. It's dry dirt. Let's just drive and go really fast and then it rains. And then holy cow, that dry lakebed is no longer a dry lakebed. And the trafficability of that same terrain, it was beautiful a few hours ago or a few minutes ago was no longer trafficable.

The reality is, combat vehicles especially will have to maneuver in terrain that is highly complex off road for tactical survivability reasons. In combat, the roads or terrain will obviously continue to be degraded through the activity of both sides, and so consequently, you have to deal with that. You know, if roads are out or bridges are out, it's like, okay, well then deal with it.

So we have to have the flexibility to have our maneuver forces be able to drive wherever they need to. And we don't want them to be constrained to roads and trails. This is particularly important, as we saw in, for example, the COIN fight over the last two decades where IEDs were such a problem.

Tim Haynes
To clarify, a COIN fight is a counterinsurgency fight where governments are trying to defeat rebellions and an IED is an improvised explosive device - basically a bomb made from whatever random parts someone can get their hands on.

Stuart Young
Well, the IEDs were always on the roads. Well, then don't drive on the roads, right? But that wasn't really the way we solve that problem. It provides you operational flexibility. It provides the ability to put more challenges for your adversary to prevent you from doing things. And quite frankly, that's where you can hide and give you that survivability that ultimately we think is important for the military.

Tim Haynes
Doctor Trent Mills is a colonel in the US Army and is the Army advisor to the DARPA director. He spent significant time and energy over the last few years engaging with the Army and other services to help accelerate understanding and adoption of autonomous capabilities.

Trent Mills
We're open now to conversations like this, like, okay, well, what does autonomy really bring? We're not looking at autonomy as an option or as a tool in the toolbox. “Let's sprinkle some autonomy on it.” We don't think of it that way. We are appreciating how autonomy is going to change the paradigms. The very way we see warfare has to be seen through a lens of autonomy, not as an optional thing to plug in.

And when you start to do that and you do that all the way across the different warfighting functions of sustainment, of logistics, of communications, when you start to bring autonomy in, is the thing you see the world through. You start coming with better ideas. So it's inspiring; it's a really kind of optimistic time for DARPA to start unleashing these things because we're ready.

We have a whole population of – I call them "baby general officers" - that have grown up for 20 years of, "this  doesn't work,” right, to now going – now, they're junior officers – now they're like, "we got to do things differently, guys. We are not going to do that again."

We don't think in single domains anymore and we include cyber and space. So autonomy applies to everything. It's not just a secret sauce. It's like I said at the very beginning, it's the paradigm through which we see the battlefield and the way to win.

One of our more prominent four stars, who runs Army Futures Command, for years, he said, "no blood on first contact," because we know we're going to lose a lot of people, but not on the first contact. That's for the robots. And he'll say it just like that and say, you guys figure that out. “You guys” meaning the enterprise, you know, all the smart people.

The human domain is the Army. We're not platform specific. We arm the human. In the other services, we put humans in the machine. We wrap machines around humans, because that's the primary weapon system, is the human being.

Tim Haynes
The first Grand Challenge was born with the intent to meet the warfighter needs - vehicles that can be driven without people in them to keep soldiers out of harm's way. The lessons learned from that first Challenge continue to inform Stuart's work.

Stuart Young
The problem with the world is there's an infinite number of edge cases, and so you can't deal with them until you discover them, and then you discover them, sometimes catastrophically. So that was the thing I took away from the first one, and I was extremely motivated to like, okay, this is a pretty crazy hard problem.

And then fast forward to when I was pitching RACER. Ironically, the director asked me the question – it’s like, well, “I thought we solved this problem in the Grand Challenge.” And I said, well, actually we didn't. We solved a different problem. And so the Grand Challenge essentially was an oversimplification of the problem that we are trying to deal with in RACER. But it was a perfect motivation, and it took DARPA to motivate the industry to get started on that problem, which was more akin to taking advantage of structure in the environment and then solving the on road driving problem, and also exposed the fact that, "hey, this off road problem is actually a lot harder than we thought."

Primarily, the difference is the amount of a priori information and the structure in the environment. So in the self-driving approach, and even back in the original Grand Challenge, there was quite a bit of knowledge about the course that they were going to go on. So they had ways to essentially follow the yellow brick road, you know, follow breadcrumbs along that trail and then basically stay on the trail. But they were essentially paralyzed if they would have to go off that trail. The part of the problem that RACER’s dealing with - that is our norm. Like, our norm is to not be on trails.

Tim Haynes
The technology landscape has changed drastically since the Grand Challenge. Advances in compute power, sensor capabilities, and AI algorithms making RACER possible are the same trifecta advancing most of the tech landscape. What’s setting RACER apart is its grounding in the physical world.

Stuart Young
We realized we need more miles on vehicles, and it wasn't just putting miles on vehicles for the purpose of miles. It's miles on vehicles for the purposes of learning where the systems work, where the systems don't work. Going after those edge cases I talked about earlier.

I mean, these are off road vehicles for a reason, and there's nobody on board. So we're only damaging metal and hardware. We can tend to be aggressive. And that was a paradigm shift that wasn't happening before -- it's that aggressive nature. The ability to repair things quickly and have a high cycle time in the field was another contributing factor in addition to the technology.

Tim Haynes
For as much as it's critical to get RACER going as many miles as possible, it might be somewhat counterintuitive to learn that this off road autonomy program isn't focused on hardware development.

Stuart Young
RACER is a software program. You know, we're building autonomy stacks. We're building algorithms. We're building data sets. We're building models, you know, neural network based models. So the data gets used to train our models. There's manual tuning, which is not very scalable. So we have to develop labeling pipelines that are efficient. We also do self-supervised learning approaches. All of that together is essentially the autonomy stack that results in the system being able to execute, driving the vehicle in that terrain.

Tim Haynes
Processing multiple sensor feeds through an autonomy stack to do real time decision making is incredibly resource intensive. That's not a deal breaker for online AI tools like ChatGPT, where computing resources are virtually unlimited. But autonomous platforms have to use edge computing, which means they're limited to their own compute power and energy, so efficiency is critical. RACER platforms have to be capable of operating totally off the grid in GPS- and communications-denied environments, while still safely fulfilling their missions.

We shared a high level overview of the two RACER platforms earlier. Stuart's going to provide a few more details.

Stuart Young
We set this program up that we wanted to test very aggressively and very hard, and so we needed a vehicle that would be reliable, and we tried to find one, and there really wasn't one that would do what we wanted to do, so we had to build it. We started with as much stock as we could and then we tricked it out.

We started with like a Polaris side by side four seater, which was very capable, and we were able to ruggedize all of our sensing and compute to be able to get after that. And that was really important for us evaluating the algorithms of our performer teams. And we were able to recover and quickly repair it when we were too aggressive, which was actually also a key component to the program, which is: push hard, be aggressive, and when you have a problem, you learn from it and you fix the vehicle and then you keep moving on.

We also put it on a tracked vehicle, which was a Textron M5, and that was to prove the agnostic nature of the autonomy to different types of platforms. So we go from a wheeled Ackerman-steered vehicle to a large, heavy skid steer vehicle, and that transitioned to those different types of platforms. In theory, this shouldn't affect the autonomy front end like the perception and planning, but should only really affect the controls back end.

We very quickly had the system running on that different style. Now that we've done that, it's critically important because we understand the nuance and transitioning to other types of vehicles. So I believe we truly are agnostic to the vehicles now. The Army, the Marines have interest in putting this on some of their program of record vehicles, and that is definitely not a barrier to the system.

It's usually more of, where can we put our sensors and the compute associated with it, but it's not about the vehicle. So the autonomy is generally applicable to any type of land vehicle.

Tim Haynes
The RACER program aims to not only be platform and payload agnostic, but also easily adaptable to any environment.

Stuart Young
We've tested it in - proper tested in - four different terrain biomes. We've been to the desert, the Mojave Desert, so we've tested it at National Training Center in Fort Irwin, California a couple times. So that area is replete with obviously rocks and sand. There's not a lot of vegetative, what we call different classes of semantic segmentation. It's basically rocks or sand and that kind of thing.

It's not like brush and stuff you've got to go around. So it's very geometric in nature. And the problem that you're trying to solve there. Then we've gone to places like Camp Roberts, California, which is Central Coast, California, which is - obviously, California is a big state - so still a very different biome.

We've also gone to Fort Cavazos, Texas, which is hill country Texas. It's a lot more deciduous, a lot more trees. The Central Coast also had trees and very significant ditches and elevating type of terrain.

How much can we take the models that we build from one environment and move it to another, which is a key part of that resiliency that I was talking about. So we built models when we went to our fourth experiment at Fort Cavazos, Texas. That was the first time that we had been there. We did not build a model on that environment with any data from that environment by design. We wanted to see how bad our model would be, and the reality is it did pretty well right out of the box, which was a little bit surprising, but also gratifying. But it didn't take long for us to make some very small changes to our system. And then it performed much better.

So that adaptation, how fast the systems can adapt, what's the manner of adaptation, is something that we're really trying to understand. So the biomes that we've uncovered so far are not representative of all biomes in the world, but they are distinctly different. And that's been our goal. And we will continue to do that as we finish the program and go to some other types of vegetative environments, you know, pine forests and that kind of thing.

Tim Haynes
As the RACER program has continued to push boundaries and tackle the next edge case, it's progressed through multiple phases and performer teams.

Stuart Young
In phase one in the program, we had initially three performers. We had Carnegie Mellon University and a consortium they put together. We had the University of Washington, and we had the NASA Jet Propulsion Laboratory in Pasadena. We always had envisioned that we needed to not only prove this technology capability, but we also needed to provide a mechanism for the services to be able to buy this capability when it existed.

And so working with the performer teams, two of them spun out companies. University of Washington spun out Overland AI and Jet Propulsion Laboratory spun out a company, Field AI. Those companies are off and running, doing really well. The ability to transition this in a scalable way, you know, universities aren't usually set up for that. So you really needed industry to be involved so that the DoD and other dual use can take advantage of that. And so that is helping somewhat organically, although that was always a desired goal of the program to facilitate that.

So I think there's a lot of disruptive energy that we're creating and showing what is possible, the state of the art, and then trying to show the operational evidence that this is actually ready to go, maybe earlier than they anticipated.

Tim Haynes
Colonel Mills weighs in with his perspective.

Trent Mills
What the Army has learned from watching RACER and DARPA over the last several years, really - it's really kind of multifaceted. But the biggest success stories for me are how we've informed the requirements process. Warren, Michigan, where our folks that do kind of the autonomy integration into our ground vehicle systems, they are generating requirements documents that will drive acquisition and procurement for the Army for the next several years.

And a lot of the content of those really boring, dense documents are lessons learned from what RACER has revealed. There are lots of examples of how what RACER has done has shifted the appetite, shifted the ambition of the Army. And that's hard to do because the Army has a fight tonight, we can't wait, we have to fight now.

So when we see it, when RACER proves something back in June and whatever. And here's the metrics. We're taking notes. The Army is like, "oh, okay, we can do that now. Let's put that in the requirements document." So then we can tell the acquisition community to go get that stuff. Because we know. We saw it.

What signals my optimism is that I've seen the Army do it. I've seen the Army take it seriously. I've seen senior officers talk about it in an intelligent way, not just, “we're going to do some autonomy.” That was kind of par for the course even a few years ago. Now our general officers are speaking in a way - they understand what autonomy stacks are. They understand what the network is going to require in order to support this thing, which is really making me optimistic about that going forward.

Tim Haynes
You might assume Stuart's taking a victory lap with RACER, but that's far from the case.

Stuart Young
Just because you have a driver's license doesn't mean you can drive a tractor trailer or a motorcycle, right? The Department of Transportation, they give you, you know, a license to drive those in special conditions. And so similarly, we have shown that we can drive in the conditions that we've tested it, but we know there's other conditions that we still need to get after.

So for example, driving at night, we need to do a better job of that. We've done it, but we can do better. So we are trying to drive the number of interventions down to zero, which means the autonomy has to get better for those edge cases so that when it encounters those edge cases, it either can deal with them or it can react more appropriately.

So it could be the robot is just like looking at something and it doesn't know what to do, and it needs help. Or it may have crashed and flipped over or something like that. So clearly those are different degrees of intervention, and we do characterize those differently. It's a two hour recovery versus it's two seconds for a user operator to say, hey, back up and go around and do this. So it's really going after the adaptation of the system to other environments. And driving those interventions to zero is where we're still focused on continuing to push on that. There's still a lot of meat left on that problem.

Tim Haynes
As our conversation comes to a close, we'll give the floor to Stuart for his thoughts on his experience at DARPA.

Stuart Young
Coming to DARPA was something I obviously always wanted to do, and it has not for a minute disappointed. This is the greatest job in the government. It's the most stressful job in the government. But the beauty of it is, it's all self-motivated stress because I am empowered to solve a very complicated problem for national security. I'm given all the tools to do it, and I'm only limited by my time and my ideas and my team's ideas to solve that problem.

Tim Haynes
That's all for this episode of Voices from DARPA. For more information on Stuart Young and his programs and for more RACER videos, check the show notes or visit DARPA.mil. As always, thanks for listening.