Uber launches an ‘AV Labs’ division to gather driving data for robotaxi partners

Uber has more than 20 autonomous vehicle partners, and they all want one thing: data. So the company says it’s going to make that available through a new division called Uber AV Labs.

Despite the name, Uber is not returning to developing its own robotaxis, which it stopped doing after one of its test vehicles killed a pedestrian in 2018. (Uber ultimately sold off the division in 2020 in a complex deal with Aurora.) But it will send its own cars out into cities adorned with sensors to collect data for partners like Waymo, Waabi, Lucid Motors, and others — though no contracts are signed just yet.

Broadly speaking, self-driving cars are in the middle of a shift away from rules-based operation and toward relying more on reinforcement learning. As that happens, real-world driving data has become hugely valuable for training these systems.

Uber told TechCrunch the autonomous vehicle companies that want this data the most are the ones that have already been collecting a lot of it themselves. It’s a sign that, like many of the frontier AI labs, they’ve come to realize that “solving” the most extreme edge cases is a volume game.

A physical limit

Right now, the size of an autonomous vehicle company’s fleet creates a physical limit to how much data it can collect. And while many of these companies create simulations of real-world environments to hedge against edge cases, nothing beats driving on actual roads — and driving a lot — when it comes to discovering all the strange, difficult, and flat-out unexpected scenarios that cars wind up in.

Waymo provides an example of this gap. The company has had autonomous vehicles in operation or in testing for a decade, and yet its current robotaxis have recently been caught illegally passing stopped school buses.

Having access to a larger pool of driving data could help robotaxi companies solve some of those problems before or as they creep up, Uber’s chief technology officer Praveen Neppalli Naga told TechCrunch in an exclusive interview.

And Uber wont be charging for it. At least not yet.

“Our goal, primarily, is to democratize this data, right? I mean, the value of this data and having partners’ AV tech advancing is far bigger than the money we can make from this,” he said.

Uber’s VP of engineering Danny Guo said the lab has to build the basic data foundation first before it figures out the product market fit. “Because if we don’t do this, we really don’t believe anybody else can,” Guo said. “So as someone who can potentially unlock the whole industry and accelerate the whole ecosystem, we believe we have to take on this responsibility right now.”

Screws and sensors

The new AV Labs division is starting out small. So far, it just has one car (a Hyundai Ioniq 5, though Uber says it is not married to a single model), and Guo told TechCrunch that his team was still literally screwing on sensors like lidars, radars, and cameras.

“We don’t know if the sensor kit will fall off, but that’s the scrappiness we have,” he said with a laugh. “I think it will take a while for us to say, deploy 100 cars to the road to start collecting data. But the prototype is there.”

Partners won’t receive raw data. Once the Uber AV Labs fleet is up and running, Naga said the division will “have to massage and work on the data to help fit to the partners.” This “semantic understanding” layer is what the driving software at companies like Waymo will be pulling from to improve a robotaxi’s real-time path planning.

Even then, Guo said there will likely be an interstitial step taken, where Uber will essentially plug a partner’s driving software into the AV Labs cars to be run in “shadow mode.” Any time the Uber AV Labs driver does something different from what the autonomous vehicle software does in shadow mode, Uber will flag that to the partner company.

This will not only help discover shortcomings in the driving software, but also help train the models to drive more like a human and less like a robot, Guo said.

The Tesla approach

If this approach sounds familiar, it is because it’s essentially what Tesla has been doing to train its own autonomous vehicle software over the last decade. Uber’s approach lacks the same scale, though, as Tesla has millions of customer cars driving on roads around the world every day.

That doesn’t bother Uber. Guo said he expects to do more targeted data collection based on the needs of the autonomous vehicle companies.

“We have 600 cities that we can pick and choose [from]. If the partner tell us a particular city they’re interested in, we can just deploy our [cars],” he said.

Naga said the company expects to grow this new division to a few hundred people within a year, and that Uber wants to move quickly. And while he sees a future in which Uber’s whole fleet of ride-hail vehicles could be leveraged to collect even more training data, he knows the new division has to start somewhere.

“From our conversations with our partners, they’re just saying: ‘give us anything that will be helpful.’ Because the amount of data Uber can collect just outweighs everything that they can possibly do with their own data collection,” Guo said.