Inside the battle to own the data that will drive the automotive industry of tomorrow.
You never forget your first ride in a self-driving car. The precision and reliability of an autonomous car, far greater than a human driver can provide, is a revelation. It’s a visceral confirmation that a bold new transportation reality is right around the corner.
I took my inaugural ride in 2013 when I rode inside a self-driving Nissan Leaf alongside Tetsuya Iijima, general manager of intelligent transportation systems engineering at Nissan. As the Leaf found its way around a test course, Iijima told me that on-board sensors would allow tomorrow’s cars to navigate without the need for pre-mapping the environment. Self-driving cars didn’t need to have the world around them pre-rendered into a form they could understand. These cars could be safer and more reliable than humans simply by gathering and processing data in real time from a battery of sensors, like lidar, radar, and video. They would “see” the world, and react.
“[IF THE CAR] CAN’T SEE THE CURBS OR LANE MARKINGS, IT’S ALMOST LIKE PUTTING A BLIND PERSON BEHIND THE WHEEL.”
That was the prehistoric age of self-driving. It turns out that sensors are only half the picture. Today, autonomous vehicles and their sensors are three years more advanced, yet there is a growing consensus among auto industry pros and analysts—including Nissan’s own engineers—that maps will make or break the age of self-driving cars.
“Nominally, you can do autonomous driving without high-definition maps,” says Sam Abuelsamid, senior research analyst at Navigant Research. “But if you get into challenging situations like rainy or snowy conditions, or can’t see the curbs or lane markings, it’s almost like putting a blind person behind the wheel.” Therein lies the rub. For self-driving cars to gain real market acceptance, they need to have “anytime, anywhere capability,” Abuelsamid says.
That’s why the technology to map our world—not just on Google Maps, but in full 3D—is such a hot commodity. In December 2015, for example, BMW, Daimler, and Volkswagen’s Audi acquired the digital mapping company HERE for $3.1 billion. Last week I met with HERE chief executive Edzard Overbeek in the company’s Berlin office, his first interview with outsiders since taking the company’s reins nearly four months ago. It’s no coincidence that Overbeek, a 30-year IT veteran, was hired based on his experience in the last 15 years as a top executive at Cisco, the world’s largest networking company. Overbeek envisions the three-dimensional rendering of the real world as the world’s next great digital network. He calls it the “location index.”
THE INTERNET OF PLACES
Internet search indexing is dominated by Google. The social index—the way people record and share personal experiences and derive meaning from it, and the very platforms that make online social interactions possible—is a bit more fragmented, but it’s mostly in the hands of Facebook. What Overbeek wants to dominate is the index for locations: how self-driving cars derive meaning from lane markings, traffic signs, and the entire built environment. It’s something that goes way beyond the street names, intersections, and points of interest in today’s navigation systems. It becomes the essential platform for getting around. “If you think about cars, drones, or advanced robotics, all these things that need to move from A to B, you need navigation technology” Overbeek says.
HERE has a three-decade head start in mapping roadways and points of interest, with that data served up to the cloud and licensed to automakers. If you’ve used a car’s onboard navigation system in recent years—to find a destination, or locate the closest gas station or coffee shop—then HERE is probably the company that tracked down that information, made sure it’s accurate, and put it into a form that your car can use. Nearly 30 brands of vehicles, representing about 80 percent of the market, access HERE’s data for use inside navigation systems designed by the auto manufacturer. In 2015, Jaguar Land Rover became the first car company to have HERE design the user interface for its navigation system in addition to supplying the data.
HERE’s maps, which can power autonomous vehicles, capture billions of 3D points to model the road surface.
HERE
To obtain those points of interest, as well as the the richer, more detailed data needed for the new era of self-driving cars, HERE deploys about 400 vehicles equipped with a tower of spinning sensors to map the roadways of the world. These vehicles use lidar (laser-based radar), GPS locaters, and cameras capturing 16.8-megapixel images to produce data point clouds with uncanny two-centimeter precision. Altogether, HERE says, it collects 700,000 data points per second. A team of nearly 3,000 specialists, based mostly in Mumbai, India, cull and correct the data used in HERE maps, which the company shares with third-party tools and sells to companies like Garmin, Yahoo, Facebook, and Amazon.
At HERE’s office in Fargo, ND, analysts receive one-terabyte hard drives pulled from the vehicles returning from U.S. highways, streets, and cul-de-sacs. First, they scrub it for quality issues, like sunlight glare on images, and confirm that faces and license plates are blurred for privacy. A separate team processes road attributes, like stop signs and speed limits, for later use in turn-by-turn navigation. Maps are then cross-referenced with government databases to detect inconsistencies or possible illegal situations. Finally, engineers process lidar data into three-dimensional renderings of the road with the centimeter-level accuracy needed for a car to confidently drive itself.
“THE CAR WILL BECOME A SERVER ON WHEELS.”
No doubt you’ve heard of Google’s Street View vehicles, the ones that drive around taking photos so you can see what someone’s house looks like as if you are standing right there in front of it. HERE’s work is the next step: mapping the world not only in three dimensions, but breaking it down in such a way that a self-driving car can tell a bush from a person and a stop sign from a yield sign. The company is far from alone in collecting extraordinarily precise data about the real world. TomTom, Google, Zenrin, and Baidu understand the importance of gathering 3D point clouds of roadways for use in self-driving cars and have begun their own projects. Apple probably does it, too, though they’d never tell.
WHO OWNS THE MAP
Overbeek believes that having Daimler, BMW, and Audi as shareholders gives HERE a secret weapon in the battle for better maps. That weapon: your Bimmer or Benz.
“The car will become a server on wheels,” Overbeek says. “It will have lidar, radar, ultrasound, sonar, and infrared—technology that can be used to provide sensor information into our cloud-based platform.” Our cars will have not only these sensors, but excellent onboard computing capability thanks to companies like Nvidia, which sells automotive-grade fast processors. Processing power will be critical because a self-driving car will be merging its real-time sensor data with its mapping data to stitch together a picture of the world.
All of those car-based sensors will produce more data than can be effectively transmitted to the cloud, so the vehicle’s onboard computer—based on routes, driver commands and other specific driving conditions—will need to be selective about the data it sends and receives between the car and the cloud to complete the vehicle’s picture of the world.
“We bring processing capability as well as artificial intelligence and voice recognition to the car,” said Danny Shapiro, senior director of automotive at Nvidia. “With onboard real-time processing, the car can sense when an update is required to autonomously complete a route and only communicate the critical differences.” Shapiro foresees a future when digital maps are so good that if every car in a certain region (say a dedicated urban autonomous zone) were completely self-driving, then the roadway won’t need physical street signs or traffic lights and yet driving would be much safer than it is today.
Google and Apple want to stake out turf in vehicles, either by supplying data (as Google does for several automakers today) or via a wholesale takeover of dashboards with systems like Android Auto and Apple CarPlay. TomTom doesn’t want to be left out, so is working with Bosch, the world’s largest automotive supplier, to license its data to carmakers.
Google and Apple are notorious for scooping consumer behavior data for their own purposes. Both companies are also developing self-driving cars. A year ago, while speaking at a business conference in Berlin, Rupert Stadler, chief executive at Audi, said, “The only person who needs access to the data onboard is the customer.” Dieter Zetsche, Daimler’s CEO, has repeatedly expressed concern that Google’s incursion into autos is primarily targeted at what people are doing in cars, not a great driving experience.
Meanwhile, carmakers want a shared platform for creating elusive standards—for protocols related to communications between cars—but also for the critical elements of trust and safety. It’s a free for all, with automakers, tech giants and mapping companies all keen to own a big piece of the action.
Overbeek said that HERE’s automotive shareholders are willing to reach into their deep pockets to invest in these location technologies. “They realize this is a disruption,” he says. The human-piloted car in your driveway today will give way to tomorrow’s network of connected, automated, and electrified vehicles that will rely on maps. “It will come their way whether they like it or not.”