Sensor-assisted cars that can avoid injuring pedestrians and cyclists are here, and will become the standard in a few years
by Steven Goodridge
“A robot may not injure a human being or, through inaction, allow a human being to come to harm.”
– Isaac Asimov’s First Law of Robotics
An interesting thing happened on the way to the self-driving vehicle: Smart collision prevention technology such as automatic emergency braking became practical for regular cars. After premiering in high-end vehicles with adaptive cruise control and lane departure warning systems, radar and camera systems have become commodity products now available in about a quarter of 2015 model year cars. As volumes have increased and the technology refined with custom silicon, prices have plummeted, with the typical cost of a smart camera or radar sensor now around $150 for an automaker to add to a car. And increasingly, the use of the such smart-car sensors has become focused on preventing collisions and saving lives, including those of pedestrians and bicyclists, who make up about half of fatal road crash victims worldwide.
Raising the Bar
Beginning in January 2016, any automaker hoping to earn the Euro NCAP 5-star safety rating for their product will be required to equip the car with automatic emergency braking capabilities that include pedestrian detection. The National Highway Transportation Safety Agency (NHTSA) recently announced a similar plan for its own five-star safety rating, as has the Insurance Institute for Highway Safety (IIHS) . Volvo, not surprisingly, now includes its pedestrian and bicyclist detection system with full-automatic braking as standard equipment on all of their new cars. Lotta Jakobsson, a Senior Technical Specialist at Volvo, describes the motivation for the technology: “Statistics show that about 90 percent of all accidents are caused by distracted drivers. Since City Safety remains alert even if the driver is distracted or tired, it helps to bring collision figures down considerably.” Although driver training and vigorous enforcement of traffic laws are essential to preventing crashes, mistakes will always be made by human drivers, bicyclists and pedestrians. Automated systems can reduce the human cost of those errors. For instance, a March 2015 analysis of car-bike crashes conducted by IIHS concluded that equipping traditional cars with automatic crash avoidance systems programmed to address the five most common car-bike crash types could prevent or mitigate up to 47 percent of car-bike crashes.
Consumer Interest
Do Americans really want R2-D2 riding along as back-seat driver? According to an April 2015 study by the Boston Consulting Group, drivers will demand automatic crash avoidance features to get lower insurance premiums. The insurance industry has measured reduced crashes and claim costs for cars with the technology. An IIHS Highway Loss Data Institute (HLDI) study revealed that Honda Accord models featuring a new forward collision warning system reduced claims for injuries to occupants of the equipped vehicles by 27 percent and claims for injuries to other road users by 40 percent. Insurers are now passing a portion of these savings on as reduced premiums for such cars. As Americans replace their aging vehicles and automatic collision systems become standard features, the majority of registered cars on US roads are likely to be equipped with the technology sometime between 2025 and 2030.
Entry Level Technology
The first bicyclist-and-pedestrian-friendly crash avoidance features to reach the car market are forward collision warning (FCW) systems and automatic emergency braking (AEB) systems. FCW systems detect when the driver is on a collision course with a motor vehicle, bicycle, or pedestrian ahead and alert the driver with a visible and audible warning if they have not begun braking in time to avoid a potential crash. AEB systems go one step further by applying the brakes to slow or stop the car and thereby prevent or mitigate the collision if the driver fails to brake effectively, or at all. Most current FCW and AEB systems combine a forward-looking radar unit in the grill and a camera behind the windshield to determine the range and position of objects ahead and identify which ones require a response. These are the same or similar sensors as those used for adaptive cruise control and lane departure warning. First generation FCW and AEB systems should be well suited to prevent car-bike rear-ending crashes, where the car approaches a bicyclist from behind traveling in the same lane ahead in the same direction. Junction and crossing type collisions involving pedestrians and cyclists coming from the sides of the vehicle’s path require wider angle sensing and pose more complex recognition challenges.
Automotive Radar
Radar sensors operate regardless of lighting conditions and in practically all weather except dense fog. Electronically steered radar units operating at 77 GHz provide detection and ranging of multiple vehicles and other objects simultaneously at ranges of up to 200 meters within a narrow (15-20 degrees) field of view. Such systems can detect pedestrians, bicyclists and motorcycle users in the roadway at ranges of up to 40-100 meters depending on their size. For shorter range sensing over a wide (120 degree) angle, 24 GHz radar is typically used. This wider angle is particularly important for detection and recognition of pedestrians and bicyclists who may be crossing or about to cross in front of the vehicle in urban situations. A disadvantage of 24 GHz and 77 GHz radars for pedestrian recognition and tracking is their limited angular resolution, resulting from the limited bandwidth allowed by the FCC for those frequencies. One strategy to increase angular resolution for pedestrian sensing is to use a more sophisticated ultra-wide-band radar system operating at 79 GHz, which will allow much wider bandwidth for automotive radar. The solution employed by US car manufacturers today, however, is to complement the radar with camera-based and/or LIDAR sensing.
Computer Vision
Camera systems provide high resolution sensing for shape recognition and position estimation. A variety of algorithms exist for detection and recognition of pedestrians in video. Some collision prevention systems depend entirely on monocular video analysis for detection of hazards; others, such as Subaru’s EyeSight system, use stereo vision for range perception without radar. Visible light cameras can be affected easily by darkness, glare and weather, making them less robust than radar. Infrared cameras address some of these limitations, but are more expensive than visible light cameras. Sensor fusion of video and radar data can leverage the complementary strengths of each mode for more robust operation.
LIDAR
The ultimate sensor for automated situational awareness may be laser radar, aka LIDAR. The sweeping beam of LIDAR provides a very high resolution range image, which is favored by developers of fully autonomous cars for its value in self-localization and object tracking. LIDAR is its own light source, operating under all types of ambient outdoor illumination, but has a shorter range than radar. Increasing the power and range of LIDAR usually requires a bulkier form factor, such as the large cannisters seen atop many prototype autonomous cars. Inconspicuously packaged short-range LIDAR is being used by a number of car companies (including Volvo in their City Safety product) to complement radar and camera sensing for improved resolution of range data.
Sonar
At very short ranges, ultrasonic sensors outperform other technologies. Sonar sensors have been used in back-up safety systems on trucks for decades, and are now used in the parking-assist systems on many new cars. These sensors have the potential to prevent thousands of tragic backover collisions involving children.
Radar, vision and sonar sensors for 360 degree awareness. [Image provided courtesy of Analog Devices]
360 Degree Awareness
Comprehensive crash avoidance technology will require 360 degree sensing around the vehicle. Automakers envision a combination of multiple radar, sonar, camera and/or LIDAR units to provide the type of traffic detection required in each direction. These include close-range sonar for parking assist and backover prevention, medium range radar for blind spot sensing, lane change and cross traffic detection, video for lane departure warning and object recognition, and long range radar for same and opposite direction vehicle detection. As hardware continue to costs decline with electronics integration and simplified packaging, the number of sensors that appear on new cars will quickly multiply in the next few years.
Limitations
Multiple threat collision mode involving sight obstruction. [FWHA image]
Despite distraction-free, continuous operation, even the best modern vehicle sensors cannot prevent all crashes. Many crashes involving bicyclists and pedestrians involve occlusion of sight lines between the motor vehicle and the other party until just before collision. One example is the multiple-threat pedestrian crash, where the view between a motor vehicle and a pedestrian crossing the roadway is blocked by another vehicle that has stopped in the adjacent lane. A similar situation exists when traffic screens the view between a left turning driver and a thru bicyclist. Existing left turn assist systems cannot see through cars and operate only at low speed. Research into improved left turn assist and occluded view collision prevention systems has focused on using vehicle-to-vehicle (V2V) and infrastructure-to-vehicle communication to supplement the sensors on a single vehicle. This strategy provides measurement of pedestrian and bicyclist activity by sensors at one or more locations and transmitting it to multiple vehicles that may encounter movement conflicts. However, due to a variety of liability, privacy, security and safety challenges, initial V2V communications deployment is still a few years away.
Unlike human beings, modern vehicle sensors cannot infer the intent of human beings near the roadway or predict dangerous behavior. Existing technology cannot tell the difference between a waiting adult and a confused person or child who might dart suddenly into traffic; just discriminating between humans, animals, and mailboxes can be difficult enough. Crash avoidance systems on the market today cannot compensate for reckless or drunk driving, or persistent distracted driving. Although technology is in development to deter and detect such behaviors, there is less consumer enthusiasm and market demand for technology that watches drivers directly than for technology that looks outside the car.
The current crop of forward collision warning and automatic braking systems are optimized for relatively low vehicle speeds where sensing and stopping distances can be kept short. This may work well enough in European nations where streets through most built-up areas feature speed limits of 30 mph or even just 20 mph. But in the US, most US pedestrian and bicyclist fatalities occur on roads posted 40 mph or higher. Much of our urbanization developed around high speed thoroughfares, so pedestrians and bicyclists commonly travel upon and across roads with speed limits of 35 to 55 mph. This requires reliable detection and tracking of bicyclists and pedestrians at longer distances, where the resolution of today’s narrowband automotive radar systems can become a limiting factor. Moving forward, it will be critical for regulatory authorities such as NHTSA to develop meaningful performance metrics for bicycle and pedestrian detection by collision avoidance systems operated under realistic US thoroughfare conditions.
The limitations of automated crash avoidance systems mean that all of the proven engineering, education and enforcement safety measures used to support pedestrian and bicyclist safety (such as lower speeds) will continue to be important. Street environments that improve human drivers’ ability to see and yield to pedestrians and cyclists will also improve the reliability of automated systems. Engineering that rewards safe pedestrian and bicyclist behavior by making it convenient and easy will continue to reduce hazardous conflicts as automation increases.
Risk Compensation
One concern that has been expressed about automated crash avoidance systems is the possibility that drivers will compensate by being less vigilant, relying on the technology to save them during distractions. Efficacy studies performed to date show real reductions in crash rates, but more data will be required to rule out the possibility of risk compensation. The worst case scenario would be for drivers’ risk compensation to cancel out the technology’s reduction in car-car crash risk, but for the technology to fail to detect bicycles and pedestrians reliably, resulting in an increase in car-bike and car-pedestrian crashes.
Alternatively, if smart car technology does reduce the crash risk to pedestrians and bicyclists, and these road users become more confident and travel more, this would be a very good thing. Traffic safety concerns are a common reason people give when asked what factors limit their bicycling and walking. The health and societal benefits of increasing walking and cycling generally outweigh the risks today, and this will be only more true in the future. One of the most commonly cited fears given by potential utility bicyclists – the fear being struck from behind by a careless, drunk or drowsy driver – depicts a relatively uncommon crash type in urban areas, but is one of the easiest types of collision that can be prevented by automatic braking systems. Elimination of this risk could make many bicyclists to feel more empowered on normal streets, and shift the focus of bicycle traffic safety to the more common intersection conflicts. And if parents knew that they could count on motor traffic to stop and yield at crosswalks, stop signs and red lights, rather than just barreling through them, many more would allow their children to walk independently to school and other destinations. Vehicle automation may be a limited but critical part of a holistic approach to improving the walkability and bikeability of our communities.
Next Steps
As the NHTSA moves ahead with its AEB recommendations, it will be important for pedestrians, bicyclists, and related stakeholders and advocacy organizations to give the government and car manufacturers a clear message that approved AEB systems must detect pedestrians and bicyclists reliably under road conditions that are realistic in the USA. Similar attention to this issue is needed at IIHS. While pedestrian and bicyclist detection performance in AEB systems has been a key interest in Europe and Asia, it does not yet appear to be on American institutions’ radar.
About the Author
Steven Goodridge is a board member of BikeWalk NC and holds a Ph.D. and MS in electrical engineering from NC State University, where he studied autonomous robotic vehicle systems, computer vision and sensor fusion. Dr. Goodridge enjoys teaching traffic bicycling classes and commuting to work by bike in Cary.