Study shines headlights on consumer driverless vehicle safety deficiencies

In a presentation at the recent Network and Distributed System Security Symposium in San Diego, researchers from UC Irvine’s Donald Bren School of Information & Computer Sciences described the real-world implications of what previously was only theorized: that low-cost and highly deployable malicious attacks can make traffic signs undetectable to artificial intelligence algorithms in some autonomous vehicles while making nonexistent signs appear out of nowhere to others. Both types of assaults can result in cars ignoring road commands, triggering unintended emergency braking, speeding and other violations.

The scientists said that their study, which involved the three most representative AI attack designs, was the first large-scale evaluation of traffic sign recognition systems in top-selling consumer vehicle brands.

“Waymo has been delivering more than 150,000 autonomous rides per week, and there are millions of Autopilot-equipped Tesla vehicles on the road, which demonstrates that autonomous vehicle technology is becoming an integral part of daily life in America and around the world,” said co-author Alfred Chen, UC Irvine assistant professor of computer science. “This fact spotlights the importance of security, since vulnerabilities in these systems, once exploited, can lead to safety hazards that become a matter of life and death.”

The lead author of the study, Ningfei Wang, a research scientist at Meta who performed this work as a Ph.D. student in computer science at UC Irvine, said that his team’s attack vectors of choice were stickers that had swirling, multicolored designs that confuse AI algorithms used for traffic sign recognition in driverless vehicles.

“These stickers can be cheaply and easily produced by anyone with access to an open-source programming language such as Python and image processing libraries,” Wang said. “Those tools combined with a computer with a graphics card and a color printer are all someone would need to foil TSR systems in autonomous vehicles.”

He added that an interesting discovery made during the project relates to the spatial memorization design common to many of today’s commercial TSR systems. While this feature makes a disappearing attack (seeming to remove a sign from the vehicle’s view) more difficult, Wang said, it makes spoofing a fake stop sign “much easier than we expected.”

Chen noted that the research was the first of its type on this security threat in real-world scenarios with commercially available vehicles.

“Academics have studied driverless vehicle security for years and have discovered various practical security vulnerabilities in the latest autonomous driving technology,” he said. “But these studies have been limited mostly to academic setups, leaving our understanding of such vulnerabilities in commercial autonomous vehicle systems highly limited. Our study fills this critical gap.”

Chen said that by focusing on a small subset of existing research in this area, his group was able to uncover various broken assumptions, inaccuracies and false claims. For example, no prior academic studies realized the common existence of spatial memorization design in commercial TSR systems. When Chen’s team members modeled such a design in previously devised academic study setups, they uncovered results that directly challenge earlier observations and claims made in the state-of-the-art research community.

“We believe this work should only be the beginning, and we hope that it inspires more researchers in both academia and industry to systematically revisit the actual impacts and meaningfulness of such types of security threats against real-world autonomous vehicles,” Chen said. “This would be the necessary first step before we can actually know if, at the society level, action is needed to ensure safety on our streets and highways.”

Joining Chen and Wang on this project were former UC Irvine graduate students Takami Sato and Yunpeng Luo; current UC Irvine graduate student Shaoyuan Xie; and Kaidi Xu, assistant professor of computer science at Drexel University. The work was supported by the National Science Foundation and the U.S. Department of Transportation’s CARMEN+ University Transportation Center, of which UC Irvine is a member.