Arguing credibly about the safety of any autonomous vehicle (AV) requires somewhat more proof than a company like Waymo, Uber or GM declaring that its robocars are safe for commercial deployment. The automaker must be able to demonstrate that its AI-driven vehicles meet specific and rigorous standards.
The next questions then become:
Which safety standards?
Who’s developing the standards?
How are vehicles being tested?
Who’s going to “assess” the safety of the AVs?
Do we trust the assessors?
There are no easy answers. Further, safety standards, especially for AI-driven vehicles, come with additional twists and numerous caveats.
Put simply, AI-based autonomous vehicles are running on machine-learning algorithms housed in black boxes. The inner workings are probabilistic in nature, and it’s almost impossible to determine why they make whatever decisions they make.
This being so, what strategies do tech companies and car OEMs use to verify their AVs’ safety?
Will the public accept that the traditionally self-certifying automotive industry certifies the safety of its AVs all by itself? Someone might remember the bang-up certification job done by the FAA (which is apparently less than independent than previously thought) on Boeing's 737 Max MCAS system.
Can any safety standards keep up with rapidly evolving algorithms and software code deployed by self-driving cars?
Isn’t it prudent to think that AV safety standards might be short-lived in usefulness and destined to become obsolete too soon, too often?
UL 4600 draft standard
Against this backdrop, Underwriters Labs, currently developing a “ Standard for Safety for the Evaluation of Autonomous Products” — UL 4600— said the members on its Standards Technical Panel (STP) met in person for the first time on June 12 and 13, to review and discuss the initial draft standard.
EE Times last week caught up with Phil Koopman, co-founder & CTO at Edge Case Research, a principal technical contributor to the draft.
The minutes of the first meeting are yet to be made public. Koopman, however, described the first meeting as “very positive and constructive.” He said the members hit all the main issues of UL 4600 that must find solutions.
Who are on the roster?
According to the UL website, the UL 4600 group lists some 30 or so STP members with voting rights. They include four chip vendors — Nvidia, Renesas, Intel and Infineon — and commercial AV users and developers, among them General Motors, Uber, Nio, Bosch, Argo AI and Aurora. Both the U.S. Department of Transportation (DoT) and Pennsylvania DoT are sending representatives.
Interestingly, the UL 4600 STP also includes three insurance companies: AXA, Liberty Mutual and Munich Re America.
In pursuit of transparency and broader inclusion, the UL 4600 group is seeking any responsible party willing to be registered as a “stakeholder.” Once registered and approved, stakeholders can request to review and comment on the draft standard. While stakeholders have no voting rights, adding them is important, explained Koopman, to ensure “a very open procedure” and to signal that AV safety “is a matter of public policy.”
UL 4600 vs. ISO 26262 and ISO/PAS 21448 (SOTIF)
UL 4600 is a relative newcomer to automotive safety standards development. ISO 26262 already exists, while ISO/PAS 21448 (safety of the intended functionality or “SOTIF”) are well into development. The most frequently asked question about UL 4600 is why the automotive world needs yet another standard?
Stressing that the UL 4600 group is closely in touch with leaders in ISO 26262 and ISO/PAS 21488, Koopman made it clear that “resolving potential overlap is an ongoing activity.”
click for larger image (Source: Phil Koopman, Edge Case Research)
Developing safety standards based on the assumption that systems will have no responsible human driver is what separates UL 4600 from other standards.
In contrast, “existing standards such as ISO 26262 and ISO/PAS 21448 were envisioned for vehicles that ultimately have a human driver responsible for safe operation of the vehicle,” Koopman noted. In his opinion, the technology in robocars and other autonomous systems “exceeds the scope of these and other traditional safety standards. Those standards are necessary, but not sufficient.”
Did you think of that?
In other words, in developing self-driving cars, Koopman believes automotive design engineers will soon discover numerous issues they hadn’t even thought about before. Speaking of “the pervasive implications of vehicles not having a responsible human driver,” Koopman put those items under the rubric of “Did you think of that?”
Expect the UL 4600 to be much more prescriptive compared to other standards.
While ISO 26262 and SOTIF provide safety as a “target” to shoot for, UL 4600 offers a “bullseye,” said Koopman. For instance, UL 4600 will expect from automotive designers a lot of details, such as, “if you are doing X, don’t forget to do Y.” Other standards show “how to get to safety,” but UL 4600 prescribes “where you end up with your system.”