Safety is the top priority for autonomous vehicle (AV) and advanced driver assistance system (ADAS) developers as they engineer the vehicles of the future. The industry is being challenged on its ability to significantly reduce, if not eliminate the 1.4M car-related deaths that occur worldwide each year. The challenge for stakeholders is threefold: First, there is a lack of clear and uniform safety standards and compliance protocols. Second, there is no multi-body regulatory framework between industry and the public sector. Third, there is an ever-growing cloud of public misinformation. Here, I will explore how startups are approaching safety in this space. However, to fully appreciate the intentions behind the initiatives these teams are taking, you must first form an understanding of the interplay between the major industry and regulatory groups.

“[The industry] aspires to have a zero-accident future, but as long as there are human drivers mixed in on the roads with automated vehicles, there's going to be accidents. I think the trick is figuring out this question of how safe is safe enough, and how do you accept that?” - Jack Weast, AV Standards at Intel

Institutions Responsible For Automotive Safety & Compliance

There are four major organizations that play a role in defining automotive safety standards, outlining methods of development that ensure safety, consistent quality, and help manufacturers stay compliant with the latest standards.

Society Of Automotive Engineers

The Society of Automotive Engineers (SAE) is a global body of engineering professionals that develops standards for mobility industries such as automotive, aerospace, and commercial vehicles. The association was founded by early automotive engineers including Henry Ford. The organization’s role in AV development is primarily in defining and mapping the five levels of autonomation. In doing so, SAE sets the benchmarks that other more granular standards must build on top of to clarify claims of certain AV and ADAS capabilities.

International Standards Organization

The International Organization for Standardization (ISO) is a non-governmental organization (NGO) that develops, publishes, and promotes worldwide industrial and commercial standards. The American National Standards Institute (ANSI) is often confused with ISO in relation to these standards. To clarify, ANSI is the U.S. division of ISO and promotes the dissemination of standards that are imposed at the global level. ISO is directing its focus on producing abundant, wide-ranging standards for functional AV safety. Functional safety is the realm of componentry analysis and how both traditional and AV components work together in the system in both hazardous and non-hazardous conditions. ISO gives developers a specified design process, specific technical requirements, and approved validation methods. The two cardinal standards from ISO concerning AVs are ISO 26262, and more recently, SOTIF (ISO/PAS 21448).

Underwriters Laboratories & Edge Case Research

Underwriters Laboratories (UL) is a private safety certification company and Edge Case Research (ECR) is a startup that offers consulting and tools to address safety concerns around full-stack autonomy. Together they are collaborating on the UL 4600 standards, which will publish later this year. The pioneering of standards by UL and ECR is a new phenomenon in the AV industry. However, ECR CTO Phil Koopman is a venerable figure in the space, having contributed for decades. Matched up with UL, the duo is seeking to fill crucial gaps in the safety standard landscape.

Insurance Institute For Highway Safety

The Insurance Institute for Highway Safety (IIHS) is a U.S. nonprofit research organization dedicated to reducing the number of deaths, injuries and property damage caused by cars. It is backed by over one hundred auto insurance companies. The IIHS is best known for its crash safety ratings, which are segmented into two classes. The first class, crashworthiness, defines how well a vehicle protects its occupants in a crash. The second class is concerned with crash avoidance and mitigation. These mitigation ratings are a key point of focus in ranking the safety of AVs and registering the necessary liability insurance.

US Government Organizations

The aforementioned non-governmental groups are working together in sharing their knowhow to provide a complete picture of the necessary controls for safe AV design. On the other hand, governmental regulatory agencies fit in by establishing frameworks for integrating AVs with public road systems, and most importantly, ensuring safety standards are followed.

Compared to other public concerns like digital privacy, the government is functioning well with industry players, as it stands to gain on this front. Why? The status quo is being challenged. The regulators are facing increasing pressure from support for programs like Road To Zero and Vision Zero, which seek to eliminate all roadway deaths. As a growing number of U.S. cities have espoused these targets, support/funding for AV development and regulatory progress have followed in lockstep.

Federal Department Of Transportation

The Federal Department of Transportation (USDOT), as overseen by the Secretary of Transportation, is the parent administration setting holistic public policies to maintain control over the evolving transportation system and national infrastructure. All of those listed hereafter are under the umbrella of the USDOT, each with different focal areas. The collective stance on AVs and ADAS is packaged into Automated Vehicle 3.0 that was published in 2018. The main takeaway from this document is that the DOT holds the power to greenlight trials and make recalls.

The organization requires each manufacturer to submit a “safety assessment certification” detailing how it is approaching autonomous vehicle safety. Additionally, manufacturers must publicize their cybersecurity and data privacy plans. As a criterion, vehicle software must be capable of being updated through over-the-air means so improvements can be distributed to vehicle owners on the fly. Finally, the Federal Automakers Vehicles Policy maintains that premarket safety assurance tools including premarket testing records, data, and analyses are reported by manufacturers to demonstrate that a new vehicle has met standards before being deployed on public roads.

The National Highway Traffic Safety Administration

The National Highway Traffic Safety Administration (NHTSA) is the most important sub-committee of the USDOT in regards to AVs. It is synergistically operating with the USDOT in creating and publishing nationwide AV policy. The NHTSA issues the Federal Motor Vehicle Safety Standards (FMVSS) to embolden these policies. The NHTSA outlines its most recent breakdown of best practices and the benefits of AVs here.

To assist in the regulatory oversight, the NHTSA requests that each entity testing autonomous vehicles to submit safety assessment letters that will outline how it is meeting the guidelines and addressing such issues as data recording, privacy, system safety, cybersecurity, and crashworthiness. In addition to its authority to issue federal vehicle safety standards and order recalls of defective vehicles, the NHTSA also leverages tools such as letters of interpretation, exemptions from current standards, and rulemakings to issue new standards or amend existing standards.

Federal Motor Carrier Safety Administration

The Federal Motor Carrier Safety Administration (FMCSA) is responsible for overseeing the safety of vehicles and their operation in interstate commerce relating to the design, development, testing, and integration of AVs. The FMCSA comes to the table when trucking/freight becomes a point of consideration.

Federal Transit Administration

The Federal Transit Administration (FTA) provides financial and technical assistance to local public transportation systems facilitating research and pilots. The FTA’s latest program is its Integrated Mobility Innovation fund that is providing $15 million for transit pilots involving autonomy for buses and shuttles.

The Federal Highway Administration

The Federal Highway Administration (FHWA) is the steward of the nation’s highways, bridges, and tunnels. On the AV front, the organization is championing the dialogue between the public and other stakeholders about how to best ensure that our road system evolves in tandem with our vehicles.

State DOTs

The individual state DOTs ensure that their respective state regulations fit into national policy.  In doing so, they oversee the development of local infrastructure to adapt to an AV-enabled system. Any vehicle operating on public roads is subject to dual regulation by the federal government and the states in which it is registered and driven. Traditionally, the National Highway Traffic Safety Administration (NHTSA) has regulated auto safety, while states have licensed automobile drivers, established traffic regulations, and regulated automobile insurance.

Nevada was the first state to permit the public operation of AVs in 2011. Since then, 28 other states have enacted legislation. 12 states have some form of highly-autonomous testing or deployment on public roads, and nine states have extensive AV testing or deployment on public roads.

We can’t discuss state policy and enforcement without discussing the imperfect California DoT Disengagement Reports. By law, companies testing on California’s public roads must disclose total miles driven and the frequency of human intervention (disengagement). The problem is a lack of uniform standards and understanding as to what constitutes a “disengagement”. At Dynamo, we suspect over the next three years, stakeholders will work to clearly define what constitutes a disengagement, as opaque standards only give way to meaningless safety communication requirements. Speaking to our portfolio, teams would love a clear definition as they work to make “the safest system possible.”

Lastly, as a reference, the IIHS lists the current breakdown of policy around development and deployment in different states.

Understanding The Major AV Safety Standards

SAE Levels Of Driving Automation

SAE released its most recent graphic demarcating the different levels of autonomy in 2018. The infographic helps to eliminate confusion by providing clarity and employing terms more commonly used by consumers. As of 2019, the industry currently sits right in the middle of the graphic, somewhere between levels 2 and 3. Tesla’s current lineup of vehicles is representative of this. This awkward stage is a point of contention among developers and regulators, with many developers opting to “skip” level 3 altogether. Levels 4 and 5 represent what generally comes to mind when the public thinks about autonomous vehicles—cars that can drive themselves. The only difference between levels 4 and 5 being ubiquitous application. Level 5 designates the ability of a vehicle to function in all conditions and geographies.

SELF DRIVE Act

The Self Drive Act is a federal bill that provides a universal framework for AVs, preempting state legislature unless state laws are identical to federal law. Before being enacted, technology development and state regulation had outpaced the NHTSA’s federal oversight. After being approved unanimously 54-0 in 2018, the bill facilitated unprecedented ease of pilot programs across the country. The bill revised the Federal Motor Vehicle Safety Standards (FMVSS) to account for advances in technology and the evolution of highly automated vehicles while prioritizing the protection of consumers.

Automated Vehicle 3.0

Automated Vehicle 3.0 is the latest installment in the DOT’s master guidebook to advance multi-modal safety, reduce policy uncertainty, and define the process for developers to collaborate with the DOT. The hallmarks of this expansion include clarification around the roles of stakeholders and how to best manage communication, along with guidelines for encouraging state and local AV testing. It is a proactive approach that remains technology-agnostic and encourages stakeholder engagement by supporting the development of independent technical standards.

ISO 26262

ISO 26262 encompasses functional safety in the event of hardware and system failures. The focus lies in outlining protocols to eliminate electronic systems malfunctions. It is also linked to other factors including driver exploits and the performance limitations of sensors or systems. Now, the majority of accidents involving AVs in “autopilot” modes have been caused by software and system architecture or engineering limitations, and not electronic malfunctions. Thus, this has prompted other standards to focus on non-electronic issues.

SOTIF (ISO/PAS 21448)

SOTIF was later introduced to supplement ISO 26262 by providing guidelines for guaranteeing the safety of the intended functionality (SOTIF) in the absence of a failure. SOTIF provides clarity and benchmarks around verification of testing, validation of simulations measures, and sensor design. The goal of SOTIF is to reduce potential unknown, unsafe conditions, by outlining the need for AI to accurately and probabilistically triage decisions. Since accounting for edge cases presents such a challenge, UL 4600 zeroes in on that niche. SOTIF can be thought of as an idealistic standard, whereas other standards like UL 4600 are prescriptive and practical.

UL 4600: “Developing The Safety Standard For The Evaluation Of Autonomous Products”

The focus of UL 4600 is to create an end-product safety case specifically outlining what a safe vehicle looks like without a human operator, no matter what underlying technology is used. This focus allows for an agile standard that can adapt to rapidly evolving technologies. The standard encompasses cars, mining, agricultural vehicles, shuttles, maintenance, and unmanned aerial vehicles (UAVs).

Building consistently safe self-driving vehicles is a great challenge but when solved will have a massive impact on our economy and society. It is promising to see the mosaic of institutions, both private and public, seeking to ensure that all parties are compliant in implementation and transparent in ongoing communications.