Urgent egress and passenger overrides

Should a passenger be able to command an unsafe vehicle behavior?

Incidents with Waymo robotaxis involving occupant harassment and burning cars have brought a question I wrote about a few years ago to public attention: What if you need to get out of the vehicle RIGHT NOW in a robotaxi -- is that allowed? And should an occupant be permitted to command the robotaxi to override road safety rules in self defense? What are the implications?

In any automated system there will be times when an occupant wants to override the automation, and especially when they want to exit a moving automated vehicle. Reasons might include: wanting to re-open transit vehicle doors if a passenger was unable to exit in time at their stop; an attack of claustrophobia; wanting to get away from another passenger due to personal safety concerns; or even needing to escape a cabin fire. (Do you really want to have to be fumbling with the ride hail app to find the “cancel trip” menu while there is smoke pouring into the cabin from a battery fire?) Some egress requests might constitute misuse or abuse, such as stopping a vehicle to intentionally block traffic, or intentionally accessing an off-limits area such as a bridge with no pedestrian infrastructure.

Creating a complete list of all possible motivations is difficult, and weighing the merits of all such egress attempts in advance seems intractable. Nonetheless, there are times when a passenger desire to exit a moving vehicle should be honored, although the vehicle should likely at least stop before permitting an exit.

In still other situations passengers might want to force an otherwise stopped vehicle to move. One reason might be fear for personal safety if threatened by malicious actors while stopped at a traffic light. Another reason might be overriding a police stop if the vehicle occupant suspects a stopping officer is instead a criminal imposter, at least until legitimacy of the police stop can be confirmed via contact with an emergency dispatcher.[1]

Another special situation is one in which a passenger has a compelling reason to order an AV to operate outside its ODD or with degraded equipment in an emergency, even if doing so will result in a reduced safety margin. For example, an AV might be programmed not to drive through heavy smoke, but doing so might be required to escape a burning town in a wildfire situation. The AV occupant might want to take the chance of driving rather than remaining in the burning town.[2]

Human drivers have the authority to deal with these situations so long as they are willing to accept the responsibility. Do you start driving when someone is trying to forcibly break into your car at a traffic light even if you might injure that malicious actor by doing so? The choice – and responsibility for consequences – falls upon the person driving a manually driven car.

The question is: to what degree should an AV permit operator overrides of safety-relevant behaviors? A complication is that there might not be a responsible individual in a vehicle to exercise control. What if a passenger is allowed to override some behaviors of the vehicle, but that passenger is impaired, or not capable of exercising mature judgment? Should an 8 year old riding solo be able to command vehicle safety overrides?[3]

Answers as to how much control a passenger should have over AV operation will depend on how stringent qualifications are for a passenger to be capable of mature decision making. It is easy to say there must be one qualified driver if there are any passengers in an AV and that manual controls must be available if needed. However, requiring a qualified driver undermines the potential benefits that AVs might provide for those who are not capable of driving or should not be driving at a particular time.

If other than unimpaired licensed drivers are permitted to override AV behaviors, there will be difficult tradeoffs as to what overrides might be permitted. Likely an 8 year old child should not be permitted to exit a school vehicle in the middle of a highway to avoid going to school. On the other hand, a 14 year old[4] might be considered mature enough to demand an emergency stop if the vehicle tries to drive into flood waters, or initiate an emergency exit with their younger sibling if the cabin fills with smoke from a vehicle battery fire.

Even if a passenger is an adult licensed to drive, should that adult be permitted to override vehicle behavior if drunk or otherwise impaired? If not, should the vehicle disable override capability if the passenger is drunk? Or should it be illegal to enter an AV with override capability when drunk? If drunks can’t catch a robotaxi ride home because they need to be sober to ride, then what is the point?

While it can be an interesting exercise to conjure extreme situations, the issue of passenger overrides and egress can also be as simple as a passenger saying “I want to get out now” when the vehicle is stopped at a red traffic light but not at the end of the scheduled trip. Should the passenger be able to unlock doors and exit? Or should the passenger be kept locked inside the vehicle until the end of the trip? Should there be a workaround available such as changing the destination? If so, should the passenger have permission to do this if some authority figure such as a parent input the original destination? Where should the threshold be drawn at which such a passenger request is denied both in context (speeding down a highway vs. stopped) or passenger maturity (a passenger one day before turning 18 years old but with no driver’s license vs. grade school age child)?

There is the possibility that remote operators will need to mediate requests for overriding AV behavior either routinely or if there is doubt as to the competence of passengers to make reasonable decisions. However, any such remote operators can be expensive, will have problems scaling, and might result in wait times long enough to impair safety by delaying decisions in urgent situations.[5]

For AVs to be deployed at scale, designers will need to decide how much authority passengers have to override vehicle behavior, and whether emergency manual vehicle controls will be required even in vehicles that are intended to be completely automated. There will be no perfect policy choice, but not setting a consistent policy is also a policy choice.

---

[1] Yes, this is a thing. Report of an accused police imposter pulling over a van full of legitimate police detectives: https://bit.ly/3DOSiFy

[2] This consideration has become especially relevant for residents of California. See: https://www.insideedition.com/how-drive-through-fire-48422

[3] One might say that no 8 year old should ride in an AV solo. But if that is the case, what exactly is the cut-off age? Some public high school systems rely on public mass transit instead of dedicated school buses, so any AV public transit vehicle will have under-age ridership. Is training or perhaps even a “rider license” required to ride in an AV and use the override controls? This topic gets complex quickly.

[4] Some states issue driver licenses to 14 year-olds in special cases. Would such a driver license be required in this case? See: https://www.thedrive.com/news/39184/americas-rarest-drivers-license-lets-14-year-olds-hit-the-road-legally

[5] The usual solution proposed is remote customer service operators that intervene when needed. Those proposing that passengers need have no control because remote operators can solve all safety problems need to spend more time waiting in customer service phone waiting queues. An additional consideration is the likely disruption to emergency response services during a natural disaster that will also require simultaneous attention to numerous AV passenger distress situations. New Year’s Eve screening of requests from potentially drunk passengers will also be challenging.

Thanks to Fred Perkins from the Center for Auto Safety for bringing this topic up in discussions on UL 4600.

This is an adapted excerpt (Section 10.4.4) from my book: How Safe is Safe Enough? Measuring and Predicting Autonomous Vehicle Safety