Blame-Free Robotaxi Crashes Are Still Crashes
The tricky logic behind robotaxi crash metrics
Which robotaxi crashes should "count" for robotaxi safety metrics? Some say that all crashes should count. Some say that only at-fault crashes should count. And some discussions about robotaxi safety say things like “no fatal crashes” when they should be saying “no at-fault fatal crashes” — which are two quite different things.
The arguments for and against using blame as a filter for which crashes count is tricky. So let’s try to sort that out. For example, when some robotaxi company claims that their vehicles harm fewer people than human-driven cars, should that be based on all crashes or just at-fault crashes?
Some robotaxi industry supporters, especially some vocal ones advocating for Waymo, want only at-fault crashes to count. Others (including me) have held that this metric is too easily misleading as the only metric. I'll try to summarize the arguments and propose a metric approach based on my recent work on redefining safety principles for embodied AI safety.
For and against blame as a metric filter
The blame-matters camp seems to be either stating or implicitly employing variations of the following ideas:
If the robotaxi didn't cause the crash, why should the company be penalized in the court of public opinion?
If all cars were robotaxis and none of them are ever at fault in a crash, that might mean there will be far fewer crashes than with all human drivers.
Using blame highlights how bad human drivers are compared to robotaxis, at least while there are so few robotaxis that most crashes involve a human road user.
Since more information is available for robotaxi crashes via their on-board sensors, that data enables us to determine blame and switch to at-fault crashes as a more meaningful metric.
Insurance claims data hinges on blame, and that data is readily available.
(Blame-metric proponents are welcome to comment with additional ideas or corrections if I got this wrong. But this is what I’ve taken away from numerous articles and discussions.)
The non-blame camp tends to make the following points:
Fatalities/mile is the usual metric society is used to dealing with, so any other metric will lead to misunderstandings (as it quite clearly already has). Arguably, fatalities/population is even better as an epidemiological metric. Blame simply does not enter into these metrics.
Ignoring not-at-fault crashes removes a significant incentive to avoid mishaps even though they might be caused by others (i.e., removes some business incentive for strong responder role capabilities). A robotaxi might say they care about responder role safety behaviors, but I believe you get what you incentivize, and you get what you measure. Ignoring responder role crashes in the primary safety metric promoted by your company means you are not as strongly incentivized to avoid not-at-fault crashes. (And even if your company is special and puts effort into things not measured and not incentivized, it is unrealistic to expect all companies struggling to catch up to you will do the same.)
Blame is not all-or-nothing, because multiple road users can make mistakes that contribute to a mishap. Moreover, tort law proportional blame assignment rules vary dramatically according to state laws, leading to inconsistent tallies that are unlikely to reflect a reasonable engineering design approach to proportional blame. (That does not mean the laws are broken; it means that the intended goals of metrics in tort law situations is different than the use of such metrics for safety engineering.)
Blame assignment is not a strong improver for an unsafe system. The primary intent of blame in the legal system is to allocation financial consequences. But if insurance covers the loss, the signal to a manufacturer is attenuated if they can make money even if somewhat higher insurance rates. Insurance is only a weak motivation for acceptable safety.
Vehicles (and drivers) can behave in ways that are unsafe without triggering blame. This happens all the time on roadways, with the phantom braking problem for automated driving systems being a very real issue. They might even behave in ways that actively shed blame onto other road users.
In severe harm cases we are more likely to have to wait months or years for a blame determination until a court case and appeals play out. What should the blame determination be while that is happening?
Blame is slippery and difficult to assign accurately (follow-up on this below). Moreover, trusting an involved party who has access to mishap data that is not released publicly to unilaterally determine blame is asking the fox to watch the henhouse. There is simply too much incentive to distort blame assignments.
Why blame is so problematic
As a thought experiment, if a robotaxi design brakes unexpectedly or brakes harder than human drivers, it might be prone to being hit from behind by human drivers who have driving habits calibrated to human driver behavior. You can say that none of the crashes are the robotaxi's fault (no at-blame crashes). But what if a robotaxi has twice the total crash rate of human drivers because of this. Do the twice-as-many-people injured in those crashes somehow not count? Would you want to be a passenger in a vehicle who is twice as likely to be injured even if other drivers were always at fault? Somehow that does not seem like a vehicle that is "safer."
Another issue about blame is that it requires careful analysis of the situation. In principle a robotaxi company has the information. But they have strong incentive to find reasons to blame others. Some of the crash reports that blame others are laughably transparent in doing so. (For example, saying that a tow truck had an
“improperly” positioned towed vehicle, so of course the robotaxi company had no other option but to hit it. Twice on the same trip with two different robotaxis. Fortunately no injuries — but give me a break! Later there was a recall to fix the issue, which by definition means that this was a safety defect.)
Why should we believe blame assignment done by a vested interest? There have been many incidents with severe spin placed on blame. This should not be a surprise when mishap reports are written by the legal department as a CYA exercise and not as an engineering report. (Remember the Cruise pedestrian dragging mishap in which Cruise attempted a public cover-up of the part of the mishap dragging sequence that seemed to be pretty clearly their fault?) We should not trust blame allocations made without transparent, qualified independent review, regardless of which company is making the claim, and there is no system in place to do that right now on a routine basis.
Insurance company blame assignment might have somewhat better consistency in vehicle-on-vehicle crashes, but brings other concerns. Robotaxi-on-pedestrian/bicyclist crashes might not get full benefit of the adversarial court system for determining blame. Who pays the lawyers to advocate for a pedestrian without relevant insurance coverage to assign blame to a robotaxi when there are only moderate injuries and little potential settlement money at stake? What about victims pulled into arbitration due to sharp business tactics by the robotaxi company? Without a full-blown court case (which might be contested and appealed for years), we don't know how any crash will turn out on blame. The robotaxi companies have a big war chest to pay lawyers, so we can expect results to be biased against blaming them.
Moreover, it can take years to get accurate blame-based data while court cases play out, opening an opportunity to claim "no blame" incorrectly in the short-term. It might even be months (or years) before a court case is brought based on a mishap, so there is always the risk of under-counting blame even if all pending court cases are considered. And if there is a settlement, that leaves the issue of blame open to speculating whether a settlement was made to reduce net financial risk or to avoid an at-fault jury verdict. Basing blame on tort liability outcomes means that a company can literally buy their way out of admitting fault.
Yet another issue with blame is that proportional blame can be tricky to assign, and is also routinely evaded by robotaxi company crash reports. As an example, consider a real robotaxi injury crash in which a robotaxi made the mistake of turning in front of an oncoming car that was speeding and got t-boned, with injuries to one person in each vehicle. The robotaxi said the other driver was "at most fault” and therefore this crash didn't count against them. That did not make it any less dangerous to be in the robotaxi that got hit when making an ill-advised left turn. Being able to blame the other driver does not turn a stupid driving decision into a safe one.
The final issue is that blame does not give credit for good defensive driving technique and avoiding crashes due to mistakes by others. A car that is never to blame but keeps getting hit is not as safe as one that is never to blame and also manages to avoid getting hit. This is the so-called responder role behavior, which Waymo has rightfully stated is important to safety.
In principle, there might be a neutral, objective, facts-based assignment of blame for every crash. But that mechanism does not exist, and would be difficult to create for both technical and business reasons.
While any of the above issues might not happen to be at play for any particular robotaxi design at some particular time, if we are going to create metrics they need to work for all participants in an industry. If they don’t, we can expect some industry players to cut corners on safety so long as the metrics look good.
For these reasons I have long held that blame should not be included in a harm/mile metric, such as a positive risk balance metric. If you only get to pick one metric, fatalities/mile is the one. At-fault-fatalities/mile has too much potential to miss counting fatalities that matter in ways that misrepresent safety.
Bookkeeping the crashes
If you use blame to bookkeep the crashes, you need to deal with the question of where the fatalities go when nobody is at fault. Did the deaths simply not happen? Somehow a scorecard of Win/Lose/Rained-out does not seem appropriate, especially if we later find out that the nobody’s-fault crashes increase dramatically with robotaxis in the mix. (We don’t know whether or not this will happen, but if we don’t track them we’ll never know how that turns out.)
On the other hand, if you use crashes without blame, some interesting math issues pop up. A robotaxi in a 3-vehicle crash with one fatality must necessarily accrue 0.33 fatalities regardless of blame. This might seem unfair if the robotaxi didn't "cause" the crash. But this also means if the robotaxi did cause the crash, it is still only 0.33 fatalities, which might also seem unfair. No matter how you work it, someone will be unhappy. But, as I discuss with an extended set of examples in my new book (see section 10.2.4, in which I work examples and talk about this topic at length), this is the only math that corresponds to the notion of fatalities/mile = total fleet fatalities divided by total fleet miles. Anything else ends up over-counting or under-counting harm.
By this math, at this point Waymo has accrued just shy of 0.5 fatalities: 0.33 in a three-vehicle crash including a motorcycle, and 0.14 from a seven-car crash, totaling 0.47 fatalities. Assuming random independent arrivals and a national average one fatal crash per 100 million miles (pre-pandemic rough historical fatality rates), we would expect a 63% chance of one fatal crash at 100 million miles for a human driver baseline. While a detailed analysis can get quite complex, in broad strokes their current fatality rate (0.47) tracks close enough to be considered roughly comparable to a human driver (0.63-0.7 depending on number fatalities in a crash) so far — but with far too little data to have high confidence in how the long-term fatality rate will turn out.
If you are an advocate of only-blame counting, you are implicitly bolstering the argument that an all-robotaxi fleet will mean no road deaths because they will never be at fault in a crash. And certainly this is a message being sent by the industry whether they say they are intentionally doing so or not. But in a world in which some crashes have no clear blame, avoiding blame is not avoiding crashes.
It might seem intuitive that if nobody is to blame there will be fewer (or perhaps no) crashes, but that will not automatically be the case. For the next decade and more, there will be a healthy mix of human drivers. Behavioral incompatibilities in a mixed human/robotaxi fleet might (or might not) increase crash rates depending on behaviors of different brands of robotaxis. And even if, decades from now, all vehicles are robotaxis, there might be emergent effects that cause fewer at-blame crashes without reducing total harm as machine learning gets clever at avoiding blame. One hopes avoiding blame will help, but it is not a guaranteed outcome.
¿Por Qué No Los Dos?
In another stream of work, I have been developing the idea that safety will never work as a single metric, and instead needs to be a multi-constraint satisfaction exercise. As an example, Robotaxis should not have a high rate of negligent behavioral crashes even if total harm goes down.
The way out of the debate on blame is to recognize that it is simply a different metric and define it in a useful way. The question is not whether blame or non-blame crashes are better. One metric clearly needs to be a non-blame crashes for human drivers as an epidemiological metric (fatalities per mile, or fatalities per population).
But we can also have an at-blame metric. The comparison between the two can be enlightening, ultimately answering questions such as whether adding robotaxis simply resulted in blame-shifting to human drivers or in fact net improved overall traffic safety.
However, for blame-based metrics to work, we need to first resolve the numerous threats to validity of such an approach mentioned above. A solid proposal to do this would be welcome, with some progress made at this point but with more work needing to be done by proponents of at-fault metrics. This includes addressing at least:
Objective methods for assigning blame with transparent review. What are the methods, what is the mechanism for delivering them, and how does it get paid for in a way that does not introduce inherent bias?
Dealing with apportioning blame in a way that results in a meaningful metric, which in many states will differ from tort negligence apportionment rules.
Accounting for mishaps which are, or might yet be, pending in the court system.
Accounting for biases in court-based, arbitration-based, and insurance-based blame allocation due to an imbalance of negotiating power in blame determinations.
Accounting for mishap settlement agreements which fail to disclose blame determinations and typically do not admit to any blame as a condition of payment.
Ensuring there is a metrics-based mechanism for incentivizing defensive driving and de-incentivizing blame shedding behavior.
Phil Koopman has been working on self-driving car safety for almost 30 years. For more on this and related topics as they applied to any embodied AI system, see his new book: Embodied AI Safety.
If you actually use SOTIF, you will count all crashes.
There is a key example of how not focusing on fault significantly improved safety. In multiple countries in Europe the notion of fault was changed such that in a motor vehicle versus a bicyclist crash the motorist would face full financial responsibility regardless of fault. No more arguing about “they were wearing dark clothing”, “reckless because no helmet”, or “they came out of nowhere”. Key thing is that the change resulted in significantly fewer crashes, a great success.