New from the Lab·The Compass — an open moral reasoning standard for AI, tested across frontier modelsExplore →
Production AI Institute · PSF v1.1 open standard
AI Right-To-KnowAI Data Use IndexCheck My AI ToolsPolicy Change WatchAgent ReadinessPublic BenchmarkContactGlobal standard · Worldwide
Incident Registry
CriticalAutomotive·2018·Uber

Uber Self-Driving Car Kills Pedestrian in Arizona

An Uber self-driving test vehicle struck and killed a pedestrian in Tempe, Arizona — the first fatal autonomous vehicle crash. Investigation found the vehicle's safety system had detected the pedestrian but classified her as a 'false positive' and disabled the emergency braking system. The backup safety driver was watching a TV show at the time.

D6 · Human OversightD5 · Deployment Safety

What happened

Elaine Herzberg was struck by an Uber self-driving Volvo SUV while crossing a road in Tempe, Arizona at night. NTSB investigation found the vehicle's radar and lidar had detected the pedestrian approximately 6 seconds before impact. However, the system's classification algorithm cycled through several object categories (vehicle, bicycle, other) before settling on 'other' — and the emergency braking system had been disabled by Uber to reduce 'erratic vehicle behaviour'. The backup human safety driver was streaming The Voice on her phone. Uber had also disabled Volvo's standard autonomous emergency braking.

PSF Analysis

How the Production Safety Framework maps to this failure

A catastrophic D6 failure compounded by a D5 violation. The human oversight role was nominal — the safety driver was present but not monitoring the road. Worse, Uber had deliberately disabled Volvo's factory emergency braking system as a way to reduce 'false positive' interventions, without replacing it with an equivalent validated control. This is precisely the D5 failure pattern: removing a safety control without validated replacement. The classification uncertainty (6 seconds of cycling through object categories) also indicates D4 observability was insufficient — there was no alert mechanism for classification instability.

Controls that would have prevented this

Specific PSF controls mapped to each failure point

1
D6 · Human Oversight
Require active engagement from safety drivers — not passive presence. Monitor driver attentiveness.
2
D5 · Deployment Safety
Never disable safety-critical subsystems (emergency braking) without a validated compensating control.
3
D6 · Human Oversight
Define explicit intervention criteria and train safety drivers on when and how to intervene.

Outcome

Elaine Herzberg died from injuries. Uber suspended its self-driving programme. Uber ATG was ultimately sold to Aurora. Arizona revoked Uber's permit to test autonomous vehicles. Safety driver Rafaela Vasquez was charged with negligent homicide in 2020.

autonomous-vehiclesafety-criticalhuman-oversightemergency-braking

Related incidents

High2024
Air Canada Chatbot Bereavement Fare
D1D5
High2025
Law Firm AI Hallucinated Fake Case Law in Court Filings
D2D6
High2026
Binnall Law Claude Console Phantom Citations in Federal Court
D2D5D6
NEXT STEP

Map this failure back to the standard

Use the PSF domains behind this incident to define review gates, remediation evidence, and safer production requirements.

Read the PSF →← All incidents