By Colin Barnden
What’s at stake?
Who ensures the public’s safety when automakers test experimental automated driving technology on public roads? The answer may actually be nobody.
A BMW iX “test vehicle” crashed near Römerstein, Germany, on Aug. 15, killing one and injuring nine, including a child.
The iX is BMW’s flagship electric SUV and the first model to be built on the automaker’s iNEXT platform. It is one of the most advanced road vehicles ever developed, features many state-of-the-art systems, and first entered production in July 2021.
As reported by Reuters, the test vehicle crashed head-on after it veered into oncoming traffic on a country road. Much of the reporting by the mainstream media has focused on whether the vehicle featured autonomous driving capabilities, and if so, whether that technology caused the crash.
Published images confirm the crashed vehicle did have an “autonomous driving” identifying label on it and also was marked as an “electric test vehicle.” Further reports provide corroboration, with additional images from the accident scene.
BMW confirmed to the AP that one of its test vehicles was involved in a collision near the southwestern German town of Reutlingen but denied that the vehicle was fully autonomous.
Perhaps the most revealing aspect of this incident was BMW’s response, with its press department rushing to assert the vehicle involved was neither operating autonomously nor had any self-driving features. The company said its test vehicle was “equipped with Level 2 driver assistance systems,” and that “the driver always remains responsible.”
I’m skeptical, and here’s why.
BMW announced a partnership with Mobileye six years ago with design intent to bring highly (SAE J3016 Level 4) and fully (SAE J3016 Level 5) automated driving technologies into series production by 2021. The July 2016 announcement further states:
The BMW iNEXT model will be the foundation for BMW Group’s autonomous driving strategy and set the basis for fleets of fully autonomous vehicles, not only on highways but also in urban environments for the purpose of automated ridesharing solutions.
The iX was the foundation for BMW’s Level 4 and Level 5 development strategy. The crashed vehicle was a test-level iX.
The incident looks to me like a public road test gone wrong, and I was unconvinced by BMW’s attempts to deflect media attention away from its car and toward the driver. So I went digging.
(Editor’s note: In the email exchange with the Ojo-Yoshida Report, BMW spokesperson confirmed the vehicle involved was BMW iX, but stressed that the vehicle model is “not a precursor to what has been announced as ‘BMW New Class’ starting in 2025.” He said, “The code ‘BMW iNext platform’ was used before the name BMW iX was launched.”)
Smoking gun?
I found a YouTube labeled “BMW iX: Driver Assistant Professional real-life test on highway and country roads.” Note that the strobing purple lights on the instrument cluster are illuminators for the driver monitoring system (DMS), which in the iX is believed to have been supplied by Seeing Machines.
Automotive DMS uses 940nm infrared light, which is invisible to the human eye but detected by the RGB-IR image sensor in modern digital cameras. The iX has pretty much the most advanced DMS in any production vehicle, perhaps second only to Ford in terms of its sophistication.
The video shows a mostly hands-free automated driving function available on both divided and undivided highways, including winding country roads. There is no obvious operational design domain limit, and the system appears to switch seamlessly between hands-on and hands-off functions.
This is by far the most advanced automated driving technology I have seen from an automaker in a series-production passenger vehicle. I sat and watched the video open-mouthed. I am extremely impressed.
But is it wise to enable a hands-free automated driving function on undivided, winding country roads? Is that really safe? Who gets to decide if that is safe? And how safe is safe enough?
The risks of hands-off driving on undivided roads evidently outweigh the benefits. For instance, what if the automated steering did the wrong thing at the worst time?
Recommended:
BMW iX Crash: Getting at the Cause
Let’s look at an example from the video.
The screen shot below is taken at 10:43. Note the driver’s hands are off the steering wheel and the vehicle is traveling at 100 kph.
The image shown below is from one second later.
Hands-off driving at 100 kph, on an undivided winding country road, in close proximity to on-coming heavy trucks, in dazzling sunlight. What could possibly go wrong?
Humans have a neuromuscular lag of about a half-second, so based on the real-world example presented here, the driver would have to react instantaneously to correct any erroneous automated steering input to avoid a head-on collision with that truck. But that’s essentially impossible with hands off the steering wheel.
From what I can determine, the video was shot by a motoring journalist. Based on road signs and place names, the location is somewhere in the vicinity of Munich — BMW’s worldwide headquarters. Using readings on the instrument cluster, the distance traveled to gather all the video footage was about 150km, and the clock shows times between 09:20 and 16:10.
In other words, this is probably a video of a test drive by an auto journalist.
I’m curious to know whether the video shows operation of a type-approved standard series-production BMW iX or whether this is a test-level vehicle, like the one that crashed, running some form of experimental automated driving software. The complete absence of design domain limits leading to the ability to operate hands-off on undivided country roads is especially relevant to the fatal crash.
The existence of this video raises critical questions about what constitutes safe testing of automated driving technology on public roads, what constitutes safe operation of that technology, and what safety procedures are established at BMW — and all automakers — to ensure the safety of other vehicles’ occupants and vulnerable road users who are in proximity to these test-level vehicles.
Recommended:
L3 Cars: Unsafe at Any Speed?
One dead, nine injured
Let’s assemble the pieces of the puzzle.
- Press reports confirm the fatal collision happened on the B28, an undivided country road near the town of Römerstein.
- Photographs confirm “autonomous driving” labels on the crashed iX test vehicle.
- The iX is the platform for BMW’s Level 4 and Level 5 development strategy.
- A video surfaces of an unrelated test drive demonstrating hands-off driving capability at 100kph on undivided country roads.
- BMW claimed the iX test vehicle lacked autonomous capability and declared “the driver always remains responsible.”
So, what might we be looking at here?
My best guess is that the BMW/Mobileye partnership succeeded in developing an extremely sophisticated test-level automated driving system, which BMW operated on German public roads using a fleet of iX-based test vehicles.
Thus far, no problem. That’s innovation; that’s development; that’s progress.
But the stated production design intent was for Level 4 and Level 5 operation, so best practice would require public road testing in accordance with SAE J3018 and with a suitable Safety Management System in place.
Argo AI, the Pittsburgh-based AV platform developer, has been independently audited for best testing practices, by TÜV SÜD. But has BMW? Has Mobileye? Has any automaker?
Is any European regulatory body or safety agency overseeing what automakers actually do on public roads — or how they test experimental technology?
The March 2018 fatal collision in Tempe, Ariz., between an Uber test-level vehicle and a pedestrian clearly demonstrated the risks of testing experimental automated driving technology on public roads. The National Transportation Safety Board’s exhaustive investigation into the crash cited a lack of regulatory oversight and Uber’s inadequate safety culture, among other issues.
The search for answers in BMW’s case must now focus on BMW’s safety culture. For example, how did a BMW “test vehicle” operating on public roads come to have an 18-month-old child onboard? Press reports indicate the child was related to the driver. Was this “test vehicle” being used for a family road trip?
(Editor’s note: BMW explained to the Ojo-Yoshida Report via email that the test vehicle involved in the crash was driven by a trained test driver, who went through a comprehensive internal training program. When testing models that are already available on the market in the kind of day-to-day driving customers do, BMW said, “test drivers are permitted to take additional people such as family members outside working hours. But test cars can only be driven by authorized individuals.”)
Complacency
Perhaps the BMW/Mobileye automated driving system really was so reliable that some simply believed they had “solved” self-driving. Complacency set in — until that crash on the B28 near Römerstein.
Did the system tragically do the wrong thing at the worst time and steer the iX test vehicle toward oncoming traffic on an undivided country road? Unlike in the hundreds of YouTube videos made by Tesla’s Full Self-Driving evangelists — and even with the support of one of the most sophisticated driver-monitoring systems available — did the human driver simply not have sufficient time to react?
As of this writing, details of the incident are scant. BMW fell back on L2 as its reminder that the driver is ultimately responsible.
BMW could be exploiting a L2 loophole as a way to avoid more stringent AV oversight, much as Tesla describes its vehicle as SAE Level 2, despite its design intent for Level 4 autonomy.
The question is why that loophole persists.
Other questions remain unanswered. We don’t know whether the vehicle was in hands-free driving mode at the time of the crash. And while the iX’s ADAS features include brake assist and adaptive cruise control, we don’t know if the test vehicle accelerated or braked on its own.
To these questions, BMW responded via email: “The police and public prosecutors are currently investigating. We are in close contact with the responsible authorities and support them where necessary. Please understand that we cannot comment on ongoing investigations.”
Bottom line:
The safest driver is an alert, engaged, and unimpaired human driver, with hands on the wheel, eyes on the road, and mind on the task of driving. Mitigating distraction, drowsiness, and impairment is the role of a DMS. Road safety policy needs to be focused there.
Colin Barnden is principal analyst at Semicast Research. He can reached at colin.barnden@semicast.net.
Copyright permission/reprint service of a full Ojo-Yoshida Report story is available for promotional use on your website, marketing materials and social media promotions. Please send us an email at talktous@ojoyoshidareport.com for details.