GM Super Cruise instrument cluster (Image: GM)
What’s at stake:
Cars with features such as highway hands-free operation are designed to work, in principle, “collaboratively” with a human driver. The big caveat is that most carmakers know next to nothing about our real-world driving behavior. At issue is how human drivers and partially automated vehicles can collaborate when neither side knows jack about the other.
Consider the moment when a car disengages its automated features and asks the carbon-based life form behind the wheel to take over. Suddenly, the driver must take charge, regardless of whether he/she is – cognitively or physically – ready.
This is carmakers’ decidedly one-sided expectation, for which human drivers are ill-prepared.
“Collaborative driving” is built on a fallacy. When either a machine or a human driver has very little idea what the other side is doing or is poised to do next, two-way collaboration becomes next to impossible.
Real-world data
On questions of machine-to-human driving handovers, there is an ample body of solid academic work and technical papers. This line of research has raised such issues as human drivers’ exhibiting “mode confusion,” or getting “out of the loop.” A prominent concern is that humans tend to “over-trust” the technology in partially automated vehicles.
Missing from such work are real-world data on human behavior when driving a partially automated vehicle, such as:
- What is a typical immediate reaction to a vehicle’s takeover request?
- What would the driver do next?
- Will the driver’s visual attention change?
- What’s the first thing the driver looks for? And what next?
- How long before the driver achieves vehicle control?
- When will the driver reach steady-state manual driving?
This new AVT study provides a window on actual driver behavior associated with system-initiated transitions of control under real-world, naturalistic driving conditions.
Bryan Reimer
The Advanced Vehicle Technology (AVT) Consortium, an academic-industry partnership, recently published results of a real-world study, focused on human behavior while using the Super Cruise’ partial automation system.
This new AVT publication “provides a window on actual driver behavior associated with system-initiated transitions of control under real-world, naturalistic driving conditions,” explained Bryan Reimer, a research scientist in the MIT Center for Transportation and Logistics, in a recent interview with the Ojo-Yoshida Report.
Reimer believes the study provides “objective data for OEMs, suppliers, regulators and other stakeholders” to begin understanding how to develop a safer collaborative driving experience and its likely tradeoffs – “balancing convenience, comfort, and safety.”
The set-up
The AVT’s set-up was straightforward. Using data collection methodologies in a “naturalistic” environment, “fourteen drivers drove for one month each a Cadillac CT6 equipped with Super Cruise, a partial automation system that, when engaged, enables hands-free driving,” according to the AVT. The vehicles were equipped with data acquisition systems continuously recording data from: (i) Controller Area Network (CAN) bus to determine vehicle kinematics, driver interaction with the vehicle controllers, and the state of in-vehicle automation systems, (ii) Global Positioning System (GPS) to record location; (iii) four 720p video cameras that captured (30fps) the driver’s face, vehicle cabin, instrument cluster, and the view of the forward roadway, according the published paper.
The study was designed to “characterize the occurrences of automation-initiated disengagement requests and assess driver response and recovery behaviors.”
The study also quantified “changes in visual attention prior and immediately after automation disengagements,” and analyzed “moment-to-moment distributional properties of driver glance and hand-on-wheel behaviors.”
Lead author Pnina Gershon, a research scientist at MIT, explained that the study provides “insights on how drivers dynamically allocate their visual attention and the time it takes them to regain control following automation-initiated disengagement.”
What did they measure?
Most notable in the AVT study is that it annotated driver’s glance location frame by frame at 30 frames per second. Researchers classified glances in four categories: : (i) Road (glances directed to the windscreen); (ii) Instrument Cluster (instrument cluster/steering wheel location); (iii) Down & Center Stack (center stack, in-car multimedia, and the down or lap location); (iv) Mirrors (left, right windows or mirrors, and rearview mirror).
The study also annotated frame by frame the driver’s steering wheel control level, based on hand positioning. Categories used were: (i) No hands (driving hands-free); (ii) One hand; (iii) Two hands; and (iv) Not visible.
Findings
The most revealing human behavior the study uncovered is that at the crucial moment when Super Cruise notified the driver of automation disengagement and asked the driver to take over, the driver took his eyes off the road and started darting glances elsewhere inside the car, especially at the Instrument Cluster.
When Super Cruise notified the driver of automation disengagement, the driver took his eyes off the road and started looking for more information.
It is sobering to note that the data shows an “overall lower and more fragmented visual attention to the road” by drivers, especially when a potentially time-critical driving situation demands undivided focus.
The AVT data shows that following Super Cruise-initiated disengagement, the proportion of glances to the Road decreased (Q50Before=0.91, Q50After=0.69; Q85Before=1.0, Q85After=0.79), the proportions of glances to the Instrument Cluster increased (Q50Before=0.14, Q50After=0.25; Q85Before=0.34, Q85After=0.45), and mean glance duration to the Road decreased by 4.86 sec in Q85.
The study also found that the number of transitions between glance locations increased by 43 percent, following disengagement.
Such changes in drivers’ glance behavior are a serious matter. They show frequent transitions between a limited number of glance locations, while shortening drivers’ long on-road glances by 73%.
The authors of the AVT study believe this reaction reflects drivers’ desire to search for information that explains the disengagement and/or automation state. They call this “information seeking behavior.”
The effect of the takeover request does not end when drivers take control of the steering wheel.
Pnina Gershon
Gershon also pointed out another surprise. The study discovered that “the effect of the takeover request does not end when drivers take control of the steering wheel.” The shift in glance behavior persists for quite a while. “We continue to see the impact on driver visual attention for a couple of additional seconds,” she noted.
How many seconds do you need?
The study showed that when driving hands-free, takeover time was significantly longer (2.4 sec) compared to driving with at least one hand on the wheel (1.8 sec).
The AVT study dug deeper into the takeover period during hands-free driving by identifying “lag, response, and recovery time periods.”
As Gershon explained, “lag” refers to time from the onset of an automation disengagement request until there is a change in behavior (glance/hands-on-wheel). The “response” period is characterized by a change in behavior from the lag period followed by a recovery period. The “recovery period” is represented by stable behavior that does not change throughout the window of assessment.
The AVT analysis of moment-to-moment visual attention and steering wheel control during takeover showed that “on average it took drivers 6.1 seconds to start the recovery of glance behavior to the road and 1.5 seconds for … at least one hand on the steering wheel.”
How many seconds are needed for a human to take charge when the car asks for help is a focus of intense discussion among automotive experts. The most often-cited guess is at least three seconds.
We asked Gershon if the AVT study supports a three-second estimate.
She said, “This is a good and hard question and really relates to how safe is safe enough!”
She explained, “The adequate amount of time for a takeover is related to the time budget the driver has to prevent adverse outcomes, the driving context, the driver state, and the driver capabilities. Within our data, we find that when driving hands-free on the highway takeover took an average of 2.4 seconds.”
But she stressed: “I still see it as an open question that requires more data, and more context.” She noted that “there’s a lot more research coming from AVT that will help deepen our understanding.”
Recommended:
Podcast: How to Build a ‘Human-Centered Vehicle’
Takeaway
Given the magnitude of changes happening in understanding drivers’ “glance behavior,” system and HMI design becomes critical.
How can carmakers minimize the lag during which drivers adjust their gaze to respond to changes in the vehicle automation state?
The AVT study suggests “designs that provide easy to ingest and adequate information associated with automation disengagement” are critical to “smoother transitions and faster recovery.”
AVT, founded in 2015, today has more than 30 sponsors, including automakers, insurance companies, tier-1 suppliers, and research organizations. Its goal is to provide OEMs with high-fidelity data, enabling manufacturers to build “a human-centered [vehicle] system,” according to Reimer.
Reimer, a key architect of the AVT Consortium, suspects that some members “who have had access to this and other comparative data for some time are already making changes to their market advantages.”
What tradeoffs?
Designing partially automated vehicles is “all about balancing convenience, comfort, and safety,” noted Reimer.
How much convenience should carmakers provide drivers by deploying partial automation? How comfortable should the driver feel?
Does the driver trust the system for an alert that something might have gone wrong? Or will the driver feel assured that the system is there to protect him despite its limited automation capabilities?
How much safer will automation make driving?
Perhaps most important of all, Reimer pointed out, “How [will] all of these factors come together to support a level of calibrated trust in the collaborative driving experience.”
What, indeed, is “calibrated trust”?
Reimer believes each OEM is trying to strike a balance. The AVT hopes that carmakers can use its research to set benchmarks, identify areas where systems face difficult challenges to improvement and “ultimately derive a stronger safety benefit from assistive and automated driving,” Reimer concluded.
Bottom line:
Safety experts have long warned us about the dilemma of partially automated driving. But vehicles with hands-free highway driving are fast becoming a reality. This technology rush compels vehicle designers to anticipate how human drivers behave at safety-critical moments and develop a human-machine interface that can at least ease that handover
Junko Yoshida is the editor in chief of The Ojo-Yoshida Report. She can be reached at junko@ojoyoshidareport.com.
Copyright permission/reprint service of a full Ojo-Yoshida Report story is available for promotional use on your website, marketing materials and social media promotions. Please send us an email at talktous@ojoyoshidareport.com for details.