By Colin Barnden
What’s at stake?
Magna and Seeing Machines have signed an agreement to supply driver and occupant monitoring systems that are fully integrated inside the rearview mirror. The exclusive arrangement has profound implications for the accelerated adoption of this critical safety technology to meet a flood of impending regulatory requirements.
Automakers are simply out of time. Perhaps they’ve been sidetracked by the race to solve “full self-driving,” or maybe it’s complacency or possibly they’re just asleep at the wheel. Whatever the cause, many automotive OEMs have failed utterly to keep up with the changing regulatory outlook for distraction, drowsiness and impairment monitoring.
It’s not that they didn’t care about the regulations that are coming into effect. A more likely explanation is that automakers – and their existing network of Tier One suppliers – simply lacked the technical expertise to integrate complex infrared optical components, image processors and CMOS image sensors safely inside the cabin of a vehicle.
Let’s briefly review the relevant regulatory changes bearing down on the industry:
- Euro NCAP’s Safety Assist Assessment Protocol for Safe Driving (version 10.1) comes into effect in January 2023. BMW is the only automaker anywhere close to being ready for the new testing protocols.
- Revisions to the European Commission’s General Safety Regulation require new vehicles to provide driver drowsiness and attention warning (DDAW) systems. For vehicles going through type approval, DDAW is a requirement starting mid-2024. From mid-2026, no vehicle with four or more wheels (including cars, vans, buses, and light- and heavy-duty trucks) can be legally sold in Europe without a DDAW system.
- Changes to U.N. Regulation 157 for Automated Lane Keeping Systems (ALKS) come into force in January 2023. The revised ALKS protocol permits operation up to 130 kph on divided highways (increased from 60 kph in the first iteration) but specifically requires monitoring of the driver’s attention state.
- The Standardization Administration of China has mandated the monitoring of driver attention and driver fatigue, in addition to detecting phone handling, smoking, eyes closed and abnormal head pose. The mandate applies to buses, cars, and light and heavy trucks.
In addition, the Insurance Institute for Highway Safety recently reported the findings of a survey which showed that, despite warnings, many people in the United States treat partially automated vehicles as self-driving.
Recommended: Can Seeing Machines Become the ‘Mobileye’ of In-Cabin Vision?
After years of “autonowashing,” or the gap between the autonomous-vehicle hoopla and the reality of available systems, and misleading marketing by various automakers, consumer confusion about the real-world capabilities of partial automation systems was inevitable – and driver monitoring is the only plausible solution.
Is the auto industry facing regulatory disaster? No. The Magna-Seeing Machines partnership solves the problems of integrating driver and occupant monitoring systems to meet regulatory requirements in one fell swoop.
As we’ve reported, mounting complex optical components into the rearview mirror requires overcoming various problems, such as heat dissipation, vibration and camera orientation.
However, Magna and Seeing Machines are the first to solve all of these challenges, and the resulting mirror solution is a “get-out-of-jail-free” card for every automaker that has been slow to react to the changing regulatory environment for distraction, drowsiness and impairment monitoring.
Which, frankly speaking, is just about all of them.
The mirror takes it all
Patent WO 2022/187805-A1, entitled “Interior Rearview Mirror Assembly With Driver Monitoring System,” was granted to Magna on Sept. 9, 2022. The image below is taken from the patent and shows the typical camera location for monitoring systems.
Let’s briefly review each location in turn, to help understand why packaging the optical components, image processor and image sensor inside the rearview mirror is such an important achievement.
- (1) A-Pillar: This location is unsuitable for occupant monitoring and interior (whole-cabin) sensing. Additionally, the view of the driver is distorted by siting the image sensor here.
- (2) Rearview Mirror: Its many advantages are spelled out below.
- Overhead Console (OHC): The image sensor has a fixed orientation that is ideal for interior sensing but is unsuitable for monitoring the driver.
- (4) Steering Column: This is the location of choice for mounting the image sensor in systems such as GM Super Cruise and Ford BlueCruise. However, this location is unsuitable for both occupant monitoring and interior sensing, while the sensor’s view of the driver is intermittently occluded as the steering wheel rotates. Further, the steering wheel rake (vertical orientation) may position the image sensor to exclude any view of the driver’s face and eyes at all.
- (5) Center Stack.
- (6) Center Stack Low: These spots are suitable for driver and occupant monitoring, but not interior sensing. Using them requires a redesign of the entire center console on a model-by-model basis to successfully integrate the infrared components, image sensor and image processor.
Also common, but not shown, is siting the image sensor within the instrument cluster. This solution is suitable for driver monitoring, but unsuitable for occupant monitoring and interior sensing.
The benefits of the mirror location (2) include its suitability for driver/occupant monitoring as well as interior sensing. Drivers themselves orient the image sensor toward the driver’s eyes when lining up the view from the mirror glass to the rear window. The infrared optical components, image sensor and image processor are all integrated inside the mirror package, and the mirror location typically already has power and CAN bus connections for the ADAS forward camera.
As the market’s first complete solution, the momentum now firmly rests with Magna and Seeing Machines to meet automakers’ requirements to comply with regulations for driver and occupant monitoring.
According to Magna, Fisker, Honda and Volkswagen were the first three automakers to design-in the mirror solution. On automotive timescales, the industry is now working to incredibly tight deadlines, so other automakers are likely to make many design decisions in the next six months.
It is hard to think of a single compelling reason not to install monitoring into the mirror, and in the short term this location could account for more than three-quarters of units installed annually. Although there remain some technological challenges still to be overcome, in time the trend for driver and occupant monitoring looks set to be (and with apologies to Abba) “The mirror takes it all.”
Immersive user experience
The image below is also taken from the patent for the Magna mirror, showing the full circuit diagram.
Magna has worked with OmniVision to integrate the 0X05B 5-megapixel RGB-IR image sensor into its design. It is also working with Osram to integrate 940nm infrared LEDs and with Texas Instruments to integrate the Sitara processor executing Seeing Machines software.
The mirror thus looks to have been designed to operate as a “zonal controller” processing video data locally and feeding driver and occupant signals and metadata to central compute processors from Qualcomm and Nvidia over a CAN-FD bus.
Signals and metadata are low-bandwidth and extremely high-value, and this topology would be vastly more efficient in terms of power consumption and low latency than transmitting gigabytes of raw video data to a domain controller.
The automotive industry suddenly looks to be in the early stages of a trend to create the digital cockpit of the future. This trend encompasses the confluence of several technologies, including eye gaze, gesture and voice, powerful AI and deep-learning accelerators, along with 5G car-to-cloud connectivity and augmented-reality heads-up displays.
Ultimately, I see this becoming a four-way race between Apple (with CarPlay and Siri), Amazon (with Alexa), Google (with Android Automotive OS and Assistant) and Cerence, to define what the competitors dub an “immersive user experience.”
Seeing Machines held an Investor Day in London earlier this month that included the debut of its driver and occupant monitoring software running from the mirror location. A screenshot of the demonstration is shown below.
The event can be viewed in full here, with details on driver and occupant monitoring software beginning at 35:50.
Bottom line:
History shows that Magna, in partnership with Mobileye, achieved about a 70 percent share of the global ADAS forward-camera market. Magna again has a first-mover advantage, this time working with Seeing Machines for driver and occupant monitoring integrated inside the mirror. If history were to repeat, this collaboration could rapidly establish a similar 70 percent share of the global in-cabin monitoring market, leaving all competitors trailing in its wake.
Colin Barnden is principal analyst at Semicast Research. He can reached at colin.barnden@semicast.net.
Copyright permission/reprint service of a full Ojo-Yoshida Report story is available for promotional use on your website, marketing materials and social media promotions. Please send us an email at talktous@ojoyoshidareport.com for details.