Source: Seeing Machines
By Bolaji Ojo
What’s at stake:
Seeing Machines asks: “Can 3D-vision sensors compete with the maturity, simplicity and low cost of 2D sensors” for in-cabin sensing and monitoring? The solution developed by the Australian company in conjunction with STMicroelectronics and Sunex is an innovative automotive sensor that offers a cost-effective solution leveraging Airy3D’s DepthIQ optical decoder.
Seeing Machines this week unveiled the results of a product development collaboration with STMicroelectronics and Sunex that leverages innovations from Airy3D to meet automotive OEMs’ need for optimal and cost-efficient in-cabin sensing and monitoring solution.
The design rolled out by the company partly resolves the controversy over the use of 3D-vision sensors vs. 2D sensors inside vehicles. The dispute over the two offerings rests on the issues of the number of devices (cameras and other components) required for effective in-cabin monitoring, efficiencies, and – naturally – cost. Seeing Machines’ reference camera design merges all these frontiers by offering 3D-level sensing and results, but without the associated high-costs of competing 3D sensing technologies or the unavoidable need for multiple 2D sensing cameras to achieve the same objective, according to Timothy Edwards, co-founder and head of strategy at the company.
In addition, the reference design offers the benefits of 2D in-cabin software and precision eye-tracking while simultaneously delivering the capabilities of 3D sensing. To top it, the offering leverages a sole camera module, with a “single sensor and lens,” Seeing Machines said, in a statement announcing the technology.
The development of the new 3D camera technology took several years. Seeing Machines worked closely with Airy3D to modify the Canadian company’s DepthIQ technology for the in-cabin sensing and monitoring requirements of automotive OEMs. The two companies had to eliminate cost challenges associated with effective 2D sensing cameras while aiming for the advantages of 3D minus the additional components and other equipment required by existing solutions, according to Chris Barrett, CEO of Airy3D.
“We are very pleased with the steady progress of Airy3D to adapt their unique technology to meet the rigorous requirements of in-cabin systems, and we are excited to be able to offer this technology exclusively to our automotive partners,” adds Paul McGlone, CEO of Seeing Machines, in a statement. “Airy3D’s depth sensing technology offers Seeing Machines a smooth transition to a future where vision-sensors are trusted to become the primary source of in-cabin information.”
Above: Airy3D’s breakthrough approach offers the lowest cost of optical hardware for any 3D sensing approach, requiring only a single additional lithographic fabrication step applied to the CMOS image wafer. Cost is instead pushed into the computational domain, where it is optimized away with embedded AI techniques.
By leveraging Airy3D’s depth sensing innovation, Seeing Machines was able to eliminate a major challenge that 3D sensing and monitoring alternatives to its reference design face. Although 3D camera sensing is preferred to 2D sensors, the current technologies – stereo vision, radar, ultra-wideband (UWB) sensing, time-of-flight cameras, structured light and monocular depth estimation – have significant challenges, Edwards said.
“All known existing 3D sensing technologies have serious limitations that ultimately hinder their widespread adoption for in-cabin applications,” Edwards wrote, in a new Whitepaper titled 3D Sensing for In-Cabin Monitoring: Solving the In-Cabin Sensing Puzzle. “Seeing Machines has identified a technology that we feel can overcome these challenges and deliver robust, dependable, accurate, and cost-effective 3D perception within the unique constraints of the automobile cabin environment.”
The key word here is “perception.” The Seeing Machines solution does not jettison 2D-sensing completely. Instead, it augments it by using Airy3D’s DepthIQ optical decoder and software decoder, eliminating “the mechanical and optical cost drawbacks of the other 3D sensing methods.”
Depth perception
In developing its sensing offering, which is now being leveraged by Seeing Machines and its technology partners to address in-cabin monitoring cost challenges, Airy3D had to address several key challenges. The company had to make sure the solution could both optimally cover what the industry refers to as WFOV, or wide field-of-view, representing about 160 degrees of the cabin. Simultaneously, the solution to this problem must also yield the correct depth sensitivity of the vehicle, Seeing Machines said.
Having resolved this, the companies also had to ensure the TDM or transmissive diffraction mask – described by Airy3D in a product briefing document as “a few microns-thick optical layer added on top of image sensors to encode depth information in the light while preserving high-quality 2D images” – could operate at the infrared wavelengths of the interior monitoring systems, and, finally, ensure that the developed solutions would function under automotive lighting and temperature extremes.
Research conducted by Seeing Machines and its partners has “yielded a first prototype camera and also sharpened our understanding of the technical issues required to further fine-tune the technology,” according to Edwards.
This implies a final solution is still being developed. Indeed, the work on the in-cabin sensing solution continues. The addition of silicon partner STMicroelectronics and Sunex, a provider of imaging and projection optics solutions, is expected to accelerate the development work. Seeing Machines said it has been working for years with Airy3D to offer solutions to the automotive industry, which is now emphasizing opportunities from in-cabin sensing and monitoring.
Edwards’ Whitepaper identified the different existing in-cabin sensing technologies and the challenges facing the industry in deciding which of these to adopt. He emphasized that the industry is pushing for “more sophisticated and dependable sensing technologies within vehicle interiors.” But why has this become such a huge passion for the automotive industry? [Click here to access the Whitepaper.]
The answer lies in the desire for greater in-cabin security, monitoring and personalization for the automotive driver and other occupants. Accommodating these objectives at a cost-efficient rate is a major challenge for design engineers at a time researchers say the market for in-cabin sensing is set to explode over the next decade. IDTechEx, for example, is projecting the sector will increase strongly between 2020 and 2034, racking up a compounded annual growth rate (CAGR) of 18.9 percent, during the period.
In a recently updated report, the research firm noted that driver monitoring systems (DMS) will become mandatory in many regions, hence efforts by automakers to get ahead of any relevant legislation. While occupant monitoring systems (OMS) may not immediately become a regulatory requirement, it makes sense to co-develop it with DMS, according to Yulin Wang, a senior technology analyst with IDTechEx, in a report.
“DMS monitors driver conditions like drowsiness and distraction, while OMS focuses on tracking passengers, such as detecting unattended children or monitoring vital signs,” Wang said. “As of early 2025, the leading DMS technology is the combination of NIR and RGB cameras, as NIR light does not distract the driver. To reduce costs, many automotive OEMs are opting to combine NIR and RGB cameras. OMS, on the other hand, is generally non-mandatory, but many automotive manufacturers are integrating sensors such as radar.”
Cost driver
Seeing Machines’ Edwards similarly identified cost as a major impediment that OEMs are grappling with. If this can be further whittled down, the incorporation of sensing devices in automobiles will accelerate, he said.
OEMs are especially concerned that the current in-cabin sensing and monitoring solutions rely heavily on the addition of more cameras in sensor modules. In addition to the cameras, design engineers must also contend with the proliferation of other sensor chips, optical elements, metal, glass, other components, connectors and wires. The combination of these is driving up costs, observers said.
Edwards said Seeing Machines decided to start its search for a solution by returning to the question of the problem the industry is trying to solve, leaving the issue of existing solutions and their challenges till later. One objective that OEMs aim to achieve with in-cabin sensing and monitoring is the creation of the vehicle’s interior perception map, or IPM, which offers a set of data on “3D range of information, but also colors and textures of shapes,” Edwards noted.
The company then raised the question of whether achieving the objective of valid IPM perspective would be better served by determining the optimal set of devices required, that is, 2D cameras, RADARs, UWB sensors, software, and the incorporation of 3D-vision sensing. Seeing Machines’ conclusion is that the industry can extract lesser value from Radar and UWB, leaving optical systems (2D vision sensors and 3D vision sensors) “to deliver the long tail of concepts desired by OEMs, to differentiate their vehicle brands.”
In essence, the best solutions will come from 2D or 3D vision sensors, Edwards concluded. However, on the technology front, 3D vision sensors offer the best innovation advantages, while carrying one significant disadvantage: cost. The company said it believes “the cost issues associated with 3D sending will soon be overcome and that there is significant future potential for the right kind of 3D sensing to unlock levels of safety, comfort, and convenience that, due to a mixture of physics and software complexity, 2D systems will be unable to compete with.”
That’s why the company is pushing the reference camera design it has developed using Airy3D’s DepthIQ optical decoder and in partnership with ST and Sunex, Edwards added. “Airy3D’s breakthrough approach offers the lowest cost of optical hardware for any 3D sensing approach, requiring only a single additional lithographic fabrication step applied to the CMOS image wafer,” he said. “Cost is instead pushed into the computational domain, where it is optimized away with embedded AI techniques.”
To download and read the full Seeing Machines Whitepaper click here.
Bottom line:
The battle line is drawn on the technologies that will provide what OEMs need urgently for in-cabin sensing and monitoring: a cost-effective and innovative solution. There are many competing options, but 3D sensing appears to have an edge, except that it is also being hobbled by the same challenge of high deployment cost. Will Seeing Machines’ reference camera design – based on Airy3D’s DepthIQ optical decoder and developed in conjunction with ST and Sunex – be the breakthrough solution the industry is looking for?
Related articles:
Mitsubishi Deal Boosts Seeing Machines in SDV Market
Bolaji Ojo is publisher and managing editor of the Ojo-Yoshida Report. He can be reached at [email protected].
Copyright permission/reprint service of a full Ojo-Yoshida Report story is available for promotional use on your website, marketing materials and social media promotions. Please send us an email at [email protected] for details.