In the EX90, Volvo Cars is both introducing and standardizing a DUS, which combines a dual camera interior monitoring system with a capacitive steering wheel. Powered by Smart Eye’s advanced AI algorithms, which are capable of analyzing driver eye, face, head, and body movements, the system enables a deeper understanding on the state of all vehicle occupants, with an emphasis on the driver. Here, the system monitors the driver’s eye patterns for any signs that they are not entirely concentrated on the driving task.

The integration of Smart Eye’s system will allow the Volvo EX90 to warn the driver, and act when necessary, while remaining non-intrusive if the driver does not need assistance.

A core aim of the partnership is the development of a DMS that detects when a driver is distracted and alerts them accordingly. The system will also provide monitoring for the entire cabin interior. Here, the solution would detect other risks, such as a baby or small child being left behind in the vehicle.

The development of the system will see both companies pool their experiences in biometrics, artificial intelligence and radar technology within the framework of the joint venture. Likewise, they envisage object recognition as a future use case of its driver monitoring system – with users receiving a notification on their smartphone to alert them of any objects left in the car.

With industry-leading accuracy, Phasya’s AI-based software analyzes biometric signals such as eye tracking or heart rate to monitor cognitive states including drowsiness, cognitive load, and distraction. The company’s algorithms are necessary components in all Driver Monitoring Systems (DMS), to understand the driver states impacting safety, driving experience, and comfort. Phasya’s solutions can also be deployed within industries such as aviation, rail, and consumer electronics. The solution is built on more than a decade of cutting-edge research in collaboration with The University of Liège, Belgium.

Rationale for the acquisition
Phasya’s technology and long experience within development and validation of algorithms to assess cognitive states is highly complementary to Tobii´s expertise and eye tracking technology. This will enhance Tobii’s Driver Monitoring System, Tobii DMS. It also opens new opportunities for using cognitive state information in several markets, such as VR, AR, Scientific Research, and Gaming, providing substantial synergies. The company currently collaborates with numerous customers within automotive, technology, and aviation, as well as with leading industry associations providing key relationships for Tobii and Phasya to build on.

Phasya is in its early pre-revenue commercialization phase. By leveraging its resources and capabilities, Tobii will be able to accelerate the commercialization of Phasya’s offering. Tobii expects the business to scale rapidly, driven by integrations of the algorithms in Tobii´s existing solutions as well as in new applications, and with the mass market adoption for automotive DMS expected in 2024/25.

The solution is the first integrated video processing chain, which combines innovative features based on the OmniVision OV2312 RGB-IR sensor, supporting exceptional day and night performance.

Smart Eye’s AI-based eye, mouth and head tracking technology provides EuroNCAP performance linked with full cabin monitoring and driver monitoring, featuring distraction, drowsiness and incapacitated driver detection, combined with driver identification and spoof-proof processing. The cabin monitoring also includes occupancy detection for all seats, combined with out of position, seat belt status and forgotten baby detection.

The action detection allows an understanding of occupant actions like driver hands on steering wheel, interaction with mobile device, calling, drinking, and eating. These actions may have an impact on the interaction between the vehicle and the occupant and will be a pre-requisite for higher levels of vehicle autonomy.

M.Brain measures the driver’s condition on a real-time basis by detecting the brainwaves around the ears through earpiece sensors that are worn. The key is the software technology that analyzes and determines the data from the brainwaves. Hyundai Mobis is committed to R&D and has even adopted machine learning to interpret the brainwave signals.

M.Brain can also be interworked with a smartphone app and provide notification that the driver is losing attention. The accident prevention technology also provides alerts for different sensory organs, such as sight (LEDs around the driver’s seat), touch (vibrating seat), hearing (headrest speaker), etc.

Hyundai Mobis plans to apply various bio-healthcare technologies to public transportation with a view to contributing to public safety. M. Brain will test- apply in Gyeonggi-do’s public buses first. 

The global in-vehicle healthcare market has now taken its first step. Heartbeat measurements or eye tracking technologies are introduced. Meanwhile, brainwave-based technology shows infinite potential for development as it is capable of measuring massive amounts of data, which is why Hyundai Mobis’ M.Brain is considered an innovative technology.

Hyundai Mobis is showing progress in developing autonomous driving healthcare technology using biosignals. At the CES in 2018, the company presented DDREM (Departed Driver Rescue & Exit Maneuver), which works to prevent accidents that occur as a result of drowsy driving. Hyundai Mobis then succeeded in developing the eye tracking DSW (Driver State Warning) system in 2019, and ROA (Rear Occupant Alert) system to detect infants in the backseat using radar last year.

MINIEYE researches autonomous driving perception technology and planning and control technology. Its mass production solutions are applicable to scenarios that include both driving and parking. In the driving scenario, it focuses on a full-domain solution that integrates out-of-cabin and in-cabin sensing, including ADAS solutions for L2+ and below and functions such as DMS, OMS and in-cabin interaction. In the parking scenario, it offers APA and AVP.

It is reported that by relying on advanced R&D capabilities and rich mass production experience, MINIEYE is accelerating the large-scale implementation of autonomous driving products. At present, MINIEYE’s front-end mass production customers include domestic autonomous brand car companies such as Dongfeng, Liuzhou Automobile, Geely Commercial Vehicle, SAIC, BYD, Weichai, Jianghuai, Heavy Duty Automobile, Shaanxi Automobile, etc., and joint venture brand car companies such as Jiangxi Isuzu and Jiangling Ford. MINIEYE has delivered 230,000 ADAS products in the first half of 2021, up 245% year on year. It expects 500,000 devices to be delivered by the end of 2021.

The cameras aim to increase road safety and reduce the rate of accidents caused by driver drowsiness and disengagement.

Nearly a third of fatal crashes occur as a result of reckless driving. Detecting the early signs of drowsiness is important to ensuring that the driver is alert enough to operate the vehicle. When detecting risks, such as drowsiness or distraction, the DMS camera alerts both the driver and the vehicle’s integrated safety systems – preventing injury and helping drivers stay focused on the road ahead.

The American firm’s new cameras leverage advanced technologies to enhance road safety – including global shutters and infrared LEDs. LI-AR0144IVEC-GMSL2-055H is equipped with ON Semiconductor’s In-Vehicle Experience Camera Module (IVEC) – AR0144, while LI-OV2311-IR-GMSL2-050H is equipped with OmniVision’s 2MP CMOS image sensor – OV2311.

Implementing technologies from ON Semiconductor and OmniVision helps the cameras capture image data at a higher resolution. This improves several DMS capabilities – such as gesture recognition, eye tracking, head roll movement, machine vision, augmented reality, and more.

The companies are merging to create a transatlantic AI juggernaut that will lead and accelerate the growth and development of the rapidly evolving automotive Interior Sensing market, as well as the Media Analytics and Human Factors Research markets.

Smart Eye has over two decades of experience building AI-based eye tracking and delivering automotive-grade Driver Monitoring Systems (DMS), as proven by 84 production contracts with 12 of the 20 largest global OEMs. Spun out of MIT Media Lab in 2009, Emotion AI pioneer Affectiva has, in recent years, broadened its technology to in-cabin sensing, using machine learning and computer vision to gain a deep, human-centric understanding of what is happening in a vehicle.

Rather than continuing as competitors in the automotive market, the companies are joining forces. By combining their best-of-breed technologies and two highly skilled and complementary teams, Smart Eye and Affectiva will bring to market unparalleled, automotive-grade Interior Sensing AI, better and faster than the competition. The combined company’s solution will not only improve automotive safety, saving human lives around the world, it will also provide differentiated mobility experiences that enhance wellness, comfort and entertainment.

In addition to strengthening Smart Eye’s automotive offering, Affectiva brings significant incremental revenues and profit from its industry-leading Media Analytics business. Used by 70 percent of the world’s largest advertisers, Affectiva’s enterprise-grade technology analyzes consumers’ emotional reactions to brand content and experiences. This technology has synergies with Smart Eye’s Research Instruments solutions, which provide the world’s leading research organizations with high-fidelity eye tracking systems for human factors research.

The integration of Affectiva’s AI with Smart Eye’s eye tracking systems can provide human factors researchers and marketers with a more holistic and insightful view of people’s behaviors.

The acquisition expands Smart Eye’s global presence. Affectiva’s Boston office gives Smart Eye a footprint within the vibrant technology innovation ecosystem on the US East Coast, and its Cairo office provides access to highly skilled local talent. Smart Eye is headquartered in Gothenburg (Sweden), with offices in Detroit (US), Tokyo (Japan), Chongqing (China), and soon Germany.

TriEye’s sensing solution proves effective in challenging visibility conditions, such as high-glare from the sun, shade patterns from the surroundings, or approaching vehicle headlights – detection scenarios that are paramount for the safety and reliability of in-cabin monitoring. Beyond enhanced visibility, the companies will evaluate the unique added value of SWIR for remote material sensing in comparison to other sensor modalities. With TriEye’s technology, Continental Engineering Services will be able to offer its customers valuable sensing solutions such as enhanced user identification capabilities or precise seat belt detection.

InGaAs-based SWIR cameras have been around for decades, serving the science, aerospace, and defense industries, but have not yet been used for mass-market applications due to their high costs and large form factor. Based on advanced nanophotonics research, TriEye and its strategic manufacturing partner enable the fabrication of a cost-effective, CMOS-based HD SWIR sensor at scale, that is mass-produced and small in size.

Among the companies that are collaborating with TriEye is the leading sports car manufacturer Porsche, in addition to the global automotive supplier Denso. The evaluation of TriEye sensing technology by CES, Porsche, Denso, and additional TriEye customers, reinforces the rise of SWIR as a key element in advanced vision solutions. More and more applications are harnessing SWIR spectrum image data to solve complex challenges that become clear when looking beyond the visible.

Stay Aware for Everyone (SAFE) Act: The bill – the Stay Aware for Everyone (SAFE) Act – will tackle the threat of distracted driving, a problem that is only increasing with the proliferation of “driver assistance” technologies that can encourage complacency if misused on the road. The SAFE Act will specifically require the DOT to study how driver-monitoring systems can prevent driver distraction, driver disengagement, automation complacency, and the foreseeable misuse of advanced driver-assist systems.

The legislation also requires a rulemaking to mandate the installation of driver-monitoring systems based on the results of this study, which shall incorporate appropriate privacy and data security safeguards. A copy of the SAFE Act can be found HERE. Senator Amy Klobuchar (D-Minn.) is also a cosponsor of this legislation.

The Canadian mobility technology supplier is combining its expertise in cameras and mirrors to help automakers make roads safer and reduce the risk of distracted driving. The system, which has yet to be named, alerts drivers when they take their eyes off the road for an extended period of time. Integrated into the vehicle’s interior mirror, it utilizes camera-based technology to monitor the driver’s head, eye, and body movements to detect distracted behavior, drowsiness, and fatigue. Likewise, the system also accounts for normal driving actions, such as looking in the side-view mirrors.

When distracted behavior is detected, the driver is alerted through customizable audible or visual notifications. Integrating the camera into the interior mirror ensure the driver’s view is not obstructed, offering the best vantage point for additional occupant monitoring features. The system is scalable to incorporate additional features – including child presence detection, seat belt detection, and facial recognition to enable user preference memory settings. The National Highway Traffic Safety Administration (NHTSA) marks each April as Distracted Driving Month, reminding road users that distracted driving has become a leading cause of accidents and fatalities on roadways in the U.S. The NHTSA estimated that, in 2019, distracted driving killed 3,142 people – a 10% increase from 2018.

Given the 60% spike in global consumer demand for SUVs over the past five years, Vayyar’s production-ready in-car sensor is designed to cover up to three rows. It can also accommodate the rapidly growing number of cars with sunroofs, thanks to uniform rear ceiling installation. Monitoring every seat and footwell, plus the trunk area, the sensor represents a breakthrough safety solution, protecting the most vulnerable vehicle occupants.

Up to 200 children die each year from heatstroke after being left unattended in cars. These tragedies can occur in minutes, since on days as cool as 22°C, the inside of a car, especially one equipped with a sunroof, can quickly reach 47°C.

Vayyar’s full-cabin sensor is fully operational in all lighting and weather conditions and maintains privacy at all times, since no cameras are involved. The ultra-wide field of view and high resolution provided by each sensor’s 48 transceivers are unmatched in the industry, enabling the platform to support multiple advanced applications simultaneously, while replacing numerous single-function sensors.

Given SUV cabin size and the fact that parents of young children comprise the majority of SUV owners, three-row coverage is a critical aspect of in-car sensing. Alternative solutions require two to three single-function sensors to cover the entire SUV cabin, driving up the cost associated with providing larger vehicle safety.

Until now, vehicles with sunroofs have also presented unique challenges in terms of in-cabin sensor positioning, with no option to fit sensors in the ceiling centre. Now OEMs can install just one Vayyar sensor in the rear ceiling area, covering the entire interior. 

Vayyar offers OEMs and Tier 1 suppliers a flagship Child Presence Detection (CPD) and enhanced Seat Belt Reminder (SBR) combo, essential for meeting Euro NCAP’s more stringent 2023 and 2025 safety scoring protocols. The platform does so by detecting occupants, classifying children and adults, and detecting posture and position.

The robust, automotive-grade solution also addresses incoming Euro NCAP safety requirements. Vayyar’s AEC-Q100 qualified and ASIL-B compliant platform provides up to 10 in-cabin points for 2023 onwards.

Multifunctionality on a single-chip platform enables automakers to move past the one sensor per function legacy industry approach. Uneconomical and unsustainable, this has traditionally burdened vehicles with more wiring, hardware, software and integration efforts. Instead, one sensor supports many safety applications, significantly reducing complexity and costs.

A data-rich 4D Point Cloud Application Programming Interface (API) provided by Vayyar’s platform enables customers to independently deploy numerous additional safety features over its point cloud layer. These may include Occupant Status (OS), optimised airbag deployment, seat belt pre-tensioning, eCall, gesture recognition, out-of-position detection, intruder detection among others.

Over-the-air (OTA) software updates also allow automakers to add new features long after a vehicle is built, providing added value by reducing development risks and time to market.

Looking for a way to measure how aware drivers are of what’s happening on the road around them when they’re using partial automation, IIHS researchers dressed an enormous stuffed bear in a high-visibility jacket and strapped it to the back of a car. They then compared whether three different groups of drivers noticed the bizarre tailpiece when the vehicle overtook them as they motored down an interstate in Maryland.

That measure is useful because today’s partial automation systems still aren’t fully autonomous.

Under the classification system developed by SAE International, automation is classified from 0 (no automation) to 5 (fully self-driving). The highest level available in production vehicles today is Level 2. These systems continuously control acceleration, braking and steering to keep the vehicle traveling at a set speed in the center of its lane while maintaining a selected following distance from the vehicle ahead.

As a number of highly publicized crashes have demonstrated, however, Level 2 systems sometimes struggle to handle common road features and driving scenarios, so they require constant supervision by the driver. Research has also shown that it’s harder for drivers to remain focused on the road when they no longer have to make the constant steering and speed adjustments they’re accustomed to making when driving manually. Scientists need a robust way to study this problem in real-world driving conditions in order to understand whether the technology affects all drivers the same way.

To test the teddy bear concept, IIHS observed 31 drivers as they drove a 2019 Mercedes-Benz C300 for about an hour on a Maryland stretch of I-70. One set of drivers who regularly used similar Level 2 systems in their own vehicles drove with the Mercedes-Benz’s partially automated feature switched on, as did another group of drivers who had little or no experience with these types of systems. A third group that was also unfamiliar with Level 2 automation completed the trip in the same vehicle but left the automation switched off.

During the trip, the vehicle with the pink teddy bear overtook each driver three times and remained in view in front for approximately 30 seconds. Cameras inside the test vehicle recorded where the driver was looking as well as the view of the road through the windshield. After the trip, the drivers were asked whether they had seen anything odd about any vehicles they had encountered and, if they had, how many times they had seen it.

The researchers observed clear differences in how successfully the three groups of drivers took note of the hard-to-miss bear, suggesting that the method effectively provided an objective measure of situational awareness.

More than twice as many inexperienced Level 2 automation users driving with the system turned on failed to recall the bear at all compared with the other groups. Almost all of the frequent Level 2 automation users who had the vehicle’s system switched on noticed the bear. They were also more likely than the other groups to correctly recall the number of times it had appeared during the drive.

The researchers also analyzed the video data and found that drivers who better recalled the bear also tended to spend more time scanning the forward roadway as well as looking out of the driver and front passenger windows, where the bear emerged into view as the vehicle carrying it passed them. In contrast, drivers who missed the bear tended to spend more time looking straight ahead.

Though the sample was small, these results suggest that partially automated driving systems have the potential to improve drivers’ situational awareness after they have gained experience using the technology. Inexperienced drivers may have more difficulty keeping track of what’s going on around them when using the unfamiliar technology than when driving without it.

On the other hand, previous studies have shown that familiarity with automation can also have drawbacks. Users can become complacent as their trust in the technology grows. That makes them more likely to check their phones or fiddle with the navigation or audio system than they would do while driving manually — a problem that has already been implicated in several serious crashes.

The updated SoC combines sensor fusion on the real-time domain with up to ASIL C metrics and an architecture optimized for smart computer vision. It offers OEMs and Tier 1s a high-performance, low-power solution that supports the latest NCAP 2020 requirements as well as the roadmap to NCAP 2025 3 Stars at competitive system costs.

The updated R-Car V3H joins Renesas’ expanding lineup of SoCs within the open and flexible Renesas autonomy platform, which offers OEMs and Tier 1s complete scalability from entry-level NCAP applications up to highly automated driving systems. Customers can also combine the R-Car V3H SoC with Renesas’ high-performance low-power RH850 MCU, integrated power management ICs, and power transistor devices to access the key components required for more complex safety compliant systems.

The highly integrated SoC supports up to ASIL C metrics safety goals on the real-time domain, reducing the need for an external safety microcontroller (MCU) to manage sensor fusion and final decision actions. The R-Car V3H features a suite of proven IPs that support the perception stack, sensor fusion with radar and/or Lidar, and ISP with up to eight MP cameras support, allowing OEMs and Tier 1s to achieve quicker time to market with lower bill of materials costs.

Eyeris’ and TI‘s Collaboration Offers a Complete, Advanced In-Cabin Monitoring System

The TDA4 SoC processor enables efficient inference of the Eyeris deep neural network (DNN) portfolio from multiple RGB-IR cameras simultaneously. This complete in-cabin monitoring solution, integrated into the D3 Engineering DesignCore platform is designed for rugged, field-deployable and production-intent units. In addition, RGB-IR image sensors are quickly becoming an industry requirement because of their unique ability to accurately capture the in-cabin space under the widest range of lighting conditions.

Eyeris DNNs are carefully designed for functional safety standards, flexible camera placements, and efficient inference on low-power edge processors. Eyeris in-cabin sensing portfolio of AI algorithms includes driver monitoring system (DMS) and occupant monitoring system (OMS) features that adhere to global NCAP requirements.

TI’s TDA4VM processors include embedded analytics acceleration to enable efficient deep learning performance within a low-power envelope. Additionally, these processors are able to perform safety-critical operations up to an ASIL D compliant level while also performing non-safety critical operations on one chip and function at an automotive operating junction temperature of 125 degrees Celsius. The SoC has built-in ISP and vision processing accelerators that help achieve more efficient system performance and reduce development costs.

Automakers and tier-1 suppliers such as Toyota, Jaguar Land Rover, Honda, Hyundai Mobis, Karma Automotive, Panasonic Automotive, Bosch, Veoneer and Mitsubishi Motors have licensed Eyeris in-cabin sensing AI solutions. Eyeris expects its in-cabin sensing AI software portfolio to go into vehicle mass production in late 2022.

Seeing Machines estimates that its entry into OMS opens an incremental market opportunity, worth up to a total of A$1.5 billion through to 2030, with an estimated revenue opportunity for the Company exceeding A$350 million.

This expansion is consistent with Seeing Machines’s “safety first” focus and leverages important breakthroughs in achieving uncompromised head, eye, and face tracking for driver monitoring with a Wide Field of View (WFOV) camera system. It will extend the Company’s highly effective attention, distraction, impairment, identification, and other human state measures from the vehicle driver to vehicle passenger(s) concurrently.

The introduction of a WFOV interior OMS will continue to support even the most challenging Euro NCAP DMS and semi-automated driving requirements which are traditionally achieved today with a narrow view (driver exclusive) camera system, while at the same time extending safety, comfort, and convenience feature to passengers. OMS capabilities range from enabling more reliable and cost-effective passive safety systems to fully AI enabled interiors able to anticipate and care for the needs of both the driver and passenger(s).

Further, the approach, enabled by Seeing Machines’s Occula® Neural Processing Unit (NPU) technology, allows a DMS to be expanded into an WFOV Interior camera system offering an array of additional high value interior and occupant sensing features with little increase in camera, illumination, or embedded processing cost. This can be achieved with readily available and proven automotive image sensors and other components, greatly enhancing the value of the Company’s existing product offering which remains consistent with the previously announced “three pillars” delivery options. This includes:

1. A market-leading Xilinx FPGA based FOVIO Chip incorporating Occula® NPU acceleration for the lowest cost, fastest time to market, standalone DMS/OMS system processing solution, targeting smart camera integration into a range of suitable locations including instrument panels and rear-view mirrors
2. Accelerated and non-accelerated Software Libraries (e-DME) which support a wide array of processing platforms, and integration sites including Infotainment Systems and ADAS domain modules, including the previously announced Qualcomm SnapDragon solution.
3. Occula® NPU ASIC licensing to automotive system on Chip (SOC) suppliers desiring to embed a uniquely optimized acceleration solution for achieving uncompromised human and object detection and tracking performance for camera-based DMS and interior/occupant monitoring systems at the lowest possible cost of processing.

Demonstrations of Seeing Machines combined Driver and Interior/Occupant monitoring technology are presently available consistent with potential 2023 model introductions.

Accounting for an estimated 80% of all car accidents and 25% of accident-related fatalities, distracted driving is one of the greatest threats on the road today. The SAFE act will require the United States Department of Transportation to conduct research into the use of Driver Monitoring Solutions (DMS) to reduce accidents, driver disengagement, and driver complacency due to automation. The bill would also require the department to issue a final ruling to mandate the installation of driver monitoring systems in all new passenger vehicles within two years of the ruling.

Eyesight Technologies has created three main solutions related to driver and passenger safety and experience inside the vehicle. Driver Sense, the company’s driver monitoring system (DMS), utilizes AI and computer vision to detect signs of driver drowsiness or distraction, as well as monitoring for dangerous actions such as the improper wearing of a seatbelt, holding of a cellphone and smoking behind the wheel. Fleet Sense – is the company’s aftermarket solution, providing driver monitoring solutions for fleets and telematics system providers. Finally, Cabin Sense, the company’s occupancy monitoring system, monitors the car’s interior, including passengers and objects left behind, to create a safer and personalized in-car environment.

To see the full regulation please visit: https://www.markey.senate.gov/imo/media/doc/SAFE%20Act.pdf

The OVM9284 is built on OmniVision’s OmniPixel®3-GS global-shutter pixel architecture, which provides best-in-class quantum efficiency at the 940 nm wavelength for the highest quality driver images in near or total darkness. The integrated OmniVision image sensor has a 3 micron pixel and a 1/4″ optical format, along with 1280 x 800 resolution.

The choice to use Melexis’ sensor was made to overcome the limitations of infrared tracking technology. For instance, when building the system, Melexis developed algorithms using proprietary strategies based on data-driven machine-learning approaches. This allows the sensor to be more efficient, coming with a lower cost, lower power usage and a compact design, allowing it to be integrated into the rear-view mirror assembly. The use of machine-learning in Melexis’ ToF sensor allows it to collect data and, from that data, enhance the system’s tracking capabilities. This means that the more frequently the sensor is used, the more accurately it can track the movement of the driver’s head and eyes. This accuracy is achieved in conjunction with the system’s VGA camera that comes with a 640px x 480px array based on Sony’s Depthsense technology and a maximum full-resolution framerate of 135FPS, allowing up to 4 raw phases per frame.

Melexis’ use of machine-learning technology in conjunction with Eyeware’s advanced eye-tracking technology presents its system as an interesting new entry to the increasingly competitive market for driver monitoring systems. It will also be interesting to see how the Time-of-Flight sensor utilized in their system compares to conventional solutions, such as infrared.

You can find out more about driver monitoring systems here

As a pioneer in the field of eye-tracking, Eyeware uses 3D time-of-flight cameras to overcome the limitations of infrared-based tracking technology. The company has developed its algorithms using proprietary strategies that are based on data-driven machine learning approaches, making it applicable in systems using low power, cost-effective and compact sensors