Researchers use AI to determine drivers' health and mental state in self-driving cars

Researchers at Western University are partnering with Mitacs Globalink to advance technology in self-driving cars by using artificial intelligence (AI).

This summer, hundreds of international students are working with Mitacs, a non-profit organization funded by the Government of Canada and provincial governments to assist organizations in cutting-edge innovation.

Harshita Mangotra is a third-year undergraduate student from India. During her internship with Mitacs, she discovered how to understand a driver’s health and mental state by looking at their eye pupils through an inexpensive in-vehicle camera.

“We found out from various studies that some psychological factors of facial attributes help in understanding the mental state and we can calculate if it’s a high cognitive load or if he is highly exhausted, mentally exhausted, or not,” she said.

Mangotra explained that the technology monitors pupil size and facial changes which indicate if a driver is angry, under stress, has high blood pressure, or is too distracted to drive.

“The AI model then predicted the change and size of the pupil and the research from this AI model will be used as an input to another AI model, which will calculate the cognitive load,” said Mangotra.

Using AI, the camera is fed video frames from the driver, she added.

In the automotive industry, there are five levels of automated driving. Researchers at Western University are working with Mitacs intending to advance to Level 3, where drivers can legally take their eyes off the road, but this has yet to be approved on Canadian roads.

Cameras that currently use Level 2 only monitor a driver’s visual attention, which means they will be alerted to put their hands back on the wheel if their eyes are off the road for too long.

“I was thinking this is something that is needed in Canada,” said Professor Soodeh Nikan, who has a background working in the automotive industry.

“The more control the automated system has, the more freedom the driver is given,” Nikan said. “Yet, when the car encounters something unexpected, it will still alert the driver to intervene, and our goal is to make sure the driver is in the right state to resume that control when it does happen.”

Mangotra is working under Professor Nikan, who currently works with Western University’s Faculty of Electrical and Computer Engineering.

“My students are working on different aspects of this,” she said. “They're focusing on the facial attributes. Another student is focusing on body attributes for activities of the driver, what activities they are doing, and what makes the readiness of the driver harder,” said Nikan.

Mangotra’s new technology can be used with any self-driving car and is expected to be ready for simulated driver testing later this year, according to Mitacs.

“Mitacs is very proud to support students through our Globalink Research Internship program so that research like Harshita’s will ultimately help people in Ontario, in Canada, and across the globe,” said Mitacs CEO John Hepburn in a statement. “The Mitacs Globalink Research Internship promotes strategic global partnerships and helps participants gain research experience in Canada, advancing innovation and creating attractive opportunities for international students who often decide to further their education here.”