Artificial intelligence that allows self-driving cars to detect people and objects hidden around blind corners has been developed by researchers at MIT.
The imaging system—dubbed CornerCameras—was built by AI researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) for seeing around obstructions using standard camera technology.
Using information about light reflections, MIT's artificial intelligence system is able to measure the speed and trajectory of hidden objects in real time using footage from smartphone cameras.
"The technology has a range of applications, from firefighters finding people in burning buildings to self-driving cars detecting pedestrians in their blind spots," an MIT spokesperson tells Newsweek. "What's impressive is that this approach works using footage from a smartphone camera, such as an iPhone 8."
Current approaches for seeing around obstructions involve specialized lasers, but CornerCameras can detect objects from the small amount of light that they reflect onto the ground.
The fuzzy shadows, referred to as the penumbra, are stitched together using AI in order to reveal information about the objects.
"Even though those objects aren't actually visible to the camera, we can look at how their movements affect the penumbra to determine where they are and where they're going," said Ph.D. graduate Katherine Bouman, lead author on the paper detailing the system.
"In this way, we show that walls and other obstructions with edges can be exploited as naturally occurring 'cameras' that reveal the hidden scenes beyond them," she said.
Other recent AI advances to come out of CSAIL include a virtual reality system for robots that allows humans to control them remotely.
Bouman, who co-wrote the paper with MIT professors Fredo Durand, Bill Freeman, Antonio Torralba and Greg Wornell, will present the latest work at the International Conference on Computer Vision, in Venice later this month.
To the surprise of the research team, the CornerCameras AI system was tested in rainy weather and continued to work.
"Given that the rain was literally changing the color of the ground, I figured that there was no way we'd be able to see subtle differences in light on the order of a tenth of a percent," Bouman said. "But because the system integrates so much information across dozens of images, the effect of the raindrops averages out, and so you can see the movement of the objects even in the middle of all that activity."
Despite the success in poor weather, the system displayed limitations when used in low-light conditions. The researchers hope to overcome this through further research, while also adapting the system for use on moving objects.
The first step will be to test it on a wheelchair, with the eventual goal of testing it on cars and other vehicles.
"The notion to even try to achieve this is innovative in and of itself, but getting it to work in practice shows both creativity and adeptness," said professor Marc Christensen, dean of the Lyle School of Engineering at Southern Methodist University, who was not involved in the research.
"This work is a significant step in the broader attempt to develop revolutionary imaging capabilities that are not limited to line-of-sight observation," he said.
Uncommon Knowledge
Newsweek is committed to challenging conventional wisdom and finding connections in the search for common ground.
Newsweek is committed to challenging conventional wisdom and finding connections in the search for common ground.
About the writer
Anthony Cuthbertson is a staff writer at Newsweek, based in London.
Anthony's awards include Digital Writer of the Year (Online ... Read more
To read how Newsweek uses AI as a newsroom tool, Click here.
