Seeing the Trees, the Forest, and Much More

Your portable phone can beat you at chess, but can it recognize a horse? Bristling with cameras, microphones, and other sensors, today’s machines are nevertheless essentially deaf and blind; they do not have senses to interact with their environment. In the meantime, vast amounts of valuable sensory data is captured, transmitted, and inexpensively stored every day. TV programs and movies, fMRI scans, planetary surveys, footage from security cameras, and digital photographs pile up and lie fallow on hard drives around the globe. It is all too much for humans to organize and access by hand. Someone has appropriately called this the “data deluge.” Automating the process of analyzing sensory data and transforming it into actionable information is one of the most useful and difficult challenges of modern engineering