The brain behind this device is the 28-year-old Indian-born Pranav Mistry, a researcher at the Media Lab, Massachusetts Institute of Technology.
The "Sixth Sense" device (patented by MIT) comprises a pocket projector, mirror and web camera bundled in a wearable pendant-like mobile. The projector can turn anything into a touch screen. The webcam (and colour-coded finger-gloves worn on the index finger and thumb) can recognise the movements of a user's hands, which enables gesture-commands.
A "square frame" gesture, for instance, will prompt the device to click a photo. The device can also recognise a book the user selects from a bookstore -- either by image recognition or radio frequency identification tags -- and project information, like an Amazon rating, onto it.
The system can also project a keyboard to type on, detect items on grocery shelves and compare online prices. A newspaper can prompt the device to search for news video clips (the device's smartphone uses an internet connection to retrieve information).
The user can also stop by any surface or wall and flick through the photos she has taken. The device allows a user to draw icons or symbols in the air using the index finger and it recognises those symbols as interaction instructions. For instance, drawing an "@" symbol will let a user check his/her mail.
"The possibilities are immense but it's a work in progress," admitted Mistry in a telephonic chat from the US. The device costs around $350 (around Rs 17,500) to build.
Mistry admits that his concept broadly comes under the ambit of "wearable computing" -- the cellphone falls in the same category, he points out. But he explained that information is confined traditionally on paper or digitally on a screen -- they do not communicate. The Sixth Sense device bridges this gap, "bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures".
You can even, says Mistry, use this contraption to teach games like football or ping pong (one of Mistry's hobbies, besides doodling, trekking, mountaineering, listening to ghazals and even watching Tom & Jerry). "Your imagination is the only limitation," he added.
Mistry gets around 500 emails everyday about his concept, of which, "around 400 emails are from Indians".
"Many CEOs of small Indian companies, including some pharma companies from Hyderabad, have evinced interest in my project," he said, but declined to name them. "I don't understand how companies work, but I would want to make the prototype cheaper for India."
And India, to which he is "surely coming back", is the focus of his interests. Mistry was born in Gujarat, where he studied computer engineering and later did a Masters in Design at IIT Bombay. He has been researching at MIT to help him get closer to his childhood dream of melding the flexibility of the digital world with the physical one.
"We need to dream our own dreams," he said. He should know, having always dreamt about bringing meaningful computing to the masses. For instance, he believes India should leverage the power of technology, rather than giving every villager a computer which is difficult to operate.
He also thinks Indians don't need to learn how to use keyboards, mouse, etc. One of his earlier projects, "Sandesh," addresses the communication needs of such people. It suggests a new system using the public switched telephone network and simple interaction methods. Sandesh contains a message-receiving unit in villages and kiosks in cities with visual aids. It uses print or sound-based media to convey messages.
Another of his projects, called "Akshar", is basically an attempt to have a mechanism for inputting Indic scripts in digital devices like mobile phones, kiosks, interactive TVs or personal computers. "I use this to talk to my mother," says Mistry, explaining that data entry methods for Indian languages like Hindi, Gujarati and in general are not keyboard-friendly and their entry using QWERTY keyboards is complicated.