It’s almost the same with the Mac. Use the mouse or trackpad to move a pointer to a button and click. Again, something happens. Easy enough, right? What could be easier? How about simply moving your hand or fingers, or blinking, or smiling? Don’t laugh.
This kind of interface might be nearer than you think.
Microsoft Goes Visual Multitouch
Just the other day I read about a Microsoft research project which developed a wristwatch-like sensor device which could be used to remote control computing devices by hand.
It seems clever enough. The sensor on your hand would control device movements.
Instead of using an electronic glove to communicate to a device, hand motions would suffice sans the gloves.
This multitouch technology removes the hand from a touch surface and enables a user to control devices with hand movements.
I have a better idea. Every iMac, MacBook, iPhone, and iPad come with built-in iSight cameras that are always looking at you while you view the screen.
Let’s take multitouch technology and marry it to facial recognition and hand gestures (something of a mashup of hand, eye, face, and fingers) to control devices, play games, and communicate and control devices remotely.
Instead of wearing electronic gloves that communicate to a computer, or attaching a device to the wrist or fingertips, let the iSight camera become the interface.
A facial recognition app could be modified to view your face and look for specific facial changes. Blinks, twitches, smiles, frowns, etc.; all of which could be attached to specific commands.
Enter your hand into the camera’s view, and the app would interpret hand gestures for movement, pointing, and clicking, ala Minority Report.
Macs, iPhone, and iPad as well as PCs and Android OS devices already have facial recognition to unlock or use a device. How much more is required to assign facial and hand gestures to specific actions or controls.
This is an idea whose time has come. Siri is decent for voice recognition but has yet to be deeply integrated deeply into everyday actions and interaction. The term multitouch would need to change, though. Nothing is being touched.