It’s called FingerIO, and by using an active sonar system, any surface around your phone could be used as an extension of the touchscreen, where gestures and traces drawn on them would register on the display. For example, tracing a word on the table top next to your phone, would see that word appear on the screen, even though you’re not directly touching it.
Although any physical surface could be used, the active sonar is strong enough that it could recognize mid-air gestures, and those made through fabric. In a demonstration video, a phone’s volume is raised and lowered using a mid-air gesture when the phone itself is inside a pocket. The same applies for a smartwatch, where gestures made on your arm, or in the air surrounding the device, activate features.
There’s no need for any sensors on your hand, or a special thimble-type finger extension, because FingerIO uses sensors already inside the phone to create the active sonar field. It’s rather simple — the speakers emit inaudible sounds, and echoes are picked up by the dual microphones. Clever modulation then increases the accuracy. The team used a Samsung Galaxy S4 as a test device, and found its audio and microphone setup is good for an average accuracy of 8mm.
Gestures are great, but when they can be made in the general vicinity of a device, there’s a risk of unwanted interaction. To avoid this, FingerIO has a double swipe activation command, made within five centimeters of the device, measuring at least four centimeters for each straight-line swipe. That way, pointing to something in front of you doesn’t suddenly make a call to Uncle Tom.
The group has published a paper on the technology, which points out several drawbacks, including devices requiring a third microphone to enable 3D tracking, and the high power consumption of an always-on sonar system. The test Galaxy S4 lasted for just four hours using the technology. However, it’s early days for FingerIO, and regardless of the current limitations, the active sonar system has been shown to work on an existing smartphone, and a custom-made smartwatch, rather than simply being a concept that may never actually work at all.
