Future Phones to Read Your Voice, Gestures

Buttons are on their way out.

Five years from now, it is likely that the mobile phone you will be holding will be a smooth, sleek brick — a piece of metal and plastic with a few grooves in it and little more.

Like the iPhone, it will be mostly display; unlike the iPhone, it will respond to voice commands and gestures as well as touch.

"So much of how we understand technology is visually driven," says Rachel Hinman, a strategist with Adaptive Path, a user-experience and design-consulting firm. "Mobile interface design has to mimic the touch, sight, gesture and auditory feeds that we use to interact with our environment."

That means speaking to your phone rather than typing, pointing with your finger instead of clicking on buttons, and gesturing instead of touching. You could listen to music, access the internet, use the camera and shop for gadgets by just telling your phone what you want to do, by waving your fingers at it, or by aiming its camera at an object you're interested in buying.

Over the last few years, advances in display technology and processing power have turned smartphones into capable, if tiny, computers. As a result, phones have gone beyond traditional audio communication and texting to support a wide range of multimedia and office applications.

The one thing that hasn't changed, until recently, is the tiny keypad. Sure, there have been some tweaks, such as T9 predictive text input that cuts down on the time it takes to type, a QWERTY keyboard instead of a 12-key one, or the touchscreen version of a keyboard found on the iPhone. But fundamentally, the act of telling your phone what to do still involves a lot of thumb-twiddling.

Experts say the industry needs a new wave of interface technologies to transform how we relate to our phones. The traditional keypads and scroll wheels will give way to haptics, advanced speech recognition and motion sensors.

Touchscreens Everywhere

Until Apple's iPhone came along, keypads were a standard feature on all mobile phones. The iPhone paved the way for a range of touchscreen-based phones, including the T-Mobile G1 and the upcoming BlackBerry Storm.

"What's a shame is everyone's following Apple now," says Gus Desbarats, chairman of The Alloy, a UK-based product-design firm. To take the next step in phone design, mobile-phone companies will have to make radical changes in how they create their products. The Alloy's two touchscreens phone prototype

"Most user interface experiences today start around a new generation of microprocessor or new screen technology," says Desbarats. "Those things are important, but companies need to ask themselves: What kind of devices do we want people to really have?"

Take the touchscreen. So far, even iPhone clones require navigation across multiple screens to complete a task.

That will change as touchscreens become more sophisticated and cheaper. Instead of a single large screen that is fragile and smudged by fingerprints, phone designers could create products with multiple touch screens.

A prototype from The Alloy shows two touchscreens folded into a clamshell with the keypad and other functionality available on one of the screens and the ability to manipulate photos and other information on the other.

Speak to Me

Users could also interact with their phone by simply speaking to it using technology from companies such as Cambridge, Massachusetts-based Vlingo.

Vlingo's application allows users to command their phones by voice. That could enable you to speak the URLs for web pages or dictate e-mail messages.

Natural speech recognition has long been challenging for human-computer interface researchers. Most devices with speech-recognition capabilities require users to speak commands in an artificially clear, stilted way. They also tend to have high error rates, leading to user disenchantment.

Unlike conventional voice-recognition technologies, which require specific applications built to recognize selected language commands, Vlingo uses a more open-ended approach.

User voice commands are captured as audio files and transferred over the wireless connection to a server, where they're processed. The technology personalizes itself for each individual user, recognizing and training itself based on the individual user's speech patterns.

"If you say Boston and it shows up as Austin you can correct it on screen," says Vlingo CEO Dave Grannan. "And when you make the correction you are training the system."

It also means no need to memorize a list of commands or change the way you speak.

The technology has already found a major partner in Yahoo, which offers voice-enabled search on BlackBerry phones. Vlingo's completely voice-powered user interface is also available on Research In Motion phones, such as the BlackBerry Curve and Pearl.

Vlingo hopes to expand its services to additional platforms such as Symbian, Android and feature phones over the next few months.

Speedier Keypads

Even the traditional keypad is set to get a face lift. Typing on a touchscreen keypad is slow and difficult, even for those without stubby fingers or long nails.

That's where Swype comes in. It allows users to use trace a continuous motion on an onscreen QWERTY keypad instead of tapping individual characters

For instance, instead of typing the word infinity, users can just draw a line through each of the characters.Swype keypad

It's fast and remarkably error-free. "This is a game-changer," says Swype founder Cliff Kushler as he draws a line through the characters on the keypad to create the word infinity. "You have a subconscious awareness of where things are on the keyboard and people can ramp up on this to 40-50 words per minute."

When Kushler is done, the line on his touch screen keypad traces through all of the letters of the word infinity — a path that, coincidentally, resembles the infinity symbol.

The 55-year old Kushler is no texting addict, but he has changed the way we interact with our phones before.

About a decade ago, Kushler helped invent the T9 predictive-text input system, which lets people enter text on 12-button phone keypads more quickly than the old multitap system (press 2 once for A, twice for B, and so on).

Briefly a college drop-out, Kushler had set out to create an interface that would make it easier for disabled people to interact with their phones. But along the way, his research morphed into T9, which became a runaway success: It is now available on 3 billion phones.

He hopes Swype will have a similar impact on touchscreen phones. His instincts are certainly good: Researchers at IBM Almaden research center have developed a similar technology, called ShapeWriter, which is available for download from the iPhone App Store.

Swype CEO Mike McSherry says he is negotiating with handset makers to integrate the technology into phones.

Slowly Moving Forward

In a few years, mobile phones are also likely to come with embedded micro-projection displays that will allow the device to project a screen or a keyboard onto a table or any surface so users can navigate using the virtual interface, says Desbarats.

User experience will also improve, as phones integrate disparate features, such as GPS and camera, to provide a better interface. Nokia's "point and find" technology, for instance, is based on image-recognition technology. The feature allows users to point their phone's camera at objects and then uses image recognition to help identify them and bring up more information.

But don't rush to the stores yet. While the industry's direction seems clear, it will take a long time before these innovations hit the market.

The 18-month average lead time for a handset to make it to market increases the pressure on companies. Manufacturers shy away from really experimental interfaces, fearing that such changes will be off-putting to consumers. Instead, they make devices that push the envelope just a little bit at a time.

"It isn't about the technology but it is about the company's ability to use the technology in a compelling and profitable way," says Adaptive Path's Hinman.

In the short term, look for small, incremental improvements as mobile-phone manufacturers strive to make their handsets evermore usable by cleverly utilizing technology and capabilities that already exist.

"The thing about the future is it is built out of bits that are there in the present," says Desbarats. "In the future we will be looking at how to put those bits together in different ways to create innovative solutions."