Discovering Natural Language Commands in Multimodal Interfaces

Explore a conference talk on improving user experiences with multimodal interfaces that support speech input. Learn about an innovative approach for generating and ranking natural language command examples to help users discover what to say and how to say it. Examine a prototype touch- and speech-based image editing tool that demonstrates this method. Investigate different UI augmentations for presenting command suggestions and their impact on promoting discovery and encouraging speech input usage. Gain insights from an online user study evaluating these alternatives and their effectiveness in addressing interpretation errors and user frustration in multimodal interfaces.