Analysis of natural gestures for controlling robot teams on multi-touch tabletop surfaces

Abstract

Multi-touch technologies hold much promise for the com-mand and control of mobile robot teams. To improve the ease of learning and usability of these interfaces, we con-ducted an experiment to determine the gestures that people would naturally use, rather than the gestures they would be instructed to use in a pre-designed system. A set of 26 tasks with differing control needs were presented sequentially on a DiamondTouch to 31 participants. We found that the task of controlling robots exposed unique gesture sets and consider-ations not previously observed, particularly in desktop-like applications. In this paper, we present the details of these findings, a taxonomy of the gesture set, and guidelines for designing gesture sets for robot control. Author Keywords Human-robot interaction, human-computer interaction, robot control, multi-touch, tabletop interface, gesture