Natural User Interface for Roombots

Roombots (RB) are self-reconfigurable modular robots designed to study robotic reconfiguration on a structured grid and adaptive locomotion off grid. One of the main goals of this platform is to create adaptive furniture inside living spaces such as homes or offices. To ease the control of RB modules in these environments, we propose a novel and more natural way of interaction with the RB modules on a RB grid, called the Natural Roombots User Interface. In our method, the user commands the RB modules using pointing gestures. The user's body is tracked using multiple Kinects. The user is also given real-time visual feedback of their physical actions and the state of the system via LED illumination electronics installed on both RB modules and the grid. We demonstrate how our interface can be used to efficiently control RB modules on simple point-to-point grid locomotion and conclude by discussing future extensions

Human motion analysis and simulation tools: a survey

Computational systems to identify objects represented in image sequences and tracking their motion in a fully automatic manner, enabling a detailed analysis of the involved motion and its simulation are extremely relevant in several fields of our society. In particular, the analysis and simulation of the human motion has a wide spectrum of relevant applications with a manifest social and economic impact. In fact, usage of human motion data is fundamental in a broad number of domains (e.g.: sports, rehabilitation, robotics, surveillance, gesture-based user interfaces, etc.). Consequently, many relevant engineering software applications have been developed with the purpose of analyzing and/or simulating the human motion. This chapter presents a detailed, broad and up to date survey on motion simulation and/or analysis software packages that have been developed either by the scientific community or commercial entities. Moreover, a main contribution of this chapter is an effective framework to classify and compare motion simulation and analysis tools

From teleoperation to the cognitive human-robot interface

Robots are slowly moving from factories to mines, construction sites, public places and homes. This new type of robot or robotized working machine – field and service robots (FSR) – should be capable of performing different kinds of tasks in unstructured changing environments, not only among humans but through continuous interaction with humans. The main requirements for an FSR are mobility, advanced perception capabilities, high "intelligence" and easy interaction with humans. Although mobility and perception capabilities are no longer bottlenecks, they can nevertheless still be greatly improved. The main bottlenecks are intelligence and the human - robot interface (HRI). Despite huge efforts in "artificial intelligence" research, the robots and computers are still very "stupid" and there are no major advancements on the horizon. This emphasizes the importance of the HRI. In the subtasks, where high-level cognition or intelligence is needed, the robot has to ask for help from the operator. In addition to task commands and supervision, the HRI has to provide the possibility of exchanging information about the task and environment through continuous dialogue and even methods for direct teleoperation. The thesis describes the development from teleoperation to service robot interfaces and analyses the usability aspects of both teleoperation/telepresence systems and robot interfaces based on high-level cognitive interaction. The analogue in the development of teleoperation interfaces and HRIs is also pointed out. The teleoperation and telepresence interfaces are studied on the basis of a set of experiments in which the different enhancement-level telepresence systems were tested in different tasks of a driving type. The study is concluded by comparing the usability aspects and the feeling of presence in a telepresence system. HRIs are studied with an experimental service robot WorkPartner. Different kinds of direct teleoperation, dialogue and spatial information interfaces are presented and tested. The concepts of cognitive interface and common presence are presented. Finally, the usability aspects of a human service robot interface are discussed and evaluated.reviewe

Evaluating Physical Rehabilitation with a Kinect Based Low Cost System

In this thesis a low cost low power embedded system is presented which enables the patients to evaluate the performance of their physical rehabilitation. The system uses the Asus Xtion as a motion sensor which is connected to a portable Beagleboard XM (an open source single board small computer). In order to enable the skeleton tracking functionality on the BeagleBoard XM, the Beckon SDK (provided by Omek Interactive) is used. The physical rehabilitation system uses the Beckon SDK to retrieve the joint positions from each frame and compute elbow and shoulder angles. Initially the instructor records the desired exercise and then the patient performs the same exercise. To identify the start and end of the exercise, an additional gesture has been introduced. Next, to remove redundant and noisy data from sequences, curve extraction and median filter algorithms are used. Then a comparison algorithm known as DTW (Dynamic Time Warping) is used to compare the angular sequences and compute a total score which represents a quantitative evaluation of the exercise. For visual feedback a QT interface is used. Instructions are provided on the screen to tell both the instructor and patient about what to do next. Real time angular data graphs as well as frames per second (FPS) are also displayed on screen for the user's benefit. Experiments were conducted to compare the system performance between low cost embedded systems and high end desktop machines. The skeleton tracking performance is also evaluated using a VICON motion capture system.Computer Scienc

The Effects of Visual Affordances and Feedback on a Gesture-based Interaction with Novice Users

This dissertation studies the roles and effects of visual affordances and feedback in a general-purpose gesture interface for novice users. Gesture interfaces are popularly viewed as intuitive and user-friendly modes of interacting with computers and robots, but they in fact introduce many challenges for users not already familiar with the system. Affordances and feedback – two fundamental building blocks of interface design – are perfectly suited to address the most important challenges and questions for novices using a gesture interface: what can they do? how do they do it? are they being understood? has anything gone wrong? Yet gesture interfaces rarely incorporate these features in a deliberate manner, and there are presently no well-adopted guidelines for designing affordances and feedback for gesture interaction, nor any clear understanding of their effects on such an interaction. A general-purpose gesture interaction system was developed based on a virtual touchscreen paradigm, and guided by a novel gesture interaction framework. This framework clarifies the relationship between gesture interfaces and the application interfaces they support, and it provides guidance for selecting and designing appropriate affordances and feedback. Using this gesture system, a 40-person (all novices) user study was conducted to evaluate the effects on interaction performance and user satisfaction of four categories of affordances and feedback. The experimental results demonstrated that affordances indicating how to do something in a gesture interaction are more important to interaction performance than affordances indicating what can be done, and also that system status is more important than feedback acknowledging user actions. However, the experiments also showed unexpectedly high interaction performance when affordances and feedback were omitted. The explanation for this result remains an open question, though several potential causes are analyzed, and a tentative interpretation is provided. The main contributions of this dissertation to the HRI and HCI research communities are 1) the design of a virtual touchscreen-based interface for general-purpose gesture interaction, to serve as a case study for identifying and designing affordances and feedback for gesture interfaces; 2) the method and surprising results of an evaluation of distinct affordance and feedback categories, in particular their effects on a gesture interaction with novice users; and 3) a set of guidelines and insights about the relationship between a user, a gesture interface, and a generic application interface, centered on a novel interaction framework that may be used to design and study other gesture systems. In addition to the intellectual contributions, this work is useful to the general public because it may influence how future assistive robots are designed to interact with people in various settings including search and rescue, healthcare and elderly care