A Comparison of Visualisation Methods for Disambiguating Verbal Requests in Human-Robot Interaction

Picking up objects requested by a human user is a common task in human-robot interaction. When multiple objects match the user's verbal description, the robot needs to clarify which object the user is referring to before executing the action. Previous research has focused on perceiving user's multimodal behaviour to complement verbal commands or minimising the number of follow up questions to reduce task time. In this paper, we propose a system for reference disambiguation based on visualisation and compare three methods to disambiguate natural language instructions. In a controlled experiment with a YuMi robot, we investigated real-time augmentations of the workspace in three conditions -- mixed reality, augmented reality, and a monitor as the baseline -- using objective measures such as time and accuracy, and subjective measures like engagement, immersion, and display interference. Significant differences were found in accuracy and engagement between the conditions, but no differences were found in task time. Despite the higher error rates in the mixed reality condition, participants found that modality more engaging than the other two, but overall showed preference for the augmented reality condition over the monitor and mixed reality conditions

Probabilistic Global Scale Estimation for MonoSLAM Based on Generic Object Detection

This paper proposes a novel method to estimate the global scale of a 3D reconstructed model within a Kalman filtering-based monocular SLAM algorithm. Our Bayesian framework integrates height priors over the detected objects belonging to a set of broad predefined classes, based on recent advances in fast generic object detection. Each observation is produced on single frames, so that we do not need a data association process along video frames. This is because we associate the height priors with the image region sizes at image places where map features projections fall within the object detection regions. We present very promising results of this approach obtained on several experiments with different object classes.Comment: Int. Workshop on Visual Odometry, CVPR, (July 2017

Kinect Range Sensing: Structured-Light versus Time-of-Flight Kinect

Recently, the new Kinect One has been issued by Microsoft, providing the next generation of real-time range sensing devices based on the Time-of-Flight (ToF) principle. As the first Kinect version was using a structured light approach, one would expect various differences in the characteristics of the range data delivered by both devices. This paper presents a detailed and in-depth comparison between both devices. In order to conduct the comparison, we propose a framework of seven different experimental setups, which is a generic basis for evaluating range cameras such as Kinect. The experiments have been designed with the goal to capture individual effects of the Kinect devices as isolatedly as possible and in a way, that they can also be adopted, in order to apply them to any other range sensing device. The overall goal of this paper is to provide a solid insight into the pros and cons of either device. Thus, scientists that are interested in using Kinect range sensing cameras in their specific application scenario can directly assess the expected, specific benefits and potential problem of either device.Comment: 58 pages, 23 figures. Accepted for publication in Computer Vision and Image Understanding (CVIU

Optimization of the Single Point Active Alignment Method (SPAAM) with a Random Forest for accurate Visual Registration

The research addresses the visual calibration of head-mounted displays such as the HoloLens. The HoloLens is an optical see-through viewing device that allows a user to experience the real world populated with virtual objects. These virtual objects need to be correctly aligned with physical objects in the environment to experience a visually appropriate scene. However, several factors, such as an outside-in tracking system, tracking errors, the user\u27s eye position, and others degrade the alignment between the virtual and physical object. A popular calibration method to correct this misalignment is the so-called Single Point Active Alignment Method (SPAAM) [1]. It allows one to improve the alignment by measuring and correcting the alignment error. Nonetheless, one encounters alignment errors since, SPAAM assumes a constant error between the physical object, the display, and the user\u27s eye. Modern low-cost tracking systems such as based on RGB-D cameras (e.g., Kinect) come with dynamic errors. Consequently, SPAAM cannot yield the required accuracy; theoretically, dynamic errors require a dynamic calibration. The objective of this research is to study the improvement a dynamic error calibration can yield regarding alignment and registration accuracy. To improve the visual experience for a user, a random forest method will be adopted for this purpose. The hypothesis is that the random forest can dynamically select the best SPAAM calibration matrix with respect to the relative position of the user and a physical object. Experimental results demonstrate improvement by a factor of four; thus, indicate that random forest is an appropriate method to mitigate object misalignment due to dynamic tracking errors

Realization Of A Spatial Augmented Reality System - A Digital Whiteboard Using a Kinect Sensor and a PC Projector

Recent rapid development of cost-effective, accurate digital imaging sensors, high-speed computational hardware, and tractable design software has given rise to the growing field of augmented reality in the computer vision realm. The system design of a 'Digital Whiteboard' system is presented with the intention of realizing a practical, cost-effective and publicly available spatial augmented reality system. A Microsoft Kinect sensor and a PC projector coupled with a desktop computer form a type of spatial augmented reality system that creates a projection based graphical user interface that can turn any wall or planar surface into a 'Digital Whiteboard'. The system supports two kinds of user inputs consisting of depth and infra-red information. An infra-red collimated light source, like that of a laser pointer pen, serves as a stylus for user input. The user can point and shine the infra-red stylus on the selected planar region and the reflection of the infra-red light source is registered by the system using the infra-red camera of the Kinect. Using the geometric transformation between the Kinect and the projector, obtained with system calibration, the projector displays contours corresponding to the movement of the stylus on the 'Digital Whiteboard' region, according to a smooth curve fitting algorithm. The described projector-based spatial augmented reality system provides new unique possibilities for user interaction with digital content

SELF-IMAGE MULTIMEDIA TECHNOLOGIES FOR FEEDFORWARD OBSERVATIONAL LEARNING

This dissertation investigates the development and use of self-images in augmented reality systems for learning and learning-based activities. This work focuses on self- modeling, a particular form of learning, actively employed in various settings for therapy or teaching. In particular, this work aims to develop novel multimedia systems to support the display and rendering of augmented self-images. It aims to use interactivity (via games) as a means of obtaining imagery for use in creating augmented self-images. Two multimedia systems are developed, discussed and analyzed. The proposed systems are validated in terms of their technical innovation and their clinical efficacy in delivering behavioral interventions for young children on the autism spectrum

Projecting physical objects into a virtual space using the Kinect and Oculus Rift

Master's Project (M.S.) University of Alaska Fairbanks, 2015Virtualized Reality as a field of research has been increasing over the last couple of decades. Initially, it required large camera arrays, expensive equipment, and custom software to implement a virtualized reality system. With the release of the Kinect and the Oculus Rift development kits, however, the average person now has the potential to acquire the hardware and software needed to implement a virtualized reality system. This project explores the possibility of using the Kinect and Oculus Rift together to display geometry based on real-world objects in a virtual environment

Innovative Technologies for Medical Education

This chapter aims to assess the current practices of anatomy education technology and provides future directions for medical education. It begins by presenting a historical synopsis of the current paradigms for anatomy learning followed by listing their limitations. Then, it focuses on several innovative educational technologies, which have been introduced over the past years to enhance the learning. These include E-learning, mobile apps, and mixed reality. The chapter concludes by highlighting future directions and addressing the barriers to fully integrating the technologies in the medical curriculum. As new technologies continue to arise, this process-oriented understanding and outcome-based expectations of educational technology should be embraced. With this view, educational technology should be valued in terms of how well the technological process informs and facilitates learning, and the acquisition and maintenance of clinical expertise