A Survey of User Interfaces for Robot Teleoperation

Robots are used today to accomplish many tasks in society, be it in industry, at home, or as helping tools on tragic incidents. The human-robot systems currently developed span a broad variety of applications and are typically very different from one another. The interaction techniques designed for each system are also very different, although some effort has been directed in defining common properties and strategies for guiding human-robot interaction (HRI) development. This work aims to present the state-of-the-art in teleoperation interaction techniques between robots and their users. By presenting potentially useful design models and motivating discussions on topics to which the research community has been paying little attention lately, we also suggest solutions to some of the design and operational problems being faced in this area

An inertial motion capture framework for constructing body sensor networks

Motion capture is the process of measuring and subsequently reconstructing the movement of an animated object or being in virtual space. Virtual reconstructions of human motion play an important role in numerous application areas such as animation, medical science, ergonomics, etc. While optical motion capture systems are the industry standard, inertial body sensor networks are becoming viable alternatives due to portability, practicality and cost. This thesis presents an innovative inertial motion capture framework for constructing body sensor networks through software environments, smartphones and web technologies. The first component of the framework is a unique inertial motion capture software environment aimed at providing an improved experimentation environment, accompanied by programming scaffolding and a driver development kit, for users interested in studying or engineering body sensor networks. The software environment provides a bespoke 3D engine for kinematic motion visualisations and a set of tools for hardware integration. The software environment is used to develop the hardware behind a prototype motion capture suit focused on low-power consumption and hardware-centricity. Additional inertial measurement units, which are available commercially, are also integrated to demonstrate the functionality the software environment while providing the framework with additional sources for motion data. The smartphone is the most ubiquitous computing technology and its worldwide uptake has prompted many advances in wearable inertial sensing technologies. Smartphones contain gyroscopes, accelerometers and magnetometers, a combination of sensors that is commonly found in inertial measurement units. This thesis presents a mobile application that investigates whether the smartphone is capable of inertial motion capture by constructing a novel omnidirectional body sensor network. This thesis proposes a novel use for web technologies through the development of the Motion Cloud, a repository and gateway for inertial data. Web technologies have the potential to replace motion capture file formats with online repositories and to set a new standard for how motion data is stored. From a single inertial measurement unit to a more complex body sensor network, the proposed architecture is extendable and facilitates the integration of any inertial hardware configuration. The Motion Cloud’s data can be accessed through an application-programming interface or through a web portal that provides users with the functionality for visualising and exporting the motion data

The use of wearable inertial measurement units to assess gait and balance outcomes related to fall risk among older adults

Due to the prevalence and associated health, social and economic costs of falls among older adults, this thesis originally aimed to identify a more robust and objective way of assessing fall risk factors with the use of wearable inertial measurement units (IMU). However, due to unforeseen circumstances, the direction of the thesis had to be changed. Therefore, the thesis aimed to investigate whether gait and balance outcomes related to fall risk, when measured with wearable IMUs are sensitive to conditions which may replicate clinical and habitual environments. In Study one, a systematic scoping review was conducted to identify characteristic differences between fallers and non-fallers with the use of IMUs. The lower trunk was the most common anatomical location, whilst walking a predetermined distance indoors was the most common test used with IMUs to distinguish between fallers and non-fallers. In Study two, seventeen older and seventeen younger adults performed multiple walking and standing tasks in a laboratory. Older adults had a lower root mean square of the IMU acceleration signal, harmonic ratio and greater step time asymmetry compared to younger adults. The use of a cognitive dual task caused gait to be slower and less symmetrical among older and younger adults. Trunk displacement to quantify trunk sway during quiet standing was greater among older adults and increased as standing conditions became more difficult. Older adults exhibited distinct differences in gait when walking indoors and outdoors. The results of Study two suggested that IMUs may identify differences between older and younger adults regarding walking speed and time to completion of clinical tests, even when a stopwatch could not. In Study three, twenty older and twenty younger adults had IMUs attached to different anatomical locations during waking hours. There were differences in all gait variables when walking supervised in the laboratory and unsupervised in habitual indoor environments for both older and younger adults. There were also large differences in gait variables when walking indoors and outdoors. These results suggest the need for future studies in continuous, outdoor and unsupervised free-living conditions, with regards to fall risk assessments. This thesis demonstrates that gait and balance outcomes related to fall risk, when measured using wearable IMUs, are sensitive to conditions resembling habitual and clinical environments among both older and younger adults. This could prove valuable for the enhancement of future fall risk research

ST18 Autonomous enhancement strategies (detection of wall impacts and development of a self-cleaning laser scanner)

Fatalities and major accidents continue to occur in mines. While technology has continued to play a pivotal role in assisting mining companies to safely and successfully conduct their operations, a high level of innovation is still required to attain greater technological advancements. The industry needs competent and skilled personnel to design, implement and rectify processes and systems within the mining industry to make autonomous systems safe and reliable. Epiroc Australia (Epiroc) is in manufacturing front-end loaders that are highly optimised for underground mining environments to perform the load-haul-dump (LHD) function of the mining process. This family of machines is known as Scoop Trams (ST) and comes in various bucket capacities from seven metric tonnes up to 18. To complement the ST-family of LHD loaders, Epiroc has developed an autonomous system for the ST18 to enhance safety, workplace environment and productivity for clients in underground operations. The autonomous system's challenges in an underground mining operation system impact aspects that include the environment, network systems and dust and rock structure instability. The autonomous system utilises the Kalman filter for position localisation using the odometer wheel sensor, articulation angle sensor, Inertial Measurement Unit (IMU) and two LIDAR scanners. The Kalman filter runs recursively to continually update the loader's position by optimally estimating the system variables (speed, attitude, gyroscopic data, acceleration and the laser scan data). If any of the sensors fails, the automation system will detect this and stop the machine to prevent any damage from occurring. This research paper focused on the reliability issues caused by path tracking and localisation errors affected by sensor contamination or failure. The focus was on the laser scanner sensor and the IMU, two major components that stop the autonomous operation. The paper looked at current technologies on the market for cleaning contaminated laser scanners and how the IMU data is currently used for impact detection. After identifying the gap, a solution for laser contamination was developed to clean a dirty laser scanner to minimise stoppages. An application was developed to utilise IMU data from the loader to detect and minimise machine damage by detecting wall impact events. The paper discussed the methods used in the development, the testing and verification of these systems. Further research into a long term replacement for the laser scanner was conducted. In the future, the IMU data use was discussed on how the autonomous algorithm could be enhanced to improve continually without degrading the environment

Expressy : Using a Wrist-worn Inertial Measurement Unit to Add Expressiveness to Touch-based Interactions

Expressiveness, which we define as the extent to which rich and complex intent can be conveyed through action, is a vital aspect of many human interactions. For instance, paint on canvas is said to be an expressive medium, because it affords the artist the ability to convey multifaceted emotional intent through intricate manipulations of a brush. To date, touch devices have failed to offer users a level of expressiveness in their interactions that rivals that experienced by the painter and those completing other skilled physical tasks. We investigate how data about hand movement – provided by a motion sensor, similar to those found in many smart watches or fitness trackers – can be used to expand the expressiveness of touch interactions. We begin by introducing a conceptual model that formalizes a design space of possible expressive touch interactions. We then describe and evaluate Expressy, an approach that uses a wrist-worn inertial measurement unit to detect and classify qualities of touch interaction that extend beyond those offered by today’s typical sensing hardware. We conclude by describing a number of sample applications, which demonstrate the enhanced, expressive interaction capabilities made possible by Expressy

Exploring the Application of Wearable Movement Sensors in People with Knee Osteoarthritis

People with knee osteoarthritis have difficulty with functional activities, such as walking or get into/out of a chair. This thesis explored the clinical relevance of biomechanics and how wearable sensor technology may be used to assess how people move when their clinician is unable to directly observe them, such as at home or work. The findings of this thesis suggest that artificial intelligence can be used to process data from sensors to provide clinically important information about how people perform troublesome activities

Phrasing Bimanual Interaction for Visual Design

Architects and other visual thinkers create external representations of their ideas to support early-stage design. They compose visual imagery with sketching to form abstract diagrams as representations. When working with digital media, they apply various visual operations to transform representations, often engaging in complex sequences. This research investigates how to build interactive capabilities to support designers in putting together, that is phrasing, sequences of operations using both hands. In particular, we examine how phrasing interactions with pen and multi-touch input can support modal switching among different visual operations that in many commercial design tools require using menus and tool palettes—techniques originally designed for the mouse, not pen and touch. We develop an interactive bimanual pen+touch diagramming environment and study its use in landscape architecture design studio education. We observe interesting forms of interaction that emerge, and how our bimanual interaction techniques support visual design processes. Based on the needs of architects, we develop LayerFish, a new bimanual technique for layering overlapping content. We conduct a controlled experiment to evaluate its efficacy. We explore the use of wearables to identify which user, and distinguish what hand, is touching to support phrasing together direct-touch interactions on large displays. From design and development of the environment and both field and controlled studies, we derive a set methods, based upon human bimanual specialization theory, for phrasing modal operations through bimanual interactions without menus or tool palettes

Wearable and IoT technologies application for physical rehabilitation

This research consists in the development an IoT Physical Rehabilitation solution based on wearable devices, combining a set of smart gloves and smart headband for use in natural interactions with a set of VR therapeutic serious games developed on the Unity 3D gaming platform. The system permits to perform training sessions for hands and fingers motor rehabilitation. Data acquisition is performed by Arduino Nano Microcontroller computation platform with ADC connected to the analog measurement channels materialized by piezo-resistive force sensors and connected to an IMU module via I2C. Data communication is performed using the Bluetooth wireless communication protocol. The smart headband, designed to be used as a first- person-controller in game scenes, will be responsible for collecting the patient's head rotation value, this parameter will be used as the player's avatar head rotation value, approaching the user and the virtual environment in a semi-immersive way. The acquired data are stored and processed on a remote server, which will help the physiotherapist to evaluate the patients' performance around the different physical activities during a rehabilitation session, using a Mobile Application developed for the configuration of games and visualization of results. The use of serious games allows a patient with motor impairments to perform exercises in a highly interactive and non-intrusive way, based on different scenarios of Virtual Reality, contributing to increase the motivation during the rehabilitation process. The system allows to perform an unlimited number of training sessions, making possible to visualize historical values and compare the results of the different performed sessions, for objective evolution of rehabilitation outcome. Some metrics associated with upper limb exercises were also considered to characterize the patient’s movement during the session.Este trabalho de pesquisa consiste no desenvolvimento de uma solução de Reabilitação Física IoT baseada em dispositivos de vestuário, combinando um conjunto de luvas inteligentes e uma fita-de-cabeça inteligente para utilização em interações naturais com um conjunto de jogos terapêuticos sérios de Realidade Virtual desenvolvidos na plataforma de jogos Unity 3D. O sistema permite realizar sessões de treino para reabilitação motora de mãos e dedos. A aquisição de dados é realizada pela plataforma de computação Arduino utilizando um Microcontrolador Nano com ADC (Conversor Analógico-Digital) conectado aos canais de medição analógicos materializados por sensores de força piezo-resistivos e a um módulo IMU por I2C. A comunicação de dados é realizada usando o protocolo de comunicação sem fio Bluetooth. A fita-de-cabeça inteligente, projetada para ser usada como controlador de primeira pessoa nos cenários de jogo, será responsável por coletar o valor de rotação da cabeça do paciente, esse parâmetro será usado como valor de rotação da cabeça do avatar do jogador, aproximando o utilizador e o ambiente virtual de forma semi-imersiva. Os dados adquiridos são armazenados e processados num servidor remoto, o que ajudará o fisioterapeuta a avaliar o desempenho dos pacientes em diferentes atividades físicas durante uma sessão de reabilitação, utilizando uma Aplicação Móvel desenvolvido para configuração de jogos e visualização de resultados. A utilização de jogos sérios permite que um paciente com deficiências motoras realize exercícios de forma altamente interativa e não intrusiva, com base em diferentes cenários de Realidade Virtual, contribuindo para aumentar a motivação durante o processo de reabilitação. O sistema permite realizar um número ilimitado de sessões de treinamento, possibilitando visualizar valores históricos e comparar os resultados das diferentes sessões realizadas, para a evolução objetiva do resultado da reabilitação. Algumas métricas associadas aos exercícios dos membros superiores também foram consideradas para caracterizar o movimento do paciente durante a sessão