From virtual demonstration to real-world manipulation using LSTM and MDN

Robots assisting the disabled or elderly must perform complex manipulation tasks and must adapt to the home environment and preferences of their user. Learning from demonstration is a promising choice, that would allow the non-technical user to teach the robot different tasks. However, collecting demonstrations in the home environment of a disabled user is time consuming, disruptive to the comfort of the user, and presents safety challenges. It would be desirable to perform the demonstrations in a virtual environment. In this paper we describe a solution to the challenging problem of behavior transfer from virtual demonstration to a physical robot. The virtual demonstrations are used to train a deep neural network based controller, which is using a Long Short Term Memory (LSTM) recurrent neural network to generate trajectories. The training process uses a Mixture Density Network (MDN) to calculate an error signal suitable for the multimodal nature of demonstrations. The controller learned in the virtual environment is transferred to a physical robot (a Rethink Robotics Baxter). An off-the-shelf vision component is used to substitute for geometric knowledge available in the simulation and an inverse kinematics module is used to allow the Baxter to enact the trajectory. Our experimental studies validate the three contributions of the paper: (1) the controller learned from virtual demonstrations can be used to successfully perform the manipulation tasks on a physical robot, (2) the LSTM+MDN architectural choice outperforms other choices, such as the use of feedforward networks and mean-squared error based training signals and (3) allowing imperfect demonstrations in the training set also allows the controller to learn how to correct its manipulation mistakes

Online Group-exercises for Older Adults of Different Physical Abilities

In this paper we describe the design and validation of a virtual fitness environment aiming at keeping older adults physically and socially active. We target particularly older adults who are socially more isolated, physically less active, and with less chances of training in a gym. The virtual fitness environment, namely Gymcentral, was designed to enable and motivate older adults to follow personalised exercises from home, with a (heterogeneous) group of remote friends and under the remote supervision of a Coach. We take the training activity as an opportunity to create social interactions, by complementing training features with social instruments. Finally, we report on the feasibility and effectiveness of the virtual environment, as well as its effects on the usage and social interactions, from an intervention study in Trento, Ital

Study of the Feasibility of a Virtual Environment for Home User Cybersecurity

This research focuses on the average home computer user’s ability to download, install and manage a virtual machine software program. The findings of this research is to be used as a foundation to the possibility of using a virtual machine software program as another form of defense for the home user’s computer. Virtual machines already have various uses, some in the cybersecurity field; this possibility could add another useful application for the software program. This research is conducted by monitoring volunteers’ ability to download, install, set up, and perform basic instructions on the virtual environment. It was from the volunteers’ experience that I hoped to gain an understanding on if more people could be able to manage a virtual environment. The findings point towards that it is possible for an average home computer user to be able to handle a virtual environment. There are steps that could help the user become more familiar with the virtual environment, but there is an openness demonstrated by the volunteers towards using a virtual machine software program as another layer of their computer cybersecurity

Virtual Reality-Based Home Visualization and Interaction

The disclosure describes virtual reality (VR) techniques to create a virtual home for visualizing an appearance and functionality of a home setup. Per the techniques, a home can be visualized in virtual reality before completion or at any later time. The techniques, which can be implemented in a smart home application, enable users to preview furniture, appliances, etc. as placed within their virtual home. The user can virtually place furniture in their homes, examine different furniture placement styles and make an informed decision before making a purchase. The techniques also enable testing smart home device automation behaviors in the virtual home. The use of virtual reality offers an immersive experience by allowing users to interact with a digital environment in three dimensions

A Framework for Interactive Teaching of Virtual Borders to Mobile Robots

The increasing number of robots in home environments leads to an emerging coexistence between humans and robots. Robots undertake common tasks and support the residents in their everyday life. People appreciate the presence of robots in their environment as long as they keep the control over them. One important aspect is the control of a robot's workspace. Therefore, we introduce virtual borders to precisely and flexibly define the workspace of mobile robots. First, we propose a novel framework that allows a person to interactively restrict a mobile robot's workspace. To show the validity of this framework, a concrete implementation based on visual markers is implemented. Afterwards, the mobile robot is capable of performing its tasks while respecting the new virtual borders. The approach is accurate, flexible and less time consuming than explicit robot programming. Hence, even non-experts are able to teach virtual borders to their robots which is especially interesting in domains like vacuuming or service robots in home environments.Comment: 7 pages, 6 figure

Healthcare PANs: Personal Area Networks for trauma care and home care

The first hour following the trauma is of crucial importance in trauma care. The sooner treatment begins, the better the ultimate outcome for the patient. Generally the initial treatment is handled by paramedical personnel arriving at the site of the accident with an ambulance. There is evidence to show that if the expertise of the on-site paramedic team can be supported by immediate and continuous access to and communication with the expert medical team at the hospital, patient outcomes can be improved. After care also influences the ultimate recovery of the patient. After-treatment follow up often occurs in-hospital in spite of the fact that care at home can offer more advantages and can accelerate recovery. Based on emerging and future wireless communication technologies, in a previous paper [1] we presented an initial vision of two future healthcare settings, supported by applications which we call Virtual Trauma Team and Virtual Homecare Team. The Virtual Trauma Team application involves high quality wireless multimedia communications between ambulance paramedics and the hospital facilitated by paramedic Body Area Networks (BANs) [2] and an ambulance-based Vehicle Area Network (VAN). The VAN supports bi-directional streaming audio and video communication between the ambulance and the hospital even when moving at speed. The clinical motivation for Virtual Trauma Team is to increase survival rates in trauma care. The Virtual Homecare Team application enables homecare coordinated by home nursing services and supported by the patient's PAN which consists of a patient BAN in combination with an ambient intelligent home environment. The homecare PAN provides intelligent monitoring and support functions and the possibility to ad hoc network to the visiting health professionals’ own BANs as well as high quality multimedia communication links to remote members of the virtual team. The motivation for Virtual Homecare Team is to improve quality of life and independence for patients by supporting care at home; the economic motivation is to replace expensive hospital-based care with homecare by virtual teams using wireless technology to support the patient and the carers. In this paper we develop the vision further and focus in particular on the concepts of personal and body area networks

Object Manipulation in Virtual Reality Under Increasing Levels of Translational Gain

Room-scale Virtual Reality (VR) has become an affordable consumer reality, with applications ranging from entertainment to productivity. However, the limited physical space available for room-scale VR in the typical home or office environment poses a significant problem. To solve this, physical spaces can be extended by amplifying the mapping of physical to virtual movement (translational gain). Although amplified movement has been used since the earliest days of VR, little is known about how it influences reach-based interactions with virtual objects, now a standard feature of consumer VR. Consequently, this paper explores the picking and placing of virtual objects in VR for the first time, with translational gains of between 1x (a one-to-one mapping of a 3.5m*3.5m virtual space to the same sized physical space) and 3x (10.5m*10.5m virtual mapped to 3.5m*3.5m physical). Results show that reaching accuracy is maintained for up to 2x gain, however going beyond this diminishes accuracy and increases simulator sickness and perceived workload. We suggest gain levels of 1.5x to 1.75x can be utilized without compromising the usability of a VR task, significantly expanding the bounds of interactive room-scale VR

Mixed reality participants in smart meeting rooms and smart home enviroments

Human–computer interaction requires modeling of the user. A user profile typically contains preferences, interests, characteristics, and interaction behavior. However, in its multimodal interaction with a smart environment the user displays characteristics that show how the user, not necessarily consciously, verbally and nonverbally provides the smart environment with useful input and feedback. Especially in ambient intelligence environments we encounter situations where the environment supports interaction between the environment, smart objects (e.g., mobile robots, smart furniture) and human participants in the environment. Therefore it is useful for the profile to contain a physical representation of the user obtained by multi-modal capturing techniques. We discuss the modeling and simulation of interacting participants in a virtual meeting room, we discuss how remote meeting participants can take part in meeting activities and they have some observations on translating research results to smart home environments

Fluid leadership in a multi-user virtual environment educational project with teenagers: Schome Park

This paper examines leadership practices in a virtual community, the Schome Park project. Schome Park, based at the Open University, UK, was the first European closed (i.e. protected) island in Teen Second Life, a multi-user 3D virtual environment. This fully realised, complex interactive 3D environment has no imposed narrative and offers significant engagement for educational projects. The Schome (‘not school not home’) third space community – i.e. not placed in the first space of home or second space of work/school (Oldenburg, 1989) - was set up with the explicit aim of challenging the instructional models and pedagogic practices of the formal, state educational system. In this disembodied environment identities, represented in the virtual world by personalised avatars, possess usefully ambivalent valences. Often adults will join ‘inworld’ educational events organised and delivered by the younger members of the community. Schome makes flexible use of a wiki (collaboratively designed website), asynchronous discussion fora and other communicative media to support learning processes and enhance the development of a physically distanced yet authentic learning community. The authors propose that the community design in these new spaces created an opportunity for leaders to emerge regardless of contextual hierarchy and to forge a developing culture. The paper makes use of evidence from varied datasets to examine manifestations of leadership in the community and issues arising. Young people have been engaged in proposing, planning, executing and reflecting on teaching and learning and governance without deference to adults. Our analysis contributes to understandings of the development of leadership within carefully designed educational online communities and some of the challenges involved for adults in facilitating an appropriately supportive environment for young people. While aware that this innovative experiment continues to face many challenges, we propose that the design of the project offers much to encourage an approach to education in which collaborative, situated engagement in learning and teaching is perceived as a more fruitful model for the twenty-first century than reproduction of traditional hierarchies of teachers and the taught of conventional classrooms