Acting rehearsal in collaborative multimodal mixed reality environments

This paper presents the use of our multimodal mixed reality telecommunication system to support remote acting rehearsal. The rehearsals involved two actors, located in London and Barcelona, and a director in another location in London. This triadic audiovisual telecommunication was performed in a spatial and multimodal collaborative mixed reality environment based on the 'destination-visitor' paradigm, which we define and put into use. We detail our heterogeneous system architecture, which spans the three distributed and technologically asymmetric sites, and features a range of capture, display, and transmission technologies. The actors' and director's experience of rehearsing a scene via the system are then discussed, exploring successes and failures of this heterogeneous form of telecollaboration. Overall, the common spatial frame of reference presented by the system to all parties was highly conducive to theatrical acting and directing, allowing blocking, gross gesture, and unambiguous instruction to be issued. The relative inexpressivity of the actors' embodiments was identified as the central limitation of the telecommunication, meaning that moments relying on performing and reacting to consequential facial expression and subtle gesture were less successful

A survey on human performance capture and animation

With the rapid development of computing technology, three-dimensional (3D) human body models and their dynamic motions are widely used in the digital entertainment industry. Human perfor- mance mainly involves human body shapes and motions. Key research problems include how to capture and analyze static geometric appearance and dynamic movement of human bodies, and how to simulate human body motions with physical e�ects. In this survey, according to main research directions of human body performance capture and animation, we summarize recent advances in key research topics, namely human body surface reconstruction, motion capture and synthesis, as well as physics-based motion sim- ulation, and further discuss future research problems and directions. We hope this will be helpful for readers to have a comprehensive understanding of human performance capture and animatio

Exploring the effectiveness of an augmented reality dressing room

In this paper, we describe our experience with the design of an augmented reality dressing room in which 3D models of a dress are overlaid with a color image from a camera to provide the function of a sort of virtual mirror. In such a way, the customer can move around to understand if a dress suits and fits them well. The project is implemented in Unity 4 Pro in combination with the Microsoft Kinect 2 for the tracking process. Design issues and technical implementation as well as the prospects for further development of the techniques are discussed. To assess the validity of our proposal, we have conducted a user study using 47 participants with different levels of experience with video games and devices used to play them. The empirical method used is qualitative. To this end, we used questionnaire-based surveys. The obtained results suggest that our solution represents a viable means to simulate dressing rooms, and participants in the study found the interaction with our 3D models to be natural for understanding if a dress suits and fits them well. Overall, the participants found our application very useful from a practical point of view

Fall prevention intervention technologies: A conceptual framework and survey of the state of the art

In recent years, an ever increasing range of technology-based applications have been developed with the goal of assisting in the delivery of more effective and efficient fall prevention interventions. Whilst there have been a number of studies that have surveyed technologies for a particular sub-domain of fall prevention, there is no existing research which surveys the full spectrum of falls prevention interventions and characterises the range of technologies that have augmented this landscape. This study presents a conceptual framework and survey of the state of the art of technology-based fall prevention systems which is derived from a systematic template analysis of studies presented in contemporary research literature. The framework proposes four broad categories of fall prevention intervention system: Pre-fall prevention; Post-fall prevention; Fall injury prevention; Cross-fall prevention. Other categories include, Application type, Technology deployment platform, Information sources, Deployment environment, User interface type, and Collaborative function. After presenting the conceptual framework, a detailed survey of the state of the art is presented as a function of the proposed framework. A number of research challenges emerge as a result of surveying the research literature, which include a need for: new systems that focus on overcoming extrinsic falls risk factors; systems that support the environmental risk assessment process; systems that enable patients and practitioners to develop more collaborative relationships and engage in shared decision making during falls risk assessment and prevention activities. In response to these challenges, recommendations and future research directions are proposed to overcome each respective challenge.The Royal Society, grant Ref: RG13082

Design and Development of the eBear: A Socially Assistive Robot for Elderly People with Depression

There has been tremendous progress in the field of robotics in the past decade and especially developing humanoid robots with social abilities that can assist human at a socio-emotional level. The objective of this thesis is to develop and study a perceptive and expressive animal-like robot equipped with artificial intelligence in assisting the elderly people with depression. We investigated how social robots can become companions of elderly individuals with depression and improve their mood and increase their happiness and well-being. The robotic platform built in this thesis is a bear-like robot called the eBear. The eBear can show facial expression and head gesture, can understand user\u27s emotion using audio-video sensory inputs and machine learning, can speak and show relatively accurate visual speech, and make dialog with users. the eBear can respond to their questions by querying the Internet, and even encourage them to physically be more active and even perform simple physical exercises. Besides building the robot, the eBear was used in running a pilot study in which seven elderly people with mild to severe depression interacted with the eBear for about 45 minutes three times a week over one month. The results of the study show that interacting with the eBear can increase happiness and mood of these human users as measured by Face Scale, and Geriatric Depression Scale (GDS) score systems. In addition, using Almere Model, it was concluded that the acceptance of the social agent increased over the study period. Videos of the users interaction with the eBear was analyzed and eye gaze, and facial expressions were manually annotated to better understand the behavior changes of users with the eBear. Results of these analyses as well as the exit surveys completed by the users at the end of the study demonstrate that a social robot such as the eBear can be an effective companion for the elderly people and can be a new approach for depression treatment