Analysis of the interaction between elderly people and a simulated virtual coach

The EMPATHIC project develops and validates new interaction paradigms for personalized virtual coaches (VC) to promote healthy and independent aging. To this end, the work presented in this paper is aimed to analyze the interaction between the EMPATHIC-VC and the users. One of the goals of the project is to ensure an end-user driven design, involving senior users from the beginning and during each phase of the project. Thus, the paper focuses on some sessions where the seniors carried out interactions with a Wizard of Oz driven, simulated system. A coaching strategy based on the GROW model was used throughout these sessions so as to guide interactions and engage the elderly with the goals of the project. In this interaction framework, both the human and the system behavior were analyzed. The way the wizard implements the GROW coaching strategy is a key aspect of the system behavior during the interaction. The language used by the virtual agent as well as his or her physical aspect are also important cues that were analyzed. Regarding the user behavior, the vocal communication provides information about the speaker's emotional status, that is closely related to human behavior and which can be extracted from the speech and language analysis. In the same way, the analysis of the facial expression, gazes and gestures can provide information on the non verbal human communication even when the user is not talking. In addition, in order to engage senior users, their preferences and likes had to be considered. To this end, the effect of the VC on the users was gathered by means of direct questionnaires. These analyses have shown a positive and calm behavior of users when interacting with the simulated virtual coach as well as some difficulties of the system to develop the proposed coaching strategy.The research presented in this paper is conducted as part of the project EMPATHIC that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement no 769872

Layered evaluation of interactive adaptive systems : framework and formative methods

Peer reviewedPostprin

Emotional recognition in computing

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University 8/4/2010.Emotions are fundamental to human lives and decision-making. Understanding and expression of emotional feeling between people forms an intricate web. This complex interactional phenomena, is a hot topic for research, as new techniques such as brain imaging give us insights about how emotions are tied to human functions. Communication of emotions is mixed with communication of other types of information (such as factual details) and emotions can be consciously or unconsciously displayed. Affective computer systems, using sensors for emotion recognition and able to make emotive responses are under development. The increased potential for emotional interaction with products and services, in many domains, is generating much interest. Emotionally enhanced systems have potential to improve human computer interaction and so to improve how systems are used and what they can deliver. They may also have adverse implications such as creating systems capable of emotional manipulation of users. Affective systems are in their infancy and lack human complexity and capability. This makes it difficult to assess whether human interaction with such systems will actually prove beneficial or desirable to users. By using experimental design, a Wizard of Oz methodology and a game that appeared to respond to the user’s emotional signals with human-like capability, I tested user experience and reactions to a system that appeared affective. To assess users’ behaviour, I developed a novel affective behaviour coding system called ‘affectemes’. I found significant gains in user satisfaction and performance when using an affective system. Those believing the system responded to emotional signals blinked more frequently. If the machine failed to respond to their emotional signals, they increased their efforts to convey emotion, which might be an attempt to ‘repair’ the interaction. This work highlights how very complex and difficult it is to design and evaluate affective systems. I identify many issues for future work, including the unconscious nature of emotions and how they are recognised and displayed with affective systems; issues about the power of emotionally interactive systems and their evaluation; and critical ethical issues. These are important considerations for future design of systems that use emotion recognition in computing.EPSRC project grant (R81374/01

Micro ad-hoc Health Social Networks (uHSN). Design and evaluation of a social-based solution for patient support

Objective: To contribute the design, development, and assessment of a new concept: Micro ad hoc Health Social Networks (uHSN), to create a social-based solution for supporting patients with chronic disease. Design: After in-depth fieldwork and intensive co-design over a 4-year project following Community-Based Participatory Research (CBPR), this paper contributes a new paradigm of uHSN, defining two interaction areas (the “backstage” the sphere invisible to the final user, where processes that build services take place; and the “onstage” the visible part that includes the patients and relatives), and describes a new transversal concept, i.e., “network spaces segments, ” to provide timely interaction among all involved profiles and guaranteeing qualitative relationships. This proposal is applicable to any service design project and to all types of work areas; in the present work, it served as a social-based solution for supporting patients with chronic disease in two real-life health scenarios: a Parkinson disease patient association and a Stroke rehabilitation service in a hospital. These two scenarios included the following main features: thematic (related to the specific disease), private, and secure (only for the patient, relatives, healthcare professional, therapist, carer), with defined specific objectives (around patient support), small size (from tens to hundreds of users), ability to integrate innovative services (e.g., connection to hospital information service or to health sensors), supported by local therapeutic associations, and clustered with preconfigured relationships among users based in network groups. Measurements: Using a mixed qualitative and quantitative approach for 6 months, the performance of the uHSN was assessed in the two environments: a hospital rehabilitation unit working with Stroke patients, and a Parkinson disease association providing physiotherapy, occupational therapy, psychological support, speech therapy, and social services. We describe the proposed methods for evaluating the uHSN quantitatively and qualitatively, and how the scientific community can replicate and/or integrate this contribution in its research. Results: The uHSN overcomes the main limitations of traditional HSNs in the main areas recommended in the literature: privacy, security, transparency, system ecology, Quality of Service (QoS), and technology enhancement. The qualitative and quantitative research demonstrated its viability and replicability in four key points: user acceptance, productivity improvement, QoS enhancement, and fostering of social relations. It also meets the expectation of connecting health and social worlds, supporting distance rehabilitation, improving professionals’ efficiency, expanding users’ social capital, improving information quality and immediacy, and enhancing perceived peer/social/emotional support. The scientific contributions of the present paper are the first step not only in customizing health solutions that empower patients, their families, and healthcare professionals, but also in transferring this new paradigm to other scientific, professional, and social environments to create new opportunities