Multimodal Shared-Control Interaction for Mobile Robots in AAL Environments

This dissertation investigates the design, development and implementation of cognitively adequate, safe and robust, spatially-related, multimodal interaction between human operators and mobile robots in Ambient Assisted Living environments both from the theoretical and practical perspectives. By focusing on different aspects of the concept Interaction, the essential contribution of this dissertation is divided into three main research packages; namely, Formal Interaction, Spatial Interaction and Multimodal Interaction in AAL. As the principle package, in Formal Interaction, research effort is dedicated to developing a formal language based interaction modelling and management solution process and a unified dialogue modelling approach. This package aims to enable a robust, flexible, and context-sensitive, yet formally controllable and tractable interaction. This type of interaction can be used to support the interaction management of any complex interactive systems, including the ones covered in the other two research packages. In the second research package, Spatial Interaction, a general qualitative spatial knowledge based multi-level conceptual model is developed and proposed. The goal is to support a spatially-related interaction in human-robot collaborative navigation. With a model-based computational framework, the proposed conceptual model has been implemented and integrated into a practical interactive system which has been evaluated by empirical studies. It has been particularly tested with respect to a set of high-level and model-based conceptual strategies for resolving the frequent spatially-related communication problems in human-robot interaction. Last but not least, in Multimodal Interaction in AAL, attention is drawn to design, development and implementation of multimodal interaction for elderly persons. In this elderly-friendly scenario, ageing-related characteristics are carefully considered for an effective and efficient interaction. Moreover, a standard model based empirical framework for evaluating multimodal interaction is provided. This framework was especially applied to evaluate a minutely developed and systematically improved elderly-friendly multimodal interactive system through a series of empirical studies with groups of elderly persons

Towards elderly social integration using a multimodal human-computer interface

This paper presents a multimodal prototype application that aims to promote the social integration of the elderly. The application enables communication with their social network through conferencing and social media services, using natural interaction modalities, like speech, touch and gestures. We begin by discussing the requirements and design guidelines that were taken into account for the development of the prototype. We also present the key elements of the development stage and the results of a usability study conducted with ten elderly volunteers. The usability study reveals that such a multimodal solution can simplify accessibility to the considered services. Results indicate that this system is simpler, more natural and more enjoyable than the current user interfaces. Furthermore, the natural interaction modalities of the proposed prototype, allow elderly to be more efficient and have a better user experience, thus contributing with an easier and faster way for this population to join the information era.info:eu-repo/semantics/acceptedVersio

A Multimodal corpus to check on pragmatic competence for Mild Cognitive Impaired aging people

International audienceThis article presents a multimodal video corpus with the principal aim to model and predict the effects of aging in Mild Cognitive Impairment situation on pragmatic and communicative skills. We take as observable variables the verbal pragmatic markers and non-verbal pragmatic markers. This approach, at the interface of the psycholinguistics, cognitive sciences and rehabilitation medicine (speech-language pathology and therapy) is part of a longitudinal research process in an ecological situation (interviews conducted by close intimate of the elderly). In the first part of the article we present the linguistic, cognitive and social characteristics of aging in its continuum up to mild cognitive impairment and pathological disorders such as Alzheimer's disease. In the second part, we develop a multimodal approach, in particular to inform and enrich speech and language therapy knowledge. Finally, we present our experimental design and preliminary results on two female participants over 75 years of age with mild cognitive impairment Our general findings indicate that with aging, verbal pragmatic markers acquire an interactive function that allows people with Mild Cognitive Impairment to maintain intersubjective relationships with their interlocutor. In addition, at the non-verbal level, gestural manifestations are increasingly mobilized over time with a preference for non-verbal pragmatic markers with a referential function and an interactive function. One such non-verbal manifestation compensates for naming deficits, planning difficulties, discursive hitches; while another optimizes and maintains the interaction with the interlocutor

Improving elderly access to audiovisual and social media, using a multimodal human-computer interface

With the growth of Internet and especially, the proliferation of social media services, an opportunity has emerged for greater social and technological integration of the elderly. However, the adoption of new technologies by this segment of the population is not always straightforward mainly due to the physical and cognitive difficulties that are typically associated with ageing. Thus, for elderly to take advantage of new technologies and services that can help improve their quality of life, barriers must be broken by designing solutions with those needs in mind from the start. The aim of this work is to verify whether Multimodal Human-Computer Interaction (MHCI) systems designed with Universal Accessibility principles, taking into account elderly specific requirements, facilitate the adoption and access to popular Social Media Services (SMSs) and Audiovisual Communication Services, thus potentially contributing to the elderly social and technological integration. A user study was initially conducted in order to learn about the limitations and requirements of elderly people with existing HCI, concerning particularly SMSs and Audiovisual Communication Services, such as Facebook or Windows Live Messenger (WLM). The results of the study, basically a set of new MHCI requirements, were used to inform further development and enhancement of a multimodal prototype previously proposed for mobility-impaired individuals, now targeting the elderly. The prototype allows connecting users with their social networks through a text, audio and video communication service and integrates with SMSs, using natural interaction modalities, like speech, touch and gesture. After the development stage a usability evaluation study was conducted. The study reveals that such multimodal solution could simplify accessibility to the supported services, through the provision of simpler to use interfaces, by adopting natural interaction modalities and by being more satisfying to use by the elderly population, than most of the current graphical user interfaces for those same services, such as Facebook.Com o crescimento da Internet e, especialmente, das redes sociais surge a oportunidade para uma maior integração social e tecnológica dos idosos. No entanto, a adoção de novas tecnologias por essa população nem sempre é simples, principalmente devido às dificuldades físicas e cognitivas que estão associadas com o envelhecimento. Assim, e para que os idosos possam tirar proveito das novas tecnologias e serviços que podem ajudar a melhorar sua qualidade de vida, essas barreiras devem ser ultrapassadas desenhando soluções de raiz com essas necessidades em mente. O objetivo deste trabalho é verificar se interfaces humano-computador multimodais desenhadas com base em princípios de Acessibilidade Universal, tendo em conta requisitos específicos da população idosa, proporcionam um acesso simplificado a serviços de média social e serviços de comunicação audiovisuais, potencialmente contribuindo para a integração social e tecnológica desta população. Um estudo com utilizadores foi inicialmente conduzido a fim de apurar as necessidades especiais desses utilizadores com soluções de software, mais especificamente serviços de média social e serviços de conferência, como o Facebook ou o Windows Live Messenger. Os resultados do estudo foram utilizados para planear o desenvolvimento de um protótipo multimodal proposto anteriormente para utilizadores com mobilidade reduzida. Este permite ligar utilizadores às suas redes sociais através de um serviço de conferência por texto, áudio e vídeo, e um serviço integrado de média social, usando modalidades de interação natural, como o toque, fala e gestos. Após a fase de desenvolvimento foi realizado um estudo de usabilidade. Esse estudo revelou que este tipo de soluções pode simplificar a acessibilidade aos serviços considerados, dado ter interfaces mais simples, por adotar modalidades de interação mais naturais e por ser mais gratificante do que a maioria das interfaces gráficas atuais para os mesmos serviços, como por exemplo o Facebook

Flexible context aware interface for ambient assisted living

A Multi Agent System that provides a (cared for) person, the subject, with assistance and support through an Ambient Assisted Living Flexible Interface (AALFI) during the day while complementing the night time assistance offered by NOCTURNAL with feedback assistance, is presented. It has been tailored to the subject’s requirements profile and takes into account factors associated with the time of day; hence it attempts to overcome shortcomings of current Ambient Assisted Living Systems. The subject is provided with feedback that highlights important criteria such as quality of sleep during the night and possible breeches of safety during the day. This may help the subject carry out corrective measures and/or seek further assistance. AALFI provides tailored interaction that is either visual or auditory so that the subject is able to understand the interactions and this process is driven by a Multi-Agent System. User feedback gathered from a relevant user group through a workshop validated the ideas underpinning the research, the Multi-agent system and the adaptable interface

User Interface Abstraction for enabling TV set based Inclusive Access to the Information Society

199 p.The television (TV) set is present in most homes worldwide, and is the most used Information and Communication Technology (ICT). Despite its large implantation in the market, the interactive services consumption on TV set is limited. This thesis focuses on overcoming the following limiting factors: (i) limited Human Computer Interaction and (ii) lack of considering user’s real life context in the digital television (dTV) service integration strategy. Making interactive services accessible to TV set’s large user base, and especially to the most vulnerable ones, is understood as the path to integrate the mankind with the information society. This thesis explores the use of user interface abstraction technologies to reach the introduced goals. The main contributions of this thesis are: (i) an approach to enable the universally accessible remote control of the TV set, (ii) an approach for the provision of universally accessible interactive services through TV sets, and (iii) an approach for the provision of universally accessible services in the TV user’s real life context. We have implemented the contributing approaches for different use cases, and we have evaluated them with real users, achieving good results

Flexible context aware interface for ambient assisted living

A Multi Agent System that provides a (cared for) person, the subject, with assistance and support through an Ambient Assisted Living Flexible Interface (AALFI) during the day while complementing the night time assistance offered by NOCTURNAL with feedback assistance, is presented. It has been tailored to the subject’s requirements profile and takes into account factors associated with the time of day; hence it attempts to overcome shortcomings of current Ambient Assisted Living Systems. The subject is provided with feedback that highlights important criteria such as quality of sleep during the night and possible breeches of safety during the day. This may help the subject carry out corrective measures and/or seek further assistance. AALFI provides tailored interaction that is either visual or auditory so that the subject is able to understand the interactions and this process is driven by a Multi-Agent System. User feedback gathered from a relevant user group through a workshop validated the ideas underpinning the research, the Multi-agent system and the adaptable interface

Papilla Vaters Tumor in Elderly: an Interdisciplinary Issue

Tumors of the papilla Vater are very rare. Papilla Vater's tumors are benign or Malignant tumors in the ampulla of Vater and periampullary region. Blockage of ampulla leads to the development of obstructive jaundice; intermittent cholangitis, epigastric discomfort and weight loss. Treatment possibilities include endoscopic ampulectomy, surgical transduodenal excision of tumors of the ampulla and pancreatoduodenectomy (PDE). Prognosis depends on histological typing of the tumor and their clinical stage. We report a case of papilla Vater's tumor in elderly with comorbidities based on literature review. A 68-year-old female patient was referred for evaluation of intra and extra hepatic bile duct dilatation noted on abdominal ultrasonography. She complained of intermittent epigastric and right upper abdominal pain, with yellowish skin for two months. The laboratory findings showed leukocytosis, hyperbilirubinemia, abnormal liver function test, and high Ca 19-9. An endoscopic retrograde cholangiopancreatography (ERCP) revealed a distal obstruction caused by papilla Vater's tumor. Abdominal computed tomography (CT) with contrast, revealed a dilated common bile duct and pancreatic duct. The histologic evaluation was highly suggestive for dysplasia. She is now on a schedule for a Whipple procedure. To make a true diagnosis and optimal treatment of papilla Vater's tumor is multimodal. By doing a comprehensive geriatric assessment, with a careful modality selection, a Whipple procedure can be performed in elderly (65 years) safely. The post operative morbidity and mortality depends on their multi morbidity. Surgical, endoscopic, or radiologic biliary decompression; relief of gastric outlet obstruction; and adequate pain control may improve the quality of life but do not affect overall survival rate. By building a great interdisciplinary teamwork, the quality of life increased as follows