State of the art of audio- and video based solutions for AAL

Working Group 3. Audio- and Video-based AAL ApplicationsIt is a matter of fact that Europe is facing more and more crucial challenges regarding health and social care due to the demographic change and the current economic context. The recent COVID-19 pandemic has stressed this situation even further, thus highlighting the need for taking action. Active and Assisted Living (AAL) technologies come as a viable approach to help facing these challenges, thanks to the high potential they have in enabling remote care and support. Broadly speaking, AAL can be referred to as the use of innovative and advanced Information and Communication Technologies to create supportive, inclusive and empowering applications and environments that enable older, impaired or frail people to live independently and stay active longer in society. AAL capitalizes on the growing pervasiveness and effectiveness of sensing and computing facilities to supply the persons in need with smart assistance, by responding to their necessities of autonomy, independence, comfort, security and safety. The application scenarios addressed by AAL are complex, due to the inherent heterogeneity of the end-user population, their living arrangements, and their physical conditions or impairment. Despite aiming at diverse goals, AAL systems should share some common characteristics. They are designed to provide support in daily life in an invisible, unobtrusive and user-friendly manner. Moreover, they are conceived to be intelligent, to be able to learn and adapt to the requirements and requests of the assisted people, and to synchronise with their specific needs. Nevertheless, to ensure the uptake of AAL in society, potential users must be willing to use AAL applications and to integrate them in their daily environments and lives. In this respect, video- and audio-based AAL applications have several advantages, in terms of unobtrusiveness and information richness. Indeed, cameras and microphones are far less obtrusive with respect to the hindrance other wearable sensors may cause to one’s activities. In addition, a single camera placed in a room can record most of the activities performed in the room, thus replacing many other non-visual sensors. Currently, video-based applications are effective in recognising and monitoring the activities, the movements, and the overall conditions of the assisted individuals as well as to assess their vital parameters (e.g., heart rate, respiratory rate). Similarly, audio sensors have the potential to become one of the most important modalities for interaction with AAL systems, as they can have a large range of sensing, do not require physical presence at a particular location and are physically intangible. Moreover, relevant information about individuals’ activities and health status can derive from processing audio signals (e.g., speech recordings). Nevertheless, as the other side of the coin, cameras and microphones are often perceived as the most intrusive technologies from the viewpoint of the privacy of the monitored individuals. This is due to the richness of the information these technologies convey and the intimate setting where they may be deployed. Solutions able to ensure privacy preservation by context and by design, as well as to ensure high legal and ethical standards are in high demand. After the review of the current state of play and the discussion in GoodBrother, we may claim that the first solutions in this direction are starting to appear in the literature. A multidisciplinary 4 debate among experts and stakeholders is paving the way towards AAL ensuring ergonomics, usability, acceptance and privacy preservation. The DIANA, PAAL, and VisuAAL projects are examples of this fresh approach. This report provides the reader with a review of the most recent advances in audio- and video-based monitoring technologies for AAL. It has been drafted as a collective effort of WG3 to supply an introduction to AAL, its evolution over time and its main functional and technological underpinnings. In this respect, the report contributes to the field with the outline of a new generation of ethical-aware AAL technologies and a proposal for a novel comprehensive taxonomy of AAL systems and applications. Moreover, the report allows non-technical readers to gather an overview of the main components of an AAL system and how these function and interact with the end-users. The report illustrates the state of the art of the most successful AAL applications and functions based on audio and video data, namely (i) lifelogging and self-monitoring, (ii) remote monitoring of vital signs, (iii) emotional state recognition, (iv) food intake monitoring, activity and behaviour recognition, (v) activity and personal assistance, (vi) gesture recognition, (vii) fall detection and prevention, (viii) mobility assessment and frailty recognition, and (ix) cognitive and motor rehabilitation. For these application scenarios, the report illustrates the state of play in terms of scientific advances, available products and research project. The open challenges are also highlighted. The report ends with an overview of the challenges, the hindrances and the opportunities posed by the uptake in real world settings of AAL technologies. In this respect, the report illustrates the current procedural and technological approaches to cope with acceptability, usability and trust in the AAL technology, by surveying strategies and approaches to co-design, to privacy preservation in video and audio data, to transparency and explainability in data processing, and to data transmission and communication. User acceptance and ethical considerations are also debated. Finally, the potentials coming from the silver economy are overviewed.publishedVersio

Acceptance of ambient assisted living (AAL) technologies among older Australians : a review of barriers in user experience

One of the great challenges facing Australian society is that of an ageing population. Amongst the issues involved in this drastic demographic change, the most significant aspect is the demand for older Australians to live independently at home. The development of Ambient Assisted Living (AAL) technologies aims to address this issue. The advancement of AAL applications have been done to support the users with their daily-life activities and health concerns by providing increased mobility, security, safety in emergencies, health-monitoring, improved lifestyle, and fall-detection through the use of sensors. However, the optimum uptake of these technologies among the end-users (the elderly Australians) still remains a big concern. Thus, there is an elevated need to understand the needs and preferences of the seniors in order to improve the acceptance of AAL applications. The aim of this study is to investigate the barriers and perceptions in the use of AAL applications amongst older Australians. Focus groups and quantitative surveys have been conducted to provide a detailed analysis of these impediments. The results show that there are different factors that restrict the use of these technologies along with the fact that elderly people have certain preferences when using them. An understanding of these factors has been gained and suggestions have been made to increase the acceptance of AAL devices. This work gives useful insights towards the design of AAL solutions according to user needs

Revisiting the Technology Challenges and Proposing Enhancements in Ambient Assisted Living for the Elderly

Several social and technical trends support the elderly’s desire to live independently in their preferred environment, despite their increasing medical needs, and enhance their quality of life at home. Ambient-assisted living (AAL) has the capabilities to support the elderly and to decrease their dependency on formal or informal caregivers. We provide a review of the technological challenges that were identified as inhibiting factors in the past decade and then present recent technological advances, e.g., cloud computing, machine learning, artificial intelligence, the Internet of Things. We also fill the gap in the current literature in regard to specific AAL solutions and propose fourth-generation AAL technology design. We find that most informal caregivers are family members who are medically untrained and that the use of advanced analytical processes on AAL-generated data could significantly increase symptom identification. We also present the implications and remaining challenges along with recommendations for future research

Distributed Computing and Monitoring Technologies for Older Patients

This book summarizes various approaches for the automatic detection of health threats to older patients at home living alone. The text begins by briefly describing those who would most benefit from healthcare supervision. The book then summarizes possible scenarios for monitoring an older patient at home, deriving the common functional requirements for monitoring technology. Next, the work identifies the state of the art of technological monitoring approaches that are practically applicable to geriatric patients. A survey is presented on a range of such interdisciplinary fields as smart homes, telemonitoring, ambient intelligence, ambient assisted living, gerontechnology, and aging-in-place technology. The book discusses relevant experimental studies, highlighting the application of sensor fusion, signal processing and machine learning techniques. Finally, the text discusses future challenges, offering a number of suggestions for further research directions

A Mobile Healthcare Solution for Ambient Assisted Living Environments

Elderly people need regular healthcare services and, several times, are dependent of physicians’ personal attendance. This dependence raises several issues to elders, such as, the need to travel and mobility support. Ambient Assisted Living (AAL) and Mobile Health (m-Health) services and applications offer good healthcare solutions that can be used both on indoor and in mobility environments. This dissertation presents an ambient assisted living (AAL) solution for mobile environments. It includes elderly biofeedback monitoring using body sensors for data collection offering support for remote monitoring. The used sensors are attached to the human body (such as the electrocardiogram, blood pressure, and temperature). They collect data providing comfort, mobility, and guaranteeing efficiency and data confidentiality. Periodic collection of patients’ data is important to gather more accurate measurements and to avoid common risky situations, like a physical fall may be considered something natural in life span and it is more dangerous for senior people. One fall can out a life in extreme cases or cause fractures, injuries, but when it is early detected through an accelerometer, for example, it can avoid a tragic outcome. The presented proposal monitors elderly people, storing collected data in a personal computer, tablet, or smartphone through Bluetooth. This application allows an analysis of possible health condition warnings based on the input of supporting charts, and real-time bio-signals monitoring and is able to warn users and the caretakers. These mobile devices are also used to collect data, which allow data storage and its possible consultation in the future. The proposed system is evaluated, demonstrated and validated through a prototype and it is ready for use. The watch Texas ez430-Chronos, which is capable to store information for later analysis and the sensors Shimmer who allow the creation of a personalized application that it is capable of measuring biosignals of the patient in real time is described throughout this dissertation

A smart home environment to support safety and risk monitoring for the elderly living independently

The elderly prefer to live independently despite vulnerability to age-related challenges. Constant monitoring is required in cases where the elderly are living alone. The home environment can be a dangerous environment for the elderly living independently due to adverse events that can occur at any time. The potential risks for the elderly living independently can be categorised as injury in the home, home environmental risks and inactivity due to unconsciousness. The main research objective was to develop a Smart Home Environment (SHE) that can support risk and safety monitoring for the elderly living independently. An unobtrusive and low cost SHE solution that uses a Raspberry Pi 3 model B, a Microsoft Kinect Sensor and an Aeotec 4-in-1 Multisensor was implemented. The Aeotec Multisensor was used to measure temperature, motion, lighting, and humidity in the home. Data from the multisensor was collected using OpenHAB as the Smart Home Operating System. The information was processed using the Raspberry Pi 3 and push notifications were sent when risk situations were detected. An experimental evaluation was conducted to determine the accuracy with which the prototype SHE detected abnormal events. Evaluation scripts were each evaluated five times. The results show that the prototype has an average accuracy, sensitivity and specificity of 94%, 96.92% and 88.93% respectively. The sensitivity shows that the chance of the prototype missing a risk situation is 3.08%, and the specificity shows that the chance of incorrectly classifying a non-risk situation is 11.07%. The prototype does not require any interaction on the part of the elderly. Relatives and caregivers can remotely monitor the elderly person living independently via the mobile application or a web portal. The total cost of the equipment used was below R3000

SHELDON Smart habitat for the elderly.

An insightful document concerning active and assisted living under different perspectives: Furniture and habitat, ICT solutions and Healthcare

Progress in ambient assisted systems for independent living by the elderly

One of the challenges of the ageing population in many countries is the efficient delivery of health and care services, which is further complicated by the increase in neurological conditions among the elderly due to rising life expectancy. Personal care of the elderly is of concern to their relatives, in case they are alone in their homes and unforeseen circumstances occur, affecting their wellbeing. The alternative; i.e. care in nursing homes or hospitals is costly and increases further if specialized care is mobilized to patients’ place of residence. Enabling technologies for independent living by the elderly such as the ambient assisted living systems (AALS) are seen as essential to enhancing care in a cost-effective manner. In light of significant advances in telecommunication, computing and sensor miniaturization, as well as the ubiquity of mobile and connected devices embodying the concept of the Internet of Things (IoT), end-to-end solutions for ambient assisted living have become a reality. The premise of such applications is the continuous and most often real-time monitoring of the environment and occupant behavior using an event-driven intelligent system, thereby providing a facility for monitoring and assessment, and triggering assistance as and when needed. As a growing area of research, it is essential to investigate the approaches for developing AALS in literature to identify current practices and directions for future research. This paper is, therefore, aimed at a comprehensive and critical review of the frameworks and sensor systems used in various ambient assisted living systems, as well as their objectives and relationships with care and clinical systems. Findings from our work suggest that most frameworks focused on activity monitoring for assessing immediate risks while the opportunities for integrating environmental factors for analytics and decision-making, in particular for the long-term care were often overlooked. The potential for wearable devices and sensors, as well as distributed storage and access (e.g. cloud) are yet to be fully appreciated. There is a distinct lack of strong supporting clinical evidence from the implemented technologies. Socio-cultural aspects such as divergence among groups, acceptability and usability of AALS were also overlooked. Future systems need to look into the issues of privacy and cyber security