Hardware for recognition of human activities: a review of smart home and AAL related technologies

Activity recognition (AR) from an applied perspective of ambient assisted living (AAL) and smart homes (SH) has become a subject of great interest. Promising a better quality of life, AR applied in contexts such as health, security, and energy consumption can lead to solutions capable of reaching even the people most in need. This study was strongly motivated because levels of development, deployment, and technology of AR solutions transferred to society and industry are based on software development, but also depend on the hardware devices used. The current paper identifies contributions to hardware uses for activity recognition through a scientific literature review in the Web of Science (WoS) database. This work found four dominant groups of technologies used for AR in SH and AAL—smartphones, wearables, video, and electronic components—and two emerging technologies: Wi-Fi and assistive robots. Many of these technologies overlap across many research works. Through bibliometric networks analysis, the present review identified some gaps and new potential combinations of technologies for advances in this emerging worldwide field and their uses. The review also relates the use of these six technologies in health conditions, health care, emotion recognition, occupancy, mobility, posture recognition, localization, fall detection, and generic activity recognition applications. The above can serve as a road map that allows readers to execute approachable projects and deploy applications in different socioeconomic contexts, and the possibility to establish networks with the community involved in this topic. This analysis shows that the research field in activity recognition accepts that specific goals cannot be achieved using one single hardware technology, but can be using joint solutions, this paper shows how such technology works in this regard

Assessment of ambient assisted living systems for patients with mild cognitive impairment

According to the World Health Organization, about 50 million people worldwide suffer from dementia. Ten million new cases added every year. Mild Cognitive Impairment (MCI) affects more than 15% of the population aged 65. Technological solutions, such as smart home technology with ubiquitous computing devices, 24/7 telemedical observation and support can alleviate the growing problem and lower pressure on the healthcare system. This approach is also preferable for homecare patients in distant and rural areas. MCI patients are mostly home-based. Ambient Assisted Living (AAL) systems provide tools for automatic registration of vital signs and other medically and socially important information. AAL system for MCI patients is a logical answer to the problem. At the same time, many of the proposed AAL systems are proprietary, technically complicated and have a high price tag for implementation and service. Also, some proposed technical solutions not entirely reflect the opinion of healthcare stakeholders. The current study was proposed as a way to bridge the possible differences in the positions. An online anonymous questionnaire for healthcare professionals was created to prove or disprove the number of interconnected hypotheses about the necessity and feasibility of AAL system for MCI patients. The main focus was made on the hypotheses: "There is necessity of AAL systems for the healthcare" and "AAL systems are capable of providing assistance for patients with Mild Cognitive Impairment". The questionnaire was presented to more than three hundred potential respondents. Around a hundred and twenty agreed to fill it, and sixty completed the whole questionnaire. Results were analyzed to produce some directions guideline for future technical applications of AAL systems for MCI patients and future research. Descriptive statistics show support for the implementation of general AAL and variants for MCI patients. Comparative analysis of ordinal data for specific groups of respondents is done with help of non-parametric tests. Mann–Whitney–Wilcoxon test and Kruskal-Wallis test are applied. Table questions results are analyzed with chisquare for frequency tables. Group analysis demonstrated relative positive uniformity in of responses in the support of AAL of MCI patients.Segundo a Organização Mundial da Saúde, cerca de 50 milhões de pessoas em todo o mundo sofrem de demência. Dez milhões de novos casos adicionados a cada ano. O comprometimento cognitivo leve (MCI) afeta mais de 15% da população com 65 anos. Soluções tecnológicas, como tecnologia de casa inteligente com dispositivos de computação onipresentes, observação e suporte telemédico 24 horas por dia, 7 dias por semana, podem aliviar o problema crescente e diminuir a pressão sobre o sistema de saúde. Essa abordagem também é preferível para pacientes de cuidados domiciliares em áreas distantes e rurais. Os pacientes com CCL são, em sua maioria, domiciliares. Os sistemas Ambient Assisted Living (AAL) fornecem ferramentas para registro automático de sinais vitais e outras informações médicas e socialmente importantes. O sistema AAL para pacientes com MCI é uma resposta lógica para o problema. Ao mesmo tempo, muitos dos sistemas AAL propostos são proprietários, tecnicamente complicados e têm um alto preço para implementação e serviço. Além disso, algumas soluções técnicas propostas não refletem inteiramente a opinião das partes interessadas na área da saúde. O presente estudo foi proposto como forma de colmatar as possíveis diferenças nas posições. Um questionário anônimo online para profissionais de saúde foi criado para comprovar ou refutar o número de hipóteses interligadas sobre a necessidade e viabilidade do sistema AAL para pacientes com CCL. O foco principal foi feito nas hipóteses: "Há necessidade de sistemas de AAL para a saúde" e "Os sistemas de AAL são capazes de prestar assistência a pacientes com Comprometimento Cognitivo Leve". O questionário foi apresentado a mais de trezentos respondentes potenciais. Cerca de cento e vinte concordaram em preenchê-lo e sessenta preencheram todo o questionário. Os resultados foram analisados para produzir algumas diretrizes para futuras aplicações técnicas de sistemas AAL para pacientes com MCI e pesquisas futuras. Estatísticas descritivas mostram suporte para a implementação de AAL geral e variantes para pacientes com CCL. A análise comparativa de dados ordinais para grupos específicos de respondentes é feita com a ajuda de testes não paramétricos. Aplicam-se os testes de Mann-Whitney-Wilcoxon e Kruskal-Wallis. Os resultados das questões da tabela são analisados com qui-quadrado para tabelas de frequência. A análise do grupo demonstrou relativa uniformidade positiva nas respostas no suporte de AAL de pacientes com CCL.Selon l'Organisation mondiale de la santé, environ 50 millions de personnes dans le monde souffrent de démence. Dix millions de nouveaux cas ajoutés chaque année. Les troubles cognitifs légers (MCI) touchent plus de 15 % de la population âgée de 65 ans. Les solutions technologiques, telles que la technologie de la maison intelligente avec des appareils informatiques omniprésents, l'observation et le soutien télémédicaux 24 heures sur 24, 7 jours sur 7, peuvent atténuer le problème croissant et réduire la pression sur le système de santé. Cette approche est également préférable pour les patients en soins à domicile dans les régions éloignées et rurales. Les patients MCI sont pour la plupart à domicile. Les systèmes Ambient Assisted Living (AAL) fournissent des outils pour l'enregistrement automatique des signes vitaux et d'autres informations importantes sur le plan médical et social. Le système AAL pour les patients MCI est une réponse logique au problème. Dans le même temps, bon nombre des systèmes AAL proposés sont propriétaires, techniquement compliqués et ont un prix élevé pour la mise en oeuvre et le service. De plus, certaines solutions techniques proposées ne reflètent pas entièrement l'opinion des acteurs de santé. L'étude actuelle a été proposée comme un moyen de combler les différences possible dans les positions. Un questionnaire anonyme en ligne destiné aux professionnels de la santé a été créé pour prouver ou réfuter le nombre d'hypothèses interconnectées sur la nécessité et la faisabilité du système AAL pour les patients MCI. L'accent a été mis principalement sur les hypothèses: "Il existe une nécessité de systèmes AAL pour les soins de santé" et "Les systèmes AAL sont capables de fournir une assistance aux patients atteints de troubles cognitifs légers". Le questionnaire a été présenté à plus de trois cents répondants potentiels. Environ cent vingt ont accepté de le remplir, et soixante ont rempli tout le questionnaire. Les résultats ont été analysés pour produire des lignes directrices pour les futures applications techniques des systèmes AAL pour les patients MCI et l'avenir de la recherche. Les statistiques descriptives montrent un soutien à la mise en oeuvre de l'AAL général et des variantes pour les patients MCI. L'analyse comparative des données ordinales pour des groupes spécifiques de répondants est effectuée à l'aide de tests non paramétriques. Le test de Mann-Whitney-Wilcoxon et le test de Kruskal-Wallis sont appliqués. Les résultats des questions de tableau sont analysés avec le chi carré pour les tableaux de fréquence. L'analyse de groupe a démontré une uniformité positive relative dans les réponses à l'appui de l'AAL des patients MCI

State of the art of audio- and video based solutions for AAL

Working Group 3. Audio- and Video-based AAL ApplicationsIt is a matter of fact that Europe is facing more and more crucial challenges regarding health and social care due to the demographic change and the current economic context. The recent COVID-19 pandemic has stressed this situation even further, thus highlighting the need for taking action. Active and Assisted Living (AAL) technologies come as a viable approach to help facing these challenges, thanks to the high potential they have in enabling remote care and support. Broadly speaking, AAL can be referred to as the use of innovative and advanced Information and Communication Technologies to create supportive, inclusive and empowering applications and environments that enable older, impaired or frail people to live independently and stay active longer in society. AAL capitalizes on the growing pervasiveness and effectiveness of sensing and computing facilities to supply the persons in need with smart assistance, by responding to their necessities of autonomy, independence, comfort, security and safety. The application scenarios addressed by AAL are complex, due to the inherent heterogeneity of the end-user population, their living arrangements, and their physical conditions or impairment. Despite aiming at diverse goals, AAL systems should share some common characteristics. They are designed to provide support in daily life in an invisible, unobtrusive and user-friendly manner. Moreover, they are conceived to be intelligent, to be able to learn and adapt to the requirements and requests of the assisted people, and to synchronise with their specific needs. Nevertheless, to ensure the uptake of AAL in society, potential users must be willing to use AAL applications and to integrate them in their daily environments and lives. In this respect, video- and audio-based AAL applications have several advantages, in terms of unobtrusiveness and information richness. Indeed, cameras and microphones are far less obtrusive with respect to the hindrance other wearable sensors may cause to one’s activities. In addition, a single camera placed in a room can record most of the activities performed in the room, thus replacing many other non-visual sensors. Currently, video-based applications are effective in recognising and monitoring the activities, the movements, and the overall conditions of the assisted individuals as well as to assess their vital parameters (e.g., heart rate, respiratory rate). Similarly, audio sensors have the potential to become one of the most important modalities for interaction with AAL systems, as they can have a large range of sensing, do not require physical presence at a particular location and are physically intangible. Moreover, relevant information about individuals’ activities and health status can derive from processing audio signals (e.g., speech recordings). Nevertheless, as the other side of the coin, cameras and microphones are often perceived as the most intrusive technologies from the viewpoint of the privacy of the monitored individuals. This is due to the richness of the information these technologies convey and the intimate setting where they may be deployed. Solutions able to ensure privacy preservation by context and by design, as well as to ensure high legal and ethical standards are in high demand. After the review of the current state of play and the discussion in GoodBrother, we may claim that the first solutions in this direction are starting to appear in the literature. A multidisciplinary 4 debate among experts and stakeholders is paving the way towards AAL ensuring ergonomics, usability, acceptance and privacy preservation. The DIANA, PAAL, and VisuAAL projects are examples of this fresh approach. This report provides the reader with a review of the most recent advances in audio- and video-based monitoring technologies for AAL. It has been drafted as a collective effort of WG3 to supply an introduction to AAL, its evolution over time and its main functional and technological underpinnings. In this respect, the report contributes to the field with the outline of a new generation of ethical-aware AAL technologies and a proposal for a novel comprehensive taxonomy of AAL systems and applications. Moreover, the report allows non-technical readers to gather an overview of the main components of an AAL system and how these function and interact with the end-users. The report illustrates the state of the art of the most successful AAL applications and functions based on audio and video data, namely (i) lifelogging and self-monitoring, (ii) remote monitoring of vital signs, (iii) emotional state recognition, (iv) food intake monitoring, activity and behaviour recognition, (v) activity and personal assistance, (vi) gesture recognition, (vii) fall detection and prevention, (viii) mobility assessment and frailty recognition, and (ix) cognitive and motor rehabilitation. For these application scenarios, the report illustrates the state of play in terms of scientific advances, available products and research project. The open challenges are also highlighted. The report ends with an overview of the challenges, the hindrances and the opportunities posed by the uptake in real world settings of AAL technologies. In this respect, the report illustrates the current procedural and technological approaches to cope with acceptability, usability and trust in the AAL technology, by surveying strategies and approaches to co-design, to privacy preservation in video and audio data, to transparency and explainability in data processing, and to data transmission and communication. User acceptance and ethical considerations are also debated. Finally, the potentials coming from the silver economy are overviewed.publishedVersio

Graph-based representation of behavior in detection and prediction of daily living activities

Various surveillance systems capture signs of human activities of daily living (ADLs) and store multimodal information as time line behavioral records. In this paper, we present a novel approach to the analysis of a behavioral record used in a surveillance system designed for use in elderly smart homes. The description of a subject's activity is first decomposed into elementary poses - easily detectable by dedicated intelligent sensors - and represented by the share coefficients. Then, the activity is represented in the form of an attributed graph, where nodes correspond to elementary poses. As share coefficients of poses are expressed as attributes assigned to graph nodes, their change corresponding to a subject's action is represented by flow in graph edges. The behavioral record is thus a time series of graphs, which tiny size facilitates storage and management of long-term monitoring results. At the system learning stage, the contribution of elementary poses is accumulated, discretized and probability-ordered leading to a finite list representing the possible transitions between states. Such a list is independently built for each room in the supervised residence, and employed for assessment of the current action in the context of subject's habits and a room purpose. The proposed format of a behavioral record, applied to an adaptive surveillance system, is particularly advantageous for representing new activities not known at the setup stage, for providing a quantitative measure of transitions between poses and for expressing the difference between a predicted and actual action in a numerical way

State of the Art of Audio- and Video-Based Solutions for AAL

It is a matter of fact that Europe is facing more and more crucial challenges regarding health and social care due to the demographic change and the current economic context. The recent COVID-19 pandemic has stressed this situation even further, thus highlighting the need for taking action. Active and Assisted Living technologies come as a viable approach to help facing these challenges, thanks to the high potential they have in enabling remote care and support. Broadly speaking, AAL can be referred to as the use of innovative and advanced Information and Communication Technologies to create supportive, inclusive and empowering applications and environments that enable older, impaired or frail people to live independently and stay active longer in society. AAL capitalizes on the growing pervasiveness and effectiveness of sensing and computing facilities to supply the persons in need with smart assistance, by responding to their necessities of autonomy, independence, comfort, security and safety. The application scenarios addressed by AAL are complex, due to the inherent heterogeneity of the end-user population, their living arrangements, and their physical conditions or impairment. Despite aiming at diverse goals, AAL systems should share some common characteristics. They are designed to provide support in daily life in an invisible, unobtrusive and user-friendly manner. Moreover, they are conceived to be intelligent, to be able to learn and adapt to the requirements and requests of the assisted people, and to synchronise with their specific needs. Nevertheless, to ensure the uptake of AAL in society, potential users must be willing to use AAL applications and to integrate them in their daily environments and lives. In this respect, video- and audio-based AAL applications have several advantages, in terms of unobtrusiveness and information richness. Indeed, cameras and microphones are far less obtrusive with respect to the hindrance other wearable sensors may cause to one’s activities. In addition, a single camera placed in a room can record most of the activities performed in the room, thus replacing many other non-visual sensors. Currently, video-based applications are effective in recognising and monitoring the activities, the movements, and the overall conditions of the assisted individuals as well as to assess their vital parameters. Similarly, audio sensors have the potential to become one of the most important modalities for interaction with AAL systems, as they can have a large range of sensing, do not require physical presence at a particular location and are physically intangible. Moreover, relevant information about individuals’ activities and health status can derive from processing audio signals. Nevertheless, as the other side of the coin, cameras and microphones are often perceived as the most intrusive technologies from the viewpoint of the privacy of the monitored individuals. This is due to the richness of the information these technologies convey and the intimate setting where they may be deployed. Solutions able to ensure privacy preservation by context and by design, as well as to ensure high legal and ethical standards are in high demand. After the review of the current state of play and the discussion in GoodBrother, we may claim that the first solutions in this direction are starting to appear in the literature. A multidisciplinary debate among experts and stakeholders is paving the way towards AAL ensuring ergonomics, usability, acceptance and privacy preservation. The DIANA, PAAL, and VisuAAL projects are examples of this fresh approach. This report provides the reader with a review of the most recent advances in audio- and video-based monitoring technologies for AAL. It has been drafted as a collective effort of WG3 to supply an introduction to AAL, its evolution over time and its main functional and technological underpinnings. In this respect, the report contributes to the field with the outline of a new generation of ethical-aware AAL technologies and a proposal for a novel comprehensive taxonomy of AAL systems and applications. Moreover, the report allows non-technical readers to gather an overview of the main components of an AAL system and how these function and interact with the end-users. The report illustrates the state of the art of the most successful AAL applications and functions based on audio and video data, namely lifelogging and self-monitoring, remote monitoring of vital signs, emotional state recognition, food intake monitoring, activity and behaviour recognition, activity and personal assistance, gesture recognition, fall detection and prevention, mobility assessment and frailty recognition, and cognitive and motor rehabilitation. For these application scenarios, the report illustrates the state of play in terms of scientific advances, available products and research project. The open challenges are also highlighted. The report ends with an overview of the challenges, the hindrances and the opportunities posed by the uptake in real world settings of AAL technologies. In this respect, the report illustrates the current procedural and technological approaches to cope with acceptability, usability and trust in the AAL technology, by surveying strategies and approaches to co-design, to privacy preservation in video and audio data, to transparency and explainability in data processing, and to data transmission and communication. User acceptance and ethical considerations are also debated. Finally, the potentials coming from the silver economy are overviewed