Applications of Affective Computing in Human-Robot Interaction: state-of-art and challenges for manufacturing

The introduction of collaborative robots aims to make production more flexible, promoting a greater interaction between humans and robots also from physical point of view. However, working closely with a robot may lead to the creation of stressful situations for the operator, which can negatively affect task performance. In Human-Robot Interaction (HRI), robots are expected to be socially intelligent, i.e., capable of understanding and reacting accordingly to human social and affective clues. This ability can be exploited implementing affective computing, which concerns the development of systems able to recognize, interpret, process, and simulate human affects. Social intelligence is essential for robots to establish a natural interaction with people in several contexts, including the manufacturing sector with the emergence of Industry 5.0. In order to take full advantage of the human-robot collaboration, the robotic system should be able to perceive the psycho-emotional and mental state of the operator through different sensing modalities (e.g., facial expressions, body language, voice, or physiological signals) and to adapt its behaviour accordingly. The development of socially intelligent collaborative robots in the manufacturing sector can lead to a symbiotic human-robot collaboration, arising several research challenges that still need to be addressed. The goals of this paper are the following: (i) providing an overview of affective computing implementation in HRI; (ii) analyzing the state-of-art on this topic in different application contexts (e.g., healthcare, service applications, and manufacturing); (iii) highlighting research challenges for the manufacturing sector

Perspectives and research on play for children with disabilities: collected papers

This book includes eight chapters reflecting various approaches towards the theme of play for children with disabilities that characterised the work of the members of the COST Action TD1309 "LUDI-Play for Children with Disabilities". Alongside these multifaceted points of view, some theoretical aspects emerged as a common background: the ICF-CY theoretical perspective, the vision of "play for the sake of play" and play as a fundamental right

Perspectives and research on play for children with disabilities. Collected papers

This book includes eight chapters reflecting various approaches towards the theme of play for children with disabilities that characterised the work of the members of the COST Action TD1309 “LUDI–Play for Children with Disabilities”. Alongside these multifaceted points of view, some theoretical aspects emerged as a common background: the ICF-CY theoretical perspective, the vision of “play for the sake of play” and play as a fundamental right

Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

Stratégies pour le raisonnement sur le contexte dans les environnements d assistance pour les personnes âgées

Tirant parti de notre expérience avec une approche traditionnelle des environnements d'assistance ambiante (AAL) qui repose sur l'utilisation de nombreuses technologies hétérogènes dans les déploiements, cette thèse étudie la possibilité d'une approche simplifiée et complémentaire, ou seul un sous-ensemble hardware réduit est déployé, initiant un transfert de complexité vers le côté logiciel. Axé sur les aspects de raisonnement dans les systèmes AAL, ce travail a permis à la proposition d'un moteur d'inférence sémantique adapté à l'utilisation particulière à ces systèmes, répondant ainsi à un besoin de la communauté scientifique. Prenant en compte la grossière granularité des données situationnelles disponible avec une telle approche, un ensemble de règles dédiées avec des stratégies d'inférence adaptées est proposé, implémenté et validé en utilisant ce moteur. Un mécanisme de raisonnement sémantique novateur est proposé sur la base d'une architecture de raisonnement inspiré du système cognitif. Enfin, le système de raisonnement est intégré dans un framework de provision de services sensible au contexte, se chargeant de l'intelligence vis-à-vis des données contextuelles en effectuant un traitement des événements en direct par des manipulations ontologiques complexes. L ensemble du système est validé par des déploiements in-situ dans une maison de retraite ainsi que dans des maisons privées, ce qui en soi est remarquable dans un domaine de recherche principalement cantonné aux laboratoiresLeveraging our experience with the traditional approach to ambient assisted living (AAL) which relies on a large spread of heterogeneous technologies in deployments, this thesis studies the possibility of a more stripped down and complementary approach, where only a reduced hardware subset is deployed, probing a transfer of complexity towards the software side, and enhancing the large scale deployability of the solution. Focused on the reasoning aspects in AAL systems, this work has allowed the finding of a suitable semantic inference engine for the peculiar use in these systems, responding to a need in this scientific community. Considering the coarse granularity of situational data available, dedicated rule-sets with adapted inference strategies are proposed, implemented, and validated using this engine. A novel semantic reasoning mechanism is proposed based on a cognitively inspired reasoning architecture. Finally, the whole reasoning system is integrated in a fully featured context-aware service framework, powering its context awareness by performing live event processing through complex ontological manipulation. the overall system is validated through in-situ deployments in a nursing home as well as private homes over a few months period, which itself is noticeable in a mainly laboratory-bound research domainEVRY-INT (912282302) / SudocSudocFranceF

Evolutionary Service Composition and Personalization Ecosystem for Elderly Care

Current demographic trends suggest that people are living longer, while the ageing process entails many necessities, calling for care services tailored to the individual senior’s needs and life style. Personalized provision of care services usually involves a number of stakeholders, including relatives, friends, caregivers, professional assistance organizations, enterprises, and other support entities. Traditional Information and Communication Technology based care and assistance services for the elderly have been mainly focused on the development of isolated and generic services, considering a single service provider, and excessively featuring a techno-centric approach. In contrast, advances on collaborative networks for elderly care suggest the integration of services from multiple providers, encouraging collaboration as a way to provide better personalized services. This approach requires a support system to manage the personalization process and allow ranking the {service, provider} pairs. An additional issue is the problem of service evolution, as individual’s care needs are not static over time. Consequently, the care services need to evolve accordingly to keep the elderly’s requirements satisfied. In accordance with these requirements, an Elderly Care Ecosystem (ECE) framework, a Service Composition and Personalization Environment (SCoPE), and a Service Evolution Environment (SEvol) are proposed. The ECE framework provides the context for the personalization and evolution methods. The SCoPE method is based on the match between the customer´s profile and the available {service, provider} pairs to identify suitable services and corresponding providers to attend the needs. SEvol is a method to build an adaptive and evolutionary system based on the MAPE-K methodology supporting the solution evolution to cope with the elderly's new life stages. To demonstrate the feasibility, utility and applicability of SCoPE and SEvol, a number of methods and algorithms are presented, and illustrative scenarios are introduced in which {service, provider} pairs are ranked based on a multidimensional assessment method. Composition strategies are based on customer’s profile and requirements, and the evolutionary solution is determined considering customer’s inputs and evolution plans. For the ECE evaluation process the following steps are adopted: (i) feature selection and software prototype development; (ii) detailing the ECE framework validation based on applicability and utility parameters; (iii) development of a case study illustrating a typical scenario involving an elderly and her care needs; and (iv) performing a survey based on a modified version of the technology acceptance model (TAM), considering three contexts: Technological, Organizational and Collaborative environment

Формирование профиля пользователя на основе аудиовизуального анализа ситуации в интеллектуальном зале совещаний

This paper discuss the problem of personal adjustment of smart room devices and forming of a user profile based on processing of multichannel audio and video streams, which register of the current situation and meeting participants behavior in the meeting room. Estimation of preference of device usage, user interface, participant role and their activity during meeting allows us to automate the processes of smart room preparation as well as manage multimedia presentation and record devices during events. 212 records were made during several meetings in the smart room with the help of the developed system of audio and video speaker localization. The accumulated experimental data allowed us to estimate the places in the room, from which the participants asked questions most of the time. The accuracy of camera pointing on speaker in the presentation zone as well as in the rows of sits estimated by participant’s face size and its position in frame during whole recording equals 90% approximately.Рассматривается проблема персонифицированной настройки оборудования интеллектуального зала и формирования профиля пользователя на основе многоканальной обработки аудио- и видеопотоков, регистрирующих текущую ситуацию и поведение участников мероприятия в зале совещаний. Компьютерное зрение предпочтений по использованию оборудования, пользовательскому интерфейсу, роли и активности участников во время мероприятий позволяет автоматизировать процессы подготовки интеллектуального зала, управления мультимедийным презентационным и записывающим оборудованием в ходе выступлений. С помощью разработанной системы аудиовидеолокализации выступающих в ходе нескольких совещаний в интеллектуальном зале в автоматическом режиме было сделано 212 записей. Накопленные экспериментальные данные позволили оценить места в зале, с которых чаще всего задаются вопросы. Точность наведения видеокамеры на выступающего в зоне презентаций, а также в рядах кресел оценивалась по размеру и положению его лица в кадре на протяжении всей съемки и в среднем составила 90%