89 research outputs found

    Development of a real-time classifier for the identification of the Sit-To-Stand motion pattern

    Get PDF
    The Sit-to-Stand (STS) movement has significant importance in clinical practice, since it is an indicator of lower limb functionality. As an optimal trade-off between costs and accuracy, accelerometers have recently been used to synchronously recognise the STS transition in various Human Activity Recognition-based tasks. However, beyond the mere identification of the entire action, a major challenge remains the recognition of clinically relevant phases inside the STS motion pattern, due to the intrinsic variability of the movement. This work presents the development process of a deep-learning model aimed at recognising specific clinical valid phases in the STS, relying on a pool of 39 young and healthy participants performing the task under self-paced (SP) and controlled speed (CT). The movements were registered using a total of 6 inertial sensors, and the accelerometric data was labelised into four sequential STS phases according to the Ground Reaction Force profiles acquired through a force plate. The optimised architecture combined convolutional and recurrent neural networks into a hybrid approach and was able to correctly identify the four STS phases, both under SP and CT movements, relying on the single sensor placed on the chest. The overall accuracy estimate (median [95% confidence intervals]) for the hybrid architecture was 96.09 [95.37 - 96.56] in SP trials and 95.74 [95.39 \u2013 96.21] in CT trials. Moreover, the prediction delays ( 4533 ms) were compatible with the temporal characteristics of the dataset, sampled at 10 Hz (100 ms). These results support the implementation of the proposed model in the development of digital rehabilitation solutions able to synchronously recognise the STS movement pattern, with the aim of effectively evaluate and correct its execution

    A deep learning solution for real-time human motion decoding in smart walkers

    Get PDF
    Dissertação de mestrado integrado em Engenharia Biomédica (especialização em Eletrónica Médica)The treatment of gait impairments has increasingly relied on rehabilitation therapies which benefit from the use of smart walkers. These walkers still lack advanced and seamless Human-Robot Interaction, which intuitively understands the intentions of human motion, empowering the user’s recovery state and autonomy, while reducing the physician’s effort. This dissertation proposes the development of a deep learning solution to tackle the human motion decoding problematic in smart walkers, using only lower body vision information from a camera stream, mounted on the WALKit Smart Walker, a smart walker prototype for rehabilitation purposes. Different deep learning frameworks were designed for early human motion recognition and detec tion. A custom acquisition method, including a smart walker’s automatic driving algorithm and labelling procedure, was also designed to enable further training and evaluation of the proposed frameworks. Facing a 4-class (stop, walk, turn right/left) classification problem, a deep learning convolutional model with an attention mechanism achieved the best results: an offline f1-score of 99.61%, an online calibrated instantaneous precision higher than 97% and a human-centred focus slightly higher than 30%. Promising results were attained for early human motion detection, with enhancements in the focus of the proposed architectures. However, further improvements are still needed to achieve a more reliable solution for integration in a smart walker’s control strategy, based in the human motion intentions.O tratamento de distúrbios da marcha tem apostado cada vez mais em terapias de reabilitação que beneficiam do uso de andarilhos inteligentes. Estes ainda carecem de uma Interação Humano-Robô avançada e eficaz, capaz de entender, intuitivamente, as intenções do movimento humano, fortalecendo a recuperação autónoma do paciente e reduzindo o esforço médico. Esta dissertação propõe o desenvolvimento de uma solução de aprendizagem para o problema de descodificação de movimento humano em andarilhos inteligentes, usando apenas vídeos recolhidos pelo WALKit Smart Walker, um protótipo de andarilho inteligente usado para reabilitação. Foram desenvolvidos algoritmos de aprendizagem para o reconhecimento e detecção precoces de movimento humano. Um método de aquisição personalizado, incluindo um algoritmo de condução e labelização automatizados, foi projetado para permitir o conseguinte treino e avaliação dos algoritmos propostos. Perante a classificação de 4 ações (parar, andar, virar à direita/esquerda), um modelo convolucional com um mecanismo de atenção alcançou os melhores resultados: f1-score offline de 99,61%, precisão instantânea calibrada online de superior a 97 % e um foco centrado no ser humano ligeiramente superior a 30%. Com esta dissertação alcançaram-se resultados promissores para a detecção precoce de movimento humano, com aprimoramentos no foco dos algoritmos propostos. No entanto, ainda são necessárias melhorias adicionais para alcançar uma solução mais robusta para a integração na estratégia de controlo de um andarilho inteligente, com base nas intenções de movimento do utilizador

    State of the art of audio- and video based solutions for AAL

    Get PDF
    Working Group 3. Audio- and Video-based AAL ApplicationsIt is a matter of fact that Europe is facing more and more crucial challenges regarding health and social care due to the demographic change and the current economic context. The recent COVID-19 pandemic has stressed this situation even further, thus highlighting the need for taking action. Active and Assisted Living (AAL) technologies come as a viable approach to help facing these challenges, thanks to the high potential they have in enabling remote care and support. Broadly speaking, AAL can be referred to as the use of innovative and advanced Information and Communication Technologies to create supportive, inclusive and empowering applications and environments that enable older, impaired or frail people to live independently and stay active longer in society. AAL capitalizes on the growing pervasiveness and effectiveness of sensing and computing facilities to supply the persons in need with smart assistance, by responding to their necessities of autonomy, independence, comfort, security and safety. The application scenarios addressed by AAL are complex, due to the inherent heterogeneity of the end-user population, their living arrangements, and their physical conditions or impairment. Despite aiming at diverse goals, AAL systems should share some common characteristics. They are designed to provide support in daily life in an invisible, unobtrusive and user-friendly manner. Moreover, they are conceived to be intelligent, to be able to learn and adapt to the requirements and requests of the assisted people, and to synchronise with their specific needs. Nevertheless, to ensure the uptake of AAL in society, potential users must be willing to use AAL applications and to integrate them in their daily environments and lives. In this respect, video- and audio-based AAL applications have several advantages, in terms of unobtrusiveness and information richness. Indeed, cameras and microphones are far less obtrusive with respect to the hindrance other wearable sensors may cause to one’s activities. In addition, a single camera placed in a room can record most of the activities performed in the room, thus replacing many other non-visual sensors. Currently, video-based applications are effective in recognising and monitoring the activities, the movements, and the overall conditions of the assisted individuals as well as to assess their vital parameters (e.g., heart rate, respiratory rate). Similarly, audio sensors have the potential to become one of the most important modalities for interaction with AAL systems, as they can have a large range of sensing, do not require physical presence at a particular location and are physically intangible. Moreover, relevant information about individuals’ activities and health status can derive from processing audio signals (e.g., speech recordings). Nevertheless, as the other side of the coin, cameras and microphones are often perceived as the most intrusive technologies from the viewpoint of the privacy of the monitored individuals. This is due to the richness of the information these technologies convey and the intimate setting where they may be deployed. Solutions able to ensure privacy preservation by context and by design, as well as to ensure high legal and ethical standards are in high demand. After the review of the current state of play and the discussion in GoodBrother, we may claim that the first solutions in this direction are starting to appear in the literature. A multidisciplinary 4 debate among experts and stakeholders is paving the way towards AAL ensuring ergonomics, usability, acceptance and privacy preservation. The DIANA, PAAL, and VisuAAL projects are examples of this fresh approach. This report provides the reader with a review of the most recent advances in audio- and video-based monitoring technologies for AAL. It has been drafted as a collective effort of WG3 to supply an introduction to AAL, its evolution over time and its main functional and technological underpinnings. In this respect, the report contributes to the field with the outline of a new generation of ethical-aware AAL technologies and a proposal for a novel comprehensive taxonomy of AAL systems and applications. Moreover, the report allows non-technical readers to gather an overview of the main components of an AAL system and how these function and interact with the end-users. The report illustrates the state of the art of the most successful AAL applications and functions based on audio and video data, namely (i) lifelogging and self-monitoring, (ii) remote monitoring of vital signs, (iii) emotional state recognition, (iv) food intake monitoring, activity and behaviour recognition, (v) activity and personal assistance, (vi) gesture recognition, (vii) fall detection and prevention, (viii) mobility assessment and frailty recognition, and (ix) cognitive and motor rehabilitation. For these application scenarios, the report illustrates the state of play in terms of scientific advances, available products and research project. The open challenges are also highlighted. The report ends with an overview of the challenges, the hindrances and the opportunities posed by the uptake in real world settings of AAL technologies. In this respect, the report illustrates the current procedural and technological approaches to cope with acceptability, usability and trust in the AAL technology, by surveying strategies and approaches to co-design, to privacy preservation in video and audio data, to transparency and explainability in data processing, and to data transmission and communication. User acceptance and ethical considerations are also debated. Finally, the potentials coming from the silver economy are overviewed.publishedVersio

    Biological and biomimetic machine learning for automatic classification of human gait

    Get PDF
    Machine learning (ML) research has benefited from a deep understanding of biological mechanisms that have evolved to perform comparable tasks. Recent successes of ML models, superseding human performance in human perception based tasks has garnered interest in improving them further. However, the approach to improving ML models tends to be unstructured, particularly for the models that aim to mimic biology. This thesis proposes and applies a bidirectional learning paradigm to streamline the process of improving ML models’ performance in classification of a task, which humans are already adept at. The approach is validated taking human gait classification as the exemplar task. This paradigm possesses the additional benefit of investigating underlying mechanisms in human perception (HP) using the ML models. Assessment of several biomimetic (BM) and non-biomimetic (NBM) machine learning models on an intrinsic feature of gait, namely the gender of the walker, establishes a functional overlap in the perception of gait between HP and BM, selecting the Long-Short-Term-Memory (LSTM) architecture as the BM of choice for this study, when compared with other models such as support vector machines, decision trees and multi-layer perceptron models. Psychophysics and computational experiments are conducted to understand the overlap between human and machine models. The BM and HP derived from psychophysics experiments, share qualitatively similar profiles of gender classification accuracy across varying stimulus exposure durations. They also share the preference for motion-based cues over structural cues (BM=H>NBM). Further evaluation reveals a human-like expression of the inversion effect, a well-studied cognitive bias in HP that reduces the gender classification accuracy to 37% (p<0.05, chance at 50%) when exposed to inverted stimulus. Its expression in the BM supports the argument for learned rather than hard-wired mechanisms in HP. Particularly given the emergence of the effect in every BM, after training multiple randomly initialised BM models without prior anthropomorphic expectations of gait. The above aspects of HP, namely the preference for motion cues over structural cues and the lack of prior anthropomorphic expectations, were selected to improve BM performance. Representing gait explicitly as motion-based cues of a non-anthropomorphic, gender-neutral skeleton not only mitigates the inversion effect in BM, but also improves significantly the classification accuracy. In the case of gender classification of upright stimuli, mean accuracy improved by 6%, from 76% to 82% (F1,18 = 16, p<0.05). For inverted stimuli, mean accuracy improved by 45%, from 37% to 82% (F1,18 = 20, p<0.05). The model was further tested on a more challenging, extrinsic feature task; the classification of the emotional state of a walker. Emotions were visually induced in subjects through exposure to emotive or neutral images from the International Affective Picture System (IAPS) database. The classification accuracy of the BM was significantly above chance at 43% accuracy (p<0.05, chance at 33.3%). However, application of the proposed paradigm in further binary emotive state classification experiments, improved mean accuracy further by 23%, from 43% to 65% (F1,18 = 7.4, p<0.05) for the positive vs. neutral task. Results validate the proposed paradigm of concurrent bidirectional investigation of HP and BM for the classification of human gait, suggesting future applications for automating perceptual tasks for which the human brain and body has evolved

    Human Activity Recognition and Fall Detection Using Unobtrusive Technologies

    Full text link
    As the population ages, health issues like injurious falls demand more attention. Wearable devices can be used to detect falls. However, despite their commercial success, most wearable devices are obtrusive, and patients generally do not like or may forget to wear them. In this thesis, a monitoring system consisting of two 24×32 thermal array sensors and a millimetre-wave (mmWave) radar sensor was developed to unobtrusively detect locations and recognise human activities such as sitting, standing, walking, lying, and falling. Data were collected by observing healthy young volunteers simulate ten different scenarios. The optimal installation position of the sensors was initially unknown. Therefore, the sensors were mounted on a side wall, a corner, and on the ceiling of the experimental room to allow performance comparison between these sensor placements. Every thermal frame was converted into an image and a set of features was manually extracted or convolutional neural networks (CNNs) were used to automatically extract features. Applying a CNN model on the infrared stereo dataset to recognise five activities (falling plus lying on the floor, lying in bed, sitting on chair, sitting in bed, standing plus walking), overall average accuracy and F1-score were 97.6%, and 0.935, respectively. The scores for detecting falling plus lying on the floor from the remaining activities were 97.9%, and 0.945, respectively. When using radar technology, the generated point clouds were converted into an occupancy grid and a CNN model was used to automatically extract features, or a set of features was manually extracted. Applying several classifiers on the manually extracted features to detect falling plus lying on the floor from the remaining activities, Random Forest (RF) classifier achieved the best results in overhead position (an accuracy of 92.2%, a recall of 0.881, a precision of 0.805, and an F1-score of 0.841). Additionally, the CNN model achieved the best results (an accuracy of 92.3%, a recall of 0.891, a precision of 0.801, and an F1-score of 0.844), in overhead position and slightly outperformed the RF method. Data fusion was performed at a feature level, combining both infrared and radar technologies, however the benefit was not significant. The proposed system was cost, processing time, and space efficient. The system with further development can be utilised as a real-time fall detection system in aged care facilities or at homes of older people

    Predictive Models For Falls-Risk Assessment in Older People, Using Markerless Motion Capture

    Get PDF
    Falling in old age contributes to considerable misery for many people. Currently, there is a lack of practical, low cost and objective methods for identifying those at risk of falls. This thesis aims to address this need. The majority of the literature related to falls risk and balance impairment uses force plates to quantify postural sway. The use of such devices in a clinical setting is rare, mainly due to cost. However, some force-plate-based commercial products have been created, e.g. the Balance Master. To align the research in this thesis to both the literature and existing methods of assessing postural sway, a method is proposed which can generate sway metrics from the output of a low-cost markerless motion capture device (Kinect V2). Good agreement was found between the proposed method and the output of the Balance Master. A key reason for the lack of research into falls-risk using markerless motion capture, is the lack of an appropriate dataset. To address this issue, a dataset of clinical movements, recorded using markerless motion capture, was created. Named KINECAL, It contains the recordings of 90 participants, labelled by age and falls-risk. The data provided includes depth images, 3D joint positions, sway metrics and socioeconomic and health meta data. Many studies have noted that postural sway increases with age and conflate age-related changes with falls risk. However, if one examines sub-populations of older people, such as master athletes, It is clear that this is not necessarily true. The structure of KINECAL allows for the examination of age-related factors and falls-risk factors simultaneously. In addition, it includes labels of falls history, clinical impairment and comprehensive metadata. KINECAL was used to identify sway metrics most closely associated with falls risk, as distinct from the ageing process. Using the identified metrics, a model was developed that can identify those who would be classified as impaired by a range of clinical tests. Finally, a model is proposed, which can predict fallers by placing individuals on a scale of physical impairment. An autoencoder was used to model, healthy adult sit-to-stand movements. Using an anomaly detection approach, an individuals level of impairment can be plotted relative to this healthy standard. Using this model, the existence of two older populations (one with a high falls risk and one with a low falls risk) is demonstrated

    Development of a Healthcare Software System for the Elderly

    Get PDF
    This research focused on the implementation of a reliable intelligent fall detection system so as to reduce accidental falls among the elderly people. A video-based detection system was used because it preserved privacy while monitoring the activities of the senior citizens. Another advantage of the video-based system is that the senior citizens are able to move freely without experiencing any hassles in wearing them as opposed to portable fall detection sensors so that they can have a more independent and happy life. A scientific research method was employed to improve the existing fall detection systems in terms of reliability and accuracy. This thesis consists of four stages where the first stage reviews the literature on the current fall detection systems, the second stage investigates the various algorithms of these existing fall detection systems, the third stage describes the proposed fall detection algorithm in detecting falls using two distinct approaches. The first approach deals with the use of specific features of the silhouette, an extracted binary map obtained from the subtraction of the foreground from the background, to determine the fall angle (FA), the bounding box (BB) ratio, the Hidden Markov Models (HMM) and the combination of FA, BB, and HMM. The second approach used is the neural network approach which is incorporated in the algorithm to identify a predetermined set of situations such as praying, sitting, standing, bending, kneeling, and lying down. The fourth stage involves the evalua- tion of the developed video-based fall detection system using different metrics which measure sensitivity (i.e. the capacity of the fall detection system to detect as well as declare a fall) and specificity (i.e. the capacity of the algorithm to detect only falls) of this algorithm. The video camera was properly positioned to avoid any occluding objects and also to cover a certain range of motion of the stunt participants performing the falls. The silhouette is extracted using an approximate median filtering approach and the threshold criteria value of 30 pixels was used. Morphological filtering methods that were dilation and erosion were used to remove any spurious noises from the extracted image prior to subsequent feature analysis. Then, this extracted silhouette was scaled and quantised using 8 bits/pixel and compared to the set of predetermined scenarios using a neural network of perceptrons. This neural network was trained based on various situations and the falls of the participants which represent inputs to the neural network algorithm during the neural learning process. In this research study, the built neural network consisted of 600 inputs, as well as 10 neurons in the hidden layer together with 7 distinct outputs which represent the set of predefined situations. Furthermore, an alarm generation algorithm was included in the fall detection algorithm such that there were three states that were STATE NULL (set at 0), STATE LYING (set at 1) and STATE ALL OTHERS (set at 2) and the initial alarm count was set to 90 frames (meaning 3 seconds of recorded consecutive images at 30 frames per second). Therefore, an alarm was generated only when the in-built counter surpassed this threshold of 90 frames to signal that a fall occurred. Following the evaluation stage, it was found that the combination of the first approach fall detection algorithm method (fall angle, bounding box, and hidden Markov) was 89% with specificity and 84.2% with sensitivity which is better than individual performance. Moreover, it was found that the second approach fall detection algorithm method (neural network performance) 94.3% of the scenarios were successfully classified whereby the specificity of the developed algorithm was determined to be 94.8% and the sensitivity was 93.8% which altogether show a promising overall performance of the fall detection video-based intelligent system. Moreover, the developed fall detection system were tested using two types of handicaps such as limping and stumbling stunt participants to observe how well this detection algorithm can detect falls as in the practical situations encountered or present in elderly people. In these cases it was found that about 90.2% of the falls were detected which showed still that the developed algorithm was quite robust and reliable subjected to these two physical handicaps motion behaviours

    A model for inebriation recognition in humans using computer vision

    Get PDF
    Abstract: Inebriation is a situational impairment caused by the consumption of alcohol affecting the consumer's interaction with the environment around them...M.Sc. (Information Technology

    Localisation of humans, objects and robots interacting on load-sensing floors

    Get PDF
    International audienceLocalisation, tracking and recognition of objects and humans are basic tasks that are of high value in applications of ambient intelligence. Sensing floors were introduced to address these tasks in a non-intrusive way. To recognize the humans moving on the floor, they are usually first localized, and then a set of gait features are extracted (stride length, cadence, pressure profile over a footstep). However, recognition generally fails when several people stand or walk together, preventing successful tracking. This paper presents a detection, tracking and recognition technique which uses objects' weight. It continues working even when tracking individual persons becomes impossible. Inspired by computer vision, this technique processes the floor pressure-image by segmenting the blobs containing objects, tracking them, and recognizing their contents through a mix of inference and combinatorial search. The result lists the probabilities of assignments of known objects to observed blobs. The concept was successfully evaluated in daily life activity scenarii, involving multi-object tracking and recognition on low resolution sensors, crossing of user trajectories, and weight ambiguity. This technique can be used to provide a probabilistic input for multi-modal object tracking and recognition systems
    • …
    corecore