Evolving Spatio-temporal Data Machines Based on the NeuCube Neuromorphic Framework: Design Methodology and Selected Applications

The paper describes a new type of evolving connectionist systems (ECOS) called evolving spatio-temporal data machines based on neuromorphic, brain-like information processing principles (eSTDM). These are multi-modular computer systems designed to deal with large and fast spatio/spectro temporal data using spiking neural networks (SNN) as major processing modules. ECOS and eSTDM in particular can learn incrementally from data streams, can include ‘on the fly’ new input variables, new output class labels or regression outputs, can continuously adapt their structure and functionality, can be visualised and interpreted for new knowledge discovery and for a better understanding of the data and the processes that generated it. eSTDM can be used for early event prediction due to the ability of the SNN to spike early, before whole input vectors (they were trained on) are presented. A framework for building eSTDM called NeuCube along with a design methodology for building eSTDM using this are presented. The implementation of this framework in MATLAB, Java, and PyNN (Python) is presented. The latter facilitates the use of neuromorphic hardware platforms to run the eSTDM. Selected examples are given of eSTDM for pattern recognition and early event prediction on EEG data, fMRI data, multisensory seismic data, ecological data, climate data, audio-visual data. Future directions are discussed, including extension of the NeuCube framework for building neurogenetic eSTDM and also new applications of eSTDM

Enhancement of Robot-Assisted Rehabilitation Outcomes of Post-Stroke Patients Using Movement-Related Cortical Potential

Post-stroke rehabilitation is essential for stroke survivors to help them regain independence and to improve their quality of life. Among various rehabilitation strategies, robot-assisted rehabilitation is an efficient method that is utilized more and more in clinical practice for motor recovery of post-stroke patients. However, excessive assistance from robotic devices during rehabilitation sessions can make patients perform motor training passively with minimal outcome. Towards the development of an efficient rehabilitation strategy, it is necessary to ensure the active participation of subjects during training sessions. This thesis uses the Electroencephalography (EEG) signal to extract the Movement-Related Cortical Potential (MRCP) pattern to be used as an indicator of the active engagement of stroke patients during rehabilitation training sessions. The MRCP pattern is also utilized in designing an adaptive rehabilitation training strategy that maximizes patients’ engagement. This project focuses on the hand motor recovery of post-stroke patients using the AMADEO rehabilitation device (Tyromotion GmbH, Austria). AMADEO is specifically developed for patients with fingers and hand motor deficits. The variations in brain activity are analyzed by extracting the MRCP pattern from the acquired EEG data during training sessions. Whereas, physical improvement in hand motor abilities is determined by two methods. One is clinical tests namely Fugl-Meyer Assessment (FMA) and Motor Assessment Scale (MAS) which include FMA-wrist, FMA-hand, MAS-hand movements, and MAS-advanced hand movements’ tests. The other method is the measurement of hand-kinematic parameters using the AMADEO assessment tool which contains hand strength measurements during flexion (force-flexion), and extension (force-extension), and Hand Range of Movement (HROM)

Application of a Brain-Inspired Spiking Neural Network Architecture to Odor Data Classification

Existing methods in neuromorphic olfaction mainly focus on implementing the data transformation based on the neurobiological architecture of the olfactory pathway. While the transformation is pivotal for the sparse spike-based representation of odor data, classification techniques based on the bio-computations of the higher brain areas, which process the spiking data for identification of odor, remain largely unexplored. This paper argues that brain-inspired spiking neural networks constitute a promising approach for the next generation of machine intelligence for odor data processing. Inspired by principles of brain information processing, here we propose the first spiking neural network method and associated deep machine learning system for classification of odor data. The paper demonstrates that the proposed approach has several advantages when compared to the current state-of-the-art methods. Based on results obtained using a benchmark dataset, the model achieved a high classification accuracy for a large number of odors and has the capacity for incremental learning on new data. The paper explores different spike encoding algorithms and finds that the most suitable for the task is the step-wise encoding function. Further directions in the brain-inspired study of odor machine classification include investigation of more biologically plausible algorithms for mapping, learning, and interpretation of odor data along with the realization of these algorithms on some highly parallel and low power consuming neuromorphic hardware devices for real-world applications

Personalised modelling with spiking neural networks integrating temporal and static information.

This paper proposes a new personalised prognostic/diagnostic system that supports classification, prediction and pattern recognition when both static and dynamic/spatiotemporal features are presented in a dataset. The system is based on a proposed clustering method (named d2WKNN) for optimal selection of neighbouring samples to an individual with respect to the integration of both static (vector-based) and temporal individual data. The most relevant samples to an individual are selected to train a Personalised Spiking Neural Network (PSNN) that learns from sets of streaming data to capture the space and time association patterns. The generated time-dependant patterns resulted in a higher accuracy of classification/prediction (80% to 93%) when compared with global modelling and conventional methods. In addition, the PSNN models can support interpretability by creating personalised profiling of an individual. This contributes to a better understanding of the interactions between features. Therefore, an end-user can comprehend what interactions in the model have led to a certain decision (outcome). The proposed PSNN model is an analytical tool, applicable to several real-life health applications, where different data domains describe a person's health condition. The system was applied to two case studies: (1) classification of spatiotemporal neuroimaging data for the investigation of individual response to treatment and (2) prediction of risk of stroke with respect to temporal environmental data. For both datasets, besides the temporal data, static health data were also available. The hyper-parameters of the proposed system, including the PSNN models and the d2WKNN clustering parameters, are optimised for each individual

State of the art of audio- and video based solutions for AAL

Working Group 3. Audio- and Video-based AAL ApplicationsIt is a matter of fact that Europe is facing more and more crucial challenges regarding health and social care due to the demographic change and the current economic context. The recent COVID-19 pandemic has stressed this situation even further, thus highlighting the need for taking action. Active and Assisted Living (AAL) technologies come as a viable approach to help facing these challenges, thanks to the high potential they have in enabling remote care and support. Broadly speaking, AAL can be referred to as the use of innovative and advanced Information and Communication Technologies to create supportive, inclusive and empowering applications and environments that enable older, impaired or frail people to live independently and stay active longer in society. AAL capitalizes on the growing pervasiveness and effectiveness of sensing and computing facilities to supply the persons in need with smart assistance, by responding to their necessities of autonomy, independence, comfort, security and safety. The application scenarios addressed by AAL are complex, due to the inherent heterogeneity of the end-user population, their living arrangements, and their physical conditions or impairment. Despite aiming at diverse goals, AAL systems should share some common characteristics. They are designed to provide support in daily life in an invisible, unobtrusive and user-friendly manner. Moreover, they are conceived to be intelligent, to be able to learn and adapt to the requirements and requests of the assisted people, and to synchronise with their specific needs. Nevertheless, to ensure the uptake of AAL in society, potential users must be willing to use AAL applications and to integrate them in their daily environments and lives. In this respect, video- and audio-based AAL applications have several advantages, in terms of unobtrusiveness and information richness. Indeed, cameras and microphones are far less obtrusive with respect to the hindrance other wearable sensors may cause to one’s activities. In addition, a single camera placed in a room can record most of the activities performed in the room, thus replacing many other non-visual sensors. Currently, video-based applications are effective in recognising and monitoring the activities, the movements, and the overall conditions of the assisted individuals as well as to assess their vital parameters (e.g., heart rate, respiratory rate). Similarly, audio sensors have the potential to become one of the most important modalities for interaction with AAL systems, as they can have a large range of sensing, do not require physical presence at a particular location and are physically intangible. Moreover, relevant information about individuals’ activities and health status can derive from processing audio signals (e.g., speech recordings). Nevertheless, as the other side of the coin, cameras and microphones are often perceived as the most intrusive technologies from the viewpoint of the privacy of the monitored individuals. This is due to the richness of the information these technologies convey and the intimate setting where they may be deployed. Solutions able to ensure privacy preservation by context and by design, as well as to ensure high legal and ethical standards are in high demand. After the review of the current state of play and the discussion in GoodBrother, we may claim that the first solutions in this direction are starting to appear in the literature. A multidisciplinary 4 debate among experts and stakeholders is paving the way towards AAL ensuring ergonomics, usability, acceptance and privacy preservation. The DIANA, PAAL, and VisuAAL projects are examples of this fresh approach. This report provides the reader with a review of the most recent advances in audio- and video-based monitoring technologies for AAL. It has been drafted as a collective effort of WG3 to supply an introduction to AAL, its evolution over time and its main functional and technological underpinnings. In this respect, the report contributes to the field with the outline of a new generation of ethical-aware AAL technologies and a proposal for a novel comprehensive taxonomy of AAL systems and applications. Moreover, the report allows non-technical readers to gather an overview of the main components of an AAL system and how these function and interact with the end-users. The report illustrates the state of the art of the most successful AAL applications and functions based on audio and video data, namely (i) lifelogging and self-monitoring, (ii) remote monitoring of vital signs, (iii) emotional state recognition, (iv) food intake monitoring, activity and behaviour recognition, (v) activity and personal assistance, (vi) gesture recognition, (vii) fall detection and prevention, (viii) mobility assessment and frailty recognition, and (ix) cognitive and motor rehabilitation. For these application scenarios, the report illustrates the state of play in terms of scientific advances, available products and research project. The open challenges are also highlighted. The report ends with an overview of the challenges, the hindrances and the opportunities posed by the uptake in real world settings of AAL technologies. In this respect, the report illustrates the current procedural and technological approaches to cope with acceptability, usability and trust in the AAL technology, by surveying strategies and approaches to co-design, to privacy preservation in video and audio data, to transparency and explainability in data processing, and to data transmission and communication. User acceptance and ethical considerations are also debated. Finally, the potentials coming from the silver economy are overviewed.publishedVersio

Diseño de un exoesqueleto para rehabilitación de miembro superior accionado por una interfaz cerebro - máquina

Uno de los temas de creciente preocupación en la actualidad es el envejecimiento de la población mundial. Solo para el 2020, 13 países podrán ser considerados “súperenvejecidos” (aquellos con 20% o más de habitantes de edades mayores a 65 años), mientras que para el 2030 se sumarán otros 21 a la lista [1]. El Perú no está excluido de este fenómeno demográfico, ya que según estadísticas se estima que entre el 2015 y el 2050 el porcentaje de la población mayor de 65 años se incrementará de 6.38% a 15.69% [2]. Uno de los sectores donde el impacto será más notorio es la medicina, ya que diversas dolencias son más frecuentes en la tercera edad. Esta disciplina deberá adecuarse y buscar métodos más eficientes de atención para dar abasto a la creciente demanda. Ante esta problemática, los países más avanzados en el campo de la tecnología están empezando a dar soluciones eficientes a los problemas médicos desde el campo de la robótica y la biomecatrónica. En este ámbito, prótesis robóticas, exoesqueletos de asistencia y rehabilitación, y equipos robotizados para cirugía son algunos de los avances en el campo que prometen alcanzar óptimos resultados. En particular, los exoesqueletos suelen ser usados en la rehabilitación de pacientes con trastornos neuromusculares, como quienes han sufrido de accidente cerebrovascular o lesión de la médula espinal. Este trabajo tiene como objetivo presentar el diseño de un exoesqueleto para brazo con tres grados de libertad, tal que el usuario realice dos tipos de movimientos del hombro (flexión-extensión y abducción-aducción) y la flexión del brazo por la rotación de la articulación del codo. El exoesqueleto estará orientado a la rehabilitación de miembro superior en quienes han sufrido accidente cerebrovascular. El accionamiento del mecanismo será por señales cerebrales obtenidas por una interfaz cerebro-computadora no invasiva. De este modo, cuando el usuario piense en realizar un movimiento con el brazo (asociado a una actividad de la vida diaria), accionará los actuadores y se ejecutará un movimiento predeterminado dentro de un volumen de trabajo seguro. El movimiento contribuirá a la rehabilitación de la misma manera en que se hace con asistencia humana, pero agregando el que el paciente siempre participe activamente, lo cual es un factor motivante que puede estimular a que el paciente concientice el movimiento y estimule la plasticidad del cerebro para la recuperación de su discapacidad. En el capítulo 1 se presenta la problemática asociada con quienes han sufrido de accidente cerebrovascular y requieren de un tratamiento de rehabilitación. En el capítulo 2 se presentan un estudio del estado del arte en exoesqueletos e interfaces cerebro-computadora. En el capítulo 3 se presentan los requerimientos con los que debe cumplir y algunas características deseables para el exoesqueleto, seguidas del concepto óptimo que daría solución a la problemática. En el capítulo 4 se presentan los componentes y planos del diseño mecánico y electrónico, y los diagramas de flujo correspondientes al sistema de control. En el capítulo 5 se presentan los costos de cada componente mecánico y electrónico, así como el precio total de un prototipo del diseño presentado. En el capítulo 6 se presentan las conclusiones con respecto a este trabajo.Tesi

Diseño de un exoesqueleto para rehabilitación de miembro superior accionado por una interfaz cerebro - máquina

Uno de los temas de creciente preocupación en la actualidad es el envejecimiento de la población mundial. Solo para el 2020, 13 países podrán ser considerados “súperenvejecidos” (aquellos con 20% o más de habitantes de edades mayores a 65 años), mientras que para el 2030 se sumarán otros 21 a la lista [1]. El Perú no está excluido de este fenómeno demográfico, ya que según estadísticas se estima que entre el 2015 y el 2050 el porcentaje de la población mayor de 65 años se incrementará de 6.38% a 15.69% [2]. Uno de los sectores donde el impacto será más notorio es la medicina, ya que diversas dolencias son más frecuentes en la tercera edad. Esta disciplina deberá adecuarse y buscar métodos más eficientes de atención para dar abasto a la creciente demanda. Ante esta problemática, los países más avanzados en el campo de la tecnología están empezando a dar soluciones eficientes a los problemas médicos desde el campo de la robótica y la biomecatrónica. En este ámbito, prótesis robóticas, exoesqueletos de asistencia y rehabilitación, y equipos robotizados para cirugía son algunos de los avances en el campo que prometen alcanzar óptimos resultados. En particular, los exoesqueletos suelen ser usados en la rehabilitación de pacientes con trastornos neuromusculares, como quienes han sufrido de accidente cerebrovascular o lesión de la médula espinal. Este trabajo tiene como objetivo presentar el diseño de un exoesqueleto para brazo con tres grados de libertad, tal que el usuario realice dos tipos de movimientos del hombro (flexión-extensión y abducción-aducción) y la flexión del brazo por la rotación de la articulación del codo. El exoesqueleto estará orientado a la rehabilitación de miembro superior en quienes han sufrido accidente cerebrovascular. El accionamiento del mecanismo será por señales cerebrales obtenidas por una interfaz cerebro-computadora no invasiva. De este modo, cuando el usuario piense en realizar un movimiento con el brazo (asociado a una actividad de la vida diaria), accionará los actuadores y se ejecutará un movimiento predeterminado dentro de un volumen de trabajo seguro. El movimiento contribuirá a la rehabilitación de la misma manera en que se hace con asistencia humana, pero agregando el que el paciente siempre participe activamente, lo cual es un factor motivante que puede estimular a que el paciente concientice el movimiento y estimule la plasticidad del cerebro para la recuperación de su discapacidad. En el capítulo 1 se presenta la problemática asociada con quienes han sufrido de accidente cerebrovascular y requieren de un tratamiento de rehabilitación. En el capítulo 2 se presentan un estudio del estado del arte en exoesqueletos e interfaces cerebro-computadora. En el capítulo 3 se presentan los requerimientos con los que debe cumplir y algunas características deseables para el exoesqueleto, seguidas del concepto óptimo que daría solución a la problemática. En el capítulo 4 se presentan los componentes y planos del diseño mecánico y electrónico, y los diagramas de flujo correspondientes al sistema de control. En el capítulo 5 se presentan los costos de cada componente mecánico y electrónico, así como el precio total de un prototipo del diseño presentado. En el capítulo 6 se presentan las conclusiones con respecto a este trabajo.Tesi