Role of Muscle Synergies in Real-Time Classification of Upper Limb Motions using Extreme Learning Machines

10.1186/s12984-016-0183-0Journal of NeuroEngineering and Rehabilitation13118

Tecnologia assistiva: identificação dos requisitos do produto de órteses para membros inferiores - uma visão a partir das percepções dos usuários

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro de Comunicação e Expressão, Programa de Pós-Graduação em Design e Expressão Gráfica, Florianópolis, 2016.Segundo dados da Organização das Nações Unidas 10% da população mundial vive com alguma deficiência. Dados do Instituto Brasileiro de Geografia e Estatística indicam que o Brasil possui mais de quarenta e cinco milhões de cidadãos com algum tipo de deficiência (23,9% da população). Ou seja, praticamente um quarto da população brasileira demanda algum recurso de Tecnologia Assistiva. Especialmente nos últimos quinze anos, o Brasil evoluiu consideravelmente na atenção e suporte às pessoas com deficiência. Contudo, apesar destes significativos avanços, o Brasil ainda está algumas décadas atrasado em relação a outros países. O objetivo desta pesquisa foi identificar os Requisitos do Produto para órteses de membros inferiores a partir das percepções dos usuários. Para tal, investigou-se as necessidades dos usuários, para que estas pudessem ser convertidas em Requisitos dos Usuários e estes em Requisitos do Produto. Através de visitas técnicas a empresas e organizações especializadas na confecção de órteses, juntamente com a aplicação de questionários aos usuários, foi realizado um levantamento que indicasse quais são os requisitos mais importantes, segundo a percepção dos usuários. O questionário aplicado foi o QUEST 2.0 (Quebec User Evaluation of Satisfaction with assistive Technology). O instrumento de coleta, avalia doze aspectos, sendo oito relativos aos recursos (1.Dimensões, 2.Peso, 3.Facilidade de ajuste, 4.Segurança, 5.Durabilidade, 6.Facilidade de uso, 7.Conforto, 8.Eficácia) e quatro relacionados aos serviços associados (9.Processo de entrega, 10. Assistência técnica, 11.Serviços profissionais e 12.Serviços de acompanhamento). Em função do objetivo principal desta pesquisa, foi depositada maior atenção aos oito primeiros aspectos. Ainda que a média final de satisfação dos usuários tenha atingido um valor mediano para avançado (3,82), as variações entre os participantes, bem como os comentários complementares registrados, expõem uma variada lista de necessidades não contempladas. Quantitativamente, os aspectos Conforto, Segurança, Peso e Durabilidade foram aqueles que propiciaram o maior volume de problemas registrados. Ainda que se tenha definido uma hierarquia para os requisitos, vale ressaltar que todos os problemas registrados são muito importantes para os usuários. Dentre os problemas registrados, vale ressaltar aqueles ligados aos relatos de quadros de dor e surgimento de escaras. Certos usuários relataram momentos em que atingiam um nível insuportável de dor, fazendo com que abandonassem o uso da órtese, temporariamente ou mesmo por longos períodos. Estes casos evidenciam o grau de importância do desenvolvimento de melhorias nas órteses para membros inferiores. Oferecendo dados e informações, este estudo sugere alguns caminhos para a ampliação da inclusão das pessoas com deficiência em nossa sociedade, de forma plena e irrestrita. A pesquisa desenvolvida e relatada neste documento pode servir de embasamento para outros estudos futuros, visto que, após a definição dos Requisitos do Produto, o desenvolvimento estratégico de produto prevê a definição das Especificações de Projeto, assim como o Projeto de Produto propriamente dito.Abstract : According to the United Nations 10% of the world's population lives with a disability. Data from the Brazilian Institute of Geography and Statistics indicate that Brazil has more than forty-five million people with a disability (23.9% of the population). That is, nearly a quarter of the population takes some Assistive Technology feature. Especially in the last fifteen years, Brazil has evolved considerably in the care and support to people with disabilities. However, despite these significant advances, Brazil is still a few decades late compared to other countries. The objective of this research was to identify the product requirements for lower limb orthoses from the perceptions of users. To this end, were investigated the needs of users, so that they could be converted to User Requirements and these in Product Requirements. Through technical visits to companies and organizations specializing in orthosis, along with questionnaires to users, a survey was conducted to indicate what are the most important requirements, as perceived by the users. The questionnaire was the QUEST 2.0 (Quebec User Evaluation of Satisfaction with assistive Technology). The instrument evaluates twelve aspects, eight related to resources (1. Dimensions, 2. Weight, 3. Adjustment facility, 4. Safety, 5. Durability, 6. Ease of use, 7. Comfort, 8. Effectiveness) and four related to associated services (9. Delivery Process, 10. Technical assistance, 11. Professional services, and 12. Monitoring services). Based on the primary objective of this research it was dedicated more attention to the first eight aspects. Although the average end user satisfaction has reached a median for advanced (3.82), the variations between participants and registered further comment, expose a varied list of needs not covered. Quantitatively aspects Comfort, Safety, Weight and Durability were those that provided the bulk of recorded problems. Even if set a hierarchy of requirements, it is noteworthy that all registered issues are very important for users. Among the reported problems, it is worth mentioning those related to reports of pain cases and appearance of bedsores. Some users have reported times when they reached an unbearable level of pain, causing them to abandon the use of bracing, temporarily or for long periods. These cases show how important the development of improvements in orthoses for lower limbs. Providing data and information, this study suggests some ways to expand the inclusion of people with disabilities in our society, in full and without restrictions. The developed and research reported in this document can serve as a basis for other future studies, since, after the definition of product requirements, product development strategy provides for the definition of project specifications, as well as the product design itself

Towards the application of multi-DOF EMG-driven neuromusculoskeletal modeling in clinical practice: methodological aspects

New methods able to assess the individual ability of patients to generate motion and adaptation strategies are increasingly required for clinical applications aiming at recovering motor functions. Indeed, more effective rehabilitation treatments are designed to be personalized on the subject capabilities. In this context, neuromusculoskeletal (NMS) models represent a valuable tool, as they can provide important information about the unique anatomical, neurological, and functional characteristics of different subjects, through the computation of human internal variables, such as muscle activations, muscle forces, joint contact forces and moments. A first possible approach is to estimate these values using optimization-based NMS models. However, these models require to make assumptions on how the muscles contribute to the observed movement. More promising are instead NMS models driven by electromyographic signals (EMG), which use experimentally recorded signals that can be considered a direct representation of the subject motor intentions. This allows to account for the actual differences in an individual neuromuscular control system, without making any preliminary assumptions. Therefore these models have the potentialities to provide the level of personalization that is essential for applications in the clinical field. Although EMG-driven NMS models have been investigated in the literature, even for clinical purposes, they are mostly limited to one degree of freedom (DOF), and consider only the muscles spanning that DOF. Additionally, despite the promising results, they are still not introduced in the clinical practice; the main reason possibly being their complexity, that makes them not usable in clinical context, where standard and reliable procedures are required. The importance of EMG-driven NMS modeling for clinical applications would be even higher with the availability of multi-DOF models, as impairments usually compromise multiple joints. Nevertheless, even if a first multi-DOF EMG-driven NMS model for the lower limbs has been recently introduced in literature, its even greater complexity makes more difficult an analysis of its applicability in the clinical field. This work represents a first effort towards a critical analysis of multi-DOF EMG-driven NMS models to evaluate their possible use in clinical practice. To achieve this objective, several issues and limitations have been addressed. In the specific, the attention has been focused on two aspects: (i) making the methodology usable, to foster its adoption by multiple laboratories and research groups, and to facilitate sensitivity analyses required to assess its accuracy; (ii) highlighting the effects of some methodological aspects related to data acquisition and processing, and evaluating their impact on the accuracy of estimated parameters and muscle forces. This analysis is even more important for multi-DOF EMG-driven NMS model as it is still not present in the literature. To accomplish the first goal, a software tool (MOtoNMS) has been developed and it is freely available for the research community. It is a complete, flexible, and user-friendly tool that allows to automatically process experimental motion data from different laboratories in a transparent and repeatable way, for their subsequent use with neuromusculoskeletal modeling software. MOtoNMS generalizes data processing methods across laboratories, and simplifies and speeds up the demanding data elaboration workflow. This simplification represents an indispensable step towards an actual translation of NMS methods in clinical practice. The second part of the work has been, instead, dedicated to analyze the impact on model parameters and muscle forces prediction of different techniques for EMG data collection and processing that are feasible for clinical settings, in particular concentrating on EMGs normalization. Indeed, moving EMG-driven NMS modeling towards clinical applications that deal with multiple DOFs requires to carefully consider subject's motor limitations due to his/her mobility impairments. This results in a rethinking about the methodologies for data acquisition and processing. Therefore, the impact of using only data from walking trials on both calibration of model parameters and computing the maximum EMG values needed for the normalization step, has been assessed with two case studies. Moreover, a protocol for the collection of maximum voluntary contractions has been proposed. This protocol is suitable for multiple DOFs applications involving patients with reduced motor ability and it requires only low-cost and easy to acquire tools to make it applicable in any laboratory. The research proposed in this thesis provides tools to simplify the use of multi-DOF EMG-driven neuromusculoskeletal models and proposes analyses and procedures to evaluate the accuracy and reliability of the obtained results with the aim of pursuing clinical applications

Subject-Independent Frameworks for Robotic Devices: Applying Robot Learning to EMG Signals

The capability of having human and robots cooperating together has increased the interest in the control of robotic devices by means of physiological human signals. In order to achieve this goal it is crucial to be able to catch the human intention of movement and to translate it in a coherent robot action. Up to now, the classical approach when considering physiological signals, and in particular EMG signals, is to focus on the specific subject performing the task since the great complexity of these signals. This thesis aims to expand the state of the art by proposing a general subject-independent framework, able to extract the common constraints of human movement by looking at several demonstration by many different subjects. The variability introduced in the system by multiple demonstrations from many different subjects allows the construction of a robust model of human movement, able to face small variations and signal deterioration. Furthermore, the obtained framework could be used by any subject with no need for long training sessions. The signals undergo to an accurate preprocessing phase, in order to remove noise and artefacts. Following this procedure, we are able to extract significant information to be used in online processes. The human movement can be estimated by using well-established statistical methods in Robot Programming by Demonstration applications, in particular the input can be modelled by using a Gaussian Mixture Model (GMM). The performed movement can be continuously estimated with a Gaussian Mixture Regression (GMR) technique, or it can be identified among a set of possible movements with a Gaussian Mixture Classification (GMC) approach. We improved the results by incorporating some previous information in the model, in order to enriching the knowledge of the system. In particular we considered the hierarchical information provided by a quantitative taxonomy of hand grasps. Thus, we developed the first quantitative taxonomy of hand grasps considering both muscular and kinematic information from 40 subjects. The results proved the feasibility of a subject-independent framework, even by considering physiological signals, like EMG, from a wide number of participants. The proposed solution has been used in two different kinds of applications: (I) for the control of prosthesis devices, and (II) in an Industry 4.0 facility, in order to allow human and robot to work alongside or to cooperate. Indeed, a crucial aspect for making human and robots working together is their mutual knowledge and anticipation of other’s task, and physiological signals are capable to provide a signal even before the movement is started. In this thesis we proposed also an application of Robot Programming by Demonstration in a real industrial facility, in order to optimize the production of electric motor coils. The task was part of the European Robotic Challenge (EuRoC), and the goal was divided in phases of increasing complexity. This solution exploits Machine Learning algorithms, like GMM, and the robustness was assured by considering demonstration of the task from many subjects. We have been able to apply an advanced research topic to a real factory, achieving promising results

Towards Remote Gait Analysis: Combining Physics and Probabilistic Models for Estimating Human Joint Mechanics

The connected health movement and remote patient monitoring promise to revolutionize patient care in multiple clinical contexts. In orthopedics, continuous monitoring of human joint and muscle tissue loading in free-living conditions will enable novel insight concerning musculoskeletal disease etiology. These developments are necessary for comprehensive patient characterization, progression monitoring, and personalized therapy. This vision has motivated many recent advances in wearable sensor-based algorithm development that aim to perform biomechanical analyses traditionally restricted to confined laboratory spaces. However, these techniques have not translated to practical deployment for remote monitoring. Several barriers to translation have been identified including complex sensor arrays. Thus, the aim of this work was to lay the foundation for remote gait analysis and techniques for estimating clinically relevant biomechanics with a reduced sensor array. The first step in this process was to develop an open-source platform that generalized the processing pipeline for automated remote biomechanical analysis. The clinical utility of the platform was demonstrated for monitoring patient gait following knee surgery using continuous recordings of thighworn accelerometer data and rectus femoris electromyograms (EMG) during free-living conditions. Individual walking bouts were identified from which strides were extracted and characterized for patient evaluation. A novel, multifactorial asymmetry index was proposed based on temporal, EMG, and kinematic descriptors of gait that was able to differentiate between patients at different stages of recovery and that was more sensitive to recovery time than were indices of cumulative physical activity. The remainder of the work focused on algorithms for estimating joint moment and simulating muscle contraction dynamics using a reduced sensor array. A hybrid technique was proposed that combined both physics and probabilistic models in a complementary fashion. Specifically, the notion of a muscle synergy function was introduced that describes the mapping between excitations from a subset of muscles and excitations from other synergistic muscles. A novel model of these synergy functions was developed that enabled estimation of unmeasured muscle excitations using a measured subset. Data from thigh- and shank-worn inertial sensors were used to estimate segment kinematics and muscle-tendon unit (MTU) lengths using physics-based techniques and a model of the musculoskeletal geometry. These estimates of muscle excitation and MTU length were used as inputs for EMG-driven simulation of muscle contraction. Estimates of muscle force, power, and work as well as net joint moment from the proposed hybrid technique were compared to estimates from laboratory-based techniques. This presents the first sensor-only (four EMG and two inertial sensors) simulation of muscle contraction dynamics and joint moment estimation using machine learning only for estimating unmeasured muscle excitations. This work provides the basis for automated remote biomechanical analysis with reduced sensor arrays; from raw sensor recordings to estimates of muscle moment, force, and power. The proposed hybrid technique requires data from only four EMG and two inertial sensors and work has begun to seamlessly integrate these sensors into a knee brace for monitoring patients following knee surgery. Future work should build on these developments including further validation and design of methods utilizing remotely and longitudinally observed biomechanics for prognosis and optimizing patient-specific interventions

A multilevel framework to measure, model, promote, and enhance the symbiotic cooperation between humans and robotic devices

In the latest decades, the common perception about the role of robotic devices in the modern society dramatically changed. In the early stages of robotics, temporally located in the years of the economic boom, the development of new devices was driven by the industrial need of producing more while reducing production time and costs. The demand was, therefore, for robotic devices capable of substituting the humans in performing simple and repetitive activities. The execution of predefined basic activities in the shortest amount of time, inside carefully engineered and confined environments, was the mission of robotic devices. Beside the results obtained in the industrial sector, a progressive widening of the fields interested in robotics – such as rehabilitation, elderly care, and medicine – led to the current vision of the device role. Indeed, these challenging fields require the robot to be a partner, which works side-by-side with the human. Therefore, the device needs to be capable of actively and efficiently interacting with humans, to provide support and overcome their limits in the execution of shared activities, even in highly unpredictable everyday environments. Highly complex and advanced robots, such as surgical robots, rehabilitation devices, flexible manipulators, and service and companion robots, have been recently introduced into the market; despite their complexity, however, they are still tools to be used to perform, better or faster, very specific tasks. The current open challenge is, therefore, to develop a new generation of symbiotically cooperative robotic partners, adding to the devices the capability to detect, understand, and adapt to the real intentions, capabilities, and needs of the humans. To achieve this goal, a bidirectional information channel shall be built to connect the human and the device. In one direction, the device requires to be informed about the state of its user; in the other direction, the human needs to be informed about the state of the whole interacting system. This work reports the research activities that I conducted during my PhD studies in this research direction. Those activities led to the design, development, and assessment on a real application of an innovative multilevel framework to close the cooperation loop between a human and a robotic device, thus promoting and enhancing their symbiotic interaction. Three main levels have been identified as core elements to close this loop: the measure level, the model level, and the extract/synthesize level. The former aims at collecting experimental measures from the whole interacting system; the second aims at estimating and predicting its dynamic behavior; the last aims at providing quantitative information to both the human and the device about their performances and about how to modify their behavior to improve their interaction symbiosis. Within the measure level, the focus has been concentrated on investigating, critically comparing, and selecting the most suitable and advanced technologies to measure kinematics and dynamics quantities in a portable and minimally intrusive way. Particular attention has been paid to new emerging technologies; moreover, useful protocols and pipelines already recognized as de-facto in other fields have been successfully adapted to fit the needs of the man-machine interaction context. Finally, the design of a new sensor has been started to overcome the lack of tools capable of effectively measuring human-device interaction forces. To implement the model level, a common platform to perform integrated multilevel simulations – i.e. simulations where the device and the human are considered together as interacting entities – has been selected and extensively validated. Furthermore, critical aspects characterizing the modeling of the device, the human, and their interactions have been studied and possible solutions have been proposed. For example, modeling the mechanics and the control within the selected software platform allowed accurate estimations of their behavior. To estimate human behavior, new methodologies and approaches based on anatomical neuromusculoskeletal models have been developed, validated, and released as open-source tools for the community, to allow accurate estimates of both kinematics and dynamics at run-time – i.e. at the same time that the movements are performed. An inverse kinematics approach has been developed and validated to estimate human joint angles from the orientation measurements provided by wearable inertial systems. Additionally, a state of the art neuromusculoskeletal modeling toolbox has been improved and interfaced with the other tools of the multilevel framework, to accurately predict human muscle forces, joint moments, and muscle and joint stiffness from electromyographic and kinematic measures. To estimate and predict the interactions, contact models, parameters optimization procedures, and high-level cooperation strategies have been investigated, developed, and applied. Within the extract/synthesize level, the information provided by the other levels has been combined together to develop informative feedbacks for both the device and the human. In one direction, the device has been provided with control signals defining how to adjust the provided support to comply with the task goals and with the human current capabilities and needs. In the other direction, quantitative feedbacks have been developed to inform the human about task execution performances, task targets, and support provided by the device. This information has been provided to the user as visual feedbacks designed to be both exhaustively informative and minimally distractive, to prevent possible loss of focus. Moreover, additional feedbacks have been devised to help external observers – therapists in the rehabilitation contexts or task planners and ergonomists in the industrial field – in the design and refinement of effective personalized tasks and long-term goals. The integration of all the hardware and software tools of each level in a modular, flexible, and reliable software framework, based on a well known robotic middleware, has been fundamental to handle the communication and information exchange processes. The developed general framework has been finally specialized to face the specific needs of robotic-aided gait rehabilitation. In this context, indeed, the final aim of promoting the symbiotic cooperation is translatable in maximizing treatment effectiveness for the patients by actively supporting their changing needs and capabilities while keeping them engaged during the whole rehabilitation process. The proposed multilevel framework specialization has been successfully used, as valuable answer to those needs, within the context of the Biomot European project. It has been, indeed, fundamental to face the challenges of closing the informative loop between the user and the device, and providing valuable quantitative information to the external observers. Within this research project, we developed an innovative compliant wearable exoskeleton prototype for gait rehabilitation capable of adjusting, at run-time, the provided support according to different cooperation strategies and to user needs and capabilities. At the same time, the wearer is also engaged in the rehabilitation process by intuitive visual feedbacks about his performances in the achievement of the rehabilitation targets and about the exoskeleton support. Both researchers and clinical experts evaluating the final rehabilitation application of the multilevel framework provided enthusiastic feedbacks about the proposed solutions and the obtained results. To conclude, the modular and generic multilevel framework developed in this thesis has the potential to push forward the current state of the art in the applications where a symbiotic cooperation between robotic devices and humans is required. Indeed, it effectively endorses the development of a new generation of robotic devices capable to perform challenging cooperative tasks in highly unpredictable environments while complying with the current needs, intentions, and capabilities of the human

Multibody dynamics 2015

This volume contains the full papers accepted for presentation at the ECCOMAS Thematic Conference on Multibody Dynamics 2015 held in the Barcelona School of Industrial Engineering, Universitat Politècnica de Catalunya, on June 29 - July 2, 2015. The ECCOMAS Thematic Conference on Multibody Dynamics is an international meeting held once every two years in a European country. Continuing the very successful series of past conferences that have been organized in Lisbon (2003), Madrid (2005), Milan (2007), Warsaw (2009), Brussels (2011) and Zagreb (2013); this edition will once again serve as a meeting point for the international researchers, scientists and experts from academia, research laboratories and industry working in the area of multibody dynamics. Applications are related to many fields of contemporary engineering, such as vehicle and railway systems, aeronautical and space vehicles, robotic manipulators, mechatronic and autonomous systems, smart structures, biomechanical systems and nanotechnologies. The topics of the conference include, but are not restricted to: Formulations and Numerical Methods, Efficient Methods and Real-Time Applications, Flexible Multibody Dynamics, Contact Dynamics and Constraints, Multiphysics and Coupled Problems, Control and Optimization, Software Development and Computer Technology, Aerospace and Maritime Applications, Biomechanics, Railroad Vehicle Dynamics, Road Vehicle Dynamics, Robotics, Benchmark Problems. The conference is organized by the Department of Mechanical Engineering of the Universitat Politècnica de Catalunya (UPC) in Barcelona. The organizers would like to thank the authors for submitting their contributions, the keynote lecturers for accepting the invitation and for the quality of their talks, the awards and scientific committees for their support to the organization of the conference, and finally the topic organizers for reviewing all extended abstracts and selecting the awards nominees.Postprint (published version

Modeling the Human Knee for Assistive Technologies

In this paper we use motion capture technology together with an electromyography (EMG) driven musculoskeletal model of the knee joint to predict muscle behavior during human dynamic movements. We propose a muscle model based on infinitely-stiff tendons and show this allows speeding up 250 times the computation of muscle force and the resulting joint moment calculation with no loss of accuracy with respect to the previously developed elastic tendon model. We then integrate our previously developed method for the estimation of 3D musculotendon kinematics in the proposed EMG-driven model. This new code enabled the creation of a stand-alone EMG-driven model that was implemented and run on an embedded system for applications in assistive technologies such as myoelectrically controlled prostheses and orthoses