A survey on bio-signal analysis for human-robot interaction

The use of bio-signals analysis in human-robot interaction is rapidly increasing. There is an urgent demand for it in various applications, including health care, rehabilitation, research, technology, and manufacturing. Despite several state-of-the-art bio-signals analyses in human-robot interaction (HRI) research, it is unclear which one is the best. In this paper, the following topics will be discussed: robotic systems should be given priority in the rehabilitation and aid of amputees and disabled people; second, domains of feature extraction approaches now in use, which are divided into three main sections (time, frequency, and time-frequency). The various domains will be discussed, then a discussion of each domain's benefits and drawbacks, and finally, a recommendation for a new strategy for robotic systems

Feature Analysis for Classification of Physical Actions using surface EMG Data

Based on recent health statistics, there are several thousands of people with limb disability and gait disorders that require a medical assistance. A robot assisted rehabilitation therapy can help them recover and return to a normal life. In this scenario, a successful methodology is to use the EMG signal based information to control the support robotics. For this mechanism to function properly, the EMG signal from the muscles has to be sensed and then the biological motor intention has to be decoded and finally the resulting information has to be communicated to the controller of the robot. An accurate detection of the motor intention requires a pattern recognition based categorical identification. Hence in this paper, we propose an improved classification framework by identification of the relevant features that drive the pattern recognition algorithm. Major contributions include a set of modified spectral moment based features and another relevant inter-channel correlation feature that contribute to an improved classification performance. Next, we conducted a sensitivity analysis of the classification algorithm to different EMG channels. Finally, the classifier performance is compared to that of the other state-of the art algorithm

Interpreting Deep Learning Features for Myoelectric Control: A Comparison with Handcrafted Features

The research in myoelectric control systems primarily focuses on extracting discriminative representations from the electromyographic (EMG) signal by designing handcrafted features. Recently, deep learning techniques have been applied to the challenging task of EMG-based gesture recognition. The adoption of these techniques slowly shifts the focus from feature engineering to feature learning. However, the black-box nature of deep learning makes it hard to understand the type of information learned by the network and how it relates to handcrafted features. Additionally, due to the high variability in EMG recordings between participants, deep features tend to generalize poorly across subjects using standard training methods. Consequently, this work introduces a new multi-domain learning algorithm, named ADANN, which significantly enhances (p=0.00004) inter-subject classification accuracy by an average of 19.40% compared to standard training. Using ADANN-generated features, the main contribution of this work is to provide the first topological data analysis of EMG-based gesture recognition for the characterisation of the information encoded within a deep network, using handcrafted features as landmarks. This analysis reveals that handcrafted features and the learned features (in the earlier layers) both try to discriminate between all gestures, but do not encode the same information to do so. Furthermore, using convolutional network visualization techniques reveal that learned features tend to ignore the most activated channel during gesture contraction, which is in stark contrast with the prevalence of handcrafted features designed to capture amplitude information. Overall, this work paves the way for hybrid feature sets by providing a clear guideline of complementary information encoded within learned and handcrafted features.Comment: The first two authors shared first authorship. The last three authors shared senior authorship. 32 page

Automatic Pain Assessment by Learning from Multiple Biopotentials

Kivun täsmällinen arviointi on tärkeää kivunhallinnassa, erityisesti sairaan- hoitoa vaativille ipupotilaille. Kipu on subjektiivista, sillä se ei ole pelkästään aistituntemus, vaan siihen saattaa liittyä myös tunnekokemuksia. Tällöin itsearviointiin perustuvat kipuasteikot ovat tärkein työkalu, niin auan kun potilas pystyy kokemuksensa arvioimaan. Arviointi on kuitenkin haasteellista potilailla, jotka eivät itse pysty kertomaan kivustaan. Kliinisessä hoito- työssä kipua pyritään objektiivisesti arvioimaan esimerkiksi havainnoimalla fysiologisia muuttujia kuten sykettä ja käyttäytymistä esimerkiksi potilaan kasvonilmeiden perusteella. Tutkimuksen päätavoitteena on automatisoida arviointiprosessi hyödyntämällä koneoppimismenetelmiä yhdessä biosignaalien prosessointnin kanssa. Tavoitteen saavuttamiseksi mitattiin autonomista keskushermoston toimintaa kuvastavia biopotentiaaleja: sydänsähkökäyrää, galvaanista ihoreaktiota ja kasvolihasliikkeitä mittaavaa lihassähkökäyrää. Mittaukset tehtiin terveillä vapaaehtoisilla, joille aiheutettiin kokeellista kipuärsykettä. Järestelmän kehittämiseen tarvittavaa tietokantaa varten rakennettiin biopotentiaaleja keräävä Internet of Things -pohjainen tallennusjärjestelmä. Koostetun tietokannan avulla kehitettiin biosignaaleille prosessointimenetelmä jatku- vaan kivun arviointiin. Signaaleista eroteltiin piirteitä sekuntitasoon mukautetuilla aikaikkunoilla. Piirteet visualisoitiin ja tarkasteltiin eri luokittelijoilla kivun ja kiputason tunnistamiseksi. Parhailla luokittelumenetelmillä saavutettiin kivuntunnistukseen 90% herkkyyskyky (sensitivity) ja 84% erottelukyky (specificity) ja kivun voimakkuuden arviointiin 62,5% tarkkuus (accuracy). Tulokset vahvistavat kyseisen käsittelytavan käyttökelpoisuuden erityis- esti tunnistettaessa kipua yksittäisessä arviointi-ikkunassa. Tutkimus vahvistaa biopotentiaalien avulla kehitettävän automatisoidun kivun arvioinnin toteutettavuuden kokeellisella kivulla, rohkaisten etenemään todellisen kivun tutkimiseen samoilla menetelmillä. Menetelmää kehitettäessä suoritettiin lisäksi vertailua ja yhteenvetoa automaattiseen kivuntunnistukseen kehitettyjen eri tutkimusten välisistä samankaltaisuuksista ja eroista. Tarkastelussa löytyi signaalien eroavaisuuksien lisäksi tutkimusmuotojen aiheuttamaa eroa arviointitavoitteisiin, mikä hankaloitti tutkimusten vertailua. Lisäksi pohdit- tiin mitkä perinteisten prosessointitapojen osiot rajoittavat tai edistävät ennustekykyä ja miten, sekä tuoko optimointi läpimurtoa järjestelmän näkökulmasta.Accurate pain assessment plays an important role in proper pain management, especially among hospitalized people experience acute pain. Pain is subjective in nature which is not only a sensory feeling but could also combine affective factors. Therefore self-report pain scales are the main assessment tools as long as patients are able to self-report. However, it remains a challenge to assess the pain from the patients who cannot self-report. In clinical practice, physiological parameters like heart rate and pain behaviors including facial expressions are observed as empirical references to infer pain objectively. The main aim of this study is to automate such process by leveraging machine learning methods and biosignal processing. To achieve this goal, biopotentials reflecting autonomic nervous system activities including electrocardiogram and galvanic skin response, and facial expressions measured with facial electromyograms were recorded from healthy volunteers undergoing experimental pain stimulus. IoT-enabled biopotential acquisition systems were developed to build the database aiming at providing compact and wearable solutions. Using the database, a biosignal processing flow was developed for continuous pain estimation. Signal features were extracted with customized time window lengths and updated every second. The extracted features were visualized and fed into multiple classifiers trained to estimate the presence of pain and pain intensity separately. Among the tested classifiers, the best pain presence estimating sensitivity achieved was 90% (specificity 84%) and the best pain intensity estimation accuracy achieved was 62.5%. The results show the validity of the proposed processing flow, especially in pain presence estimation at window level. This study adds one more piece of evidence on the feasibility of developing an automatic pain assessment tool from biopotentials, thus providing the confidence to move forward to real pain cases. In addition to the method development, the similarities and differences between automatic pain assessment studies were compared and summarized. It was found that in addition to the diversity of signals, the estimation goals also differed as a result of different study designs which made cross dataset comparison challenging. We also tried to discuss which parts in the classical processing flow would limit or boost the prediction performance and whether optimization can bring a breakthrough from the system’s perspective

Neuromuscular Control Modeling: from Physics to Data-Driven Approaches

Il controllo neurale della postura umana è stato investigato a partire da un punto di vista fisico. Il paradigma di controllo intermittente è stato usato allo scopo di capire il peso di quest'ultimo nella generazione delle traiettorie del centro di pressione. Un primo contributo di questo lavoro rigurda quindi l'analisi del centro di pressione generato dal suddetto modello biomeccanico attraverso l'extended detrended fluctuation analysis, recentemente proposta in letteratura. Le proprietà di correlazione a lungo termine e disomogeneità sono risultate strettamente legate al guadagno derivativo del modello di controllo intermittente e anche al grado di intermittenza. Il paradigma di controllo è stato poi esteso verso un sistema biomeccanico più complesso, cioè un pendolo inverso doppio link con controllo intermittente alla caviglia. I contributi più significativi hanno riguardato la modellazione matematica del centro di pressione per una struttura multi-link e la verifica della sua plausibilità fisiologica. Si è poi preso in considerazione il caso della postura perturbata, integrando aspetti cinematici, dinamici e relativi all'attività muscolare. A tal fine, si è utilizzato sia un approccio fisico che basato su dati per l'identificazione dei modelli a struttura variabile Si sono prese in considerazione differenti condizioni di sperimentali, e in tutti i casi l'approccio utilizzato ha garantito un adeguato grado di interpretabilità riguardo il ruolo del sistema nervoso centrale nella regolazione del postura eretta in condizioni perturbate. La seconda parte della tesi ha riguardato la caratterizzazione del controllo motorio attraverso il segnale elettromiografico di superfice. Il primo contributo ha riguardato l'identifcazione dell'onset muscolare in condizioni di basso rapporto segnale rumore, sfruttando operatori energetico di tipo Teager-Kaiser al fine del precondizionamento del segnale mioelettrico. La versioe estesa di questo tipo di operatori è risultata particolarmente utile al miglioramento delle performance di numerosi algoritmi di detection. Si è poi proseguito con l'utilizzo di tali segnali al fine della classificazione dei gesti dell'arto superiore. In particoalre si è prerso in considerazione il problema della motion intention detection dei principali movimenti della spalla , utilizzando sia descrittori del segnale elettromiografico nel dominio del tempo e della frequenza. Quest'ultimo aspetto risulta essere un elemento di novità nel contesto scientifico in quanto si sono considerati il riconoscimento l'intezioni di movimento di otto gesti della spalla con particolare attenzione al ruolo dei descrittori del segnale per la classificazione. Infine, con approcci simili, si è preso in considerazione il problema del riconoscimento della scrittura manuale a partire dal dato elettromiografico. Tale aspetto risulta poco investigato sotto la prospettiva della pattern recognition mioelettrica, ma la sua valenza è data dalla crescente richiesta di interfacce uomo-macchina per compiti riabilitativi che coinvolgono una componente cognitiva significativa, Inoltre, vista la tendenza ad investigare il ruolo del polso per il prelievo del segnale elettromiografico al fine della realizzazione delle suddette interfacce, si è analizzato l'utilizzo dei segnali elettromiografici del polso rispetto a quelli dell'avambraccio al fine di predirre le cifre scritte dall'utente, noto che l'avambraccio risulta essere la zona di prelievo più comunemente utilizzata.The biomechanics and the neural control of the human stance was investigated starting from a physical point of view. In particular the intermittent motor control paradigm was investigated with the aim of understanding how such paradigm mirrors in the center of pressure (COP) trajectories. A first contribution given in this work of thesis regards the analysis of COP generated from intermittent controlled inverted pendulum through the extended detrended fluctuation analysis, which was recently introduced in the literature. It has been found that the long-term correlation and inohmogeniety properties of the COP time series strictly depend on the derivative gain term of the intermittent controller and on the degree of intermittency of the control action. Thus, , another contribution provided in this work of thesis regards the use of a more complex biomechanical model of the stance, e.g. a double-link inverted pendulum intermittently controlled at the ankle. In terms of novelty, it deserves to be pointed out the results regarding the mechanical modeling of the COP for a multi-link structure, and the assessment of its physiological plausibility. . On the other hand, when the perturbed posture motor task was taken into account, there was the need to enlarge the perspective, integrating kinematic, dynamic and muscle activity data. The idea of employing different sources of information to develop models of the CNS represents an important element that was investigated using tools related to hybrid system identification theory. Subjects underwent to impulsive support base translations in three different conditions: considering eyes open, closed, and performing mental counting. Although such data were essentially analyzed through a data-driven approach, the identified models guaranteed physical interpretations of the role played by the CNS in the three different conditions. The second main core of this thesis regards the characterization of the motor control using the surface electromyographic (sEMG) signals. A first contribution given in this work regarded the muscle onset detection considering low SNR scenarios. In this framework, energy operators such as the Teager-Kaiser energy operator (TKEO) and its extended version (ETKEO) were investigated as signal preconditioning steps before the application of state of the art onset detection algorithms. The latter have been significantly boosted when ETKEO was used with respect to TKEO. The use of extended energy operators for the sEMG signal preprocessing constitutes a novel element in this field that can be also further investigated in future studies. From the sEMG muscle, one can also predict which movement the subject is going to perform. This aspect can be enclosed in the motion intention detection (MID) field. In this thesis a MID problem was investigated by taking into account two important aspects: as first the study was centered on the shoulder joint movements. Secondly, the MID problem was faced under a pattern recognition perspective. This allowed to verify whether methodologies encountered in the myoelectric hand gesture recognition can be transferred in the affine field of MID In contrast to what reported in the literature, where MID problems generally consider only few movements, in this work of thesis up to eight shoulder movements have been investigated. Myoelectric pattern recognition architectures were also used in the assessment of the ten hand-written digits. Despite the handwriting can be considered a hand movement that involves fine muscular control actions, it has not been consistently investigated in the field of sEMG based hand gesture recognition. Further, since the literature supports the change from forearm to wrist in order to acquire EMG data for hand gesture recognition, it was investigated whether such exchange can be performed when a challenging classification task, as the handwriting recognition has to be performed

Techniques of EMG signal analysis: detection, processing, classification and applications

Electromyography (EMG) signals can be used for clinical/biomedical applications, Evolvable Hardware Chip (EHW) development, and modern human computer interaction. EMG signals acquired from muscles require advanced methods for detection, decomposition, processing, and classification. The purpose of this paper is to illustrate the various methodologies and algorithms for EMG signal analysis to provide efficient and effective ways of understanding the signal and its nature. We further point up some of the hardware implementations using EMG focusing on applications related to prosthetic hand control, grasp recognition, and human computer interaction. A comparison study is also given to show performance of various EMG signal analysis methods. This paper provides researchers a good understanding of EMG signal and its analysis procedures. This knowledge will help them develop more powerful, flexible, and efficient applications