STUDY OF HAND GESTURE RECOGNITION AND CLASSIFICATION

To recognize different hand gestures and achieve efficient classification to understand static and dynamic hand movements used for communications.Static and dynamic hand movements are first captured using gesture recognition devices including Kinect device, hand movement sensors, connecting electrodes, and accelerometers. These gestures are processed using hand gesture recognition algorithms such as multivariate fuzzy decision tree, hidden Markov models (HMM), dynamic time warping framework, latent regression forest, support vector machine, and surface electromyogram. Hand movements made by both single and double hands are captured by gesture capture devices with proper illumination conditions. These captured gestures are processed for occlusions and fingers close interactions for identification of right gesture and to classify the gesture and ignore the intermittent gestures. Real-time hand gestures recognition needs robust algorithms like HMM to detect only the intended gesture. Classified gestures are then compared for the effectiveness with training and tested standard datasets like sign language alphabets and KTH datasets. Hand gesture recognition plays a very important role in some of the applications such as sign language recognition, robotics, television control, rehabilitation, and music orchestration

An Interactive Digital-Twin Model for Virtual Reality Environments to Train in the Use of a Sensorized Upper-Limb Prosthesis

In recent years, the boost in the development of hardware and software resources for building virtual reality environments has fuelled the development of tools to support training in different disciplines. The purpose of this work is to discuss a complete methodology and the supporting algorithms to develop a virtual reality environment to train the use of a sensorized upper-limb prosthesis targeted at amputees. The environment is based on the definition of a digital twin of a virtual prosthesis, able to communicate with the sensors worn by the user and reproduce its dynamic behaviour and the interaction with virtual objects. Several training tasks are developed according to standards, including the Southampton Hand Assessment Procedure, and the usability of the entire system is evaluated, too

Evaluating the influence of subject-related variables on EMG-based hand gesture classification

In this study we evaluated the effect of subjectrelated variables, i.e. hand dominance, gender and experience in using, on the performances of an EMG-based system for virtual upper limb and prosthesis control. The proposed system consists in a low density EMG sensors arrangement, a purpose-built signal-conditioning electronic circuitry and a software able to classify the gestures and to replicate them via avatars. The classification algorithm was optimized in terms of feature extraction and dimensionality reduction. In its optimal configuration, the system allows to accurately discriminate five different hand gestures (accuracy = 88.85 ± 7.19%). Statistical analysis demonstrated no significant difference in classification accuracy related to hand-dominance (handedness) and to gender. In addition, maximum accuracy in dominant hand is achieved since first use of the system, whilst accuracy in classifying gestures of the non-dominant hand significantly increases with experience. These results indicate that this system can be potentially used by every trans-radial upper-limb amputee for virtual/real limb control

Métodos de classificação confiável e resiliente de movimentos de membros superiores baseado em extreme learning machines e sinais de eletromiografia de superfície

Apesar de avanços recentes, a classificação confiável de sinais de eletromiografia de superfície (sEMG) permanece uma tarefa árdua sob a perspectiva de Aprendizagem de Máquina. Sinais de sEMG possuem uma sobreposição de classes inerente à sua natureza, o que impede a separação perfeita das amostras e produz ruídos de classificação. Alternativas ao problema geralmente baseiam-se na filtragem do sEMG ou métodos de pós-processamento como o Major-Voting, soluções estas que necessariamente geram atrasos na classificação do sinal e frequentemente não geram melhoras substanciais. A abordagem deste trabalho baseia-se no desenvolvimento de métodos confiáveis e resilientes sob a perspectiva de classificação que gerem saídas mais estáveis e consistentes para o classificador baseado em Extreme Learning Machines (ELM) utilizado. Para tanto, métodos envolvendo o pré-processamento e pós-processamento, a suavização do arg max do classificador, thresholds adaptativos e um classificador binário auxiliar foram utilizados. Os sinais classificados derivam de 12 canais de sEMG envolvendo três bases de dados diferentes onde 99 ensaios compostos pela execução de 17 movimentos distintos do segmento mão-braço foram realizados. Nos melhores resultados, os métodos utilizados atingiram taxas de acerto médio global de 66,99 ± 23,6% para a base de voluntários amputados, 87,10 ± 5,89% para a base de voluntários não-amputados e taxas superiores a 99% para todas as variações de diferentes ensaios que compõe a base de dados adquirida em laboratório. Já para a taxa de acerto média ponderada por classes, nos melhores resultados foram de 53,36 ± 18,2% para a base de voluntários amputados, 77,94 ± 6,22% para a base de voluntários não-amputados e taxas superiores a 91% para os ensaios da base de dados adquirida em laboratório. Ambas as métricas de taxa de acerto consideradas superam ou equivalem-se a alternativas descritas na literatura, utilizando abordagens que não demandam grandes mudanças estruturais no classificador.Despite recent advances, reliable classification of surface electromyography (sEMG) signals remains an arduous task from the perspective of Machine Learning. sEMG signals have inherent class overlaps that prevent optimal labeling due to classification noises. Alternatives to classification ripples usually rely on stochastic sEMG filtering or post-processing methods, like Major-Voting, both solutions that insert constraints and additional delays in signal classification and often do not generate substantial improvements. The approach of this paper focuses on the development of reliable and resilient methods used in combination with an Extreme Learning Machines (ELM) classifier to generate more stable and consistent outputs. Methods of pre-processing and post-processing, a smoothed arg max version of the ELM, adaptive thresholds, and an auxiliary binary classifier were used to process signals derived from 12 EMG channels from three different databases. In total, 99 trials were performed, each one containing 17 different upper-limb movements. The proposed methods reached an average overall accuracy rate of 66.99 ± 23.6% for the amputee individuals’ database, 87.10 ± 5.89% for the non-amputee individuals’ database, and rates over 99% for all variations of our own lab-generated database. The average weighted accuracy rates were 53.36 ± 18.2% for the amputee individuals’ database, 77.94 ± 6.22% for the base of the non-amputee individuals’ database, and higher than 91% for the best-case scenario of our own lab-generated database. In both metrics considered, the results outperform, or match alternatives described in the literature using approaches that do not require significant changes in the classifier's architecture

Novel devices and strategies to investigate and counteract pain: an engineer’s point of view on pain and how to fight against it

Pain is a gift; it is a fundamental adaptive mechanism to protect ourselves from injuries and illnesses [1]. While usually pain is perceived as debilitating and intolerable and researchers are focusing on how to suppress it, its presence has a great survival value, which becomes evident in its absence. Contrary to the common belief, insensitivity to pain is a curse, which sometimes is an ill-fated consequence of some conditions, such as alcoholism, multiple sclerosis, diabetes and leprosy [2]. Pain warns us of dangers, impeding us to continue adding insult to an injury causing more serious issues; it forces us to rest and protect the affected body part until the latter has recovered completely. As such, pain represents an "unpleasant sensory and emotional experience" but "associated with actual or potential tissue damage" [3], thus necessary to avoid further repercussions. When a high-intensity stimulus that can damage tissues is applied onto the human skin, it activates pain receptors [4], called nociceptors; information reaches the brain through myelinated A and un-myelinated C nerve fibers and the pain is experienced. If pain is certainly a gift, complications arise when its natural mechanisms do not perform correctly and the unpleasant painful sensations become incessant, interfering drastically with the person’s quality of life. This is not a rare event: as an example, more than 100 million adults in the United States only are affected by chronic pain. In such cases, pain loses its role as a warning against more critical injuries and simply becomes a medical problem, requiring medical treatment. Costs of these treatments are not negligible: they range from $560 to$635 billion of US dollars per year, combining the health care cost and the productivity estimates [5]. Therefore pain has to be investigated, to be better understood and counteracted, if necessary. Easy? Not at all: pain is difficult to ascertain, is primarily assessed by means of self-report [6] and experiments on pain suffer from a lack of reproducibility and accuracy. The result is that many relevant neuro-physiological aspects on pain still remain unclear. Here is where the engineer work becomes essential: medicine needs engineering to design, develop and implement reliable and precise devices whose features help pain research. Furthermore, when possible, the engineer should propose solutions that are inexpensive, portable, easy-to-assemble and customizable to suit diverse experimental requirements, ready to be employed into different researches. A complex challenge, without a doubt, but fundamental: the reader will find throughout this thesis, final report of my PhD in Electronic Engineering, a dissertation on the development of brand new devices and strategies to study and counteract the feeling of pain. Particularly, this thesis is made of two distinct engineering projects, devoted to these two aspects respectively. i ii Part I - Pain Counteraction: PROVIRT PLP project has the aim to improve the rehabilitation of upper limb amputees who suffer from Phantom Limb Pain syndrome, a chronic pain condition, with a technology based on pattern recognition and virtual reality. A promising discover made by Ramachandran [3] showed how restoring the visual feedback of the amputated limb may have a primary importance in the pain counteraction, thus I implemented a device to maximise such illusion. PROVIRT PLP is able to read surface electromyographic signals from the amputee’s stump and coherently convert them into the movement of an avatar in a virtual reality environment. This part describes the project requisites, the design and implementation of the electronics, the observation regarding the classifiers and the pattern recognition stage and the software application that commands the virtual reality module. The system is then further improved by the means of three experimental in-vivo tests, namely Test A - Optimization of EMG-based hand gesture recognition: Supervised vs. unsupervised data preprocessing on healthy subjects and transradial amputees (published as [7]), Test B - Evaluating the influence of subject-related variables on EMG-based hand gesture classification (published as [8]) and Test C - Tuning parameters and performance evaluation. The project has been funded by INAIL (Italian government agency for the insurance against work-related injuries) and can be considered a successful compromise between gesture/intention of movement classification accuracy and ease of use for both health professionals and amputees. To date, Test D - Therapy effectiveness that will eventually shed some light on PROVIRT PLP suitability for PLP counteraction is ongoing, with promising partial results. Part II - Pain Investigation: PUSH project has the aim to define objective measures of pain, by means of robust and consistent patterns of noxious stimuli and innocuous touch. The project wants to develop a device, able to allow for reliable non-invasive investigation of the peripheral and central mechanisms related to the sense of pain, towards the definition of biomarkers for its quantitative assessment. Since fMRI is the standard tool in advanced brain research, PUSH is developed to be completely MR-compatible. This part describes the idea of a pneumatic-driven system to elicit pain, the design and implementation of the electronics, the characterization of the system and a fMRI in-vivo experiment on an adult volunteer to validate it. The project can be considered as a brand new interesting prototype for any kind of fMRI- and EEG-based work on painful mechanical stimulation, and a technical paper that describes its details is published as [9]. This work has been funded by the Department of Bioengineering, Imperial College of Science, Technology and Medicine, London, UK, the Department of Neuroscience, Physiology and Pharmacology, University College London, London, UK and the Centre for the Developing Brain, Kings College London, St Thomas’ Hospital, London, UK. Support for this research is by the EU-FP7 grants CONTEST (ITN-317488), BALANCE (ICT-601003), Symbitron (ICT-661626) and EU-H2020 grant CogIMon (ICT-23-2014). Summarizing, this research work is a step forward towards a strict cooperation between engineers and medical doctors, in the perspective of further advances in the understanding and the counteracting of the feeling of pain in humans. Two engineering devices, PROVIRT PLP and PUSH, have been developed to address and solve medical challenges, hoping they will be even a small puzzle piece in helping people achieve a better quality of life