Subtle hand gesture identification for HCI using temporal decorrelation source separation BSS of surface EMG

Hand gesture identification has various human computer interaction (HCI) applications. This paper presents a method for subtle hand gesture identification from sEMG of the forearm by decomposing the signal into components originating from different muscles. The processing requires the decomposition of the surface EMG by temporal decorrelation source separation (TDSEP) based blind source separation technique. Pattern classification of the separated signal is performed in the second step with a back propagation neural network. The focus of this work is to establish a simple, yet robust system that can be used to identify subtle complex hand actions and gestures for control of prosthesis and other HCI based devices. The proposed model based approach is able to overcome the ambiguity problems (order and magnitude problem) of BSS methods by selecting an a priori mixing matrix based on known hand muscle anatomy. The paper reports experimental results, where the system was able to reliably recognize different subtle hand gesture with an overall accuracy of 97%. The advantage of such a system is that it is easy to train by a lay user, and can easily be implemented in real time after the initial training. The paper also highlights the importance of mixing matrix analysis in BSS technique

Blind Source Separation Based Classification Scheme for Myoelectric Prosthesis Hand

For over three decades, researchers have been working on using surface electromyography (sEMG) as a means for amputees to use remaining muscles to control prosthetic limbs (Baker, Scheme, Englehart, Hutcinson, & Greger, 2010; Hamdi, Dweiri, Al-Abdallat, & Haneya, 2010; Kiguchi, Tanaka, & Fukuda, 2004). Most research in this domain has focused on using the muscles of the upper arms and shoulders to control the gross orientation and grasp of a low-degree-of-freedom prosthetic device for manipulating objects (Jacobsen & Jerard, 1974). Each measured upper arm muscle is typically mapped directly to one degree of freedom of the prosthetic. For example, tricep contraction could be used for rotation while bicep flexion might close or open the prosthetic. More recently, researchers have begun to look at the potential of using the forearm muscles in hand amputees to control a multi-fingered prosthetic hand. While we know of no fully functional hand prosthetic, this is clearly a promising new area of EMG research. One of the challenges for creating hand prosthetics is that there is not a trivial mapping of individual muscles to finger movements. Instead, many of the same muscles are used for several different fingers (Schieber, 1995)

Iterative issues of ICA, quality of separation and number of sources : a study for biosignal applications

This thesis has evaluated the use of Independent Component Analysis (ICA) on Surface Electromyography (sEMG), focusing on the biosignal applications. This research has identified and addressed the following four issues related to the use of ICA for biosignals: • The iterative nature of ICA • The order and magnitude ambiguity problems of ICA • Estimation of number of sources based on dependency and independency nature of the signals • Source separation for non-quadratic ICA (undercomplete and overcomplete) This research first establishes the applicability of ICA for sEMG and also identifies the shortcomings related to order and magnitude ambiguity. It has then developed, a mitigation strategy for these issues by using a single unmixing matrix and neural network weight matrix corresponding to the specific user. The research reports experimental verification of the technique and also the investigation of the impact of inter-subject and inter-experimental variations. The results demonstrate that while using sEMG without separation gives only 60% accuracy, and sEMG separated using traditional ICA gives an accuracy of 65%, this approach gives an accuracy of 99% for the same experimental data. Besides the marked improvement in accuracy, the other advantages of such a system are that it is suitable for real time operations and is easy to train by a lay user. The second part of this thesis reports research conducted to evaluate the use of ICA for the separation of bioelectric signals when the number of active sources may not be known. The work proposes the use of value of the determinant of the Global matrix generated using sparse sub band ICA for identifying the number of active sources. The results indicate that the technique is successful in identifying the number of active muscles for complex hand gestures. The results support the applications such as human computer interface. This thesis has also developed a method of determining the number of independent sources in a given mixture and has also demonstrated that using this information, it is possible to separate the signals in an undercomplete situation and reduce the redundancy in the data using standard ICA methods. The experimental verification has demonstrated that the quality of separation using this method is better than other techniques such as Principal Component Analysis (PCA) and selective PCA. This has number of applications such as audio separation and sensor networks

Approximate credibility intervals on electromyographic decomposition algorithms within a Bayesian framework

This thesis develops a framework to uncover the probability of correctness of algorithmic results. Specifically, this thesis is not concerned with the correctness of these algorithms, but with the uncertainty of their results arising from existing uncertainty in their inputs. This is achieved using a Bayesian approach. This framework is then demonstrated using independent component analysis with electromyographic data. Blind source separation (BSS) algorithms, such as independent component analysis (ICA), are often used to solve the inverse problem arising when, for example, attempting to retrieve the activation patterns of motor units (MUs) from electromyographic (EMG) data. BSS, or similar algorithms, return a result but do not generally provide any indication on the quality of that result or certainty one can have in it being the actual original pattern and not one strongly altered by the noise/errors in the input. This thesis uses Bayesian inference to extend ICA both to incorporate prior physiological information, thus making it in effect a semi-blind source separation (SBSS) algorithm, and to quantify the uncertainties around the values of the sources as estimated by ICA. To this end, this thesis also presents a way to put a prior on a mixing matrix given a physiological model as well as a re-parametrisation of orthogonal matrices which is helpful in pre-empting floating point errors when incorporating this prior of the mixing matrix into an algorithm which estimates the un-mixing matrix. In experiments done using EMG data, it is found that the addition of the prior is of benefit when the input is very noisy or very short in terms of samples or both. The experiments also show that the information about the certainty can be used as a heuristic for feature extraction or general quality control provided an appropriate baseline has been determined

Fused mechanomyography and inertial measurement for human-robot interface

Human-Machine Interfaces (HMI) are the technology through which we interact with the ever-increasing quantity of smart devices surrounding us. The fundamental goal of an HMI is to facilitate robot control through uniting a human operator as the supervisor with a machine as the task executor. Sensors, actuators, and onboard intelligence have not reached the point where robotic manipulators may function with complete autonomy and therefore some form of HMI is still necessary in unstructured environments. These may include environments where direct human action is undesirable or infeasible, and situations where a robot must assist and/or interface with people. Contemporary literature has introduced concepts such as body-worn mechanical devices, instrumented gloves, inertial or electromagnetic motion tracking sensors on the arms, head, or legs, electroencephalographic (EEG) brain activity sensors, electromyographic (EMG) muscular activity sensors and camera-based (vision) interfaces to recognize hand gestures and/or track arm motions for assessment of operator intent and generation of robotic control signals. While these developments offer a wealth of future potential their utility has been largely restricted to laboratory demonstrations in controlled environments due to issues such as lack of portability and robustness and an inability to extract operator intent for both arm and hand motion. Wearable physiological sensors hold particular promise for capture of human intent/command. EMG-based gesture recognition systems in particular have received significant attention in recent literature. As wearable pervasive devices, they offer benefits over camera or physical input systems in that they neither inhibit the user physically nor constrain the user to a location where the sensors are deployed. Despite these benefits, EMG alone has yet to demonstrate the capacity to recognize both gross movement (e.g. arm motion) and finer grasping (e.g. hand movement). As such, many researchers have proposed fusing muscle activity (EMG) and motion tracking e.g. (inertial measurement) to combine arm motion and grasp intent as HMI input for manipulator control. However, such work has arguably reached a plateau since EMG suffers from interference from environmental factors which cause signal degradation over time, demands an electrical connection with the skin, and has not demonstrated the capacity to function out of controlled environments for long periods of time. This thesis proposes a new form of gesture-based interface utilising a novel combination of inertial measurement units (IMUs) and mechanomyography sensors (MMGs). The modular system permits numerous configurations of IMU to derive body kinematics in real-time and uses this to convert arm movements into control signals. Additionally, bands containing six mechanomyography sensors were used to observe muscular contractions in the forearm which are generated using specific hand motions. This combination of continuous and discrete control signals allows a large variety of smart devices to be controlled. Several methods of pattern recognition were implemented to provide accurate decoding of the mechanomyographic information, including Linear Discriminant Analysis and Support Vector Machines. Based on these techniques, accuracies of 94.5% and 94.6% respectively were achieved for 12 gesture classification. In real-time tests, accuracies of 95.6% were achieved in 5 gesture classification. It has previously been noted that MMG sensors are susceptible to motion induced interference. The thesis also established that arm pose also changes the measured signal. This thesis introduces a new method of fusing of IMU and MMG to provide a classification that is robust to both of these sources of interference. Additionally, an improvement in orientation estimation, and a new orientation estimation algorithm are proposed. These improvements to the robustness of the system provide the first solution that is able to reliably track both motion and muscle activity for extended periods of time for HMI outside a clinical environment. Application in robot teleoperation in both real-world and virtual environments were explored. With multiple degrees of freedom, robot teleoperation provides an ideal test platform for HMI devices, since it requires a combination of continuous and discrete control signals. The field of prosthetics also represents a unique challenge for HMI applications. In an ideal situation, the sensor suite should be capable of detecting the muscular activity in the residual limb which is naturally indicative of intent to perform a specific hand pose and trigger this post in the prosthetic device. Dynamic environmental conditions within a socket such as skin impedance have delayed the translation of gesture control systems into prosthetic devices, however mechanomyography sensors are unaffected by such issues. There is huge potential for a system like this to be utilised as a controller as ubiquitous computing systems become more prevalent, and as the desire for a simple, universal interface increases. Such systems have the potential to impact significantly on the quality of life of prosthetic users and others.Open Acces

Proceedings of the 19th Sound and Music Computing Conference

Proceedings of the 19th Sound and Music Computing Conference - June 5-12, 2022 - Saint-Étienne (France). https://smc22.grame.f