Pattern recognition-based real-time myoelectric control for anthropomorphic robotic systems : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Mechatronics at Massey University, Manawatū, New Zealand

All copyrighted Figures have been removed but may be accessed via their source cited in their respective captions.Advanced human-computer interaction (HCI) or human-machine interaction (HMI) aims to help humans interact with computers smartly. Biosignal-based technology is one of the most promising approaches in developing intelligent HCI systems. As a means of convenient and non-invasive biosignal-based intelligent control, myoelectric control identifies human movement intentions from electromyogram (EMG) signals recorded on muscles to realise intelligent control of robotic systems. Although the history of myoelectric control research has been more than half a century, commercial myoelectric-controlled devices are still mostly based on those early threshold-based methods. The emerging pattern recognition-based myoelectric control has remained an active research topic in laboratories because of insufficient reliability and robustness. This research focuses on pattern recognition-based myoelectric control. Up to now, most of effort in pattern recognition-based myoelectric control research has been invested in improving EMG pattern classification accuracy. However, high classification accuracy cannot directly lead to high controllability and usability for EMG-driven systems. This suggests that a complete system that is composed of relevant modules, including EMG acquisition, pattern recognition-based gesture discrimination, output equipment and its controller, is desirable and helpful as a developing and validating platform that is able to closely emulate real-world situations to promote research in myoelectric control. This research aims at investigating feasible and effective EMG signal processing and pattern recognition methods to extract useful information contained in EMG signals to establish an intelligent, compact and economical biosignal-based robotic control system. The research work includes in-depth study on existing pattern recognition-based methodologies, investigation on effective EMG signal capturing and data processing, EMG-based control system development, and anthropomorphic robotic hand design. The contributions of this research are mainly in following three aspects: Developed precision electronic surface EMG (sEMG) acquisition methods that are able to collect high quality sEMG signals. The first method was designed in a single-ended signalling manner by using monolithic instrumentation amplifiers to determine and evaluate the analog sEMG signal processing chain architecture and circuit parameters. This method was then evolved into a fully differential analog sEMG detection and collection method that uses common commercial electronic components to implement all analog sEMG amplification and filtering stages in a fully differential way. The proposed fully differential sEMG detection and collection method is capable of offering a higher signal-to-noise ratio in noisy environments than the single-ended method by making full use of inherent common-mode noise rejection capability of balanced signalling. To the best of my knowledge, the literature study has not found similar methods that implement the entire analog sEMG amplification and filtering chain in a fully differential way by using common commercial electronic components. Investigated and developed a reliable EMG pattern recognition-based real-time gesture discrimination approach. Necessary functional modules for real-time gesture discrimination were identified and implemented using appropriate algorithms. Special attention was paid to the investigation and comparison of representative features and classifiers for improving accuracy and robustness. A novel EMG feature set was proposed to improve the performance of EMG pattern recognition. Designed an anthropomorphic robotic hand construction methodology for myoelectric control validation on a physical platform similar to in real-world situations. The natural anatomical structure of the human hand was imitated to kinematically model the robotic hand. The proposed robotic hand is a highly underactuated mechanism, featuring 14 degrees of freedom and three degrees of actuation. This research carried out an in-depth investigation into EMG data acquisition and EMG signal pattern recognition. A series of experiments were conducted in EMG signal processing and system development. The final myoelectric-controlled robotic hand system and the system testing confirmed the effectiveness of the proposed methods for surface EMG acquisition and human hand gesture discrimination. To verify and demonstrate the proposed myoelectric control system, real-time tests were conducted onto the anthropomorphic prototype robotic hand. Currently, the system is able to identify five patterns in real time, including hand open, hand close, wrist flexion, wrist extension and the rest state. With more motion patterns added in, this system has the potential to identify more hand movements. The research has generated a few journal and international conference publications

Biosignal‐based human–machine interfaces for assistance and rehabilitation : a survey

As a definition, Human–Machine Interface (HMI) enables a person to interact with a device. Starting from elementary equipment, the recent development of novel techniques and unobtrusive devices for biosignals monitoring paved the way for a new class of HMIs, which take such biosignals as inputs to control various applications. The current survey aims to review the large literature of the last two decades regarding biosignal‐based HMIs for assistance and rehabilitation to outline state‐of‐the‐art and identify emerging technologies and potential future research trends. PubMed and other databases were surveyed by using specific keywords. The found studies were further screened in three levels (title, abstract, full‐text), and eventually, 144 journal papers and 37 conference papers were included. Four macrocategories were considered to classify the different biosignals used for HMI control: biopotential, muscle mechanical motion, body motion, and their combinations (hybrid systems). The HMIs were also classified according to their target application by considering six categories: prosthetic control, robotic control, virtual reality control, gesture recognition, communication, and smart environment control. An ever‐growing number of publications has been observed over the last years. Most of the studies (about 67%) pertain to the assistive field, while 20% relate to rehabilitation and 13% to assistance and rehabilitation. A moderate increase can be observed in studies focusing on robotic control, prosthetic control, and gesture recognition in the last decade. In contrast, studies on the other targets experienced only a small increase. Biopotentials are no longer the leading control signals, and the use of muscle mechanical motion signals has experienced a considerable rise, especially in prosthetic control. Hybrid technologies are promising, as they could lead to higher performances. However, they also increase HMIs’ complex-ity, so their usefulness should be carefully evaluated for the specific application

Emerging ExG-based NUI Inputs in Extended Realities : A Bottom-up Survey

Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduceto the areaof XR.Peer reviewe