Surface electromyography in personalised modelling of the head and neck

Preoperative estimation of function loss is subjective and unreliable since it depends on the personal expertise of individual physicians. Moreover, each patient is unique and will respond differently to the various treatment options. The Virtual Therapy Group is developing tools to make this tough decision-making process easier, with personalised functional outcome expectations. The idea is to develop a digital doppelgänger. This virtual look-a-like should be able to adapt to individual patients by processing conventional imaging data (magnetic resonance imagining, computed tomography, ultrasound), as well as 3D video measurements to define mobility of anatomical structures, and surface electromyography (sEMG) to account for individual muscle activation patterns. By personalising the currently available generic models with the above data, we could create genuine digital replicas of individual patients. In clinical practice, some head-and-neck patients regain full function after their treatment and continue their lives with good speech and swallowing function. Others, however, do not and suffer from pathological speech and dysphagia. We think that these differences relate to variation in neural motor control and muscle innervation. Nerve anatomy differs between individuals. Some people may just have more nerve branch innervations for particular muscles than do others. With sEMG, we can record a crude estimate of muscle activations, which will hopefully enable us to map neural motor commands. This dissertation demonstrates in Chapter 3 that with features extracted from sEMG signals, we can accurately estimate 3D static lip shapes. This promising finding shows that sEMG signals can provide sufficient information on motor control. Chapter 4 demonstrates that a statistical model can adequately predict dynamic movements – visemes (groups of speech sounds that visually look the same), facial expressions, and asymmetric movements – with signals measured from 16 facial muscles. Chapter 5 describes the step from statistical models towards biomechanical models that implement real physics. These models will be advantageous because they follow physical laws and preserve real anatomy and geometry. In Chapter 6, we elaborate on the process of inverse modelling: calculating the input of muscle activations needed to generate specific functional outcomes – in our case, the 3D lip movements of functions such as speech. Unfortunately, this is a rather complicated procedure, and because of the aforementioned redundancy of the musculoskeletal system, it can lead to multiple solutions. However, we also demonstrate in this chapter that with sEMG we can reduce the solution-space and acquire more patient-specific data on muscle activation. Chapter 7 presents a technical elaboration on inverse modelling, investigating static and dynamic optimisation techniques with and without sEMG. Chapter 8 discusses the work and proposes future research directions on the basis of four main pillars in personalising the generic models. To conclude, forward modelling will be elementary for driving the model with surgical adaptations and patient-specific learnt muscle-activation strategies, so it could show us the treatment effects directly after surgery. Inverse modelling, on the other hand, could show us any potential compensatory mechanisms, which may differ from patient to patient. Some patients will be able to relearn functions; others will not. With a fully operative digital doppelgänger, clinicians will be able to perform various treatment strategies and compare treatment outcomes at the multidisciplinary meeting to agree upon the best individual treatment strategies. The doppelgänger will also be helpful during counselling, to simulate the functional sequelae of treatment and to better prepare and inform the patient

Predicting 3D lip movement using facial sEMG: a first step towards estimating functional and aesthetic outcome of oral cancer surgery

In oral cancer, loss of function due to surgery can be unacceptable, designating the tumour as functionally inoperable. Other curative treatments can then be considered. Currently, predictions of these functional consequences are subjective and unreliable. We want to create patient-specific models to improve and objectify these predictions. A first step was taken by controlling a 3D lip model with volunteer-specific sEMG activities. We focus on the lips first, because they are essential for speech, oral food transport, and facial mimicry. Besides, they are more accessible to measurements than intraoral organs. 3D lip movement and corresponding sEMG activities are measured in five healthy volunteers, who performed 19 instructions repeatedly, to create a quantitative lip model by establishing the relationship between sEMG activities of eight facial muscles bilaterally on the input side and the corresponding 3D lip displacements on the output side. The relationship between 3D lip movement and sEMG activities was accommodated in a state-space model. A good relationship between sEMG activities and 3D lip movement was established with an average root mean square error of 2.43 mm for the first-order system and 2.46 mm for the second-order system. This information can be incorporated into biomechanical models to further personalise functional outcome assessment after treatment

Simulation of facial expressions using person-specific sEMG signals controlling a biomechanical face model

Purpose: Functional inoperability in advanced oral cancer is difficult to assess preoperatively. To assess functions of lips and tongue, biomechanical models are required. Apart from adjusting generic models to individual anatomy, muscle activation patterns (MAPs) driving patient-specific functional movements are necessary to predict remaining functional outcome. We aim to evaluate how volunteer-specific MAPs derived from surface electromyographic (sEMG) signals control a biomechanical face model. Methods: Muscle activity of seven facial muscles in six volunteers was measured bilaterally with sEMG. A triple camera set-up recorded 3D lip movement. The generic face model in ArtiSynth was adapted to our needs. We controlled the model using the volunteer-specific MAPs. Three activation strategies were tested: activating all muscles (act all) , selecting the three muscles showing highest muscle activity bilaterally (act 3) —this was calculated by taking the mean of left and right muscles and then selecting the three with highest variance—and activating the muscles considered most relevant per instruction (act rel) , bilaterally. The model’s lip movement was compared to the actual lip movement performed by the volunteers, using 3D correlation coefficients (ρ). Results: The correlation coefficient between simulations and measurements with act rel resulted in a median ρ of 0.77. act 3 had a median ρ of 0.78, whereas with act all the median ρ decreased to 0.45. Conclusion: We demonstrated that MAPs derived from noninvasive sEMG measurements can control movement of the lips in a generic finite element face model with a median ρ of 0.78. Ultimately, this is important to show the patient-specific residual movement using the patient’s own MAPs. When the required treatment tools and personalisation techniques for geometry and anatomy become available, this may enable surgeons to test the functional results of wedge excisions for lip cancer in a virtual environment and to weigh surgery versus organ-sparing radiotherapy or photodynamic therapy

sEMG-assisted inverse modelling of 3D lip movement: a feasibility study towards person-specific modelling

Abstract We propose a surface-electromyographic (sEMG) assisted inverse-modelling (IM) approach for a biomechanical model of the face to obtain realistic person-specific muscle activations (MA) by tracking movements as well as innervation trajectories. We obtained sEMG data of facial muscles and 3D positions of lip markers in six volunteers and, using a generic finite element (FE) face model in ArtiSynth, performed inverse static optimisation with and without sEMG tracking on both simulation data and experimental data. IM with simulated data and experimental data without sEMG data showed good correlations of tracked positions (0.93 and 0.67) and poor correlations of MA (0.27 and 0.20). When utilising the sEMG-assisted IM approach, MA correlations increased drastically (0.83 and 0.59) without sacrificing performance in position correlations (0.92 and 0.70). RMS errors show similar trends with an error of 0.15 in MA and of 1.10 mm in position. Therefore, we conclude that we were able to demonstrate the feasibility of an sEMG-assisted inverse modelling algorithm for the perioral region. This approach may help to solve the ambiguity problem in inverse modelling and may be useful, for instance, in future applications for preoperatively predicting treatment-related function loss

The error values for <i>e</i>_<i>c</i> and <i>e</i>_<i>r</i> by volunteer with the mean optimal settings, i.e. volunteer-independent settings (see Eq (16)).

<p>The error values for <i>e</i>_<i>c</i> and <i>e</i>_<i>r</i> by volunteer with the mean optimal settings, i.e. volunteer-independent settings (see Eq (<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0175025#pone.0175025.e031" target="_blank">16</a>)).</p