We highlight two alternative, yet complementary, solutions for harnessing available neural resources for improving integration of artificial limbs (ALs) through embodiment. ‘Hard’ embodiment exploits neural and cognitive body mechanisms by closely mimicking their original biological functions. ‘Soft’ embodiment exploits these same mechanisms by recycling them to support a different function altogether

De Vignemont, F

Makin, TR

Micera, S

English

UCL Discovery

SOFT EMBODIMENT FOR ENGINEERING ARTIFICIAL LIMBS Tamar Makin1,* , Frederique de Vignemont2     , Silvestro Micera3,4  1 Institute of Cognitive Neuroscience, University College London, 17 Queen Square, London WC1N 3AZ, UK  2 Institut Jean Nicod, Département d’études cognitives, ENS, EHESS, CNRS, PSL University, UMR 8129, 29 rue d’Ulm, 75005 Paris, France 3The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant’Anna, viale Rinaldo Piaggio 34, 56025, Pontedera, Italy 4Bertarelli Foundation Chair in Translational Neuroengineering, Center for Neuroprosthetics and Institute of Bioengineering, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland *Corresponding author: t.makin@ucl.ac.uk  ABSTRACT We highlight two alternative, yet complementary, solutions for harnessing available neural resources for improving integration of artificial limbs through embodiment. ‘Hard’ embodiment exploits neural and cognitive body mechanisms by closely mimicking their original biological functions. ‘Soft’ embodiment exploits these same mechanisms by recycling them to support a different function altogether.  KEYWORDS Artificial limb, Assistive Technology, Motor Control, Neuroprosthesis, Substitution   EMBODIMENT FOR ENGINEERING As fierce warriors, many of our science fiction heroes (and villains), have benefited from increased capacities of an artificial arm after losing their biological arm in combat. Once fitted, the user is able to immediately control the device intuitively and fluently. This futuristic vision stands in stark contrast to current figures on successful adoption of upper limb prosthetic devices. For example, as many as 40% of individuals fitted with a prosthesis use it rarely or abandon it all together [1]. Contrary to the seamless integration of science fiction devices, it seems that the human brain relates to current prostheses as foreign, requiring extensive practice towards expert use, much like learning to drive a car. Indeed, despite dramatic advancements in design and interface, the proportion of skilled prosthesis users seems stubbornly stable [2].   Hypothetically, the learning curve could be greatly accelerated if instead of starting from scratch, one could hijack already existing neural resources belonging to natural hand and arm control in order to operate the new device. This process, broadly known as embodiment [3], was suggested as a powerful mechanism to improve the integration of prosthetic limbs [4]. It is often assumed that the more embodied the prosthesis is, the better. But we will argue that there is actually a cost to embodiment, if it follows too closely the body model. Beyond mimicking how the brain controls a body part, we should also exploit this existing biological infrastructure to support new solutions for improved artificial limb functionality. We outline an alternative embodiment strategy, aimed at recycling neural resources that originally evolved to control biological arms, which can be repurposed for a new function (Box 1; Figure 1). These two complementary strategies could provide a richer and more agile framework for embodied technology.      THE LIMITS OF HARD EMBODIMENT A device is considered ‘embodied’ if information about it is processed and used by the nervous system as information about one’s own body part, in order to accomplish the same function (hereafter hard embodiment). If the brain could exploit pre-existing sensorimotor pathways to control a prosthesis, then the user should be able to control the device more intuitively and fluently, and the prosthesis may even end up feeling like a part of the user’s body. However, the unique adaptations that the nervous system had undergone to control a human limb – first through evolution and refined in early life development – may not be directly exploitable for interfacing with modern-day prosthetic devices.   To begin with - given the existing hardware and software technological imperfections for biomimetic sensorimotor interface and control, presently (and in the foreseeable future), artificial limbs are unable to fully mimic a biological hand, and thus to be fully embodied. This failure could increase the cognitive burden while using a device designed for embodiment, and generate a feeling of discomfort, or even of alienation. In comparison to biological body parts specifically evolved for our nervous system control, artificial body parts cannot win.   A further limitation of hard embodiment is revealed in individuals who were born without a hand. These individuals cannot process their artificial limb as their biological limb since they have not developed the required nervous resources [5]. Similar issues might arise when considering other adaptations to the normal nervous structure and function that coincide with traumatic amputation, e.g. nerve damage, peripheral and central plasticity mechanisms. A hard embodiment framework will need to be adapted to account for erroneous biological processing on a patient-by-patient basis, which might be impractical.  Finally, with technological progress, we must think of all the advantages that robotic motor augmentation, such as extra finger and arm devices, could offer [6]. These devices      may prove  incompatible with the internal models for normal human      motor control, where our motor system translates a desired action into motor primitives. These internal models are partially based on our genetic template and early life experience, which did not include the biomechanics of an artificial augmentation. Users might prefer to switch across devices, depending on their task requirements. How can the human nervous system deal with the abrupt change of entirely distinct inputs and outputs associated with different functional attachments, which may have little overlap with the mechanics of hand motor control? Staying too close to the body model may obscure the harnessing of exciting practical solutions for future artificial devices, which could in turn free us from the biological limits of our body.    RECYCLING THE NEURAL BASIS OF THE BODY The nervous system, and sensorimotor loop in particular, allows us to process signals, integrate multisensory information and increase error-based learning. It provides an existing model for the external world and predictions on how the external worlds can be manipulated BOX 1: TWO STRATEGIES FOR EMBODIMENT OF EXTERNAL DEVICES  Which strategy is more efficient if one wants to fly? To attach wings to one’s arms to mimic birds? Or to extract the general principles that allow the bird to fly and adapt them to the available technology? We can draw a parallel with augmentative technologies for motor control, which raise similar questions.    Figure 1: Illustration for soft and hard conceptual frameworks. Concrete examples for soft embodiment are shown in Figure 2.    HARD EMBODIMENT Assumption: The way we relate to our body is the best model we have for fluent ALs coupling with our nervous system. Objective: For ALs to become a body part. Mean: To use the neural processing for ALs and for the body in the same way in order to accomplish the same functions. Limit: Reduced flexibility due to the constraints of the body template.  SOFT EMBODIMENT Assumption: There are other models than the body that can be as good or even better. Objective: For ALs to offer new opportunities for interacting with the world.  Mean: To recycle the neural processing normally dedicated to the body and put them into a new use, possibly to fulfil new functions. Limit: Time constraints for learning the new use.    by our actions [7]. It is important to realise that these existing neural loops are not doomed to function only with something that is exactly like a body and that performs the same functions as the body and in the same way. This neural toolkit constitutes a powerful potential means for information compression, error-based trial learning and reducing the cognitive load of prosthesis usage, that could actually be exploited for a new range of purposes. So far, engineers have been interested in exploiting this infrastructure for hard embodiment. But we would like to propose that in addition, (neuro)engineers should aim to create prostheses that      are not enslaved to the body template and that exploit body processing for their own purpose (Box 1). A device is considered as ‘softly embodied’ if it co-opts some of the computational bandwidth which was evolutionarily developed for sensorimotor control, resulting in faster, more intuitive and less cognitively demanding learning to control the device. Consider for example the redundancies that exist both in the central and peripheral nervous system to guide motor control of a limb. If we could recruit a sub-set of these resources to train the user to control an external device using simplified operating principles (relative to the biological body), we could better benefit from already existing technological advancements [8].  With soft embodiment, we invite engineers to redefine the function of a certain neural process or computation. This is distinct from tools, which are generally designed to take advantage of our existing neural control of movement, as actuated by our bodies. Freed from the mediating role of the body, the neural principles used to successfully control a device under the soft embodiment framework do not necessarily mimic those of body motor control to produce an analogous function. Alternatively, canonical principles of motor control could be repurposed to support new functions with the aid of new technologies.   Recycling makes sense from an evolutionary perspective insofar as it is more parsimonious than developing new neural systems. The soft embodiment framework expands on the idea that brain regions have been redeployed through evolution to support new cognitive abilities as they evolve [9]. This conceptual framework opens up alternative opportunities for harnessing soft embodiment for brain-prosthesis interfaces that do not necessarily adhere to biomimetic principles, such as teleoperated artificial limbs, extra fingers and tentacle arms. Both the nervous system and the engineers are free to come up with new solutions for how to best mobilise existing neural resources for improving artificial limb control, as demonstrated in the examples in Figure 2.   At the phenomenological level, soft embodiment may also prove to be less costly. Since the objective is no longer for the prosthesis to become a body part, there is no risk that the prosthesis might trigger maladaptive body representation, akin to clinical syndromes such as alien hand, phantom pain or dystonia. Soft embodiment will also allow us to exploit sensorimotor resources without necessarily impacting bodily awareness and our sense of body  Figure 2: Examples for soft embodiment. Left: An example for combined principles from soft and hard embodiment in the same artificial limb device. Feedback for tactile and proprioceptive information is fed to the same afferent nerve, allowing for mimicking (touch) and recycling (proprioception) of the same channel [10]. Right: An example for soft embodiment observed using brain decoding in body-selective visual cortex of prosthesis users. Pairs of stimuli that are closer together in the multidimensional representational space are clustered together under the same branch, with shorter connections indicating greater relative representational similarity. In controls (left), cosmetic prostheses are clustered with hands and functional prostheses with tools, reflecting their visual similarities. In prosthesis users (right), however, the two prostheses types (cosmetic and functional) are clustered together, with tools and hands represented dissimilarly from both prostheses. Therefore, expert users represent prosthetic devices less like biological hands, and more like other types of prosthetic devices [11].    ownership. This provides wider opportunities for implementation, for example with regards to current emerging technologies for autonomous prosthesis control [8].  FUTURE OUTLOOK Hard embodiment is presently successfully inspiring artificial limb design. However, our future environment may raise new challenges for which the body as we know it is not the best tool.      While the boundary between soft- and hard- embodiment is not clear cut, the general realm of soft embodiment opens the door to more effective artificial limbs that go beyond the body template. It is important to emphasize that hard and soft embodiments are not mutually exclusive, and could be used modularly or in collaboration to best support devices design. For example, we developed a device that uses peripheral afferent nerves both for mimicking the original (tactile) signal and for ‘hacking’ the channel to relay proprioceptive information, as demonstrated in Figure 2 [10]. By shifting the function of existing neural pathways, we will have the opportunity to more flexibly adapt to new opportunities for improved motor control. Yet, the success of soft embodiment strongly relies on the choice of the (most exploitable) neural pathways to recycle and this process might not be readily available [12]. Thus, soft embodiment requires a closer collaboration between engineers and neuroscientists.  ACKNOWLEDGEMENTS Funding. TRM: Wellcome Trust Senior Research Fellowship (215575/Z/19/Z), ERC Starting Grant (715022 EmbodiedTech). FDV: ANR-16-CE28-0015, ANR-10-LABX-0087 IEC, ANR-10-IDEX-0001-02 PSL. SM: Bertarelli Foundation, Swiss National Competence Center Research Robotics, and Swiss National Science Foundation (CHRONOS project).                  REFERENCES 1 Biddiss, E. and Chau, T. (2007) Upper-limb prosthetics: critical factors in device abandonment. Am J Phys Med Rehabil 86, 977–987 2 Carey, S.L. et al. (2017) Differences in Myoelectric and Body-Powered Upper-Limb Prostheses: Systematic Literature Review. JPO: Journal of Prosthetics and Orthotics 29, P4–P16 3 de Vignemont, F. (2018) Mind the Body, Oxford University Press. 4 Makin, T.R. et al. (2017) Neurocognitive barriers to the embodiment of technology. Nature Biomedical Engineering 1, 0014 5 Wesselink, D.B. et al. (2019) Obtaining and maintaining cortical hand representation as evidenced from acquired and congenital handlessness. eLife 8, 611 6 Kieliba, P. et al. (2020) Neural correlates of hand augmentation. BioArxiv 7 Dhawale, A.K. et al. (2017) The Role of Variability in Motor Learning. Annu Rev Neurosci 40, 479–498 8 Zhuang, K.Z. et al. (2019) Shared human–robot proportional control of a dexterous myoelectric prosthesis. Nat Mach Intell 1, 400–411 9 Dehaene, S. and Cohen, L. (2007) Cultural Recycling of Cortical Maps. Neuron 56, 384–398 10 D'Anna, E. et al. (2019) A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback. Science Robotics 4, eaau8892 11 Maimon-Mor, R.O. et al. (2020) Is an artificial limb embodied as a hand? Brain decoding in prosthetic limb users. Plos Biol In Press 12 Miehlbradt, J. et al. (2018) Data-driven body–machine interface for the accurate control of drones. Proc Natl Acad Sci USA 115, 7913–7918 

Soft Embodiment for Engineering Artificial Limbs

https://discovery.ucl.ac.uk/10113693/1/Soft-embodiment_Revisions-TICS_submitted.pdf

Soft Embodiment for Engineering Artificial Limbs

Abstract

Similar works

Full text

Available Versions

UCL Discovery