Robotic simulators for tissue examination training with multimodal sensory feedback

Tissue examination by hand remains an essential technique in clinical practice. The effective application depends on skills in sensorimotor coordination, mainly involving haptic, visual, and auditory feedback. The skills clinicians have to learn can be as subtle as regulating finger pressure with breathing, choosing palpation action, monitoring involuntary facial and vocal expressions in response to palpation, and using pain expressions both as a source of information and as a constraint on physical examination. Patient simulators can provide a safe learning platform to novice physicians before trying real patients. This paper reviews state-of-the-art medical simulators for the training for the first time with a consideration of providing multimodal feedback to learn as many manual examination techniques as possible. The study summarizes current advances in tissue examination training devices simulating different medical conditions and providing different types of feedback modalities. Opportunities with the development of pain expression, tissue modeling, actuation, and sensing are also analyzed to support the future design of effective tissue examination simulators

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications

Authority-Sharing Control of Assistive Robotic Walkers

A recognized consequence of population aging is a reduced level of mobility, which undermines the life quality of several senior citizens. A promising solution is represented by assisitive robotic walkers, combining the benefits of standard walkers (improved stability and physical support) with sensing and computing ability to guarantee cognitive support. In this context, classical robot control strategies designed for fully autonomous systems (such as fully autonomous vehicles, where the user is excluded from the loop) are clearly not suitable, since the user’s residual abilities must be exploited and practiced. Conversely, to guarantee safety even in the presence of user’s cognitive deficits, the responsibility of controlling the vehicle motion cannot be entirely left to the assisted person. The authority-sharing paradigm, where the control authority, i.e., the capability of controlling the vehicle motion, is shared between the human user and the control system, is a promising solution to this problem. This research develops control strategies for assistive robotic walkers based on authority-sharing: this way, we ensure that the walker provides the user only the help he/she needs for safe navigation. For instance, if the user requires just physical support to reach the restrooms, the robot acts as a standard rollator; however, if the user’s cognitive abilities are limited (e.g., the user does not remember where the restrooms are, or he/she does not recognize obstacles on the path), the robot also drives the user towards the proper corridors, by planning and following a safe path to the restrooms. The authority is allocated on the basis of an error metric, quantifying the distance between the current vehicle heading and the desired movement direction to perform the task. If the user is safely performing the task, he/she is endowed with control authority, so that his/her residual abilities are exploited. Conversely, if the user is not capable of safely solving the task (for instance, he/is going to collide with an obstacle), the robot intervenes by partially or totally taking the control authority to help the user and ensure his/her safety (for instance, avoiding the collision). We provide detailed control design and theoretical and simulative analyses of the proposed strategies. Moreover, extensive experimental validation shows that authority-sharing is a successful approach to guide a senior citizen, providing both comfort and safety. The most promising solutions include the use of haptic systems to suggest the user a proper behavior, and the modification of the perceived physical interaction of the user with the robot to gradually share the control authority using a variable stiffness vehicle handling

A Novel Haptic Simulator for Evaluating and Training Salient Force-Based Skills for Laparoscopic Surgery

Laparoscopic surgery has evolved from an \u27alternative\u27 surgical technique to currently being considered as a mainstream surgical technique. However, learning this complex technique holds unique challenges to novice surgeons due to their \u27distance\u27 from the surgical site. One of the main challenges in acquiring laparoscopic skills is the acquisition of force-based or haptic skills. The neglect of popular training methods (e.g., the Fundamentals of Laparoscopic Surgery, i.e. FLS, curriculum) in addressing this aspect of skills training has led many medical skills professionals to research new, efficient methods for haptic skills training. The overarching goal of this research was to demonstrate that a set of simple, simulator-based haptic exercises can be developed and used to train users for skilled application of forces with surgical tools. A set of salient or core haptic skills that underlie proficient laparoscopic surgery were identified, based on published time-motion studies. Low-cost, computer-based haptic training simulators were prototyped to simulate each of the identified salient haptic skills. All simulators were tested for construct validity by comparing surgeons\u27 performance on the simulators with the performance of novices with no previous laparoscopic experience. An integrated, \u27core haptic skills\u27 simulator capable of rendering the three validated haptic skills was built. To examine the efficacy of this novel salient haptic skills training simulator, novice participants were tested for training improvements in a detailed study. Results from the study demonstrated that simulator training enabled users to significantly improve force application for all three haptic tasks. Research outcomes from this project could greatly influence surgical skills simulator design, resulting in more efficient training

Haptics Rendering and Applications

There has been significant progress in haptic technologies but the incorporation of haptics into virtual environments is still in its infancy. A wide range of the new society's human activities including communication, education, art, entertainment, commerce and science would forever change if we learned how to capture, manipulate and reproduce haptic sensory stimuli that are nearly indistinguishable from reality. For the field to move forward, many commercial and technological barriers need to be overcome. By rendering how objects feel through haptic technology, we communicate information that might reflect a desire to speak a physically- based language that has never been explored before. Due to constant improvement in haptics technology and increasing levels of research into and development of haptics-related algorithms, protocols and devices, there is a belief that haptics technology has a promising future

The Effect of Sensory Experience and Movement Observation on Motor Adaptation to Novel Force Perturbations

On a daily basis, humans capably and effortlessly interact with their surrounding environment through the performance of accurate movements. Movements are often perturbed through the physical influence of the surrounding environment, interaction with objects, or injury, yet adaptation is both rapid and flexible. When adapting, humans are informed by their direct trial-and-error movement experience, incrementally updating predictive control on the next movement; how the brain processes sensed errors into adaptation is not fully known. We may also learn to move through the observation of the movements of others, as visually observed movement information may be transformed into motor memories that influence subsequent motor command; the neural computations underlying such a learning process are not well understood. In this thesis, I aimed to further understand how people incrementally update their predictive motor control in novel haptic environments as a function of sensation during action and during observation. Theories of motor learning suggest that adaptation scales with the size of experienced error. Previous studies have indicated the relationship between adaptation and sensed error can be modulated by statistics of the perturbing environment. In Chapter 2, we considered how the duration of experienced force perturbations might modulate adaptive strategy and found that people becomes increasingly sensitive to kinematic and dynamic sensory signals when experiencing perturbations of decreasing duration. We further found that subjects experiencing pulsatile forces adapted their steady-state feedforward prediction of dynamics with a persistently mismatched breadth when compared to the duration of experienced forces, but learned to closely match the experienced duration of full-movement forces. In the next two sections, we considered the learning effects of movement observation. In Chapter 3, we newly designed and implemented an experimental paradigm in which movement and observation were interleaved, varying the strength of perturbations and associated kinematics from trial-to-trial. Based on previous descriptions of long-term learning by observing, we hypothesized that incremental adaptation would be corrective with respect to observed errors but more modest in magnitude than gains from physical practice. Instead, we found that the incremental adaptive response of movement observation generally countered the direction of experienced forces and was similar in magnitude to the response following action, but was not error-corrective with respect to real-valued signals. Previous research had established an initial advantage when adapting to novel dynamics following observation but the learning processes influencing this effect were unknown. In Chapter 4, we newly demonstrated that the long-term movement observation resulted in adaptive changes in feedforward predictive dynamics. We found that observation generated a small, but significant, compensatory change in reach dynamics that could be characterized by a learned scaling of perturbation-appropriate kinematic signals, suggesting a transformation of visual inputs into a neural representation of environment dynamics

Novel Bidirectional Body - Machine Interface to Control Upper Limb Prosthesis

Objective. The journey of a bionic prosthetic user is characterized by the opportunities and limitations involved in adopting a device (the prosthesis) that should enable activities of daily living (ADL). Within this context, experiencing a bionic hand as a functional (and, possibly, embodied) limb constitutes the premise for mitigating the risk of its abandonment through the continuous use of the device. To achieve such a result, different aspects must be considered for making the artificial limb an effective support for carrying out ADLs. Among them, intuitive and robust control is fundamental to improving amputees’ quality of life using upper limb prostheses. Still, as artificial proprioception is essential to perceive the prosthesis movement without constant visual attention, a good control framework may not be enough to restore practical functionality to the limb. To overcome this, bidirectional communication between the user and the prosthesis has been recently introduced and is a requirement of utmost importance in developing prosthetic hands. Indeed, closing the control loop between the user and a prosthesis by providing artificial sensory feedback is a fundamental step towards the complete restoration of the lost sensory-motor functions. Within my PhD work, I proposed the development of a more controllable and sensitive human-like hand prosthesis, i.e., the Hannes prosthetic hand, to improve its usability and effectiveness. Approach. To achieve the objectives of this thesis work, I developed a modular and scalable software and firmware architecture to control the Hannes prosthetic multi-Degree of Freedom (DoF) system and to fit all users’ needs (hand aperture, wrist rotation, and wrist flexion in different combinations). On top of this, I developed several Pattern Recognition (PR) algorithms to translate electromyographic (EMG) activity into complex movements. However, stability and repeatability were still unmet requirements in multi-DoF upper limb systems; hence, I started by investigating different strategies to produce a more robust control. To do this, EMG signals were collected from trans-radial amputees using an array of up to six sensors placed over the skin. Secondly, I developed a vibrotactile system to implement haptic feedback to restore proprioception and create a bidirectional connection between the user and the prosthesis. Similarly, I implemented an object stiffness detection to restore tactile sensation able to connect the user with the external word. This closed-loop control between EMG and vibration feedback is essential to implementing a Bidirectional Body - Machine Interface to impact amputees’ daily life strongly. For each of these three activities: (i) implementation of robust pattern recognition control algorithms, (ii) restoration of proprioception, and (iii) restoration of the feeling of the grasped object's stiffness, I performed a study where data from healthy subjects and amputees was collected, in order to demonstrate the efficacy and usability of my implementations. In each study, I evaluated both the algorithms and the subjects’ ability to use the prosthesis by means of the F1Score parameter (offline) and the Target Achievement Control test-TAC (online). With this test, I analyzed the error rate, path efficiency, and time efficiency in completing different tasks. Main results. Among the several tested methods for Pattern Recognition, the Non-Linear Logistic Regression (NLR) resulted to be the best algorithm in terms of F1Score (99%, robustness), whereas the minimum number of electrodes needed for its functioning was determined to be 4 in the conducted offline analyses. Further, I demonstrated that its low computational burden allowed its implementation and integration on a microcontroller running at a sampling frequency of 300Hz (efficiency). Finally, the online implementation allowed the subject to simultaneously control the Hannes prosthesis DoFs, in a bioinspired and human-like way. In addition, I performed further tests with the same NLR-based control by endowing it with closed-loop proprioceptive feedback. In this scenario, the results achieved during the TAC test obtained an error rate of 15% and a path efficiency of 60% in experiments where no sources of information were available (no visual and no audio feedback). Such results demonstrated an improvement in the controllability of the system with an impact on user experience. Significance. The obtained results confirmed the hypothesis of improving robustness and efficiency of a prosthetic control thanks to of the implemented closed-loop approach. The bidirectional communication between the user and the prosthesis is capable to restore the loss of sensory functionality, with promising implications on direct translation in the clinical practice

Mechanisms of motor learning: by humans, for robots

Whenever we perform a movement and interact with objects in our environment, our central nervous system (CNS) adapts and controls the redundant system of muscles actuating our limbs to produce suitable forces and impedance for the interaction. As modern robots are increasingly used to interact with objects, humans and other robots, they too require to continuously adapt the interaction forces and impedance to the situation. This thesis investigated the motor mechanisms in humans through a series of technical developments and experiments, and utilized the result to implement biomimetic motor behaviours on a robot. Original tools were first developed, which enabled two novel motor imaging experiments using functional magnetic resonance imaging (fMRI). The first experiment investigated the neural correlates of force and impedance control to understand the control structure employed by the human brain. The second experiment developed a regressor free technique to detect dynamic changes in brain activations during learning, and applied this technique to investigate changes in neural activity during adaptation to force fields and visuomotor rotations. In parallel, a psychophysical experiment investigated motor optimization in humans in a task characterized by multiple error-effort optima. Finally a computational model derived from some of these results was implemented to exhibit human like control and adaptation of force, impedance and movement trajectory in a robot