Robotic simulators for tissue examination training with multimodal sensory feedback

Tissue examination by hand remains an essential technique in clinical practice. The effective application depends on skills in sensorimotor coordination, mainly involving haptic, visual, and auditory feedback. The skills clinicians have to learn can be as subtle as regulating finger pressure with breathing, choosing palpation action, monitoring involuntary facial and vocal expressions in response to palpation, and using pain expressions both as a source of information and as a constraint on physical examination. Patient simulators can provide a safe learning platform to novice physicians before trying real patients. This paper reviews state-of-the-art medical simulators for the training for the first time with a consideration of providing multimodal feedback to learn as many manual examination techniques as possible. The study summarizes current advances in tissue examination training devices simulating different medical conditions and providing different types of feedback modalities. Opportunities with the development of pain expression, tissue modeling, actuation, and sensing are also analyzed to support the future design of effective tissue examination simulators

A technology-aided multi-modal training approach to assist abdominal palpation training and its assessment in medical education

Kinaesthetic Learning Activities (KLA) are techniques for enhancing the motor learning process to provide a deep understanding of fundamental skills in particular disciplines. With KLA learning takes place by carrying out a physical activity to transform empirical achievements into representative cognitive understanding. In disciplines such as medical education, frequent hands-on practice of certain motor skills plays a key role in the development of medical students' competency. Therefore it is essential that clinicians master these core skills early on in their educational journey as well as retain them for the entirety of their career. Transferring knowledge of performing dexterous motor skills, such as clinical examinations, from experts to novices demands a systematic approach to quantify relevant motor variables with the help of medical experts in order to form a reference best practice model for target skills. Additional information (augmented feedback) on certain aspects of movements could be extracted from this model and visualised via multi-modal sensory channels in order to enhance motor performance and learning processes. This thesis proposes a novel KLA methodology to significantly improve the quality of palpation training in medical students. In particular, it investigates whether it is possible to enhance the existing abdominal palpation skills acquisition process (motor performance and learning) with provision of instructional concurrent and terminal augmented feedback on applied forces by the learner's hand via an autonomous multimodal displays. This is achieved by considering the following: identifying key motor variables with help of medical experts; forming a gold standard model for target skills by collecting pre-defined motor variables with an innovative quantification technique; designing an assessment criteria by analysing the medical experts' data; and systematically evaluating the impact of instructional augmented feedback on medical students' motor performance with two distinct assessment approaches(a machine-based and a human-based). In addition, an evaluation of performance on a simpler task is carried out using a game-based training method, to compare feedback visualisation techniques, such as concurrent visual and auditory feedback as used in a serious games environment, with abstract visualisation of motor variables. A detailed between-participants study is presented to evaluate the effect of concurrent augmented feedback on participants' skills acquisition in the motor learning process. Significant improvement on medical students' motor performance was observed when augmented feedback on applied forces were visually presented (H(2) = 6:033, p < :05). Moreover, a positive correlation was reported between computer-generated scores and human-generated scores, r = :62, p (one-tailed) < :05. This indicates the potential of the computer-based assessment technique to assist the current assessment process in medical education. The same results were also achieved in a blind-folded (no-feedback) transfer test to evaluate performance and short-term retention of skills in the game-based training approach. The accuracy in the exerted target force for participants in the game-playing group, who were trained using the game approach (Mdn = 0:86), differed significantly from the participants in control group, who trained using the abstract visualisation of the exerted force value (Mdn = 1:56), U = 61, z = -2:137, p < :05, r = -0:36. Finally, the usability of both motor learning approaches were surveyed via feedback questionnaires and positive responses were achieved from users. The research presented shows that concurrent augmented feedback significantly improves the participants' motor control abilities. Furthermore, advanced visualisation techniques such as multi-modal displays increases the participants' motivation to engage in learning and to retain motor skills

Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques.Comment: Accepted by Medical Image Analysi

User experiences of virtual reality technologies for healthcare in learning: an integrative review

The aim of this integrative review was to analyse the usage of different virtual reality (VR) technologies in learning and user experiences (UXs) of these technologies in healthcare practice and education. The integrative review was conducted in spring 2019 by searching eight international databases. The searches retrievedn = 26 original articles that were quality checked and included for the review. Three different VR technologies used in the field of healthcare education and practice were identified: haptic device simulators, computer-based simulations and head-mounted displays (HMDs). The haptic simulators were the most often used, whereas the HMD devices were the least-used technology in the field of healthcare. In immersive virtual environments, UX includes ten components. Most of the components were observed in the context of haptic devices and HMD devices, with all ten components being observed with the HMD devices. Almost all of the components were rated as positive. In conclusion, the development of VR technology has enabled the creation of the most comprehensive UXs, thus enhancing skill development, enabling remote access to training and, ultimately, improving patient safety. This review is important as it highlights the need for far more UX research within immersive virtual environments

Development of Piezoresistive Tactile Sensors and a Graphical Display System for Minimally Invasive Surgery and Robotics

Development of Piezoresistive Tactile Sensors and a Graphical Display System for Minimally Invasive Surgery and Robotics Masoud Kalantari, PhD Concordia University, 2013 This PhD work presents a new tactile and feedback systems for minimally invasive surgery (MIS)and robotics. The thesis is divided into two major sections: the tactile sensing system, and the graphical display system. In the tactile sensing system, piezoresistive materials are used as measuring elements. The first part of the thesis is focused on the theoretical modeling of piezoresistive sensing elements, which are semiconductive polymer composites. The model predicts the piezoresistive behavior in semiconductive polymer composites, including their creep effect and contact resistance. A single force sensing resistor (FSR) is, then, developed by using the semiconductive polymer composite materials. The developed FSR is used in the structure of a novel tactile sensor as the transduction element. The developed tactile sensor is designed to measure the difference in the hardness degree of soft tissues. This capability of the sensor helps surgeons to distinguish different types of tissues involved in the surgery. The tactile sensor is integrated on the extremity of a surgical tool to provide tactile feedback from the interaction between surgical instruments and the tissue during MIS. Mitral valve annuloplasty repair by MIS is of our particular interest to be considered as a potential target for the use of the developed tactile sensor. In the next step, the contact interaction of the tactile sensor with soft tissues is modelled, parametrically. Viscoelastic interaction is considered between the tactile sensor and atrial tissue in annuloplasty mitral valve repair; and a parametric solution for the viscoelastic contact is achieved. In addition to the developed sensor, a novel idea regarding measuring the indentation rate, in addition to measuring force and displacement is implemented in a new design of an array tactile sensor. It is shown that the indentation-rate measurement is an important factor in distinguishing the hardness degree of tissues with viscoelastic behaviour. The second part of the thesis is focused on the development of a three-dimensional graphical display that provides visual palpation display to any surgeon performing robotic assisted MIS. Two matrices of the developed piezoresistive force sensor are used to palpate the tissue and collect the tactile information. The collected data are processed with a new algorithm and graphically rendered in three dimensions. Consequently, the surgeon can determine the presence, location, and the size of any hidden superficial tumor/artery by grasping the target tissue in a quasi-dynamic way

A role for sensory areas in coordinating active sensing motions

Active sensing, which incorporates closed-loop behavioral selection of information during sensory acquisition, is an important feature of many sensory modalities. We used the rodent whisker tactile system as a platform for studying the role cortical sensory areas play in coordinating active sensing motions. We examined head and whisker motions of freely moving mice performing a tactile search for a randomly located reward, and found that mice select from a diverse range of available active sensing strategies. In particular, mice selectively employed a strategy we term contact maintenance, where whisking is modulated to counteract head motion and sustain repeated contacts, but only when doing so is likely to be useful for obtaining reward. The context dependent selection of sensing strategies, along with the observation of whisker repositioning prior to head motion, suggests the possibility of higher level control, beyond simple reflexive mechanisms. In order to further investigate a possible role for primary somatosensory cortex (SI) in coordinating whisk-by-whisk motion, we delivered closed-loop optogenetic feedback to SI, time locked to whisker motions estimated through facial electromyography. We found that stimulation regularized whisking (increasing overall periodicity), and shifted whisking frequency, changes that emulate behaviors of rodents actively contacting objects. Importantly, we observed changes to whisk timing only for stimulation locked to whisker protractions, possibly encoding that natural contacts are more likely during forward motion of the whiskers. Simultaneous neural recordings from SI show cyclic changes in excitability, specifically that responses to excitatory stimulation locked to whisker retractions appeared suppressed in contrast to stimulation during protractions that resulted in changes to whisk timing. Both effects are evident within single whisks. These findings support a role for sensory cortex in guiding whisk-by-whisk motor outputs, but suggest a coupling that depends on behavioral context, occurring on multiple timescales. Elucidating a role for sensory cortex in motor outputs is important to understanding active sensing, and may further provide novel insights to guide the design of sensory neuroprostheses that exploit active sensing context

Contextualising simulation: the use of patient-focused hybrid simulation for clinical skills education

This thesis documents a research programme into the use of Patient-Focused Hybrid Simulation (PFHS) for clinical skills education. PFHS is an approach to simulating clinical skills that combines a simulated patient (SP) with a part-task trainer (PTT) embedding the simulation of procedural skills within a more holistic clinical context, potentially overcoming some of the shortcomings of single modality simulation. Although promising, there remains limited evidence supporting its use. Two studies were conducted using a mixed-method approach. The first study was based on the simulation of the management of a traumatic skin laceration and consisted of two parts: 1) investigating the use of PFHS as a means of introducing clinical challenge by modifying the clinical context in which a procedure is performed; 2) exploring clinician’s perception of the use of PFHS and PTT for assessing of clinical competence. These findings suggest that by changing the clinical context in which a procedure is performed, PFHS can potentially be used to objectively simulate challenge. It also demonstrated that PFHS when compared to PTT simulations was better able to induce authentic clinical behaviour within the simulation. Central to this is the presence of a human being (SP). The second study compared the use of PFHS to patients for the training and assessment of cardiovascular examination skills. Within the limitations of this study, no significant difference was observed between PFHS and real patient-trained students in terms of their post-training performance of cardiovascular examination on real patients. There also appeared to be degree of concurrent validity between assessment of competency with PFHS and with real patients when conducted as an Objective Structured Clinical Examination (OSCE). The work presented provides additional evidence to the existing literature to support the use of PFHS in clinical skills education. However, it also raises a multitude of questions particularly of how PFHS as well as simulation in general should be used and future directions for simulation research.Open Acces