16 research outputs found
Multi-fingered haptic palpation utilizing granular jamming stiffness feedback actuators
This paper describes a multi-fingered haptic palpation method using stiffness feedback actuators for simulating tissue palpation procedures in traditional and in robot-assisted minimally invasive surgery. Soft tissue stiffness is simulated by changing the stiffness property of the actuator during palpation. For the first time, granular jamming and pneumatic air actuation are combined to realize stiffness modulation. The stiffness feedback actuator is validated by stiffness measurements in indentation tests and through stiffness discrimination based on a user study. According to the indentation test results, the introduction of a pneumatic chamber to granular jamming can amplify the stiffness variation range and reduce hysteresis of the actuator. The advantage of multi-fingered palpation using the proposed actuators is proven by the comparison of the results of the stiffness discrimination performance using two-fingered (sensitivity: 82.2%, specificity: 88.9%, positive predicative value: 80.0%, accuracy: 85.4%, time: 4.84 s) and single-fingered (sensitivity: 76.4%, specificity: 85.7%, positive predicative value: 75.3%, accuracy: 81.8%, time: 7.48 s) stiffness feedback
Pseudo-Haptics for Rigid Tool/Soft Object Interaction Feedback in Virtual Environments
This paper proposes a novel pseudo-haptics soft object stiffness simulation technique which is a marked improvement to currently used simulation methods and an effective low-cost alternative to expensive 3-DOF haptic devices. Soft object stiffness simulation is achieved by maneuvering an indenter avatar over the surface of a virtual soft object by means of an input device, such as a mouse, a joystick, or a touch-sensitive tablet. The alterations to the indenter avatar behavior produced by the proposed technique create for the user the illusion of interaction with a hard inclusion embedded in the soft object. The proposed pseudo-haptics technique is validated with a series of experiments conducted by employing three types of 2-DOF force-sensitive haptic surfaces, including a touchpad, a tablet with an S-pen input, and a tablet with a bare finger input. It is found that both the sensitivity and the positive predictive value of hard inclusion detection can be significantly improved by 33.3% and 13.9% respectively by employing tablet computers. Using tablet computers could produce results comparable to direct hand touch in detecting hard inclusions in a soft object. The experimental results presented here confirm the potential of the proposed technique for conveying haptic information in rigid tool / soft object interaction in virtual environments
VISIO-HAPTIC DEFORMABLE MODEL FOR HAPTIC DOMINANT PALPATION SIMULATOR
Vision and haptic are two most important modalities in a medical simulation. While
visual cues assist one to see his actions when performing a medical procedure, haptic
cues enable feeling the object being manipulated during the interaction. Despite their
importance in a computer simulation, the combination of both modalities has not been
adequately assessed, especially that in a haptic dominant environment. Thus, resulting
in poor emphasis in resource allocation management in terms of effort spent in
rendering the two modalities for simulators with realistic real-time interactions.
Addressing this problem requires an investigation on whether a single modality
(haptic) or a combination of both visual and haptic could be better for learning skills
in a haptic dominant environment such as in a palpation simulator. However, before
such an investigation could take place one main technical implementation issue in
visio-haptic rendering needs to be addresse
A Variable Stiffness Robotic Probe for Soft Tissue Palpation
During abdominal palpation diagnosis, a medical practitioner would change the stiffness of their fingers in order to improve the detection of hard nodules or abnormalities in soft tissue to maximize the haptic information gain via tendons. Our recent experiments using a controllable stiffness robotic probe representing a human finger also confirmed that such stiffness control in the finger can enhance the accuracy of detecting hard nodules in soft tissue. However, the limited range of stiffness achieved by the antagonistic springs variable stiffness joint subject to size constraints made it unsuitable for a wide range of physical examination scenarios spanning from breast to abdominal examination. In this letter, we present a new robotic probe based on a variable lever mechanism able to achieve stiffness ranging from 0.64 to 1.06 N â‹…m/rad that extends the maximum stiffness by around 16 times and the stiffness range by 33 times. This letter presents the mechanical model of the novel probe, the finite element simulation as well as experimental characterization of the stiffness response for lever actuation
A Variable Stiffness Robotic Probe for Soft Tissue Palpation
During abdominal palpation diagnosis, a medical practitioner would change the stiffness of their fingers in order to improve the detection of hard nodules or abnormalities in soft tissue to maximize the haptic information gain via tendons. Our recent experiments using a controllable stiffness robotic probe representing a human finger also confirmed that such stiffness control in the finger can enhance the accuracy of detecting hard nodules in soft tissue. However, the limited range of stiffness achieved by the antagonistic springs variable stiffness joint subject to size constraints made it unsuitable for a wide range of physical examination scenarios spanning from breast to abdominal examination. In this letter, we present a new robotic probe based on a variable lever mechanism able to achieve stiffness ranging from 0.64 to 1.06 N·m/rad that extends the maximum stiffness by around 16 times and the stiffness range by 33 times. This letter presents the mechanical model of the novel probe, the finite element simulation as well as experimental characterization of the stiffness response for lever actuation.This work was supported by The United Kingdom Engineering and Physical Sciences Research Council under MOTION Grant EP/N03211X/2
VISIO-HAPTIC DEFORMABLE MODEL FOR HAPTIC DOMINANT PALPATION SIMULATOR
Vision and haptic are two most important modalities in a medical simulation. While
visual cues assist one to see his actions when performing a medical procedure, haptic
cues enable feeling the object being manipulated during the interaction. Despite their
importance in a computer simulation, the combination of both modalities has not been
adequately assessed, especially that in a haptic dominant environment. Thus, resulting
in poor emphasis in resource allocation management in terms of effort spent in
rendering the two modalities for simulators with realistic real-time interactions.
Addressing this problem requires an investigation on whether a single modality
(haptic) or a combination of both visual and haptic could be better for learning skills
in a haptic dominant environment such as in a palpation simulator. However, before
such an investigation could take place one main technical implementation issue in
visio-haptic rendering needs to be addresse
Estimating and understanding motion : from diagnostic to robotic surgery
Estimating and understanding motion from an image sequence is a central topic in computer vision. The high interest in this topic is because we are living in a world where many events that occur in the environment are dynamic. This makes motion estimation and understanding a natural component and a key factor in a widespread of applications including object recognition , 3D shape reconstruction, autonomous navigation and medica! diagnosis.
Particularly, we focus on the medical domain in which understanding the human body for clinical purposes requires retrieving the organs' complex motion patterns, which is in general a hard problem when using only image data. In this thesis, we cope with this problem by posing the question - How to achieve a realistic motion estimation to offer a better clinical understanding? We focus this thesis on answering this question by using a variational formulation as a basis to understand one of the most complex motions in the human's body, the heart motion, through three different applications: (i) cardiac motion estimation for diagnostic, (ii) force estimation and (iii) motion prediction, both for robotic surgery.
Firstly, we focus on a central topic in cardiac imaging that is the estimation of the cardiac motion. The main aim is to offer objective and understandable measures to physicians for helping them in the diagnostic of cardiovascular diseases. We employ ultrafast ultrasound data and tools for imaging motion drawn from diverse areas such as low-rank analysis and variational deformation to perform a realistic cardiac motion estimation. The significance is that by taking low-rank data with carefully chosen penalization, synergies in this complex variational problem can be created. We demonstrate how our proposed solution deals with complex deformations through careful numerical experiments using realistic and simulated data.
We then move from diagnostic to robotic surgeries where surgeons perform delicate procedures remotely through robotic manipulators without directly interacting with the patients. As a result, they lack force feedback, which is an important primary sense for increasing surgeon-patient transparency and avoiding injuries and high mental workload. To solve this problem, we follow the conservation principies of continuum mechanics in which it is clear that the change in shape of an elastic object is directly proportional to the force applied. Thus, we create a variational framework to acquire the deformation that the tissues undergo due to an applied force. Then, this information is used in a learning system to find the nonlinear relationship between the given data and the applied force. We carried out experiments with in-vivo and ex-vivo data and combined statistical, graphical and perceptual analyses to demonstrate the strength of our solution.
Finally, we explore robotic cardiac surgery, which allows carrying out complex procedures including Off-Pump Coronary Artery Bypass Grafting (OPCABG). This procedure avoids the associated complications of using Cardiopulmonary Bypass (CPB) since the heart is not arrested while performing the surgery on a beating heart. Thus, surgeons have to deal with a dynamic target that compromisetheir dexterity and the surgery's precision. To compensate the heart motion, we propase a solution composed of three elements: an energy function to estimate the 3D heart motion, a specular highlight detection strategy and a prediction approach for increasing the robustness of the solution. We conduct evaluation of our solution using phantom and realistic datasets.
We conclude the thesis by reporting our findings on these three applications and highlight the dependency between motion estimation and motion understanding at any dynamic event, particularly in clinical scenarios.L’estimació i comprensió del moviment dins d’una seqüència d’imatges és un tema central en la visió per ordinador, el que genera un gran interès perquè vivim en un entorn ple d’esdeveniments dinà mics. Per aquest motiu és considerat com un component natural i factor clau dins d’un ampli ventall d’aplicacions, el qual inclou el reconeixement d’objectes, la reconstrucció de formes tridimensionals, la navegació autònoma i el diagnòstic de malalties.
En particular, ens situem en l’à mbit mèdic en el qual la comprensió del cos humà , amb finalitats clÃniques, requereix l’obtenció de patrons complexos de moviment dels òrgans. Aquesta és, en general, una tasca difÃcil quan s’utilitzen només dades de tipus visual. En aquesta tesi afrontem el problema plantejant-nos la pregunta - Com es pot aconseguir una estimació realista del moviment amb l’objectiu d’oferir una millor comprensió clÃnica? La tesi se centra en la resposta mitjançant l’ús d’una formulació variacional com a base per entendre un dels moviments més complexos del cos humà , el del cor, a través de tres aplicacions: (i) estimació del moviment cardÃac per al diagnòstic, (ii) estimació de forces i (iii) predicció del moviment, orientant-se les dues últimes en cirurgia robòtica.
En primer lloc, ens centrem en un tema principal en la imatge cardÃaca, que és l’estimació del moviment cardÃac. L’objectiu principal és oferir als metges mesures objectives i comprensibles per ajudar-los en el diagnòstic de les malalties cardiovasculars. Fem servir dades d’ultrasons ultrarà pids i eines per al moviment d’imatges procedents de diverses à rees, com ara l’anà lisi de baix rang i la deformació variacional, per fer una estimació realista del moviment cardÃac. La importà ncia rau en que, en prendre les dades de baix rang amb una penalització acurada, es poden crear sinergies en aquest problema variacional complex. Mitjançant acurats experiments numèrics, amb dades realÃstiques i simulades, hem demostrat com les nostres propostes solucionen deformacions complexes.
Després passem del diagnòstic a la cirurgia robòtica, on els cirurgians realitzen procediments delicats remotament, a través de manipuladors robòtics, sense interactuar directament amb els pacients. Com a conseqüència, no tenen la percepció de la força com a resposta, que és un sentit primari important per augmentar la transparència entre el cirurgià i el pacient, per evitar lesions i per
reduir la cà rrega de treball mental. Resolem aquest problema seguint els principis de conservació de la mecà nica del medi continu, en els quals està clar que el canvi en la forma d’un objecte elà stic és directament proporcional a la força aplicada. Per això hem creat un marc variacional que adquireix la deformació que pateixen els teixits per l’aplicació d’una força. Aquesta informació s’utilitza en un sistema d’aprenentatge, per trobar la relació no lineal entre les dades donades i la força aplicada. Hem dut a terme experiments amb dades in-vivo i ex-vivo i hem combinat l’anà lisi estadÃstic, grà fic i de percepció que demostren la robustesa de la nostra solució. Finalment, explorem la cirurgia cardÃaca robòtica, la qual cosa permet realitzar procediments complexos, incloent la cirurgia coronà ria sense bomba (off-pump coronary artery bypass grafting o OPCAB). Aquest procediment evita les complicacions associades a l’ús de circulació extracorpòria (Cardiopulmonary Bypass o CPB), ja que el cor no s’atura mentre es realitza la cirurgia. Això
comporta que els cirurgians han de tractar amb un objectiu dinà mic que compromet la seva destresa i la precisió de la cirurgia. Per compensar el moviment del cor, proposem una solució composta de tres elements: un funcional d’energia per estimar el moviment tridimensional del cor, una estratègia de detecció de les reflexions especulars i una aproximació basada en mètodes de predicció, per tal d’augmentar la robustesa de la solució. L’avaluació de la nostra solució s’ha dut
a terme mitjançant conjunts de dades sintètiques i realistes.
La tesi conclou informant dels nostres resultats en aquestes tres aplicacions i posant de relleu la dependència entre l’estimació i la comprensió del moviment en qualsevol esdeveniment dinà mic, especialment en escenaris clÃnics.Postprint (published version
Towards an efficient haptic rendering using data-driven modeling
This thesis focuses on the optimisation of haptic rendering of interactions with deformable models. The research demonstrated that data-driven techniques can produce a real-time, accurate and complex simulation experience. Applications include, but not limited to, virtual training, rapid prototyping, virtual presence, and entertainment