Realistic Haptic Rendering of Interacting Deformable Objects in Virtual Environments

International audienceA new computer haptics algorithm to be used in general interactive manipulations of deformable virtual objects is presented. In multimodal interactive simulations, haptic feedback computation often comes from contact forces. Subsequently, the fidelity of haptic rendering depends significantly on contact space modeling. Contact and friction laws between deformable models are often simplified in up to date methods. They do not allow a "realistic" rendering of the subtleties of contact space physical phenomena (such as slip and stick effects due to friction or mechanical coupling between contacts). In this paper, we use Signorini's contact law and Coulomb's friction law as a computer haptics basis. Real-time performance is made possible thanks to a linearization of the behavior in the contact space, formulated as the so-called Delassus operator, and iteratively solved by a Gauss-Seidel type algorithm. Dynamic deformation uses corotational global formulation to obtain the Delassus operator in which the mass and stiffness ratio are dissociated from the simulation time step. This last point is crucial to keep stable haptic feedback. This global approach has been packaged, implemented, and tested. Stable and realistic 6D haptic feedback is demonstrated through a clipping task experiment

Haptic Rendering of Hyperelastic Models with Friction

International audience— This paper presents an original method for inter-actions' haptic rendering when treating hyperelastic materials. Such simulations are known to be difficult due to the non-linear behavior of hyperelastic bodies; furthermore, haptic constraints enjoin contact forces to be refreshed at least at 1000 updates per second. To enforce the stability of simulations of generic objects of any range of stiffness, this method relies on implicit time integration. Soft tissues dynamics is simulated in real time (20 to 100 Hz) using the Multiplicative Jacobian Energy Decomposition (MJED) method. An asynchronous preconditioner, updated at low rates (1 to 10 Hz), is used to obtain a close approximation of the mechanical coupling of interactions. Finally, the contact problem is linearized and, using a specific-loop, it is updated at typical haptic rates (around 1000 Hz) allowing this way new simulations of prompt stiff-contacts and providing a continuous haptic feedback as well

Sensory Communication

Contains table of contents for Section 2, an introduction and reports on twelve research projects.National Institutes of Health Grant R01 DC00117National Institutes of Health Grant R01 DC02032National Institutes of Health/National Institute of Deafness and Other Communication Disorders Grant 2 R01 DC00126National Institutes of Health Grant 2 R01 DC00270National Institutes of Health Contract N01 DC-5-2107National Institutes of Health Grant 2 R01 DC00100U.S. Navy - Office of Naval Research Grant N61339-96-K-0002U.S. Navy - Office of Naval Research Grant N61339-96-K-0003U.S. Navy - Office of Naval Research Grant N00014-97-1-0635U.S. Navy - Office of Naval Research Grant N00014-97-1-0655U.S. Navy - Office of Naval Research Subcontract 40167U.S. Navy - Office of Naval Research Grant N00014-96-1-0379U.S. Air Force - Office of Scientific Research Grant F49620-96-1-0202National Institutes of Health Grant RO1 NS33778Massachusetts General Hospital, Center for Innovative Minimally Invasive Therapy Research Fellowship Gran

Frictional Contact in Interactive Deformable Environments

L\u2019uso di simulazioni garantisce notevoli vantaggi in termini di economia, realismo e di flessibilit\ue0 in molte aree di ricerca e in ambito dello sviluppo tecnologico. Per questo motivo le simulazioni vengono usate spesso in ambiti quali la prototipazione di parti meccaniche, nella pianificazione e nell\u2019addestramento di procedure di assemblaggio e disassemblaggio inoltre, di recente, le simulazioni si sono dimostrate validi strumenti anche nell\u2019assistenza e nell\u2019addestramento ai chirurghi, in particolare nel caso della chirurgia laparoscopica. La chirurgia laparoscopica, infatti, \ue8 considerata lo standard per molte procedure chirurgiche. La principale differenza rispetto alla chirurgia tradizionale risiede nella notevole limitazione che ha il chirurgo nell\u2019interagire e nel percepire l\u2019ambiente in lavora, sia nella vista che nel tatto. Questo rappresenta una forte limitazione per il chirurgo a cui \ue8 richiesta una lunga fase di addestramento prima di poter ottenere la necessaria destrezza per intervenire in laparoscopia con profitto. Queste limitazioni, d\u2019altra parte, rendono la laparoscopia il candidato ideale per l\u2019introduzione della simulazione nell\u2019addestramento. Attualmente sono disponibili in commercio dei software per l\u2019addestramento alla laparoscopia, tuttavia essi sono in genere basati su modelli rigidi, o modelli che comunque mancano del necessario realismo fisico. L\u2019introduzione di modelli deformabili migliorerebbe notevolmente l\u2019accuratezza e il realismo delle simulazioni. Nel caso dell\u2019addestramento il maggior realismo permetterebbe all\u2019utente di acquisire non solo le conoscenze motorie basilari ma anche capacit\ue0 e conoscenze di pi\uf9 alto livello. I corpi rigidi, infatti, rappresentano una buona approssimazione della realt\ue0 solo in situazioni particolari ed entro intervalli di sollecitazioni molto ristretti. Quando si considerano materiali non ingegneristici, come accade nelle simulazioni chirurgiche, le deformazioni non possono essere trascurate senza compromettere irrimediabilmente il realismo dei risultati. L\u2019uso di modelli deformabili tuttavia introduce notevole complessit\ue0 computazionale per il calcolo della fisica che regola le deformazioni e limita fortemente l\u2019uso di dati precalcolati, spesso utilizzati per velocizzare la fase di identificazione delle collisioni tra i corpi. I ritardi dovuti all\u2019uso di modelli deformabili rappresentano un grosso limite soprattutto nelle applicazioni interattive che, per consentire all\u2019utente di interagire con l\u2019ambiente, richiedono il calcolo della simulazione entro intervalli di tempo molto ridotti. In questa tesi viene affrontato il tema della simulazione di ambienti interattivi composti da corpi deformabili che interagiscono con attrito. Vengono analizzati e sviluppati differenti tecniche e metodi per le diverse componenti della simulazione: dalla simulazione di modelli deformabili, agli algoritmi di identificazione e soluzione delle collisioni e alla modellazione e integrazione dell\u2019attrito nella simulazione. In particolare vengono valutati i principali metodi che rappresentano lo stato dell\u2019arte nella modellazione di materiali deformabili. L\u2019analisi considera i fondamenti fisici su cui i modelli si basano e quindi sul grado di realismo che possono garantire in termini di deformazioni modellabili e la semplicit\ue0 d\u2019uso degli stessi (ovvero la facilit\ue0 di comprensione del metodo, la calibrazione del modello e la possibilit\ue0 di adattare il modello a situazioni differenti) ma viene considerata anche la complessit\ue0 computazionale di ciascun metodo in quanto essa rappresenta un fattore estremamente importante nella scelta e nell\u2019uso dei modelli deformabili nelle simulazioni. Il confronto dei differenti modelli e le caratteristiche identificate hanno motivato lo sviluppo di un metodo innovativo per fornire un\u2019interfaccia comune ai vari metodi di simulazione dei tessuti deformabili. Tale interfaccia ha il vantaggio di fornire dei metodi omogenei per la manipolazione dei diversi modelli deformabili. Ci\uf2 garantisce la possibilit\ue0 di scambiare il modello usato per la simulazione delle deformazioni mantenendo inalterati le altre strutture dati e i metodi della simulazione. L\u2019introduzione di tale interfaccia unificata si dimostra particolarmente vantaggiosa in quanto permette l\u2019uso di un solo metodo per l\u2019identificazione delle collisioni per tutti i differenti modelli deformabili. Ci\uf2 semplifica molto l\u2019analisi e la definizione dei requisiti di tale modulo software. L\u2019identificazione delle collisioni tra modelli rigidi generalmente precalcola delle partizioni dello spazio in cui i corpi sono definiti oppure sfrutta la suddivisione del corpo analizzato in parti convesse per velocizzare la simulazione. Nel caso di modelli deformabili non \ue8 possibile applicare tali tecniche a causa dei continui cambiamenti nella configurazione dei corpi. Dopo che le collisioni tra i corpi sono state riconosciute e che i punti di contatto sono stati identificati e necessario risolvere le collisioni tenendo conto della fisica sottostante i contatti. Per garantire il realismo \ue8 necessario assicurare che i corpi non si compenetrino mai e che nella simulazione delle collisioni tutti i fenomeni fisici di interesse coinvolti nel contatto tra i corpi vengano considerati: questi includono le forze elastiche che si esercitano tra i corpi e le forze di attrito che si generano lungo le superfici di contatto. L\u2019innovativo metodo proposto per la soluzione delle collisioni garantisce il realismo della simulazione e l\u2019integrazione con l\u2019interfaccia proposta per la gestione unificata dei modelli. Una caratteristica importante dei tessuti biologici \ue8 il comportamento anisotropico, dovuto, in genere, alla loro struttura fibrosa. In questa tesi viene proposto un nuovo metodo per aggiungere l\u2019anisotropia al comportamento dei modelli massa molla. Il metodo ha il vantaggio di mantenere la velocit\ue0 computazionale e la semplicit\ue0 di implementazione dei modelli massa molla classici e riesce a differenziare efficacemente la risposta del modello alle sollecitazioni lungo le differenti direzioni. Le tecniche descritte sono state integrate in due applicazioni che forniscono la simulazione della fisica di ambienti con corpi deformabili. La prima delle due implementa tutti i metodi descritti per la simulazione dei modelli deformabili, identifica le collisioni con precisione e le risolve fornendo la possibilit\ue0 di scegliere il modello di attrito pi\uf9 adatto, dimostrando cos\uec la fattibilit\ue0 dell\u2019approccio proposto. La limitazione principale di tale simulatore risiede nell\u2019alto tempo di calcolo richiesto per la simulazione dei singoli passi di simulazione. Tale limitazione \ue8 stata superata in una seconda implementazione che sfrutta il parallelismo intrinseco delle simulazioni fisiche per ottimizzare gli algoritmi e che, quindi, riesce a sfruttare al meglio la potenza computazionale delle architetture hardware parallele. Al fine di ottenere le prestazioni richieste per la simulazione di ambienti interattivi con ritorno di forza, la simulazione \ue8 basata su un algoritmo di identificazione delle collisioni semplificato, ma implementa gli altri metodi descritti in questa tesi. L\u2019implementazione parallela sfrutta le capacit\ue0 di calcolo delle moderne schede video munite di processori altamente paralleli e ci\uf2 permette di aggiornare la scena ogni millisecondo. Questo elimina ogni discontinuit\ue0 nel ritorno di forza reso all\u2019utente e nell\u2019aggiornamento della grafica della scena, inoltre garantisce il realismo necessario alla simulazione fisica sottostante. Le applicazioni implementate provano la fattibilit\ue0 della simulazione della fisica di interazioni complesse tra corpi deformabili. Inoltre, l\u2019implementazione parallela della simulazione rappresenta un promettente punto di partenza per la realizzazione di simulazioni interattive che potr\ue0 essere utilizzato in ambiti di ricerca differenti, quali l\u2019addestramento di chirurghi o la prototipazione rapida.The use of simulations provides great advantages in term of economy, realism, and adaptability to user requirements in many research and technological fields. For this reason simulations are currently exploited, for example, in prototyping of machinery parts, in assembly-disassembly test or training and, recently, simulations have also allowed the development of many useful and promising tools for the assistance and learning of surgical procedures. This is particularly true for laparoscopic intervention. Laparoscopy, in fact, represents the gold standard for many surgical procedures. The principal difference from standard surgery is the reduction of the surgeon ability to perceive the surgical scenario, both from visual and tactile point of view. This represents a great limitation for surgeons who undergo long training before being able to perform laparoscopic intervention with proficiency. This, on the other hand, makes laparoscopy an excellent candidate for the use of simulations for training. Some commercial training softwares are already available on the market, but they are usually based on rigid body models that completely lack the physical realism. The introduction of deformable models may leads to a great increment in terms of realism and accuracy. And, in the case of laparoscopy trainer it may allow the user to learn not only basic motor skills, but also higher level capabilities and knowledge. Rigid bodies, in fact, represents a good approximation of reality only in some situations and in very restricted ranges of solicitations. In particular, when non engineering materials are involved, as happens in surgical simulations, deformations cannot be neglected without completely loosing the realism of the environment. The use of deformable models, however, is limited for the high computational costs involved in the computation of the physics undergoing the deformations and because of the reduction in pre computable data in particular for collision detection between bodies. This represents a very limiting factor in interactive environments where, to allow the user to interactively control the virtual bodies, the simulation should be performed in real time. In this thesis we address the simulation of interactive environment populated with deformable models that interact with frictional contacts. This includes the analysis and the development of different techniques which implement the various parts of the simulation: mainly the methods for the simulation of deformable models, the collision detection and collision solution techniques but also the modelling and the integration of suitable friction models in the simulation. In particular we evaluated the principal methods that represent the state of the art in soft tissue modeling. Our analysis is based on the physical background of each method and thus on its realism in terms of deformations that the method can mimic and on the ease of use (i.e. method understanding, calibration and ability to adapt to different scenarios) but we also compared the computational complexity of different models, as it represents an extremely important factor in the choice and in the use of models in simulations. The comparison of different features in analyzed methods motivated us to the development of an innovative method to wrap in a common representation framework different methodologies of soft tissue simulation. This framework has the advantage of providing a unified interface for all the deformable models and thus it provides the ability to switch between deformable model keeping unchanged all other data structures and methods of the simulation. The use of this unique interface allows us to use one single method to perform the collision detection phase for all the analyzed deformable models, this greatly helped during the identification of requirements and features of such software module. Collision detection phase, when applied to rigid bodies, usually takes advantage of pre computation to subdivide body shapes in convex elements or to construct partitions of the space in which the body is defined to speed up the computation. When handling deformable models this is not possible because of the continuous changes in bodies shape. The collision detection method used in this work takes into account this problem and regularly adapt the data structures to the body configuration. After collisions have been detected and contact points have been identified on colliding bodies, it is necessary to solve the collision in a physics based way. To this extent we have to ensure that objects never compenetrate during the simulation and that, when solving collisions, all the physical phenomena involved in the contact of real bodies are taken into account: this include the elastic response of bodies during the contact and the frictional force exerted between each pair of colliding bodies. The innovative method for solving collision that we describe in this thesis ensures the realism of the simulation and the seamless interaction with the common framework used to integrate deformable models. One important feature of biologic tissues is their anisotropic behavior that usually comes from the fibrous structure of these tissues. In this thesis we propose a new method to introduce anisotropy in mass spring model. The method has the advantages of preserving the speed and ease of implementation of the model and it effectively introduces differentiation of the model behavior along the chosen directions. The described techniques have been integrated in two applications that allows the physical simulation of environments populated with deformable models. The first application implements all the described methods to simulate deformable models, it performs precise collision detection and solution with the possibility to chose the most suitable friction model for the simulation. It demonstrates the effectiveness of the proposed framework. The main limitation of this simulator, i.e. its high computation time, is tackled and solved in a second application that exploits the intrinsic parallelism of physical simulations to optimize the implementation and to exploit parallel architecture computational power. To obtain the performances required for an interactive environment the simulation is based on a simplified collision detection algorithm, but it features all the other techniques described in this thesis. The parallel implementation exploits graphic cards processor, a highly parallel architecture that update the scene every milliseconds. This allows the rendering of smooth haptic feedback to the user and ensures the realism of the physics simulation. The implemented applications prove the feasibility of the simulation of complex interactions between deformable models with physics realism. In addition, the parallel implementation of the simulator represents a promising starting point for the development of interactive simulations that can be used in different fields of research, such as surgeon training or fast prototyping

Sensory Communication

Contains table of contents for Section 2 and reports on five research projects.National Institutes of Health Contract 2 R01 DC00117National Institutes of Health Contract 1 R01 DC02032National Institutes of Health Contract 2 P01 DC00361National Institutes of Health Contract N01 DC22402National Institutes of Health Grant R01-DC001001National Institutes of Health Grant R01-DC00270National Institutes of Health Grant 5 R01 DC00126National Institutes of Health Grant R29-DC00625U.S. Navy - Office of Naval Research Grant N00014-88-K-0604U.S. Navy - Office of Naval Research Grant N00014-91-J-1454U.S. Navy - Office of Naval Research Grant N00014-92-J-1814U.S. Navy - Naval Air Warfare Center Training Systems Division Contract N61339-94-C-0087U.S. Navy - Naval Air Warfare Center Training System Division Contract N61339-93-C-0055U.S. Navy - Office of Naval Research Grant N00014-93-1-1198National Aeronautics and Space Administration/Ames Research Center Grant NCC 2-77

Haptics Rendering and Applications

There has been significant progress in haptic technologies but the incorporation of haptics into virtual environments is still in its infancy. A wide range of the new society's human activities including communication, education, art, entertainment, commerce and science would forever change if we learned how to capture, manipulate and reproduce haptic sensory stimuli that are nearly indistinguishable from reality. For the field to move forward, many commercial and technological barriers need to be overcome. By rendering how objects feel through haptic technology, we communicate information that might reflect a desire to speak a physically- based language that has never been explored before. Due to constant improvement in haptics technology and increasing levels of research into and development of haptics-related algorithms, protocols and devices, there is a belief that haptics technology has a promising future

Sensory Communication

Contains table of contents for Section 2, an introduction and reports on fifteen research projects.National Institutes of Health Grant RO1 DC00117National Institutes of Health Grant RO1 DC02032National Institutes of Health Contract P01-DC00361National Institutes of Health Contract N01-DC22402National Institutes of Health/National Institute on Deafness and Other Communication Disorders Grant 2 R01 DC00126National Institutes of Health Grant 2 R01 DC00270National Institutes of Health Contract N01 DC-5-2107National Institutes of Health Grant 2 R01 DC00100U.S. Navy - Office of Naval Research/Naval Air Warfare Center Contract N61339-94-C-0087U.S. Navy - Office of Naval Research/Naval Air Warfare Center Contract N61339-95-K-0014U.S. Navy - Office of Naval Research/Naval Air Warfare Center Grant N00014-93-1-1399U.S. Navy - Office of Naval Research/Naval Air Warfare Center Grant N00014-94-1-1079U.S. Navy - Office of Naval Research Subcontract 40167U.S. Navy - Office of Naval Research Grant N00014-92-J-1814National Institutes of Health Grant R01-NS33778U.S. Navy - Office of Naval Research Grant N00014-88-K-0604National Aeronautics and Space Administration Grant NCC 2-771U.S. Air Force - Office of Scientific Research Grant F49620-94-1-0236U.S. Air Force - Office of Scientific Research Agreement with Brandeis Universit