Accurate sound synthesis of 3D object collisions in interactive virtual scenarios

Questa tesi affronta lo studio di algoritmi efficienti per la sintesi di suoni risultanti dalla collisione di oggetti generici, partendo da una descrizione fisica del problema. L'obiettivo della ricerca e' lo sviluppo di strumenti in grado di aumentare l'accuratezza del feedback uditivo in ambienti di realta' virtuale attraverso un approccio basato sulla fisica, senza il bisogno quindi di far riferimento a suoni pre-registrati. Data la loro versatilita' nel trattare geometrie complesse, i metodi agli elementi finiti (FEM) sono stati scelti per la discretizzazione spaziale di generici risonatori tridimensionali. Le risultanti equazioni discrete sono riarrangiate in modo da disaccoppiare i modi normali del sistema tramite l'utilizzo di tecniche di Analisi e Sintesi Modale. Queste tecniche, infatti, portano convenientemente ad algoritmi computazionalmente efficienti per la sintesi del suono. Implementazioni di esempio di tali algoritmi sono state sviluppate facendo uso solo di software open-source: questo materiale a corredo della tesi permette una migliore riproducibilita' dei risultati di questa tesi da parte di ricercatori aventi una preparazione nel campo della sintesi audio. I risultati originali presenti in questo lavoro includono: i tecniche efficienti basate sulla fisica che aiutano l'implementazione in tempo reale di algoritmi di sintesi del suono su hardware comune; ii un metodo per la gestione efficiente dei dati provenienti da analisi FEM che, assieme ad un modello espressivo per la dissipazione, permette di calcolare l'informazione caratterizzante un oggetto risonante e salvarla in una struttura dati compatta iii una trasformazione nel dominio discreto del tempo su due diverse rappresentazioni nello spazio degli stati di filtri digitali del secondo ordine, che permette il calcolo esatto di variabili derivate come la velocita' e l'energia di un risonatore anche quando semplici realizzazioni a soli poli sono impiegate i un'efficiente realizzazione multirate di un banco parallelo di risonatori, derivata usando una suddivisione con Quadrature-Mirror-Filters (QMF). Confrontata con lavori simili presenti in letteratura, questa realizzazione permette l'uso di eccitazione nonlineare in feedback per un banco di risonatori in multirate: l'idea chiave consiste nello svolgere un cambio di stato adattivo nel banco di risonatori, muovendo i risonatori dalla frequenza di campionamento elevata, usata per il processamento della fase transiente, ad un insieme di sottofrequenze ridotte usate durante l'evoluzione in stato libero del sistema.This thesis investigates efficient algorithms for the synthesis of sounds produced by colliding objects, starting from a physical description of the problem. The objective of this investigation is to provide tools capable of increasing the accuracy of the synthetic auditory feedback in virtual environments through a physics-based approach, hence without the need of pre-recorded sounds. Due to their versatility in dealing with complex geometries, Finite Element Methods (FEM) are chosen for the space-domain discretization of generic three-dimensional resonators. The resulting state-space representations are rearranged so as to decouple the normal modes in the corresponding equations, through the use of Modal Analysis/Synthesis techniques. Such techniques, in fact, conveniently lead to computationally efficient sound synthesis algorithms. The whole mathematical treatment develops until deriving such algorithms. Finally, implementation examples are provided which rely only on open-source software: this companion material guarantees the reproducibility of the results, and can be handled without much effort by most researchers having a background in sound processing. The original results presented in this work include: i efficient physics-based techniques that help implement real-time sound synthesis algorithms on common hardware; ii a method for the efficient management of FEM data which, by working together with an expressive damping model, allows to pre-compute the information characterizing a resonating object and then to store it in a compact data structure; iii a time-domain transformation of the state-space representation of second-order digital filters, allowing for the exact computation of dependent variables such as resonator velocity and energy, even when simple all-pole realizations are used; iv an efficient multirate realization of a parallel bank of resonators, which is derived using a Quadrature-Mirror-Filters (QMF) subdivision. Compared to similar works previously proposed in the literature, this realization allows for the nonlinear feedback excitation of a multirate filter bank: the key idea is to perform an adaptive state change in the resonator bank, by switching the sampling rate of the resonators from a common highest value, used while processing the initial transient of the signals at full bandwidth, to a set of lower values in ways to enable a multirate realization of the same bank during the steady state evolution of the signals

Design and development of a VR system for exploration of medical data using haptic rendering and high quality visualization

[no abstract

Fast audio-haptic prototyping with mass-interaction physics

International audienceThis paper presents ongoing work on the topic of physical modelling and force-feedback interaction. Specifically, it proposes a frame- work for rapidly prototyping virtual objects and scenes by means of mass-interaction models, and coupling the user and these objects via an affordable multi-DoF haptic device. The modelled objects can be computed at the rate of the haptic loop, but can also operate at a higher audio-rate, producing sound. The open-source design and overall simplicity of the proposed system makes it an interesting solution for introducing both physical simulations and force-feedback interaction, and also for applications in artistic creation. This first implementation prefigures current work conducted on the develop- ment of modular open-source mass-interaction physics tools for the design of haptic and multisensory applications

Haptics Rendering and Applications

There has been significant progress in haptic technologies but the incorporation of haptics into virtual environments is still in its infancy. A wide range of the new society's human activities including communication, education, art, entertainment, commerce and science would forever change if we learned how to capture, manipulate and reproduce haptic sensory stimuli that are nearly indistinguishable from reality. For the field to move forward, many commercial and technological barriers need to be overcome. By rendering how objects feel through haptic technology, we communicate information that might reflect a desire to speak a physically- based language that has never been explored before. Due to constant improvement in haptics technology and increasing levels of research into and development of haptics-related algorithms, protocols and devices, there is a belief that haptics technology has a promising future

Virtual environments for medical training : graphic and haptic simulation of tool-tissue interactions

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2004.Includes bibliographical references (leaves 122-127).For more than 2,500 years, surgical teaching has been based on the so called "see one, do one, teach one" paradigm, in which the surgical trainee learns by operating on patients under close supervision of peers and superiors. However, higher demands on the quality of patient care and rising malpractice costs have made it increasingly risky to train on patients. Minimally invasive surgery, in particular, has made it more difficult for an instructor to demonstrate the required manual skills. It has been recognized that, similar to flight simulators for pilots, virtual reality (VR) based surgical simulators promise a safer and more comprehensive way to train manual skills of medical personnel in general and surgeons in particular. One of the major challenges in the development of VR-based surgical trainers is the real-time and realistic simulation of interactions between surgical instruments and biological tissues. It involves multi-disciplinary research areas including soft tissue mechanical behavior, tool-tissue contact mechanics, computer haptics, computer graphics and robotics integrated into VR-based training systems. The research described in this thesis addresses many of the problems of simulating tool-tissue interactions in medical virtual environments. First, two kinds of physically based real time soft tissue models - the local deformation and the hybrid deformation model - were developed to compute interaction forces and visual deformation fields that provide real-time feed back to the user. Second, a system to measure in vivo mechanical properties of soft tissues was designed, and eleven sets of animal experiments were performed to measure in vivo and in vitro biomechanical properties of porcine intra-abdominal organs. Viscoelastic tissue(cont.) parameters were then extracted by matching finite element model predictions with the empirical data. Finally, the tissue parameters were combined with geometric organ models segmented from the Visible Human Dataset and integrated into a minimally invasive surgical simulation system consisting of haptic interface devices inside a mannequin and a graphic display. This system was used to demonstrate deformation and cutting of the esophagus, where the user can haptically interact with the virtual soft tissues and see the corresponding organ deformation on the visual display at the same time.by Jung Kim.Ph.D

Enhancing the E-Commerce Experience through Haptic Feedback Interaction

The sense of touch is important in our everyday lives and its absence makes it difficult to explore and manipulate everyday objects. Existing online shopping practice lacks the opportunity for physical evaluation, that people often use and value when making product choices. However, with recent advances in haptic research and technology, it is possible to simulate various physical properties such as heaviness, softness, deformation, and temperature. The research described here investigates the use of haptic feedback interaction to enhance e-commerce product evaluation, particularly haptic weight and texture evaluation. While other properties are equally important, besides being fundamental to the shopping experience of many online products, weight and texture can be simulated using cost-effective devices. Two initial psychophysical experiments were conducted using free motion haptic exploration in order to more closely resemble conventional shopping. One experiment was to measure weight force thresholds and another to measure texture force thresholds. The measurements can provide better understanding of haptic device limitation for online shopping in terms of the availability of different stimuli to represent physical products. The outcomes of the initial psychophysical experimental studies were then used to produce various absolute stimuli that were used in a comparative experimental study to evaluate user experience of haptic product evaluation. Although free haptic exploration was exercised on both psychophysical experiments, results were relatively consistent with previous work on haptic discrimination. The threshold for weight force discrimination represented as downward forces was 10 percent. The threshold for texture force discrimination represented as friction forces was 14.1 percent, when using dynamic coefficient of friction at any level of static coefficient of friction. On the other hand, the comparative experimental study to evaluate user experience of haptic product information indicated that haptic product evaluation does not change user performance significantly. However, although there was an increase in the time taken to complete the task, the number of button click actions tended to decrease. The results showed that haptic product evaluation could significantly increase the confidence of shopping decision. Nevertheless, the availability of haptic product evaluation does not necessarily impose different product choices but it complements other selection criteria such as price and appearance. The research findings from this work are a first step towards exploring haptic-based environments in e-commerce environments. The findings not only lay the foundation for designing online haptic shopping but also provide empirical support to research in this direction

Haptic technology for micro-robotic cell injection training systems — a review

Currently, the micro-robotic cell injection procedure is performed manually by expert human bio-operators. In order to be proficient at the task, lengthy and expensive dedicated training is required. As such, effective specialized training systems for this procedure can prove highly beneficial. This paper presents a comprehensive review of haptic technology relevant to cell injection training and discusses the feasibility of developing such training systems, providing researchers with an inclusive resource enabling the application of the presented approaches, or extension and advancement of the work. A brief explanation of cell injection and the challenges associated with the procedure are first presented. Important skills, such as accuracy, trajectory, speed and applied force, which need to be mastered by the bio-operator in order to achieve successful injection, are then discussed. Then an overview of various types of haptic feedback, devices and approaches is presented. This is followed by discussion on the approaches to cell modeling. Discussion of the application of haptics to skills training across various fields and haptically-enabled virtual training systems evaluation are then presented. Finally, given the findings of the review, this paper concludes that a haptically-enabled virtual cell injection training system is feasible and recommendations are made to developers of such systems

The Shape of Damping: Optimizing Damping Coefficients to Improve Transparency on Bilateral Telemanipulation

This thesis presents a novel optimization-based passivity control algorithm for hapticenabled bilateral teleoperation systems involving multiple degrees of freedom. In particular, in the context of energy-bounding control, the contribution focuses on the implementation of a passivity layer for an existing time-domain scheme, ensuring optimal transparency of the interaction along subsets of the environment space which are preponderant for the given task, while preserving the energy bounds required for passivity. The involved optimization problem is convex and amenable to real-time implementation. The effectiveness of the proposed design is validated via an experiment performed on a virtual teleoperated environment. The interplay between transparency and stability is a critical aspect in haptic-enabled bilateral teleoperation control. While it is important to present the user with the true impedance of the environment, destabilizing factors such as time delays, stiff environments, and a relaxed grasp on the master device may compromise the stability and safety of the system. Passivity has been exploited as one of the the main tools for providing sufficient conditions for stable teleoperation in several controller design approaches, such as the scattering algorithm, timedomain passivity control, energy bounding algorithm, and passive set position modulation. In this work it is presented an innovative energy-based approach, which builds upon existing time-domain passivity controllers, improving and extending their effectiveness and functionality. The set of damping coefficients are prioritized in each degree of freedom, the resulting transparency presents a realistic force feedback in comparison to the other directions. Thus, the prioritization takes effect using a quadratic programming algorithm to find the optimal values for the damping. Finally, the energy tanks approach on passivity control is a solution used to ensure stability in a system for robotics bilateral manipulation. The bilateral telemanipulation must maintain the principle of passivity in all moments to preserve the system\u2019s stability. This work presents a brief introduction to haptic devices as a master component on the telemanipulation chain; the end effector in the slave side is a representation of an interactive object within an environment having a force sensor as feedback signal. The whole interface is designed into a cross-platform framework named ROS, where the user interacts with the system. Experimental results are presented

Towards an efficient haptic rendering using data-driven modeling

This thesis focuses on the optimisation of haptic rendering of interactions with deformable models. The research demonstrated that data-driven techniques can produce a real-time, accurate and complex simulation experience. Applications include, but not limited to, virtual training, rapid prototyping, virtual presence, and entertainment