Communicating Wave Energy: An Active Learning Experience For Students

We have conducted an education project to communicate the wave energy concept to high school students. A virtual reality system that combines both hardware and software is developed in this project to simulate the buoy-wave interaction. This first-of-its-kind wave energy unit is portable and physics-based, allowing students to conduct a number of hands-on activities. This system is the core component of an educational experience that integrates demonstration and hands-on learning, with an aim of introducing the wave energy conversion process to students in an interactive environment. Presentations have been made at two different high schools with diverse student populations, and students involved in this project rated very positively about their learning experience. As revealed by their feedback, the virtual environment and its combination with the hardware are the most important factors that help students to appreciate the knowledge in the wave energy conversion process

Criação de material didático no ensino da anatomia veterinária: angiotécnica: Creation of didactic material in the teaching of veterinary anatomy: angiotechnics

As técnicas anatômicas têm sido utilizadas para a preparação de peças para o ensino prático da anatomia, principalmente a macroscópica. Essas técnicas desenvolvidas facilitam o trabalho dos professores durante o ensino e melhoram a compreensão do conteúdo didático. Desta maneira o presente trabalho objetivou desenvolver uma peça anatômica, detalhando o processo de realização da angiotécnica, e apresentando o material como opção de ensino diferenciado das estruturas anatômicas vasculares do rim suíno. A angiotécnica foi realizada em duas etapas: primeiramente a repleção, que consiste no preenchimento de ductos dos vasos sanguíneos dos rins, por meio de injeção de soluções de resina em seu interior. A segunda etapa consistiu na eliminação do tecido orgânico da amostra com ácido HCl, para visibilizar os ductos com resina, essa etapa é chamada de corrosão. O modelo preparado apresentou claramente as artérias, veias e suas ramificações, permitindo a observação de estruturas menores como capilares, mostrando que o órgão renal é bem vascularizado. A técnica demostrou ser uma opção didática válida no ensino do morfológico devido à perfeita observação das estruturas vasculares, permitindo o manuseio prático por parte do corpo docente e discente no ensino da anatomia veterinária

Modeling and rendering for development of a virtual bone surgery system

A virtual bone surgery system is developed to provide the potential of a realistic, safe, and controllable environment for surgical education. It can be used for training in orthopedic surgery, as well as for planning and rehearsal of bone surgery procedures...Using the developed system, the user can perform virtual bone surgery by simultaneously seeing bone material removal through a graphic display device, feeling the force via a haptic deice, and hearing the sound of tool-bone interaction --Abstract, page iii

VISIO-HAPTIC DEFORMABLE MODEL FOR HAPTIC DOMINANT PALPATION SIMULATOR

Vision and haptic are two most important modalities in a medical simulation. While visual cues assist one to see his actions when performing a medical procedure, haptic cues enable feeling the object being manipulated during the interaction. Despite their importance in a computer simulation, the combination of both modalities has not been adequately assessed, especially that in a haptic dominant environment. Thus, resulting in poor emphasis in resource allocation management in terms of effort spent in rendering the two modalities for simulators with realistic real-time interactions. Addressing this problem requires an investigation on whether a single modality (haptic) or a combination of both visual and haptic could be better for learning skills in a haptic dominant environment such as in a palpation simulator. However, before such an investigation could take place one main technical implementation issue in visio-haptic rendering needs to be addresse

Patient Specific Systems for Computer Assisted Robotic Surgery Simulation, Planning, and Navigation

The evolving scenario of surgery: starting from modern surgery, to the birth of medical imaging and the introduction of minimally invasive techniques, has seen in these last years the advent of surgical robotics. These systems, making possible to get through the difficulties of endoscopic surgery, allow an improved surgical performance and a better quality of the intervention. Information technology contributed to this evolution since the beginning of the digital revolution: providing innovative medical imaging devices and computer assisted surgical systems. Afterwards, the progresses in computer graphics brought innovative visualization modalities for medical datasets, and later the birth virtual reality has paved the way for virtual surgery. Although many surgical simulators already exist, there are no patient specific solutions. This thesis presents the development of patient specific software systems for preoperative planning, simulation and intraoperative assistance, designed for robotic surgery: in particular for bimanual robots that are becoming the future of single port interventions. The first software application is a virtual reality simulator for this kind of surgical robots. The system has been designed to validate the initial port placement and the operative workspace for the potential application of this surgical device. Given a bimanual robot with its own geometry and kinematics, and a patient specific 3D virtual anatomy, the surgical simulator allows the surgeon to choose the optimal positioning of the robot and the access port in the abdominal wall. Additionally, it makes possible to evaluate in a virtual environment if a dexterous movability of the robot is achievable, avoiding unwanted collisions with the surrounding anatomy to prevent potential damages in the real surgical procedure. Even if the software has been designed for a specific bimanual surgical robot, it supports any open kinematic chain structure: as far as it can be described in our custom format. The robot capabilities to accomplish specific tasks can be virtually tested using the deformable models: interacting directly with the target virtual organs, trying to avoid unwanted collisions with the surrounding anatomy not involved in the intervention. Moreover, the surgical simulator has been enhanced with algorithms and data structures to integrate biomechanical parameters into virtual deformable models (based on mass-spring-damper network) of target solid organs, in order to properly reproduce the physical behaviour of the patient anatomy during the interactions. The main biomechanical parameters (Young's modulus and density) have been integrated, allowing the automatic tuning of some model network elements, such as: the node mass and the spring stiffness. The spring damping coefficient has been modeled using the Rayleigh approach. Furthermore, the developed method automatically detect the external layer, allowing the usage of both the surface and internal Young's moduli, in order to model the main parts of dense organs: the stroma and the parenchyma. Finally the model can be manually tuned to represent lesion with specific biomechanical properties. Additionally, some software modules of the simulator have been properly extended to be integrated in a patient specific computer guidance system for intraoperative navigation and assistance in robotic single port interventions. This application provides guidance functionalities working in three different modalities: passive as a surgical navigator, assistive as a guide for the single port placement and active as a tutor preventing unwanted collision during the intervention. The simulation system has beed tested by five surgeons: simulating the robot access port placemen, and evaluating the robot movability and workspace inside the patient abdomen. The tested functionalities, rated by expert surgeons, have shown good quality and performance of the simulation. Moreover, the integration of biomechanical parameters into deformable models has beed tested with various material samples. The results have shown a good visual realism ensuring the performance required by an interactive simulation. Finally, the intraoperative navigator has been tested performing a cholecystectomy on a synthetic patient mannequin, in order to evaluate: the intraoperative navigation accuracy, the network communications latency and the overall usability of the system. The tests performed demonstrated the effectiveness and the usability of the software systems developed: encouraging the introduction of the proposed solution in the clinical practice, and the implementation of further improvements. Surgical robotics will be enhanced by an advanced integration of medical images into software systems: allowing the detailed planning of surgical interventions by means of virtual surgery simulation based on patient specific biomechanical parameters. Furthermore, the advanced functionalities offered by these systems, enable surgical robots to improve the intraoperative surgical assistance: benefitting of the knowledge of the virtual patient anatomy

Investigation of a holistic human-computer interaction (HCI) framework to support the design of extended reality (XR) based training simulators

In recent years, the use of Extended Reality (XR) based simulators for training has increased rapidly. In this context, there is a need to explore novel HCI-based approaches to design more effective 3D training environments. A major impediment in this research area is the lack of an HCI-based framework that is holistic and serves as a foundation to integrate the design and assessment of HCI-based attributes such as affordance, cognitive load, and user-friendliness. This research addresses this need by investigating the creation of a holistic framework along with a process for designing, building, and assessing training simulators using such a framework as a foundation. The core elements of the proposed framework include the adoption of participatory design principles, the creation of information-intensive process models of target processes (relevant to the training activities), and design attributes related to affordance and cognitive load. A new attribute related to affordance of 3D scenes is proposed (termed dynamic affordance) and its role in impacting user comprehension in data-rich 3D training environments is studied. The framework is presented for the domain of orthopedic surgery. Rigorous user-involved assessment of the framework and simulation approach has highlighted the positive impact of the HCI-based framework and attributes on the acquisition of skills and knowledge by healthcare users

VISIO-HAPTIC DEFORMABLE MODEL FOR HAPTIC DOMINANT PALPATION SIMULATOR

Vision and haptic are two most important modalities in a medical simulation. While visual cues assist one to see his actions when performing a medical procedure, haptic cues enable feeling the object being manipulated during the interaction. Despite their importance in a computer simulation, the combination of both modalities has not been adequately assessed, especially that in a haptic dominant environment. Thus, resulting in poor emphasis in resource allocation management in terms of effort spent in rendering the two modalities for simulators with realistic real-time interactions. Addressing this problem requires an investigation on whether a single modality (haptic) or a combination of both visual and haptic could be better for learning skills in a haptic dominant environment such as in a palpation simulator. However, before such an investigation could take place one main technical implementation issue in visio-haptic rendering needs to be addresse

Simulateur collaboratif de chirurgie d'instrumentation du rachis scoliotique en réalité virtuelle avec interface haptique logicielle

RÉSUMÉ La scoliose est une déformation tridimensionnelle de la colonne vertébrale qui nécessite, dans les cas graves, une intervention chirurgicale invasive et très délicate visant à redresser la colonne. Les outils disponibles pour l’entraînement des médecins, tels que les cadavres et les rachis synthétiques, présentent des inconvénients majeurs : les jeunes cadavres disponibles atteints de scoliose se font rares; le réalisme du comportement biomécanique est questionnable; ces deux types d’outils ne peuvent être réutilisés; ils ne représentent pas toute la variété des cas scoliotiques. Les technologies de la réalité virtuelle et les simulations numériques peuvent offrir des solutions pour contourner ces inconvénients. Afin d’aborder cette problématique, l’objectif général de la recherche a consisté à élaborer un prototype logiciel de simulateur collaboratif de chirurgie d’instrumentation du rachis scoliotique en réalité virtuelle incluant un retour d’effort logiciel pour les manoeuvres correctrices principales de la chirurgie, offrant ainsi un outil d’entraînement et d’apprentissage alternatif aux outils traditionnels. Ce projet est entré dans la continuité des travaux de recherche d’étudiants et d’associés de recherche de la Chaire de recherche industrielle CRSNG/Medtronic en biomécanique de la colonne vertébrale, et s’est distingué principalement par la mise en place de l’aspect collaboratif pour un contexte d’entraînement réaliste avec des participants distants, ainsi que le développement et l’évaluation d’une interface haptique logicielle. La revue bibliographique a suggéré que la chirurgie orthopédique ne semble pas encore bénéficier du potentiel offert par la réalité virtuelle et les interfaces haptiques quant à la simulation et à l’entraînement virtuel autant que d’autres types de chirurgies. La plupart des chirurgies pour lesquelles des simulateurs ont été développés impliquent des organes démontrant une certaine compliance, un espace de travail relativement restreint et des forces de faibles amplitudes, pouvant être simulées à l’aide de systèmes haptiques commerciaux génériques. Au contraire, la chirurgie d’instrumentation du rachis scoliotique nécessite l’application d’efforts de grande amplitude pour des mouvements relativement lents à peu de degrés de liberté, requérant un système haptique spécifique. De plus, les modèles physiques, bien que plus complexes et lourds en termes de temps de calculs que les modèles géométriques, sont nécessaires à l’obtention d’une expérience haptique réaliste. À la lumière de ces observations, nous avons émis deux hypothèses de recherche. La première hypothèse supposait que les principales manoeuvres correctrices effectuées lors d’une chirurgie d’instrumentation du rachis scoliotique peuvent être modélisées et simulées en réalité virtuelle immersive à l’aide d’une interface haptique logicielle et d’un modèle biomécanique personnalisé à ±15 % des valeurs d’efforts réelles telles que perçues par des chirurgiens experts. La seconde hypothèse supposait qu’une boucle de rendu haptique multifréquence, basée sur un algorithme de prédiction / correction, permettra d’atteindre la fréquence minimale requise (1000 Hz) pour un retour d’effort fonctionnel dans un contexte d’entraînement réaliste.---------ABSTRACT Scoliosis is a three-dimensional deformation of the spine requiring, in severe cases, a highly delicate and invasive surgical operation to correct the spinal deformities. Available tools for surgical training, such as cadavers and synthetic spines, have major drawbacks: limited availability of young cadaveric spines with scoliosis; questionable behaviour realism; destruction after first use; limited variability in scoliotic cases for training. Virtual reality technologies and computer simulations can offer solutions to these drawbacks. To address this problem, the general objective of this research consisted in elaborating the software prototype of a collaborative virtual reality scoliosis instrumentation surgery simulator, including force feedback for the main corrective surgical manoeuvres, as an alternative training and learning tool. This project has been a continuation of previous work from graduate students and research associates of the NSERC/Medtronic Industrial Research Chair in Spine Biomechanics, and focused on setting up and testing the collaborative aspect for a realistic training context with remote participants, as well as developing and evaluating a software haptic interface. The literature review suggested that orthopaedic surgery does not seem to benefit from virtual reality technologies and haptic interfaces regarding simulation and virtual training as much as other types of surgeries. Most surgeries for which simulators have been developed involve organs with a certain compliance, a relatively confined workspace and “delicate” forces, and can be simulated with generic commercial haptic devices. On the contrary, scoliosis instrumentation surgery involves the application of high forces through moderately slow and of few degrees of freedom movements, requiring a haptic device specific to scoliosis surgery. Also, physical models, although more complex and computationally expensive than geometric models, are necessary for a realistic haptic experience. In light of these observations, we stated two hypotheses. The first hypothesis was that the main corrective manoeuvres of scoliosis instrumentation surgery can be modeled and simulated in immersive virtual reality with a software haptic interface and a patient-specific biomechanical model at ±15 % of the actual force values as perceived by expert surgeons. The second hypothesis was that a multirate haptic rendering loop, based on a prediction / correction algorithm, will achieve the minimal required update rate (1000 Hz) for a functional force feedback in a realistic training context