Advanced interaction techniques for medical models

Advances in Medical Visualization allows the analysis of anatomical structures with the use of 3D models reconstructed from a stack of intensity-based images acquired through different techniques, being Computerized Tomographic (CT) modality one of the most common. A general medical volume graphics application usually includes an exploration task which is sometimes preceded by an analysis process where the anatomical structures of interest are first identified. The main objective of this thesis is the improvement of the user experience in the analysis and exploration of medical datasets. This improvement involves the development of efficient algorithms designed both under a user-centered perspective and taking the new computing capabilities into account in order to obtain high quality results in real-time. On the analysis stage, we have focused on the identification of the bones at joints, which is particularly challenging because the bones are very close to each other and their boundaries become ambiguous in CT images. We have concentrated our efforts on reaching maximum automation of the overall process. The proposed algorithm uses an example mesh of the same bone that has to be segmented, usually from a different person, to drive the segmentation process. The algorithm is based on an energy minimization scheme to deform the initial example mesh while following the well-defined features of the volume data to be segmented in a local and adaptive way. We also present contributions on three different aspects of the exploration task: a best-view determination system and centering in virtual reality environments, a focus-and-context technique and a point selection method. In medical practice it would often be very useful to have access to a quick pre-visualization of the involved medical dataset. We have proposed a new system which allows users to obtain a set of representative views in a short time and permits the generation of inspection paths at almost no extra cost. The technique relies on the use of a multiscale entropy measure for the generation of good viewpoints and uses a complexity-based metric, the normalized compression distance, for the calculation of the representative views set. In the exploration of medical datasets, it is difficult to simultaneously visualize interior and exterior structures because the structures are commonly quite complex and it is easy to lose the context. We have developed a new interaction tool, the Virtual Magic Lantern, tailored to facilitate volumetric data inspection in a Virtual Reality environment. It behaves like a lantern whose illumination cone determines the region of interest. It addresses the occlusion management problem and facilitates the inspection of inner structures without the total elimination of the exterior structures, offering in this way, a focus+context-based visualization of the overall structures. Finally, the analysis of medical datasets may require the selection of 3D points for measurements involving anatomical structures. Although there are well-established 3D object selection techniques for polygonal models, there is a lack of techniques specifically developed for volume datasets. We present a new selection technique for Virtual Reality setups which allows users to easily select anchor points in non-necessarily segmented volume datasets rendered using Direct Volume Rendering. This new metaphor is based on the use of a ray emanating from the user, whose trajectory is enriched with its points of intersection with the on-the-fly determination of the isosurfaces along the ray path. Additionally, a visual feedback of the ray selection is offered through the use of two helper mirror views, in order to show occluded candidate points that would otherwise be invisible to the user without posterior and ad-hoc manipulation.Els avenços en la investigació en el camp de Medical Visualization permeten l’anàlisi de models volumètrics tridimensionals d’estructures anatòmiques obtinguts a partir d’imatgesmèdiques capturades mitjançant diferents tècniques, essent la Tomografia Computeritzada (TC) una de lesmés habituals. Generalment, les aplicacions informàtiques d’ajuda al diagnòstic, la simulació, etc., permeten l’exploració interactiva d’aquest tipus de models, una tasca que pot anar precedida d’un procés d’identificació (segmentació) de les estructures anatòmiques per tal de possibilitar la seva exploració. L’objectiu d’aquesta tesi és millorar l’eficiència i l’experiència de l’usuari, tant de la tasca de segmentació com de l’exploració. Per tal d’assolir-ho s’han desenvolupat diversos algorismes; dissenyats sota una perspectiva centrada en l’usuari i fent servir els darrers avenços tecnològics de las targes gràfiques, el que ens permet obtenir resultats visuals de màxima qualitat en temps real. Respecte de la tasca de segmentació, ens hem centrat en el problema de la identificació d’ossos ubicats en articulacions, en models capturats mitjançant TC. La identificació d’aquests ossos pot arribar a ser molt feixuga i costosa fent servir les tècniques clàssiques de segmentació. La recerca realitzada en elmarc de la tesi s’ha enfocat en assolir la màxima automatització possible del procés sencer. La tècnica proposada empra una malla triangular d’exemple de l’os que es vol segmentar, que es fará servir per guiar tot el procés de segmentació. L’algorisme deforma de forma local i adaptativa aquesta malla, adaptant-la a la informació present en el model volumètric en les parts en que la seva frontera està definida de forma no ambigua, i respectant la forma original en les zones en les que el model presenta algun tipus d’incertesa en la definició de la frontera, ja sigui be perque l’estructura òssia apareix totalment unida a altres estructures òssies de l’articulació o be degut a que la informació capturada no presenta una frontera ben contrastada. Per altra banda, en la pràctica clínica pot ser de molta utilitat oferir a l’usuari una previsualització ràpida del model volumètric que ha d’inspeccionar. En aquesta tesi elaborem una nova tècnica que permet obtenir en un temps acceptable un conjunt de vistes representatives d’un model volumètric, així comla generació automàtica d’una animació a l’entorn del model que facilita a l’usuari una ràpida comprensió del mateix. La tècnica desenvolupada utilitza una formulació de l’entropia multiescala per la obtenció de bones vistes i la distància de compressió normalitzada, una mètrica del camp de la teoria de la complexitat, per establir el conjunt de vistes representatives. En l’exploració de models mèdics pot ser difícil la visualització simultània d’estructures internes i externes. Per abordar aquest problema s’ha desenvolupat una nova tècnica d’interacció anomenada Virtual Magic Lantern, pensada per a facilitar la inspecció d’aquests models en entorns de realitat virtual. Aquesta metàfora d’interacció es comporta com una llanterna. El seu feix de llum determina una regió d’interès del model, que serà visualitzada emprant una funció de transferència específica permetent la visualització de les estructures internes sense eliminar el context de tot el model. En l’anàlisi de modelsmédics pot ser necessària la selecció de punts concrets per a poder realitzar algun tipus de medició entre estructures anatòmiques. Depenent del algorisme de visualització del model, determinar quin punt exactament vol seleccionar l’usuari pot no tenir un resultat únic. Per solventar aquest problema, s’ha desenvolupat una nova metàfora d’interacció per entorns de realitat virtual, que permet la selecció de punts en un model volumètric no necessàriament segmentat. Aquesta tècnica es basa en l’ús d’un raig originat en la mà de l’usuari, sobre el que es visualitzen els punts d’intersecció amb les estructures anatòmiques que travessa. Donat que la superfície d’aquestes estructures no està explícitament definida, s’ha requerit desenvolupar especialment un càlcul ràpid i precís de les seves interseccions amb el raig. Per tal de facilitar la visió dels punts interiors a superfícies opaques i enriquir la visualització global, s’afegeix sobre dos plans auxiliars la visió del volum tallat garantint la visibilitat total del conjunt de punts.Los avances en la investigación en el área de Medical Visualization permiten el análisis de modelos volumétricos tridimensionales de estructuras anatómicas, los cuales se obtienen a partir de imágenes médicas capturadas mediante diferentes técnicas de captación, siendo la Tomografía Computerizada (TC) una de las más frecuentes. Habitualmente, las aplicaciones informáticas orientadas al análisis de este tipo de modelos, bien sean para el soporte al diagnóstico, simuladores médicos o la planificación de procesos quirúrgicos, permiten la exploración interactiva de los modelos volumétricos. Dependiendo de las estructuras anatómicas que se precise analizar, puede ser necesario realizar un proceso de identificación (segmentación) de las estructuras anatómicas para posibilitar su posterior inspección. El objetivo principal de esta tesis ha consistido en el desarrollo de nuevas técnicas informáticas que mejoren la experiencia del usuario en los procesos tanto de segmentación como de exploración de un modelo volumétrico. Para alcanzar dicho objetivo, ha sido necesario el desarrollo de algoritmos eficientes diseñados teniendo particularmente en cuenta al usuario final y explotando los últimos avances en la tecnología de las tarjetas gráficas para poder obtener resultados visuales de la máxima calidad en tiempo real. En lo relativo al proceso de segmentación, nos hemos centrado en la identificación de las estructuras óseas ubicadas en articulaciones, en modelos capturadosmediante TC. La identificación de este tipo de estructuras usando los métodos tradicionales de segmentación puede llegar a ser muy tediosa, debido a que puede necesitarse mucha intervención por parte del usuario. La investigación llevada a cabo ha tenido como objetivo principal el maximizar el grado de automatización en el proceso de segmentación de este tipo de estructuras. La técnica propuesta parte de un ejemplo de la estructura ósea (malla triangular) que se quiere segmentar, generada a partir de los datos o bien de otra persona o bien de la misma persona en otras circunstancias. A partir de este ejemplo el algoritmo deforma la malla de manera local y adaptativa, adaptandola a la información presente en elmodelo volumétrico en aquellas zonas donde la frontera de la estructura está definida de forma no ambígua, y respetando la forma de la malla original en aquellas otras zonas en las cuales el modelo volumétrico presenta algún tipo de incertidumbre en la definición de la frontera, ya sea porque la estructura ósea aparece totalmente unida a otras estructuras óseas de la articulación o debido a que la información capturada no presenta una frontera bien contrastada. En lo relativo al proceso de exploración, esta tesis presenta resultados en dos vertientes distintas. Por un lado, la generación automática de una previsualización del modelo volumétrico y por el otro lado, el desarrollo de nuevas técnicas de interacción que faciliten la exploración de modelos volumétricos en entornos de realidad virtual. Ofrecer al usuario una previsualización rápida del modelo volumétrico que ha de inspeccionar, puede ser de mucha utilidad en la práctica clínica. En esta tesis elaboramos un nuevo sistema que permite obtener en un tiempo razonable un conjunto de vistas representativas del modelo volumétrico, así como la generación de una animación alrededor del modelo que facilita al usuario una rápida comprensión del mismo. Las técnicas desarrolladas se basan en el uso de la entropía multiescala para el cálculo de vistas informativas del modelo volumétrico. A partir del conjunto de vistas calculadas y mediante el uso de la distancia de compresión normalizada, una métrica de Teoría de la Complejidad, se puede calcular un subconjunto de vistas representativas del modelo volumétrico. Por otro lado, en la exploración de modelos volumétricos puede ser difícil visualizar simultáneamente estructuras anatómicas internas y externas. Esto es debido a que las estructuras son bastantes complejas, y es fácil perder la referencia respecto a otras estructuras anatómicas. En esta tesis se ha desarrollado una nueva técnica de interacción, bautizada como VirtualMagic Lantern, orientada a facilitar la inspección de modelos volumétricos en entornos de realidad virtual. Esta nueva metáfora de interacción se comporta como una linterna de mano guiada por el usuario, cuyo haz de luz define sobre el modelo volumétrico una región de interés. Esta región de interés será visualizada utilizando una función de transferencia diferente a la usada para el resto del modelo, posibilitando de esta manera la inspección de estructuras internas sin eliminar totalmente el resto delmodelo. En el análisis de modelos médicos puede ser necesaria la selección de puntos concretos para poder realizar algún tipo de medición entre estructuras anatómicas. Dependiendo del tipo de visualización del modelo, determinar qué punto exactamente quiere seleccionar el usuario puede no tener un resultado único. Para solucionar este problema, se presenta una nuevametáfora de interacción en entornos de realidad virtual para la selección de puntos anatómicos de un modelo volumétrico no necesariamente segmentado. Esta técnica se basa en el uso de un rayo originado en la mano del usuario, sobre el que son visualizados los puntos de intersección de las estructuras anatómicas que atraviesa. Dado que la superficie de estas estructuras anatómicas no está explícitamente representada en el modelo volumétrico, se ha requerido desarrollar un cálculo preciso y rápido de la intersección del rayo con estas estructuras. Para ofrecer una visualización de los puntos calculados sin ningún tipo de oclusión por parte de las estructuras anatómicas existentes en el modelo, se ha añadido a la visualización global la visualización de dos paneles auxiliares en los cuales se muestra el mismo modelo volumétrico recortado de tal manera que sean completamente visibles el conjunto de los puntos. De esta forma, se facilita al usuario la selección de los puntos calculados sin tener que realizar ningún tipo de manipulación del modelo para poder obtener una visualización en la que los puntos calculados sean visibles

Colonic content: effect of diet, meals, and defecation

This is the peer reviewed version of the following article: Bendezú, Á., Mego, M., Monclús, E., Merino, X., Accarino, A., Malagelada, J., Navazo, I., Azpiroz, F. Colonic content: effect of diet, meals, and defecation. "Neurogastroenterology and motility", Febrer 2017, vol. 29, núm. 2, which has been published in final form at [http://onlinelibrary.wiley.com.recursos.biblioteca.upc.edu/doi/10.1111/nmo.12930/full]. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.The metabolic activity of colonic microbiota is influenced by diet; however, the relationship between metabolism and colonic content is not known. Our aim was to determine the effect of meals, defecation, and diet on colonic content. Methods: In 10 healthy subjects, two abdominal MRI scans were acquired during fasting, 1 week apart, and after 3 days on low- and high-residue diets, respectively. With each diet, daily fecal output and the number of daytime anal gas evacuations were measured. On the first study day, a second scan was acquired 4 hours after a test meal (n=6) or after 4 hours with nil ingestion (n=4). On the second study day, a scan was also acquired after a spontaneous bowel movement. Results: On the low-residue diet, daily fecal volume averaged 145 ± 15 mL; subjects passed 10.6 ± 1.6 daytime anal gas evacuations and, by the third day, non-gaseous colonic content was 479 ± 36 mL. The high-residue diet increased the three parameters to 16.5 ± 2.9 anal gas evacuations, 223 ± 19 mL fecal output, and 616 ± 55 mL non-gaseous colonic content (P<.05 vs low-residue diet for all). On the low-residue diet, non-gaseous content in the right colon had increased by 41 ± 11 mL, 4 hours after the test meal, whereas no significant change was observed after 4-hour fast (-15 ± 8 mL; P=.006 vs fed). Defecation significantly reduced the non-gaseous content in distal colonic segments. Conclusion & inferences: Colonic content exhibits physiologic variations with an approximate 1/3 daily turnover produced by meals and defecation, superimposed over diet-related day-to-day variations.Peer ReviewedPostprint (author's final draft

The virtual magic lantern: an interaction metaphor for enhanced medical data inspection

In this paper we present the Virtual Magic Lantern (VML), an interaction tool tailored to facilitate volumetric data inspection. It behaves like a lantern whose virtual illumination cone provides the focal region which is visualized using a secondary transfer function or different rendering style. This may be used for simple visual inspection, surgery planning, or injure diagnosis. The VML is a particularly friendly and intuitive interaction tool suitable for an immersive Virtual Reality setup with a large screen, where the user moves a Wanda device, like a lantern pointing to the model. We show that this inspection metaphor can be efficiently and easily adapted to a GPU ray casting volume visualization algorithm. We also present the Virtual Magic Window (VMW) metaphor as an efficient collateral implementation of the VML, that can be seen as a restricted case where the lantern illuminates following the viewing direction, through a virtual window created as the intersection of the virtual lantern (guided by the Wanda device) and the bounding box of the volume.Peer ReviewedPostprint (author’s final draft

Colonic content in health and its relation to functional gut symptoms

This is the peer reviewed version of the following article: Bendezú, R. A., Barba, E., Burri, E., Cisternas, D., Accarino, A., Quiroga, S., Monclus, E., Navazo, I., Malagelada, J.-R. and Azpiroz, F. (2016), Colonic content in health and its relation to functional gut symptoms. Neurogastroenterol. Motil., 28: 849–854, which has been published in final form at [doi:10.1111/nmo.12782]. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-ArchivingGut content may be determinant in the generation of digestive symptoms, particularly in patients with impaired gut function and hypersensitivity. Since the relation of intraluminal gas to symptoms is only partial, we hypothesized that non-gaseous component may play a decisive role. Methods: Abdominal computed tomography scans were evaluated in healthy subjects during fasting and after a meal (n = 15) and in patients with functional gut disorders during basal conditions (when they were feeling well) and during an episode of abdominal distension (n = 15). Colonic content and distribution were measured by an original analysis program. Key results: In healthy subjects both gaseous (87 ± 24 mL) and non-gaseous colonic content (714 ± 34 mL) were uniformly distributed along the colon. In the early postprandial period gas volume increased (by 46 ± 23 mL), but non-gaseous content did not, although a partial caudad displacement from the descending to the pelvic colon was observed. No differences in colonic content were detected between patients and healthy subjects. Symptoms were associated with discrete increments in gas volume. However, no consistent differences in non-gaseous content were detected in patients between asymptomatic periods and during episodes of abdominal distension. Conclusions & inferences: In patients with functional gut disorders, abdominal distension is not related to changes in non-gaseous colonic content. Hence, other factors, such as intestinal hypersensitivity and poor tolerance of small increases in luminal gas may be involved.Peer ReviewedPostprint (author's final draft

End to End Colonic Content Assessment: ColonMetry Application

Colon segmentation; Colonic content; Intestinal gasSegmentación de colon; Contenido colónico; Gas intestinalSegmentació del còlon; Contingut colònic; Gas intestinalThe analysis of colonic contents is a valuable tool for the gastroenterologist and has multiple applications in clinical routine. When considering magnetic resonance imaging (MRI) modalities, T2 weighted images are capable of segmenting the colonic lumen, whereas fecal and gas contents can only be distinguished in T1 weighted images. In this paper, we present an end-to-end quasi-automatic framework that comprises all the steps needed to accurately segment the colon in T2 and T1 images and to extract colonic content and morphology data to provide the quantification of colonic content and morphology data. As a consequence, physicians have gained new insights into the effects of diets and the mechanisms of abdominal distension.This work was supported by the Spanish Ministry of Science and Innovation (Proyectos de Generación de Conocimiento), PID2021-122295OB-I00, and Agencia Estatal de Investigación and Fondos FEDER, PID2021-122136OB-C21); Ciberehd is funded by the Instituto de Salud Carlos III

Quasi-automatic colon segmentation on T2-MRI images with low user effort

About 50% of the patients consulting a gastroenterology clinic report symptoms without detectable cause. Clinical researchers are interested in analyzing the volumetric evolution of colon segments under the effect of different diets and diseases. These studies require noninvasive abdominal MRI scans without using any contrast agent. In this work, we propose a colon segmentation framework designed to support T2-weighted abdominal MRI scans obtained from an unprepared colon. The segmentation process is based on an efficient and accurate quasiautomatic approach that drastically reduces the specialist interaction and effort with respect other state-of-the-art solutions, while decreasing the overall segmentation cost. The algorithm relies on a novel probabilistic tubularity filter, the detection of the colon medial line, probabilistic information extracted from a training set and a final unsupervised clustering. Experimental results presented show the benefits of our approach for clinical use.Peer ReviewedPostprint (author's final draft

MTCut: GPU-based marching tetra cuts

Isosurface construction and rendering based on tetrahedral grids has shown to be feasible on programmable graphics hardware. In this paper we present MTCut: a volume cutting algorithm that is able to cut isosurfaces obtained by a Marching Tetrahedra algorithm on volume data. It does not require a tetrahedal representation and runs in real time for complex meshes of up to 1.8M triangles. Our algorithm takes as input the isosurface to be cut, slices it, and produces the cut geometry in response to the user interaction with a haptic device. The result is a watertight manifold that can be interactively recovered back to CPU in response to a user request.Postprint (published version

Fitted avatars: automatic skeleton adjustment for self-avatars in virtual reality

In the era of the metaverse, self-avatars are gaining popularity, as they can enhance presence and provide embodiment when a user is immersed in Virtual Reality. They are also very important in collaborative Virtual Reality to improve communication through gestures. Whether we are using a complex motion capture solution or a few trackers with inverse kinematics (IK), it is essential to have a good match in size between the avatar and the user, as otherwise mismatches in self-avatar posture could be noticeable for the user. To achieve such a correct match in dimensions, a manual process is often required, with the need for a second person to take measurements of body limbs and introduce them into the system. This process can be time-consuming, and prone to errors. In this paper, we propose an automatic measuring method that simply requires the user to do a small set of exercises while wearing a Head-Mounted Display (HMD), two hand controllers, and three trackers. Our work provides an affordable and quick method to automatically extract user measurements and adjust the virtual humanoid skeleton to the exact dimensions. Our results show that our method can reduce the misalignment produced by the IK system when compared to other solutions that simply apply a uniform scaling to an avatar based on the height of the HMD, and make assumptions about the locations of joints with respect to the trackers.This work was funded by the Spanish Ministry of Science and Innovation (PID2021-122136OB-C21). Jose Luis Ponton was also funded by the Spanish Ministry of Universities (FPU21/01927).Peer ReviewedPostprint (published version

Rendering and interacting with volume models in inmmersive environments

The recent advances in VR headsets, such as the Oculus Rift or HTC Vive, at affordable prices offering a high resolution display, has empowered the development of immersive VR applications. Unfortunately, the rendering engine and the interaction devices, are not specifically designed to deal with volumetric data. In this paper we propose an immersive VR system that uses some well-known acceleration algorithms to achieve real-time rendering of volumetric datasets in an immersive VR system. Moreover, we have incorporated different basic interaction techniques to facilitate the inspection of the volume dataset. The interaction has been designed to be as natural as possible in order to achieve the most comfortable, user-friendly virtual experience. We have conducted an informal user study to evaluate the user preferences. Our evaluation shows that our application is perceived usable, easy of learn and very effective in terms of the high level of immersion achieved.Peer ReviewedPostprint (author's final draft

Semi-automatic colonic content analysis for diagnostic

The analysis of the morphology and content of the gut is necessary in order to understand metabolic and functional gut activity and for diagnostic purposes. Magnetic resonance imaging (MRI) has become an important modality technique since it is able to visualize soft tissues using no ionizing radiation, and hence removes the need for any contrast agents. In the last few years, MRI of gastrointestinal function has advanced substantially, although scarcely any publication has been devoted to the analysis of the colon content. This paper presents a semi-automatic segmentation tool for the quantitative assessment of the unprepared colon from MRI images. This application has allowed for the analysis of the colon content in various clinical experiments. The results of the assessment have contributed to a better understanding of the functionality of the colon under different diet conditions. The last experiment carried out by medical doctors showed a marked influence of diet on colonic content, accounting for about 30% of the volume variations.Peer ReviewedPostprint (published version