40 research outputs found
Scaling Up Medical Visualization : Multi-Modal, Multi-Patient, and Multi-Audience Approaches for Medical Data Exploration, Analysis and Communication
Medisinsk visualisering er en av de mest applikasjonsrettede omrÄdene av visualiseringsforsking. Tett samarbeid med medisinske eksperter er nÞdvendig for Ä tolke medisinsk bildedata og lage betydningsfulle visualiseringsteknikker og visualiseringsapplikasjoner. Kreft er en av de vanligste dÞdsÄrsakene, og med Þkende gjennomsnittsalder i i-land Þker ogsÄ antallet diagnoser av gynekologisk kreft. Moderne avbildningsteknikker er et viktig verktÞy for Ä vurdere svulster og produsere et Þkende antall bildedata som radiologer mÄ tolke. I tillegg til antallet bildemodaliteter, Þker ogsÄ antallet pasienter, noe som fÞrer til at visualiseringslÞsninger mÄ bli skalert opp for Ä adressere den Þkende kompleksiteten av multimodal- og multipasientdata. Dessuten er ikke medisinsk visualisering kun tiltenkt medisinsk personale, men har ogsÄ som mÄl Ä informere pasienter, pÄrÞrende, og offentligheten om risikoen relatert til visse sykdommer, og mulige behandlinger. Derfor har vi identifisert behovet for Ä skalere opp medisinske visualiseringslÞsninger for Ä kunne hÄndtere multipublikumdata.
Denne avhandlingen adresserer skaleringen av disse dimensjonene i forskjellige bidrag vi har kommet med. FĂžrst presenterer vi teknikkene vĂ„re for Ă„ skalere visualiseringer i flere modaliteter. Vi introduserer en visualiseringsteknikk som tar i bruk smĂ„ multipler for Ă„ vise data fra flere modaliteter innenfor et bildesnitt. Dette lar radiologer utforske dataen effektivt uten Ă„ mĂ„tte bruke flere sidestilte vinduer. I det neste steget utviklet vi en analyseplatform ved Ă„ ta i bruk «radiomic tumor profiling» pĂ„ forskjellige bildemodaliteter for Ă„ analysere kohortdata og finne nye biomarkĂžrer fra bilder. BiomarkĂžrer fra bilder er indikatorer basert pĂ„ bildedata som kan forutsi variabler relatert til kliniske utfall. «Radiomic tumor profiling» er en teknikk som genererer mulige biomarkĂžrer fra bilder basert pĂ„ fĂžrste- og andregrads statistiske mĂ„linger. Applikasjonen lar medisinske eksperter analysere multiparametrisk bildedata for Ă„ finne mulige korrelasjoner mellom kliniske parameter og data fra «radiomic tumor profiling». Denne tilnĂŠrmingen skalerer i to dimensjoner, multimodal og multipasient. I en senere versjon la vi til funksjonalitet for Ă„ skalere multipublikumdimensjonen ved Ă„ gjĂžre applikasjonen vĂ„r anvendelig for livmorhalskreft- og prostatakreftdata, i tillegg til livmorkreftdataen som applikasjonen var designet for. I et senere bidrag fokuserer vi pĂ„ svulstdata pĂ„ en annen skala og muliggjĂžr analysen av svulstdeler ved Ă„ bruke multimodal bildedata i en tilnĂŠrming basert pĂ„ hierarkisk gruppering. Applikasjonen vĂ„r finner mulige interessante regioner som kan informere fremtidige behandlingsavgjĂžrelser. I et annet bidrag, en digital sonderingsinteraksjon, fokuserer vi pĂ„ multipasientdata. Bildedata fra flere pasienter kan sammenlignes for Ă„ finne interessante mĂžnster i svulstene som kan vĂŠre knyttet til hvor aggressive svulstene er. Til slutt skalerer vi multipublikumdimensjonen med en likhetsvisualisering som er anvendelig for forskning pĂ„ livmorkreft, pĂ„ bilder av nevrologisk kreft, og maskinlĂŠringsforskning pĂ„ automatisk segmentering av svulstdata. Som en kontrast til de allerede fremhevete bidragene, fokuserer vĂ„rt siste bidrag, ScrollyVis, hovedsakelig pĂ„ multipublikumkommunikasjon. Vi muliggjĂžr skapelsen av dynamiske og vitenskapelige âscrollytellingâ-opplevelser for spesifikke eller generelle publikum. Slike historien kan bli brukt i spesifikke brukstilfeller som kommunikasjon mellom lege og pasient, eller for Ă„ kommunisere vitenskapelige resultater via historier til et generelt publikum i en digital museumsutstilling.
VÄre foreslÄtte applikasjoner og interaksjonsteknikker har blitt demonstrert i brukstilfeller og evaluert med domeneeksperter og fokusgrupper. Dette har fÞrt til at noen av vÄre bidrag allerede er i bruk pÄ andre forskingsinstitusjoner. Vi Þnsker Ä evaluere innvirkningen deres pÄ andre vitenskapelige felt og offentligheten i fremtidige arbeid.Medical visualization is one of the most application-oriented areas of visualization research. Close collaboration with medical experts is essential for interpreting medical imaging data and creating meaningful visualization techniques and visualization applications. Cancer is one of the most common causes of death, and with increasing average age in developed countries, gynecological malignancy case numbers are rising. Modern imaging techniques are an essential tool in assessing tumors and produce an increasing number of imaging data radiologists must interpret. Besides the number of imaging modalities, the number of patients is also rising, leading to visualization solutions that must be scaled up to address the rising complexity of multi-modal and multi-patient data. Furthermore, medical visualization is not only targeted toward medical professionals but also has the goal of informing patients, relatives, and the public about the risks of certain diseases and potential treatments. Therefore, we identify the need to scale medical visualization solutions to cope with multi-audience data.
This thesis addresses the scaling of these dimensions in different contributions we made. First, we present our techniques to scale medical visualizations in multiple modalities. We introduced a visualization technique using small multiples to display the data of multiple modalities within one imaging slice. This allows radiologists to explore the data efficiently without having several juxtaposed windows. In the next step, we developed an analysis platform using radiomic tumor profiling on multiple imaging modalities to analyze cohort data and to find new imaging biomarkers. Imaging biomarkers are indicators based on imaging data that predict clinical outcome related variables. Radiomic tumor profiling is a technique that generates potential imaging biomarkers based on first and second-order statistical measurements. The application allows medical experts to analyze the multi-parametric imaging data to find potential correlations between clinical parameters and the radiomic tumor profiling data. This approach scales up in two dimensions, multi-modal and multi-patient. In a later version, we added features to scale the multi-audience dimension by making our application applicable to cervical and prostate cancer data and the endometrial cancer data the application was designed for. In a subsequent contribution, we focus on tumor data on another scale and enable the analysis of tumor sub-parts by using the multi-modal imaging data in a hierarchical clustering approach. Our application finds potentially interesting regions that could inform future treatment decisions. In another contribution, the digital probing interaction, we focus on multi-patient data. The imaging data of multiple patients can be compared to find interesting tumor patterns potentially linked to the aggressiveness of the tumors. Lastly, we scale the multi-audience dimension with our similarity visualization applicable to endometrial cancer research, neurological cancer imaging research, and machine learning research on the automatic segmentation of tumor data. In contrast to the previously highlighted contributions, our last contribution, ScrollyVis, focuses primarily on multi-audience communication. We enable the creation of dynamic scientific scrollytelling experiences for a specific or general audience. Such stories can be used for specific use cases such as patient-doctor communication or communicating scientific results via stories targeting the general audience in a digital museum exhibition.
Our proposed applications and interaction techniques have been demonstrated in application use cases and evaluated with domain experts and focus groups. As a result, we brought some of our contributions to usage in practice at other research institutes. We want to evaluate their impact on other scientific fields and the general public in future work.Doktorgradsavhandlin
Hierarchical processing, editing and rendering of acquired geometry
La reprĂ©sentation des surfaces du monde rĂ©el dans la mĂ©moire dâune machine peut dĂ©sormais ĂȘtre obtenue automatiquement via divers pĂ©riphĂ©riques de capture tels que les scanners 3D. Ces nouvelles sources de donnĂ©es, prĂ©cises et rapides, amplifient de plusieurs ordres de grandeur la rĂ©solution des surfaces 3D, apportant un niveau de prĂ©cision Ă©levĂ© pour les applications nĂ©cessitant des modĂšles numĂ©riques de surfaces telles que la conception assistĂ©e par ordinateur, la simulation physique, la rĂ©alitĂ© virtuelle, lâimagerie mĂ©dicale, lâarchitecture, lâĂ©tude archĂ©ologique, les effets spĂ©ciaux, lâanimation ou bien encore les jeux video. Malheureusement, la richesse de la gĂ©omĂ©trie produite par ces mĂ©thodes induit une grande, voire gigantesque masse de donnĂ©es Ă traiter, nĂ©cessitant de nouvelles structures de donnĂ©es et de nouveaux algorithmes capables de passer Ă lâĂ©chelle dâobjets pouvant atteindre le milliard dâĂ©chantillons. Dans cette thĂšse, je propose des solutions performantes en temps et en espace aux problĂšmes de la modĂ©lisation, du traitement gĂ©omĂ©trique, de lâĂ©dition intĂ©ractive et de la visualisation de ces surfaces 3D complexes. La mĂ©thodologie adoptĂ©e pendant lâĂ©laboration transverse de ces nouveaux algorithmes est articulĂ©e autour de 4 Ă©lĂ©ments clĂ©s : une approche hiĂ©rarchique systĂ©matique, une rĂ©duction locale de la dimension des problĂšmes, un principe dâĂ©chantillonage-reconstruction et une indĂ©pendance Ă lâĂ©numĂ©ration explicite des relations topologiques aussi appelĂ©e approche basĂ©e-points. En pratique, ce manuscrit propose un certain nombre de contributions, parmi lesquelles : une nouvelle structure hiĂ©rarchique hybride de partitionnement, lâArbre Volume-Surface (VS-Tree) ainsi que de nouveaux algorithmes de simplification et de reconstruction ; un systĂšme dâĂ©dition intĂ©ractive de grands objets ; un noyau temps-rĂ©el de synthĂšse gĂ©omĂ©trique par raffinement et une structure multi-rĂ©solution offrant un rendu efficace de grands objets. Ces structures, algorithmes et systĂšmes forment une chaĂźne capable de traiter les objets en provenance du pipeline dâacquisition, quâils soient reprĂ©sentĂ©s par des nuages de points ou des maillages, possiblement non 2-variĂ©tĂ©s. Les solutions obtenues ont Ă©tĂ© appliquĂ©es avec succĂšs aux donnĂ©es issues des divers domaines dâapplication prĂ©citĂ©s.Digital representations of real-world surfaces can now be obtained automatically using various acquisition devices such as 3D scanners and stereo camera systems. These new fast and accurate data sources increase 3D surface resolution by several orders of magnitude, borrowing higher precision to applications which require digital surfaces. All major computer graphics applications can take benefit of this automatic modeling process, including: computer-aided design, physical simulation, virtual reality, medical imaging, architecture, archaeological study, special effects, computer animation and video games. Unfortunately, the richness of the geometry produced by these media comes at the price of a large, possibility gigantic, amount of data which requires new efficient data structures and algorithms offering scalability for processing such objects. This thesis proposes time and space efficient solutions for modeling, editing and rendering such complex surfaces, solving these problems with new algorithms sharing 4 fundamental elements: a systematic hierarchical approach, a local dimension reduction, a sampling-reconstruction paradigm and a pointbased basis. Basically, this manuscript proposes several contributions, including: a new hierarchical space subdivision structure, the Volume-Surface Tree, for geometry processing such as simplification and reconstruction; a streaming system featuring new algorithms for interactive editing of large objects, an appearancepreserving multiresolution structure for efficient rendering of large point-based surfaces, and a generic kernel for real-time geometry synthesis by refinement. These elements form a pipeline able to process acquired geometry, either represented by point clouds or non-manifold meshes. Effective results have been successfully obtained with data coming from the various applications mentioned
Recommended from our members
Mobile depth sensing technology and algorithms with application to occupational therapy healthcare
This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThe UK government is striving to shift its current healthcare delivery model from clini-cianâoriented services, to that of patient and selfâcareâoriented intervention strategies. It seeks to do so through Information Communication (ICT) and Computer Mediated Re-ality Technologies (CMRT) as a key strategy to overcome the everâincreasing scarcity of healthcare resources and costs. To this end, in the UK the use of paperâbased information systems have exhibited their limitations in providing apposite care. At the national level, The Royal College of Occupational Therapists (RCOT) identify home visits and modifica-tions as key levers in a multifactorial health programme to evaluate interventions for older people with a history of falling or are identified as being prone to falling. Prescribing Assistive Equipment (AE) is one such mechanism that seeks to reduce the risk of falling whilst promoting the continued independence of physical dexterity and mobility in older adults at home. In the UK, the yearly cost of falls is estimated at ÂŁ2.3 billion. Further evidence places a 30% to 60% abandonment rate on prescribed AE by and large due to a âpoor fitâ and measurement inaccuracies.
To remain aligned with the national strategy, and assist in the eradication of measurement inaccuracies, this thesis employs Mobile Depth Sensing and Motion Track-ing Devices (MDSMTDs) to assist OTs in in the process of digitally measuring the extrin-sic fallârisk factors for the provision of AE. The quintessential component in this assess-ment lies in the measurement of fittings and furniture items in the home. To digitise and aid in this process, the artefact presented in this thesis employs stereo computerâvision and camera calibration algorithms to extract edges in 3D space. It modifies the SobelâFeldman convolution filter by reducing the magnitude response and employs the camera intrinsic parameters as a mechanism to calculate the distortion matrix for interpolation between the edges and the 3D point cloud. Further Augmented Reality User Experience (AR-UX) facets are provided to digitise current state of the art clinical guidance and over-lay its instructions onto the real world (i.e., 3D space).
Empirical mixed methods assessment revealed that in terms of accuracy, the arte-fact exhibited enhanced performance gains over current paperâbased guidance. In terms of accuracy consistency, the artefact can rectify measurement consistency inaccuracies, but there are still a wide range of factors that can influence the integrity of the point-cloud in respect of the deviceâs point-of-view, holding positions and measurement speed. To this end, OTs usability, and adoption preferences materialise in favour of the artefact. In conclusion, this thesis demonstrates that MDSMTDs are a promising alterna-tive to existing paperâbased measurement practices as OTs appear to prefer the digitalâbased system and that they can take measurements more efficiently and accurately
MeasureIt-ARCH: A Tool for Facilitating Architectural Design in the Open Source Software Blender
This thesis discusses the design and synthesis of MeasureIt-ARCH, a GNU GPL licensed software add-on developed by the author in order to add functionality to the Open Source 3D modeling software Blender that facilitates the creation of architectural drawings. MeasureIt-ARCH adds to Blender simple tools to dimension and annotate 3D models, as well as basic support for the definition and drawing of line work. These tools for the creation of dimensions, annotations and line work are designed to be used in tandem with Blender's existing modelling and rendering tool set. While the drawings that MeasureIt-ARCH produces are fundamentally conventional, as are the majority of the techniques that MeasureIt-ARCH employs to create them, MeasureIt-ARCH does provide two simple and relatively novel methods in its drawing systems. MeasureIt-ARCH provides a new method for the placement of dimension elements in 3D space that draws on the dimension's three dimensional context and surrounding geometry order to determine a placement that optimizes legibility. This dimension placement method does not depend on a 2D work plane, a convention that is common in industry standard Computer Aided Design software. MeasureIt-ARCH also implements a new approach for drawing silhouette lines that operates by transforming the silhouetted models geometry in 4D 'Clip Space'.
The hope of this work is that MeasureIt-ARCH might be a small step towards creating an Open Source design pipeline for Architects. A step towards creating architectural drawings that can be shared, read, and modified by anyone, within a platform that is itself free to be changed and improved. The creation of MeasureIt-ARCH is motivated by two goals. First, the work aims to create a basic functioning Open Source platform for the creation of architectural drawings within Blender that is publicly and freely available for use. Second, MeasureIt-ARCH's development served as an opportunity to engage in an interdisciplinary act of craft, providing the author an opportunity to explore the act of digital tool making and gain a basic competency in this intersection between Architecture and Computer Science.
To achieve these goals, MeasureIt-ARCH's development draws on references from the history of line drawing and dimensioning within Architecture and Computer Science. On the Architectural side, we make use of the history of architectural drawing and dimensioning conventions as described by Mario Carpo, Alberto PĂ©rez GĂłmez and others, as well as more contemporary frameworks for the classification of architectural software, such as Mark Bew and Mervyn Richard's BIM Levels framework, in order to help determine the scope of MeasureIt-ARCH's feature set. When crafting MeasureIt-ARCH, precedent works from the field of Computer Science that implement methods for producing line drawings from 3D models helped inform the authorâs approach to line drawing. In particular this work draws on the overview of line drawing methods produced by BĂ©nard Pierre and Aaron Hertzmann, Arthur Appel's method for line drawing using 'Quantitative Invisibility', the techniques employed in the Freestyle line drawing system created by Grabli et al. as well as other to help inform MeasureIt-ARCH's simple drawing tools.
Beyond discussing MeasureIt-ARCH's development and its motivations, this thesis also provides three small speculative discussions about the implications that an Open Source design tool might have on the architectural profession.
We investigate MeasureIt-ARCH's use for small scale architectural projects in a practical setting, using it's tool set to produce conceptual design and renovation drawings for cottages at the Lodge at Pine Cove. We provide a demonstration of how MeasureIt-ARCH and Blender can integrate with external systems and other Blender add-ons to produce a proof of concept, dynamic data visualization of the Noosphere installation at the Futurium center in Berlin by the Living Architecture Systems Group. Finally, we discuss the tool's potential to facilitate greater engagement with the Open Source Architecture (OSArc) movement by illustrating a case study of the work done by Alastair Parvin and Clayton Prest on the WikiHouse project, and by highlighting the challenges that face OSArc projects as they try to produce Open Source Architecture without an Open Source design software
On object recognition for industrial augmented reality
Some reasons are market pressure, an increase of functionality, and adaptability to an already complex environment, among others. Therefore, workers face fast-changing and challenging tasks along with all the product lifecycle that reach the human cognitive limits. Although nowadays some operations are automated, many of them still need to be carried out by humans because of their complexity.
In addition to management strategies and design for X, Industrial Augmented Reality (IAR) has proven to potentially benefit activities such as maintenance, assembly, manufacturing, and repair, among others. It is also supposed to upgrade the manufacturing processes by improving it, simplifying decision-making activities, reducing time and user movements, diminishing errors, and decreasing mental and physical effort. Nevertheless, IAR has not succeeded in breaking out of the laboratories and establishing itself as a strong solution in the industry, mainly because technical and interaction components are far from ideal. Its advance is limited by its enabling technologies. One of its biggest challenges are the methods for understanding the surroundings considering the different domain variables that affect IAR implementations. Thus, inspired by some systematical methodologies proposing that, for any problemsolving activity, it is required to define the characteristics that constrain the problem and the needs to be satisfied, a general frame of IAR was proposed through the identification of Domain Variables (DV), that are relevant characteristics of the industrial process in the previous Augmented Reality (AR) applications. These DV regard the user, parts, environment, and task that have an impact on the technical implementation and user performance and perception (Chapter 2).
Subsequently, a detailed analysis of the influence of the DV on technical implementations related to the processes intended to understand the surroundings was performed. The results of this analysis suggest that the DV influence the technical process in two ways. The first one is that they define the boundaries in the characteristics of the technology, and the second one is that they cause some issues in the process of understanding the surroundings (Chapter 3).
Further, an automatic method for creating synthetic datasets using solely the 3D model of the parts was proposed. It is hypothesized that the proposed variables are the main source of visual variations of an object in this context. Thus, the proposed method is derived from physically recreated light-matter interactions of this relevant variables. This method is aimed to create fully labeled datasets for training and testing surrounding understanding algorithms (Chapter 4).
Finally, the proposed method is evaluated in a study case of object classification of two cases: a particular industrial case, and a general classification problem (using classes of ImageNet). Results suggest that fine-tuning models with the proposed method reach comparable performance (no statistical difference) than models trained with photos. These results validate the proposed method as a viable alternative for training surrounding understanding algorithms applied to industrial cases (Chapter 5)
Image based surface reflectance remapping for consistent and tool independent material appearence
Physically-based rendering in Computer Graphics requires the knowledge of material properties other than 3D shapes, textures and colors, in order to solve the rendering equation. A number of material models have been developed, since no model is currently able to reproduce the full range of available materials. Although only few material models have been widely adopted in current rendering systems, the lack of standardisation causes several issues in the 3D modelling
workflow, leading to a heavy tool dependency of material appearance. In industry, final decisions about products are often based on a virtual prototype, a crucial step for the production pipeline, usually developed by a collaborations among several
departments, which exchange data. Unfortunately, exchanged data often tends to differ from the original, when imported into a different application. As a result, delivering consistent visual results requires time, labour and computational cost.
This thesis begins with an examination of the current state of the art in material appearance representation and capture, in order to identify a suitable strategy to tackle material appearance consistency. Automatic solutions to this problem are suggested in this work, accounting for the constraints of real-world scenarios, where the only available information is a reference rendering and the renderer used to obtain it, with no access to the implementation of the shaders. In particular, two image-based frameworks are proposed, working under these constraints.
The first one, validated by means of perceptual studies, is aimed to the remapping of BRDF parameters and useful when the parameters used for the reference rendering are available. The second one provides consistent material appearance across different renderers, even when the parameters used for the reference are unknown. It allows the selection of an arbitrary reference rendering tool, and manipulates the output of other renderers in order to be consistent with the reference
High-Quality Shadow Rendering from Complex Light Sources
V interaktivnĂch aplikacĂch jsou stĂny tradiÄnÄ zobrazovĂĄny s pomocĂ algoritmu zaloĆŸenĂœm na stĂnovĂœch mapĂĄch. NevĂœhodou toho algoritmu je, ĆŸe stĂnovĂĄ mapa, reprezentovanĂĄ texturou, mĂĄ pouze omezenĂ© rozliĆĄenĂ. To mĆŻĆŸe vĂ©st k nepÄknĂœm vizuĂĄlnĂm artefaktĆŻm objevujĂcĂch se na hranĂĄch stĂnĆŻ. Tato prĂĄce pĆedstavuje postup, kterĂœ je zaloĆŸen na vylepĆĄenĂ© deformaci textury. To umoĆŸnĂ zobrazit scĂ©nu obsahujĂcĂ sloĆŸitĂ© svÄtelnĂ© zdroje, zredukovat artefakty na hranicĂch stĂnĆŻ a takĂ© vylepĆĄit kvalitu stĂnĆŻ bez ohledu na typu scĂ©ny a jejĂ konfiguraci.In interactive applications, shadows are traditionally rendered using the shadow mapping algorithm. The disadvantage of the algorithm is limited resolution of depth texture which may lead to unpleasant visual artifacts on shadow edges. This work introduces an approach that is based on the improved texture warping. It allows for rendering a scene with the complex light sources, reduce the artifacts on the shadow boundaries and also improve the quality of the shadows regardless of the type of the scene and its configuration.