KOLAM : human computer interfaces fro visual analytics in big data imagery

In the present day, we are faced with a deluge of disparate and dynamic information from multiple heterogeneous sources. Among these are the big data imagery datasets that are rapidly being generated via mature acquisition methods in the geospatial, surveillance (specifically, Wide Area Motion Imagery or WAMI) and biomedical domains. The need to interactively visualize these imagery datasets by using multiple types of views (as needed) into the data is common to these domains. Furthermore, researchers in each domain have additional needs: users of WAMI datasets also need to interactively track objects of interest using algorithms of their choice, visualize the resulting object trajectories and interactively edit these results as needed. While software tools that fulfill each of these requirements individually are available and well-used at present, there is still a need for tools that can combine the desired aspects of visualization, human computer interaction (HCI), data analysis, data management, and (geo-)spatial and temporal data processing into a single flexible and extensible system. KOLAM is an open, cross-platform, interoperable, scalable and extensible framework for visualization and analysis that we have developed to fulfil the above needs. The novel contributions in this thesis are the following: 1) Spatio-temporal caching for animating both giga-pixel and Full Motion Video (FMV) imagery, 2) Human computer interfaces purposefully designed to accommodate big data visualization, 3) Human-in-the-loop interactive video object tracking - ground-truthing of moving objects in wide area imagery using algorithm assisted human-in-the-loop coupled tracking, 4) Coordinated visualization using stacked layers, side-by-side layers/video sub-windows and embedded imagery, 5) Efficient one-click manual tracking, editing and data management of trajectories, 6) Efficient labeling of image segmentation regions and passing these results to desired modules, 7) Visualization of image processing results generated by non-interactive operators using layers, 8) Extension of interactive imagery and trajectory visualization to multi-monitor wall display environments, 9) Geospatial applications: Providing rapid roam, zoom and hyper-jump spatial operations, interactive blending, colormap and histogram enhancement, spherical projection and terrain maps, 10) Biomedical applications: Visualization and target tracking of cell motility in time-lapse cell imagery, collecting ground-truth from experts on whole-slide imagery (WSI) for developing histopathology analytic algorithms and computer-aided diagnosis for cancer grading, and easy-to-use tissue annotation features.Includes bibliographical reference

Architectures for ubiquitous 3D on heterogeneous computing platforms

Today, a wide scope for 3D graphics applications exists, including domains such as scientific visualization, 3D-enabled web pages, and entertainment. At the same time, the devices and platforms that run and display the applications are more heterogeneous than ever. Display environments range from mobile devices to desktop systems and ultimately to distributed displays that facilitate collaborative interaction. While the capability of the client devices may vary considerably, the visualization experiences running on them should be consistent. The field of application should dictate how and on what devices users access the application, not the technical requirements to realize the 3D output. The goal of this thesis is to examine the diverse challenges involved in providing consistent and scalable visualization experiences to heterogeneous computing platforms and display setups. While we could not address the myriad of possible use cases, we developed a comprehensive set of rendering architectures in the major domains of scientific and medical visualization, web-based 3D applications, and movie virtual production. To provide the required service quality, performance, and scalability for different client devices and displays, our architectures focus on the efficient utilization and combination of the available client, server, and network resources. We present innovative solutions that incorporate methods for hybrid and distributed rendering as well as means to manage data sets and stream rendering results. We establish the browser as a promising platform for accessible and portable visualization services. We collaborated with experts from the medical field and the movie industry to evaluate the usability of our technology in real-world scenarios. The presented architectures achieve a wide coverage of display and rendering setups and at the same time share major components and concepts. Thus, they build a strong foundation for a unified system that supports a variety of use cases.Heutzutage existiert ein großer Anwendungsbereich für 3D-Grafikapplikationen wie wissenschaftliche Visualisierungen, 3D-Inhalte in Webseiten, und Unterhaltungssoftware. Gleichzeitig sind die Geräte und Plattformen, welche die Anwendungen ausführen und anzeigen, heterogener als je zuvor. Anzeigegeräte reichen von mobilen Geräten zu Desktop-Systemen bis hin zu verteilten Bildschirmumgebungen, die eine kollaborative Anwendung begünstigen. Während die Leistungsfähigkeit der Geräte stark schwanken kann, sollten die dort laufenden Visualisierungen konsistent sein. Das Anwendungsfeld sollte bestimmen, wie und auf welchem Gerät Benutzer auf die Anwendung zugreifen, nicht die technischen Voraussetzungen zur Erzeugung der 3D-Grafik. Das Ziel dieser Thesis ist es, die diversen Herausforderungen zu untersuchen, die bei der Bereitstellung von konsistenten und skalierbaren Visualisierungsanwendungen auf heterogenen Plattformen eine Rolle spielen. Während wir nicht die Vielzahl an möglichen Anwendungsfällen abdecken konnten, haben wir eine repräsentative Auswahl an Rendering-Architekturen in den Kernbereichen wissenschaftliche Visualisierung, web-basierte 3D-Anwendungen, und virtuelle Filmproduktion entwickelt. Um die geforderte Qualität, Leistung, und Skalierbarkeit für verschiedene Client-Geräte und -Anzeigen zu gewährleisten, fokussieren sich unsere Architekturen auf die effiziente Nutzung und Kombination der verfügbaren Client-, Server-, und Netzwerkressourcen. Wir präsentieren innovative Lösungen, die hybrides und verteiltes Rendering als auch das Verwalten der Datensätze und Streaming der 3D-Ausgabe umfassen. Wir etablieren den Web-Browser als vielversprechende Plattform für zugängliche und portierbare Visualisierungsdienste. Um die Verwendbarkeit unserer Technologie in realitätsnahen Szenarien zu testen, haben wir mit Experten aus der Medizin und Filmindustrie zusammengearbeitet. Unsere Architekturen erreichen eine umfassende Abdeckung von Anzeige- und Rendering-Szenarien und teilen sich gleichzeitig wesentliche Komponenten und Konzepte. Sie bilden daher eine starke Grundlage für ein einheitliches System, das eine Vielzahl an Anwendungsfällen unterstützt

Paralelismo y distribución para la optimización del proceso de visualización de volúmenes en tiempo real

Los métodos actuales de simulación, entrenamiento y planeamiento de procedimientos quirúrgicos basados en la tecnología de realidad virtual utilizan datos volumétricos producidos por escáneres de resonancia magnética (MRI) y tomografía axial computada (CT) de pacientes reales. Considerando el estado actual de la tecnología de renderizado de imágenes médicas, la visualización en tiempo real de los grandes volúmenes de información extraída de estos dispositivos presenta un desafío: desarrollar nuevos métodos de procesamiento donde se respeten las severas exigencias de resolución y calidad, además de permitir una gran fluidez en la animación para obtener un nivel suficientemente alto de realismo. Todas estas características demandan una potencia de cálculo extremadamente grande. Como solución para esta problemática, se plantea la utilización de una arquitectura de computación en paralelo y distribuida, para lograr la mayor potencia de cálculo posible y al mismo tiempo, económicamente viable. Dicha solución consiste en un sistema de distribución de cómputo específicamente diseñado para problemas de renderizado de imágenes tridimensionales orientado a la representación gráfica de volúmenes en tiempo real, a través del uso de múltiples computadoras con placas gráficas programables, interconectadas a través de una red Ethernet de alta performance.Eje: Computación gráfica, imágenes y visualizaciónRed de Universidades con Carreras en Informática (RedUNCI

Streaming and User Behaviour in Omnidirectional Videos

Omnidirectional videos (ODVs) have gone beyond the passive paradigm of traditional video, offering higher degrees of immersion and interaction. The revolutionary novelty of this technology is the possibility for users to interact with the surrounding environment, and to feel a sense of engagement and presence in a virtual space. Users are clearly the main driving force of immersive applications and consequentially the services need to be properly tailored to them. In this context, this chapter highlights the importance of the new role of users in ODV streaming applications, and thus the need for understanding their behaviour while navigating within ODVs. A comprehensive overview of the research efforts aimed at advancing ODV streaming systems is also presented. In particular, the state-of-the-art solutions under examination in this chapter are distinguished in terms of system-centric and user-centric streaming approaches: the former approach comes from a quite straightforward extension of well-established solutions for the 2D video pipeline while the latter one takes the benefit of understanding users’ behaviour and enable more personalised ODV streaming

Knife Edge Scanning Microscope Brain Atlas Interface for Tracing and Analysis of Vasculature Data

The study of the neurovascular network in the brain is important to understand brain functions as well as causes of several brain dysfunctions. Many techniques have been applied to acquire neurovascular data. The Knife-Edge Scanning Microscope (KESM), developed by the Brain Network Lab at Texas A&M University, can generate whole-brain-scale data at submicrometer resolution. The specimen can be stained with different stains, and depending on the type of stain used, the KESM can image different types of microstructures in the brain. The India ink stain allows the neurovascular network in the brain to be imaged. In order to visualize and analyze such large datasets (~ 1.5 TB per brain), a lightweight, web-based mouse brain atlas called the Knife-Edge Scanning Microscope Brain Atlas (KESMBA) was developed in the lab. The atlas serves several whole mouse brain data sets including India ink. The multi-section overlay technique used in the atlas enables 3D visualization of the structural information in the data. To solve the challenging issue of tracing micro-vessels in the brain, in this thesis a semi-automated tracing and analysis method is developed and integrated into the KESM brain atlas. Using the KESMBA interface developed in this thesis, the user can look at the 3D structure of the vessels on the brain atlas and can guide the tracing algorithm. To analyze the vasculature network traced by the user, a data analysis component is also added. This new KESMBA interface is expected to help in quickly tracing and analyzing the vascular network of the brain with minimal manual effort. In order to visualize and analyze such large data sets (~ 1.5 TB per brain), a light-weight, web-based mouse brain atlas called the Knife-Edge Scanning Microscope Brain Atlas (KESMBA) was developed in the lab. The atlas serves several whole mouse brain data sets including India ink. The multi-section overlay technique used in the atlas enables 3D visualization of the structural information in the data. To solve the challenging issue of tracing micro-vessels in the brain, in this thesis a semi-automated tracing and analysis method is developed and integrated into the KESM brain atlas. Using the KESMBA interface developed in this thesis, the user can look at the 3D structure of the vessels on the brain atlas and can guide the tracing algorithm. In order to analyze the vasculature network traced by the user, a data analysis component is also added. This new KESMBA interface is expected to help in quickly tracing and analyzing the vascular network of the brain with minimal manual effort