From Capture to Display: A Survey on Volumetric Video

Volumetric video, which offers immersive viewing experiences, is gaining increasing prominence. With its six degrees of freedom, it provides viewers with greater immersion and interactivity compared to traditional videos. Despite their potential, volumetric video services poses significant challenges. This survey conducts a comprehensive review of the existing literature on volumetric video. We firstly provide a general framework of volumetric video services, followed by a discussion on prerequisites for volumetric video, encompassing representations, open datasets, and quality assessment metrics. Then we delve into the current methodologies for each stage of the volumetric video service pipeline, detailing capturing, compression, transmission, rendering, and display techniques. Lastly, we explore various applications enabled by this pioneering technology and we present an array of research challenges and opportunities in the domain of volumetric video services. This survey aspires to provide a holistic understanding of this burgeoning field and shed light on potential future research trajectories, aiming to bring the vision of volumetric video to fruition.Comment: Submitte

Grounded Visual Analytics: A New Approach to Discovering Phenomena in Data at Scale

We introduce Grounded Visual Analytics, a new method that integrates qualitative and quantitative approaches in order to help investigators discover patterns about human activity. Investigators who develop or study systems often use log data, which keeps track of interactions their participants perform. Discovering and characterizing patterns in this data is important because it can help guide interactive computing system design. This new approach integrates Visual Analytics, a field that investigates Information Visualization and interactive machine learning, and Grounded Theory, a rigorous qualitative research method for discovering nuanced understanding of qualitative data. This dissertation defines and motivates this new approach, reviews relevant existing tools, builds the Log Timelines system. We present and analyze six case studies that use Log Timelines, a probe that we created in order explore Grounded Visual Analytics. In a series of case studies, we collaborate with a participant-investigator on their own project and data. Their use of Grounded Visual Analytics generates ideas about how future research can bridge the gap between qualitative and quantitative methods

Organizational identity design: A multimodal discourse analysis of Australian university homepages

This thesis studies web homepages to understand the complex social practice of organizational identity communication on a digital medium. It examines how designs of web homepages realize discourses of identity through the mobilization and orchestration of various semiotic resources into multimodal ensembles, addressing critical organizational visual identity elements (‘logo,’ ‘corporate name,’ ‘color,’ ‘typography,’ ‘graphic shapes,’ and ‘images’), communicative content of the page, and navigation structures. By examining these three ‘strata’ of organizational identity communication, it investigates how a homepage uses formal design elements and more abstract principles of composition, such as spatial positioning and content ordering, as resources for making meaning. The data consists of three complementary sets drawn from thirty-nine web homepages of Australian university websites in 2020. Data set #1 includes four homepages for an in-depth study of organizational identity designs; data set #2 consists of 400 images from the ‘above the fold’ web area as the most strategic space on four homepages between the years 2015 and 2021; data set #3 is comprised of eight historical versions of a selected web homepage between the years 2000 and 2021, with three most representative designs for an in-depth investigation. Grounded in the discourse-analytic approach informed by multimodal social semiotics, the thesis adopts a mixed-method approach to data analysis. It applies multimodal discourse analysis combining the Genre and Multimodality model (Bateman, 2008; Bateman et al., 2017) to document the structural design patterns and social semiotic (metafunctional) approach to address the meaning potentials of the identified patterns; (Kress & van Leeuwen, 2021); content analysis (Bell, 2001; Rose, 2016) and visual social actor framework (van Leeuwen, 2008) to identify key representational tropes and visual personae. The study reveals the role of design as a mediating tool between the participants of discourse – the rhetor-institution/designer and envisaged audiences – and offers systematic insights into the uses of semiotic resources, both material (e.g., formal design elements and navigation structures) and nonmaterial (e.g., spatial considerations and content structuring), all contributing to the production of meanings and fostering identification with such meanings in the form of association with the university’s identity. Addressing the subtle differences and shifts in the form and function of key layout structures and strategies of viewer engagement, the study concludes that is plural – each university constantly revises semiotic choices and their multimodal composition to achieve specific rhetorical purposes. Together with several visual design choices, five identified strategies of viewer engagement – proximation, alignment, equalization, objectivation, and subjectivation – promote the university as a place of opportunity, achievement, sociality, and intellectual growth for a student as an individual and as a member of the community. The current research contributes to the emerging collaboration between multimodality, organization studies, and branding, recognizing the complexities and importance of multimodal communication in web-mediated texts amidst the critically increased roles of marketization and social presence in the current higher education landscape

Visual analytics methods for retinal layers in optical coherence tomography data

Optical coherence tomography is an important imaging technology for the early detection of ocular diseases. Yet, identifying substructural defects in the 3D retinal images is challenging. We therefore present novel visual analytics methods for the exploration of small and localized retinal alterations. Our methods reduce the data complexity and ensure the visibility of relevant information. The results of two cross-sectional studies show that our methods improve the detection of retinal defects, contributing to a deeper understanding of the retinal condition at an early stage of disease.Die optische Kohärenztomographie ist ein wichtiges Bildgebungsverfahren zur Früherkennung von Augenerkrankungen. Die Identifizierung von substrukturellen Defekten in den 3D-Netzhautbildern ist jedoch eine Herausforderung. Wir stellen daher neue Visual-Analytics-Methoden zur Exploration von kleinen und lokalen Netzhautveränderungen vor. Unsere Methoden reduzieren die Datenkomplexität und gewährleisten die Sichtbarkeit relevanter Informationen. Die Ergebnisse zweier Querschnittsstudien zeigen, dass unsere Methoden die Erkennung von Netzhautdefekten in frühen Krankheitsstadien verbessern

Visual analytics for relationships in scientific data

Domain scientists hope to address grand scientific challenges by exploring the abundance of data generated and made available through modern high-throughput techniques. Typical scientific investigations can make use of novel visualization tools that enable dynamic formulation and fine-tuning of hypotheses to aid the process of evaluating sensitivity of key parameters. These general tools should be applicable to many disciplines: allowing biologists to develop an intuitive understanding of the structure of coexpression networks and discover genes that reside in critical positions of biological pathways, intelligence analysts to decompose social networks, and climate scientists to model extrapolate future climate conditions. By using a graph as a universal data representation of correlation, our novel visualization tool employs several techniques that when used in an integrated manner provide innovative analytical capabilities. Our tool integrates techniques such as graph layout, qualitative subgraph extraction through a novel 2D user interface, quantitative subgraph extraction using graph-theoretic algorithms or by querying an optimized B-tree, dynamic level-of-detail graph abstraction, and template-based fuzzy classification using neural networks. We demonstrate our system using real-world workflows from several large-scale studies. Parallel coordinates has proven to be a scalable visualization and navigation framework for multivariate data. However, when data with thousands of variables are at hand, we do not have a comprehensive solution to select the right set of variables and order them to uncover important or potentially insightful patterns. We present algorithms to rank axes based upon the importance of bivariate relationships among the variables and showcase the efficacy of the proposed system by demonstrating autonomous detection of patterns in a modern large-scale dataset of time-varying climate simulation

Typogenetic design - aesthetic decision support for architectural shape generation

Typogenetic Design is an interactive computational design system combining generative design, evolutionary search and architectural optimisation technology. The active tool for supporting design decisions during architectural shape generation uses an aesthetic system to guide the search process. This aesthetic system directs the search process toward preferences expressed interactively by the designer. An image input as design reference is integrated by means of shape comparison to provide direction to the exploratory search. During the shape generation process, the designer can choose solutions interactively in a graphical user interface. Those choices are then used to support the selection process as part of the fitness function by online classification. Enhancing human decision making capabilities in human-in-the-loop design systems addresses the complexity of architecture in respect to aesthetic requirements. On the strength of machine learning, the integral performance trade-off during multi-criteria optimisation was extended to address aesthetic preferences. The tacit knowledge and subjective understanding of designers can be used in the shape generation process based on interactive mechanisms. As a result, an integrated support system for performance-based design was developed and tested. Closing the loop from design to construction using design optimisation of structural nodes in a set of case studies confirmed the need for intuitive design systems, interfaces and mechanisms to make architectural optimisation more accessible and intuitive to handle. This dissertation investigated Typogenetic Design as a tool for initial morphological search. Novel instruments for human interaction with design systems were developed using mixed-method research. The present investigation consists of an in-depth technological enquiry into the use of interactive generative design for exploratory search as an integrated support system for performance-based design. Associated project-based research on the design potential of Typogenetic Design showcases the application of the design system for architecture. Generative design as an expressive tool to produce architectural geometries was investigated in regard to its ability to drive initial morphological search of complex geometries. The reinterpretation of processes and boosting of productivity by artificial intelligence was instrumental in exploring a holistic approach combining quantitative and qualitative criteria in a human-in-the-loop system. The shift in focus from an objective to a subjective understanding of computational design processes indicates a perspective change from optimisation to learning as a computational paradigm. Integrating learning capabilities in architectural optimisation enhances the capability of architects to explore large design spaces of emergent representations using evolutionary search. The shift from design automation to interactive generative design introduces the possibility for designers to evaluate shape solutions based on their knowledge and expertise to the computational system. At the same time, the aesthetic system is trained in adaptation to the choices made by the designer. Furthermore, an initial image input allows the designer to add a design reference to the Typogenetic Design process. Shape comparison using a similarity measure provides additional guidance to the architectural shape generation using grammar evolution. Finally, a software prototype was built and tested by means of user-experience evaluation. These participant experiments led to the specification of custom software requirements for the software implementation of a parametric Typogenetic tool. I explored semi-automated design in application to different design cases using the software prototype of Typogenetic Design. Interactive mass-customisation is a promising application of Typogenetic Design to interactively specify product structure and component composition. The semi-automated design paradigm is one step on the way to moderating the balance between automation and control of computational design systems

Advances in 3D reconstruction

La tesi affronta il problema della ricostruzione di scene tridimensionali a partire da insiemi non strutturati di fotografie delle stesse. Lo stato dell'arte viene avanzato su diversi fronti: il primo contributo consiste in una formulazione robusta del problema di struttura e moto basata su di un approccio gerarchico, contrariamente a quello sequenziale prevalente in letteratura. Questa metodologia abbatte di un ordine di grandezza il costo computazionale complessivo, risulta inerentemente parallelizzabile, minimizza il progressivo accumulo degli errori e elimina la cruciale dipendenza dalla scelta della coppia di viste iniziale comune a tutte le formulazioni concorrenti. Un secondo contributo consiste nello sviluppo di una nuova procedura di autocalibrazione, particolarmente robusta e adatta al contesto del problema di moto e struttura. La soluzione proposta consiste in una procedura in forma chiusa per il recupero del piano all'infinito data una stima dei parametri intrinseci di almeno due camere. Questo metodo viene utilizzato per la ricerca esaustiva dei parametri interni, il cui spazio di ricerca Š strutturalmente limitato dalla finitezza dei dispositivi di acquisizione. Si Š indagato infine come visualizzare in maniera efficiente e gradevole i risultati di ricostruzione ottenuti: a tale scopo sono stati sviluppati algoritmi per il calcolo della disparit… stereo e procedure per la visualizzazione delle ricostruzione come insiemi di piani tessiturati automaticamente estratti, ottenendo una rappresentazione fedele, compatta e semanticamente significativa. Ogni risultato Š stato corredato da una validazione sperimentale rigorosa, con verifiche sia qualitative che quantitative.The thesis tackles the problem of 3D reconstruction of scenes from unstructured picture datasets. State of the art is advanced on several aspects: the first contribute consists in a robust formulation of the structure and motion problem based on a hierarchical approach, as opposed to the sequential one prevalent in literature. This methodology reduces the total computational complexity by one order of magnitude, is inherently parallelizable, minimizes the error accumulation causing drift and eliminates the crucial dependency from the choice of the initial couple of views which is common to all competing approaches. A second contribute consists in the discovery of a novel slef-calibration procedure, very robust and tailored to the structure and motion task. The proposed solution is a closed-form procedure for the recovery of the plane at infinity given a rough estimate of focal parameters of at least two cameras. This method is employed for the exaustive search of internal parameters, whise space is inherently bounded from the finiteness of acquisition devices. Finally, we inevstigated how to visualize in a efficient and compelling way the obtained reconstruction results: to this effect several algorithms for the computation of stereo disparity are presented. Along with procedures for the automatic extraction of support planes, they have been employed to obtain a faithful, compact and semantically significant representation of the scene as a collection of textured planes, eventually augmented by depth information encoded in relief maps. Every result has been verified by a rigorous experimental validation, comprising both qualitative and quantitative comparisons

Semantics-aware content delivery framework for 3D Tele-immersion

3D Tele-immersion (3DTI) technology allows full-body, multimodal interaction among geographically dispersed users, which opens a variety of possibilities in cyber collaborative applications such as art performance, exergaming, and physical rehabilitation. However, with its great potential, the resource and quality demands of 3DTI rise inevitably, especially when some advanced applications target resource-limited computing environments with stringent scalability demands. Under these circumstances, the tradeoffs between 1) resource requirements, 2) content complexity, and 3) user satisfaction in delivery of 3DTI services are magnified. In this dissertation, we argue that these tradeoffs of 3DTI systems are actually avoidable when the underlying delivery framework of 3DTI takes the semantic information into consideration. We introduce the concept of semantic information into 3DTI, which encompasses information about the three factors: environment, activity, and user role in 3DTI applications. With semantic information, 3DTI systems are able to 1) identify the characteristics of its computing environment to allocate computing power and bandwidth to delivery of prioritized contents, 2) pinpoint and discard the dispensable content in activity capturing according to properties of target application, and 3) differentiate contents by their contributions on fulfilling the objectives and expectation of user’s role in the application so that the adaptation module can allocate resource budget accordingly. With these capabilities we can change the tradeoffs into synergy between resource requirements, content complexity, and user satisfaction. We implement semantics-aware 3DTI systems to verify the performance gain on the three phases in 3DTI systems’ delivery chain: capturing phase, dissemination phase, and receiving phase. By introducing semantics information to distinct 3DTI systems, the efficiency improvements brought by our semantics-aware content delivery framework are validated under different application requirements, different scalability bottlenecks, and different user and application models. To sum up, in this dissertation we aim to change the tradeoff between requirements, complexity, and satisfaction in 3DTI services by exploiting the semantic information about the computing environment, the activity, and the user role upon the underlying delivery systems of 3DTI. The devised mechanisms will enhance the efficiency of 3DTI systems targeting on serving different purposes and 3DTI applications with different computation and scalability requirements