A Modular and Open-Source Framework for Virtual Reality Visualisation and Interaction in Bioimaging

Life science today involves computational analysis of a large amount and variety of data, such as volumetric data acquired by state-of-the-art microscopes, or mesh data from analysis of such data or simulations. The advent of new imaging technologies, such as lightsheet microscopy, has resulted in the users being confronted with an ever-growing amount of data, with even terabytes of imaging data created within a day. With the possibility of gentler and more high-performance imaging, the spatiotemporal complexity of the model systems or processes of interest is increasing as well. Visualisation is often the first step in making sense of this data, and a crucial part of building and debugging analysis pipelines. It is therefore important that visualisations can be quickly prototyped, as well as developed or embedded into full applications. In order to better judge spatiotemporal relationships, immersive hardware, such as Virtual or Augmented Reality (VR/AR) headsets and associated controllers are becoming invaluable tools. In this work we present scenery, a modular and extensible visualisation framework for the Java VM that can handle mesh and large volumetric data, containing multiple views, timepoints, and color channels. scenery is free and open-source software, works on all major platforms, and uses the Vulkan or OpenGL rendering APIs. We introduce scenery's main features, and discuss its use with VR/AR hardware and in distributed rendering. In addition to the visualisation framework, we present a series of case studies, where scenery can provide tangible benefit in developmental and systems biology: With Bionic Tracking, we demonstrate a new technique for tracking cells in 4D volumetric datasets via tracking eye gaze in a virtual reality headset, with the potential to speed up manual tracking tasks by an order of magnitude. We further introduce ideas to move towards virtual reality-based laser ablation and perform a user study in order to gain insight into performance, acceptance and issues when performing ablation tasks with virtual reality hardware in fast developing specimen. To tame the amount of data originating from state-of-the-art volumetric microscopes, we present ideas how to render the highly-efficient Adaptive Particle Representation, and finally, we present sciview, an ImageJ2/Fiji plugin making the features of scenery available to a wider audience.:Abstract Foreword and Acknowledgements Overview and Contributions Part 1 - Introduction 1 Fluorescence Microscopy 2 Introduction to Visual Processing 3 A Short Introduction to Cross Reality 4 Eye Tracking and Gaze-based Interaction Part 2 - VR and AR for System Biology 5 scenery — VR/AR for Systems Biology 6 Rendering 7 Input Handling and Integration of External Hardware 8 Distributed Rendering 9 Miscellaneous Subsystems 10 Future Development Directions Part III - Case Studies C A S E S T U D I E S 11 Bionic Tracking: Using Eye Tracking for Cell Tracking 12 Towards Interactive Virtual Reality Laser Ablation 13 Rendering the Adaptive Particle Representation 14 sciview — Integrating scenery into ImageJ2 & Fiji Part IV - Conclusion 15 Conclusions and Outlook Backmatter & Appendices A Questionnaire for VR Ablation User Study B Full Correlations in VR Ablation Questionnaire C Questionnaire for Bionic Tracking User Study List of Tables List of Figures Bibliography Selbstständigkeitserklärun

Web based public participation in visual impact assessment of urban landscape.

Zhang Zongyu.Thesis (M.Phil.)--Chinese University of Hong Kong, 2001.Includes bibliographical references (leaves 101-108).Abstracts in English and Chinese.ABSTRACT IN ENGLISH --- p.i-iiABSTRACT IN CHINESE --- p.iiiACKNOWLEDGEMENTS --- p.iv-vTABLE OF CONTENTS --- p.vi-viiiLIST OF TABLES --- p.ixLIST OF FIGURES --- p.x-xiChapter CHAPTER ONE --- INTRODUCTIONChapter 1.1 --- Landscape and landscape Assessment --- p.1Chapter 1.1.1 --- The descriptive inventory approach --- p.2Chapter 1.1.2 --- Public preference models --- p.4Chapter 1.2 --- Urban Landscape --- p.5Chapter 1.3 --- Relationship between professional and public --- p.8Chapter 1.3.1 . --- Inherent conflicts --- p.9Chapter 1.3.2. --- Roles of both sides --- p.9Chapter 1.3.3 --- Collaboration between professionals and the public --- p.10Chapter CHAPTER TWO --- VISUAL IMPACT ASSESSMENTChapter 2.1 --- The needs for visual impact assessment --- p.13Chapter 2.2 --- The visual impact assessment process --- p.16Chapter 2.3 --- The information inventory in the visual impact assessment --- p.19Chapter 2.3.1 --- Landscape simulation --- p.20Chapter 2.3.2 --- Visual impacts identification --- p.22Chapter 2.4 --- Public participation --- p.23Chapter 2.4.1 --- Public preference in the urban landscape --- p.24Chapter 2.4.2 --- Public accessibility to the urban landscape planning process --- p.28Chapter CHAPTER THREE --- CAPTURING THE SYSTEM SPECIFICATIONSChapter 3.1 --- General considerations --- p.30Chapter 3.1.1 --- Function requirements --- p.30Chapter 3.1.2 --- Project management --- p.32Chapter 3.1.3 --- User interface --- p.33Chapter 3.1.4 --- Web access --- p.34Chapter 3.1.5 --- Qualification of public participation in urban planning --- p.35Chapter 3.2 --- Envisioning the proposed web based system --- p.37Chapter 3.2.1 --- Proposed virtual collaboration --- p.38Chapter 3.2.1.1 --- Improving participants' access to the web based visual impact assessment --- p.39Chapter 3.2.1.2 --- Capturing the public appreciation --- p.41Chapter 3.2.2 --- Collaboration between planners and public --- p.43Chapter CHAPTER FOUR --- SYSTEM DESIGNChapter 4.1 --- Main software or tools for developing the proposed web based system --- p.45Chapter 4.1.1 --- Arcview 3.1 or Arc/Info with 3D analyst and Internet mapping server extensions --- p.46Chapter 4:1.2 --- VRML 2.0 and Java --- p.49Chapter 4.1.3 --- Java3D API --- p.52Chapter 4.2 --- System configuration --- p.55Chapter 4.2.1. --- System architecture --- p.55Chapter 4.2.2. --- Data management --- p.57Chapter 4.2.2.1 --- Urban landscape information management --- p.57Chapter 4.2.2.2 --- Public participation --- p.64Chapter 4.2.3. --- User interface design --- p.69Chapter CHAPTER FIVE --- PROTOTYPE SYSTME AND PILOT STUDYChapter 5.1 --- General description --- p.74Chapter 5.2 --- Implementation --- p.75Chapter 5.2.1 --- Connecting the two-dimensional world with a three-dimensional virtual urban environment --- p.75Chapter 5.2.2 --- Data flow of the system for interactions between the GIS and the VRML browser --- p.77Chapter 5.3 --- Data preparation --- p.81Chapter 5.3.1 --- Constructing the terrain model --- p.81Chapter 5.3.2 --- Retrieving the landscape themes --- p.87Chapter 5.4 --- Public oriented user interface design --- p.88Chapter 5.5 --- Participation log --- p.96Chapter CHAPTER SIX --- CONCLUSIONAPPENDI

Natural landscape scenic preference: techniques for evaluation and simulation.

The aesthetic beauty of a landscape is a very subjective issue: every person has their own opinions and their own idea of what beauty is. However, all people have a common evolutionary history, and, according to the Biophilia hypothesis, a genetic predisposition to liking certain types of landscapes. It is possible that this common inheritance allows us to attempt to model scenic preference for natural landscapes. The ideal type of model for such predictions is the psychophysical preference model, integrating psychological responses to landscapes with objective measurements of quantitative and qualitative landscape variables. Such models commonly predict two thirds of the variance in the predications of the general public for natural landscapes. In order to create such a model three sets of data were required: landscape photographs (surrogates of the actual landscape), landscape preference data and landscape component variable measurements. The Internet was used to run a questionnaire survey; a novel, yet flexible, environmentally friendly and simple method of data gathering, resulting in one hundred and eighty responses. A geographic information system was used to digitise ninety landscape photographs and measure their landforms (based on elevation) in terms of areas and perimeters, their colours and proxies for their complexity and coherence. Landscape preference models were created by running multiple linear regressions using normalised preference data and the landscape component variables, including mathematical transformations of these variables. The eight models created predicted over sixty percent of variance in the responses and had moderate to high correlations with a second set of landscape preference data. A common base to the models were the variables of complexity, water and mountain landform, in particular the presence or absence of water and mountains was noted as being significant in determining landscape scenic preference. In order to fully establish the utility of these models, they were further tested against: changes in weather and season; the addition of cultural structures; different photographers; alternate film types; different focal lengths; and composition. Results showed that weather and season were not significant in determining landscape preference; cultural structures increased preferences for landscapes; and photographs taken by different people did not produce consistent results from the predictive models. It was also found that film type was not significant and that changes in focal length altered preferences for landscapes

Modeling and Simulation in Engineering

This book provides an open platform to establish and share knowledge developed by scholars, scientists, and engineers from all over the world, about various applications of the modeling and simulation in the design process of products, in various engineering fields. The book consists of 12 chapters arranged in two sections (3D Modeling and Virtual Prototyping), reflecting the multidimensionality of applications related to modeling and simulation. Some of the most recent modeling and simulation techniques, as well as some of the most accurate and sophisticated software in treating complex systems, are applied. All the original contributions in this book are jointed by the basic principle of a successful modeling and simulation process: as complex as necessary, and as simple as possible. The idea is to manipulate the simplifying assumptions in a way that reduces the complexity of the model (in order to make a real-time simulation), but without altering the precision of the results

Adaptivity of 3D web content in web-based virtual museums : a quality of service and quality of experience perspective

The 3D Web emerged as an agglomeration of technologies that brought the third dimension to the World Wide Web. Its forms spanned from being systems with limited 3D capabilities to complete and complex Web-Based Virtual Worlds. The advent of the 3D Web provided great opportunities to museums by giving them an innovative medium to disseminate collections' information and associated interpretations in the form of digital artefacts, and virtual reconstructions thus leading to a new revolutionary way in cultural heritage curation, preservation and dissemination thereby reaching a wider audience. This audience consumes 3D Web material on a myriad of devices (mobile devices, tablets and personal computers) and network regimes (WiFi, 4G, 3G, etc.). Choreographing and presenting 3D Web components across all these heterogeneous platforms and network regimes present a significant challenge yet to overcome. The challenge is to achieve a good user Quality of Experience (QoE) across all these platforms. This means that different levels of fidelity of media may be appropriate. Therefore, servers hosting those media types need to adapt to the capabilities of a wide range of networks and devices. To achieve this, the research contributes the design and implementation of Hannibal, an adaptive QoS & QoE-aware engine that allows Web-Based Virtual Museums to deliver the best possible user experience across those platforms. In order to ensure effective adaptivity of 3D content, this research furthers the understanding of the 3D web in terms of Quality of Service (QoS) through empirical investigations studying how 3D Web components perform and what are their bottlenecks and in terms of QoE studying the subjective perception of fidelity of 3D Digital Heritage artefacts. Results of these experiments lead to the design and implementation of Hannibal

Immersive analytics with abstract 3D visualizations: A survey

After a long period of scepticism, more and more publications describe basic research but also practical approaches to how abstract data can be presented in immersive environments for effective and efficient data understanding. Central aspects of this important research question in immersive analytics research are concerned with the use of 3D for visualization, the embedding in the immersive space, the combination with spatial data, suitable interaction paradigms and the evaluation of use cases. We provide a characterization that facilitates the comparison and categorization of published works and present a survey of publications that gives an overview of the state of the art, current trends, and gaps and challenges in current research

GIS-based landscape design research

Landscape design research is important for cultivating spatial intelligence in landscape architecture. This study explores GIS (geographic information systems) as a tool for landscape design research - investigating landscape designs to understand them as architectonic compositions (architectonic plan analysis). The concept ‘composition’ refers to a conceivable arrangement, an architectural expression of a mental construct that is legible and open to interpretation. Landscape architectonic compositions and their representations embody a great wealth of design knowledge as objects of our material culture and reflect the possible treatment of the ground, space, image and program as a characteristic coherence. By exploring landscape architectonic compositions with GIS, design researchers can acquire design knowledge that can be used in the creation and refinement of a design.  The research aims to identify and illustrate the potential role of GIS as a tool in landscape design research, so as to provide insight into the possibilities and limitations of using GIS in this capacity. The critical, information-oriented case of Stourhead landscape garden (Wiltshire, UK), an example of a designed landscape that covers the scope and remit of landscape architecture design, forms the heart of the study. The exploration of Stourhead by means of GIS can be understood as a plausibility probe. Here the case study is considered a form of ‘quasi-experiment’, testing the hypothesis and generating a learning process that constitutes a prerequisite for advanced understanding, while using an adjusted version of the framework for landscape design analysis by Steenbergen and Reh (2003). This is a theoretically informed analytical method based on the formal interpretation of the landscape architectonic composition addressing four landscape architectonic categories: the basic, the spatial, the symbolic and the programmatic form. This study includes new aspects to be analysed, such as the visible form and the shape of the walk, and serves as the basis for the landscape architectonic analysis in which GIS is used as the primary analytical tool.  GIS-based design research has the possibility to cultivate spatial intelligence in landscape architecture through three fields of operation: GIS-based modelling: description of existing and future landscape architectonic compositions in digital form; GIS-based analysis: exploration, analysis and synthesis of landscape architectonic compositions in order to reveal latent architectonic relationships and principles, while utilizing the processing capacities and possibilities of computers for ex-ante and ex-post simulation and evaluation; GIS-based visual representation: representation of (virtual) landscape architectonic compositions in space and time, in order to retrieve and communicate information and knowledge of the landscape design.  Though there are limitations, this study exemplifies that GIS is a powerful instrument to acquire knowledge from landscape architectonic compositions. The study points out that the application of GIS in landscape design research can be seen as an extension of the fundamental cycle of observation, visual representation, analysis and interpretation in the process of knowledge acquisition, with alternative visualisations and digital landscape models as important means for this process. Using the calculating power of computers, combined with inventive modelling, analysis and visualisation concepts in an interactive process, opened up possibilities to reveal new information and knowledge about the basic, spatial, symbolic and programmatic form of Stourhead. GIS extended the design researchers’ perception via measurement, simulation and experimentation, and at the same time offered alternative ways of understanding the landscape architectonic composition. This gave rise to the possibility of exploring new elements in the framework of landscape design research, such as the visible form and kinaesthetic aspects, analysing the composition from eyelevel perspective. Moreover, the case study showcases that GIS has the potential to measure phenomena that are often subject to intuitive and experimental design, combining general scientific knowledge of, for instance, visual perception and way-finding, with the examination of site-specific design applications. GIS also enabled one to understand the landscape architectonic composition of Stourhead as a product of time, via the analysis of its development through reconstruction and evaluation of several crucial time-slice snapshots. The study illustrates that GIS can be regarded an external cognitive tool that facilitates and mediates in design knowledge acquisition. GIS facilitates in the sense that it can address the ‘same types of design-knowledge’ regarding the basic, spatial, symbolic and programmatic form, but in a more precise, systematic, transparent, and quantified manner. GIS mediates in the sense that it influences what and how aspects of the composition can be understood and therefore enables design researchers to generate ‘new types of design-knowledge’ by advanced spatial analysis and the possibility of linking or integrating other information layers, fields of science and data sources. The research contributes to the development and distribution of knowledge of GIS-applications in landscape architecture in two ways: (1) by ‘following’ the discipline and developing aspects of it, and (2) by setting in motion fundamental developments in the field, providing alternative readings of landscape architecture designs

GIS-based landscape design research:

Landscape design research is important for cultivating spatial intelligence in landscape architecture. This study explores GIS (geographic information systems) as a tool for landscape design research - investigating landscape designs to understand them as architectonic compositions (architectonic plan analysis). The concept ‘composition’ refers to a conceivable arrangement, an architectural expression of a mental construct that is legible and open to interpretation. Landscape architectonic compositions and their representations embody a great wealth of design knowledge as objects of our material culture and reflect the possible treatment of the ground, space, image and program as a characteristic coherence. By exploring landscape architectonic compositions with GIS, design researchers can acquire design knowledge that can be used in the creation and refinement of a design.  The research aims to identify and illustrate the potential role of GIS as a tool in landscape design research, so as to provide insight into the possibilities and limitations of using GIS in this capacity. The critical, information-oriented case of Stourhead landscape garden (Wiltshire, UK), an example of a designed landscape that covers the scope and remit of landscape architecture design, forms the heart of the study. The exploration of Stourhead by means of GIS can be understood as a plausibility probe. Here the case study is considered a form of ‘quasi-experiment’, testing the hypothesis and generating a learning process that constitutes a prerequisite for advanced understanding, while using an adjusted version of the framework for landscape design analysis by Steenbergen and Reh (2003). This is a theoretically informed analytical method based on the formal interpretation of the landscape architectonic composition addressing four landscape architectonic categories: the basic, the spatial, the symbolic and the programmatic form. This study includes new aspects to be analysed, such as the visible form and the shape of the walk, and serves as the basis for the landscape architectonic analysis in which GIS is used as the primary analytical tool.  GIS-based design research has the possibility to cultivate spatial intelligence in landscape architecture through three fields of operation: GIS-based modelling: description of existing and future landscape architectonic compositions in digital form; GIS-based analysis: exploration, analysis and synthesis of landscape architectonic compositions in order to reveal latent architectonic relationships and principles, while utilizing the processing capacities and possibilities of computers for ex-ante and ex-post simulation and evaluation; GIS-based visual representation: representation of (virtual) landscape architectonic compositions in space and time, in order to retrieve and communicate information and knowledge of the landscape design.  Though there are limitations, this study exemplifies that GIS is a powerful instrument to acquire knowledge from landscape architectonic compositions. The study points out that the application of GIS in landscape design research can be seen as an extension of the fundamental cycle of observation, visual representation, analysis and interpretation in the process of knowledge acquisition, with alternative visualisations and digital landscape models as important means for this process. Using the calculating power of computers, combined with inventive modelling, analysis and visualisation concepts in an interactive process, opened up possibilities to reveal new information and knowledge about the basic, spatial, symbolic and programmatic form of Stourhead. GIS extended the design researchers’ perception via measurement, simulation and experimentation, and at the same time offered alternative ways of understanding the landscape architectonic composition. This gave rise to the possibility of exploring new elements in the framework of landscape design research, such as the visible form and kinaesthetic aspects, analysing the composition from eyelevel perspective. Moreover, the case study showcases that GIS has the potential to measure phenomena that are often subject to intuitive and experimental design, combining general scientific knowledge of, for instance, visual perception and way-finding, with the examination of site-specific design applications. GIS also enabled one to understand the landscape architectonic composition of Stourhead as a product of time, via the analysis of its development through reconstruction and evaluation of several crucial time-slice snapshots. The study illustrates that GIS can be regarded an external cognitive tool that facilitates and mediates in design knowledge acquisition. GIS facilitates in the sense that it can address the ‘same types of design-knowledge’ regarding the basic, spatial, symbolic and programmatic form, but in a more precise, systematic, transparent, and quantified manner. GIS mediates in the sense that it influences what and how aspects of the composition can be understood and therefore enables design researchers to generate ‘new types of design-knowledge’ by advanced spatial analysis and the possibility of linking or integrating other information layers, fields of science and data sources. The research contributes to the development and distribution of knowledge of GIS-applications in landscape architecture in two ways: (1) by ‘following’ the discipline and developing aspects of it, and (2) by setting in motion fundamental developments in the field, providing alternative readings of landscape architecture designs