VOICE: Visual Oracle for Interaction, Conversation, and Explanation

We present VOICE, a novel approach for connecting large language models' (LLM) conversational capabilities with interactive exploratory visualization. VOICE introduces several innovative technical contributions that drive our conversational visualization framework. Our foundation is a pack-of-bots that can perform specific tasks, such as assigning tasks, extracting instructions, and generating coherent content. We employ fine-tuning and prompt engineering techniques to tailor bots' performance to their specific roles and accurately respond to user queries, and a new prompt-based iterative scene-tree generation establishes a coupling with a structural model. Our text-to-visualization method generates a flythrough sequence matching the content explanation. Finally, 3D natural language interaction provides capabilities to navigate and manipulate the 3D models in real-time. The VOICE framework can receive arbitrary voice commands from the user and responds verbally, tightly coupled with corresponding visual representation with low latency and high accuracy. We demonstrate the effectiveness and high generalizability potential of our approach by applying it to two distinct domains: analyzing three 3D molecular models with multi-scale and multi-instance attributes, and showcasing its effectiveness on a cartographic map visualization. A free copy of this paper and all supplemental materials are available at https://osf.io/g7fbr/

Visualisation of curved tubular structures in medical databases: An application to virtual colonoscopy

Medical conditions affecting the colon are problematic to diagnose due to the difficulty in examining this particular internal organ. To date, the most widely used approach is to perform a colonoscopy; a procedure in which a small camera is inserted into the colon to examine its surface. This procedure is unpleasant and potentially dangerous for the patient, and is expensive and time consuming for the hospital. As a result, patients at risk of developing the conditions are not always screened as often as would be desirable. Over the last few years a new approach known as virtual colonoscopy has been gaining popularity. The method uses information from a CT scan to reconstruct a 3D model of the colon which can then be examined without the patient needing to undergo a colonoscopy. This approach is now commonly used when screening for polyps (an indication of colon cancer) but can not be so easily used on conditions such as Inflammatory Bowel Disease (IBD) where information beyond the shape of the surface is required. This thesis forms part of a larger project which aims to diagnose conditions such as IBD by using image processing algorithms on CT data and presenting the results to the user in an easy to interpret way. Specifically we are concerned with this visualisation stage of the system and so have developed a new visualisation approach which we call Volumetric CPR. This can be used to supplement the more traditional virtual flythrough visualisation and is applicable to IBD detection as well as screening for polyps. Our technique builds on the concept of Curved Planar Reformation (CPR), which has proved to be a practical and widely used tool for the visualisation of curved tubular structures within the human body. It has been useful in medical procedures involving the examination of blood vessels and the spine. However, it is more difficult to use it for structures such as the colon because abnormalities are smaller relative to the size of the structure and may not have such distinct density and shape characteristics. Our new approach improves on this situation by using volume rendering for hollow regions of the structure and standard CPR, for the surrounding tissue. This effectively combines grey scale contextual information with detailed colour information from the area of interest. The approach is successfully used with each of the standard CPR types and the resulting images are promising as an alternative for virtual colonoscopy. We also demonstrate how systems can effectively utilize this new visualisation in order to convey maximum information to the user. We show how overlays can be used to present surface coverage data and how sophisticated lighting models can improve the users understanding of the 3D structure. We also present details of how to integrate our visualisation into existing systems and work flows

The matrix revisited: A critical assessment of virtual reality technologies for modeling, simulation, and training

A convergence of affordable hardware, current events, and decades of research have advanced virtual reality (VR) from the research lab into the commercial marketplace. Since its inception in the 1960s, and over the next three decades, the technology was portrayed as a rarely used, high-end novelty for special applications. Despite the high cost, applications have expanded into defense, education, manufacturing, and medicine. The promise of VR for entertainment arose in the early 1990\u27s and by 2016 several consumer VR platforms were released. With VR now accessible in the home and the isolationist lifestyle adopted due to the COVID-19 global pandemic, VR is now viewed as a potential tool to enhance remote education. Drawing upon over 17 years of experience across numerous VR applications, this dissertation examines the optimal use of VR technologies in the areas of visualization, simulation, training, education, art, and entertainment. It will be demonstrated that VR is well suited for education and training applications, with modest advantages in simulation. Using this context, the case is made that VR can play a pivotal role in the future of education and training in a globally connected world

Enceladus and the Icy Moons of Saturn : July 26-29, 2016, Boulder, Colorado

Topics include (but are not limited to) geology, geophysics, geochemistry and mineralogy, active jets and their properties and origins, formation and evolution, astrobiology, and future investigations of these bodies. Comparative planetology is encouraged. A major focus will be the new results from the 2015 Cassini encounters with Enceladus and the other icy moons. Contributions from Cassini, Voyager, and groundbased studies are welcomeUniversities Space Research Association (USRA), Lunar and Planetary Institute (LPI)Local Organizing Committee, Larry Esposito, University of Colorado Carly Howett, Southwest Research Institute Laura Bloom, University of Colorado ; Science Organizing Committee Paul Schenk, Lunar and Planetary Institute [and 12 others

Toronto Augmented Reality Map: Enhancing citizen engagement with open government data using contemporary media platforms

This thesis investigates how visualization strategies and media platforms affect citizen engagement with urban public data. There is currently an international movement towards government transparency and accessible information as developed nations become more urbanized and information technology more ubiquitous. Concurrently, new media platforms (e.g., virtual and augmented reality) are evolving rapidly and show promise of mass adoption. These factors together offer design researchers a unique opportunity to develop new forms of citizen-facing media. I therefore developed an interactive augmented reality application that works with a printed map of the city of Toronto to overlay open government data as visualized digital content. An iterative practice-based research approach was used. Usability tests demonstrated that a strength of augmented reality is its facilitation of multi-user engagement. This thesis concludes by discussing how the Toronto augmented reality map can be made into an interactive citizen-facing installation in the public sphere

Searching for Optimal Runtime Assurance via Reachability and Reinforcement Learning

A runtime assurance system (RTA) for a given plant enables the exercise of an untrusted or experimental controller while assuring safety with a backup (or safety) controller. The relevant computational design problem is to create a logic that assures safety by switching to the safety controller as needed, while maximizing some performance criteria, such as the utilization of the untrusted controller. Existing RTA design strategies are well-known to be overly conservative and, in principle, can lead to safety violations. In this paper, we formulate the optimal RTA design problem and present a new approach for solving it. Our approach relies on reward shaping and reinforcement learning. It can guarantee safety and leverage machine learning technologies for scalability. We have implemented this algorithm and present experimental results comparing our approach with state-of-the-art reachability and simulation-based RTA approaches in a number of scenarios using aircraft models in 3D space with complex safety requirements. Our approach can guarantee safety while increasing utilization of the experimental controller over existing approaches

Modelling urban spatial change: a review of international and South African modelling initiatives

August 2013Urban growth and land use change models have the potential to become important tools for urban spatial planning and management. Before embarking on any modelling, however, GCRO felt it was important to take note of, and critically assess lessons to be learnt from international experience and scholarship on spatial modelling, as well as a number of South African experiments that model future urban development. In 2012, GCRO initiated preliminary research into current international and South African modelling trends through a desktop study and telephone, email and personal interviews. This Occasional paper sets out to investigate what urban spatial change modelling research is currently being undertaken internationally and within South Africa. At the international level, urban modelling research since 2000 is reviewed according to five main categories: land use transportation (LUT), cellular automata, urban system dynamics, agent-based models (ABMs) and spatial economics/econometric models (SE/EMs). Within South Africa, urban modelling initiatives are categorised differently and include a broader range of urban modelling techniques. Typologies used include: provincial government modelling initiatives in Gauteng; municipal government modelling initiatives; other government-funded modelling research; and academic modelling research. The various modelling initiatives described are by no means a comprehensive review of all urban spatial change modelling projects in South Africa, but provide a broad indication of the types of urban spatial change modelling underway. Importantly, the models may form the basis for more accurate and sophisticated urban modelling projects in the future. The paper concludes by identifying key urban modelling opportunities and challenges for short- to long-term planning in the GCR and South Africa.Written by Chris Wray, Josephine Musango and Kavesha Damon (GCRO) Koech Cheruiyot (NRF:SARChI chair in Development Planning and Modelling at Wits

Reconciling the dissonance between Historic Preservation and Virtual Reality through a Place-based Virtual Heritage system.

This study explores a problematic disconnect associated with virtual heritage and the immersive 3D computer modeling of cultural heritage. The products of virtual heritage often fail to adhere to long-standing principles and recent international conventions associated with historic preservation, heritage recording, designation, and interpretation. By drawing upon the geographic concepts of space, landscape, and place, along with advances in Geographic Information Systems, first-person serious games, and head-mounted Virtual Reality platforms this study envisions, designs, implements, and evaluates a virtual heritage system that seeks to reconcile the dissonance between Virtual Reality and historic preservation. Finally, the dissertation examines the contributions and future directions of such a Place-based Virtual Heritage system in human geography and historic preservation planning and interpretation