AVA: Towards Autonomous Visualization Agents through Visual Perception-Driven Decision-Making

With recent advances in multi-modal foundation models, the previously text-only large language models (LLM) have evolved to incorporate visual input, opening up unprecedented opportunities for various applications in visualization. Our work explores the utilization of the visual perception ability of multi-modal LLMs to develop Autonomous Visualization Agents (AVAs) that can interpret and accomplish user-defined visualization objectives through natural language. We propose the first framework for the design of AVAs and present several usage scenarios intended to demonstrate the general applicability of the proposed paradigm. The addition of visual perception allows AVAs to act as the virtual visualization assistant for domain experts who may lack the knowledge or expertise in fine-tuning visualization outputs. Our preliminary exploration and proof-of-concept agents suggest that this approach can be widely applicable whenever the choices of appropriate visualization parameters require the interpretation of previous visual output. Feedback from unstructured interviews with experts in AI research, medical visualization, and radiology has been incorporated, highlighting the practicality and potential of AVAs. Our study indicates that AVAs represent a general paradigm for designing intelligent visualization systems that can achieve high-level visualization goals, which pave the way for developing expert-level visualization agents in the future

ScaleTrotter: Illustrative Visual Travels Across Negative Scales

We present ScaleTrotter, a conceptual framework for an interactive, multi-scale visualization of biological mesoscale data and, specifically, genome data. ScaleTrotter allows viewers to smoothly transition from the nucleus of a cell to the atomistic composition of the DNA, while bridging several orders of magnitude in scale. The challenges in creating an interactive visualization of genome data are fundamentally different in several ways from those in other domains like astronomy that require a multi-scale representation as well. First, genome data has intertwined scale levels---the DNA is an extremely long, connected molecule that manifests itself at all scale levels. Second, elements of the DNA do not disappear as one zooms out---instead the scale levels at which they are observed group these elements differently. Third, we have detailed information and thus geometry for the entire dataset and for all scale levels, posing a challenge for interactive visual exploration. Finally, the conceptual scale levels for genome data are close in scale space, requiring us to find ways to visually embed a smaller scale into a coarser one. We address these challenges by creating a new multi-scale visualization concept. We use a scale-dependent camera model that controls the visual embedding of the scales into their respective parents, the rendering of a subset of the scale hierarchy, and the location, size, and scope of the view. In traversing the scales, ScaleTrotter is roaming between 2D and 3D visual representations that are depicted in integrated visuals. We discuss, specifically, how this form of multi-scale visualization follows from the specific characteristics of the genome data and describe its implementation. Finally, we discuss the implications of our work to the general illustrative depiction of multi-scale data

Vivern a virtual environment for multiscale visualization and modeling of DNA nanostructures

DNA nanostructures offer promising applications, particularly in the biomedical domain, as they can be used for targeted drug delivery, construction of nanorobots, or as a basis for molecular motors. One of the most prominent techniques for assembling these structures is DNA origami. Nowadays, desktop applications are used for the in silico design of such structures. However, as such structures are often spatially complex, their assembly and analysis are complicated. Since virtual reality (VR) was proven to be advantageous for such spatial-related tasks and there are no existing VR solutions focused on this domain, we propose Vivern, a VR application that allows domain experts to design and visually examine DNA origami nanostructures. Our approach presents different abstracted visual representations of the nanostructures, various color schemes, and an ability to place several DNA nanostructures and proteins in one environment, thus allowing for the detailed analysis of complex assemblies. We also present two novel examination tools, the Magic Scale Lens and the DNA Untwister, that allow the experts to visually embed different representations into local regions to preserve the context and support detailed investigation. To showcase the capabilities of our solution, prototypes of novel nanodevices conceptualized by our collaborating experts, such as DNA-protein hybrid structures and DNA origami superstructures, are presented. Finally, the results of two rounds of evaluations are summarized. They demonstrate the advantages of our solution, especially for scenarios where current desktop tools are very limited, while also presenting possible future research directions.Fil: Kutak, David. Masaryk University; República ChecaFil: Selzer, Matias Nicolas. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca. Instituto de Ciencias e Ingeniería de la Computación; Argentina. Universidad Nacional del Sur. Departamento de Ciencias e Ingenieria de la Computacion. Laboratorio de Investigación y Desarrollo en Visualización yComputación Gráfica; ArgentinaFil: Byska, Jan. Masaryk University; República ChecaFil: Ganuza, María Luján. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca. Instituto de Ciencias e Ingeniería de la Computación; Argentina. Universidad Nacional del Sur. Departamento de Ciencias e Ingenieria de la Computacion. Laboratorio de Investigación y Desarrollo en Visualización yComputación Gráfica; ArgentinaFil: Barisic, Ivan. Austrian Institute of Technology; AustriaFil: Kozlikova, Barbora. Masaryk University; República ChecaFil: Miao, Haichao. Austrian Institute of Technology; Austri

A Virtual Environment for Collaborative Inspection in Additive Manufacturing

Additive manufacturing (AM) techniques have been used to enhance the design and fabrication of complex components for various applications in the medical, aerospace, energy, and consumer products industries. A defining feature for many AM parts is the complex internal geometry enabled by the printing process. However, inspecting these internal structures requires volumetric imaging, i.e., X-ray CT, leading to the well-known challenge of visualizing complex 3D geometries using 2D desktop interfaces. Furthermore, existing tools are limited to single-user systems making it difficult to jointly discuss or share findings with a larger team, i.e., the designers, manufacturing experts, and evaluation team. In this work, we present a collaborative virtual reality (VR) for the exploration and inspection of AM parts. Geographically separated experts can virtually inspect and jointly discuss data. It also supports VR and non-VR users, who can be spectators in the VR environment. Various features for data exploration and inspection are developed and enhanced via real-time synchronization. We followed usability and interface verification guidelines using Nielsen's heuristics approach. Furthermore, we conducted exploratory and semi-structured interviews with domain experts to collect qualitative feedback. Results reveal potential benefits, applicability, and current limitations. The proposed collaborative VR environment provides a new basis and opens new research directions for virtual inspection and team collaboration in AM settings.Comment: Conditionally Accepted - CHI LBW 202

Second-Generation Sequencing Supply an Effective Way to Screen RNAi Targets in Large Scale for Potential Application in Pest Insect Control

The key of RNAi approach success for potential insect pest control is mainly dependent on careful target selection and a convenient delivery system. We adopted second-generation sequencing technology to screen RNAi targets. Illumina's RNA-seq and digital gene expression tag profile (DGE-tag) technologies were used to screen optimal RNAi targets from Ostrinia furnalalis. Total 14690 stage specific genes were obtained which can be considered as potential targets, and 47 were confirmed by qRT-PCR. Ten larval stage specific expression genes were selected for RNAi test. When 50 ng/µl dsRNAs of the genes DS10 and DS28 were directly sprayed on the newly hatched larvae which placed on the filter paper, the larval mortalities were around 40∼50%, while the dsRNAs of ten genes were sprayed on the larvae along with artificial diet, the mortalities reached 73% to 100% at 5 d after treatment. The qRT-PCR analysis verified the correlation between larval mortality and the down-regulation of the target gene expression. Topically applied fluorescent dsRNA confirmed that dsRNA did penetrate the body wall and circulate in the body cavity. It seems likely that the combination of DGE-tag with RNA-seq is a rapid, high-throughput, cost less and an easy way to select the candidate target genes for RNAi. More importantly, it demonstrated that dsRNAs are able to penetrate the integument and cause larval developmental stunt and/or death in a lepidopteron insect. This finding largely broadens the target selection for RNAi from just gut-specific genes to the targets in whole insects and may lead to new strategies for designing RNAi-based technology against insect damage

Geometric Abstraction for Effective Visualization and Modeling

Zusammenfassung in deutscher SpracheIn this cumulative thesis, I describe geometric abstraction as a strategy to create an integrated visualization system for spatial scientific data. The proposed approach creates a multitude of representations of spatial data in two dominant ways. Along the spatiality axis, it gradually removes spatial details and along the visual detail axis, the features are increasingly aggregated and represented by different visual objects. These representations are then integrated into a conceptual abstraction space that enables users to efficiently change the representation to adjust the abstraction level to a task in mind. To enable the expert to perceive correspondence between these representations, controllable animated transitions are provided. Finally, the abstraction space can record user interactions and provides visual indications to guide the expert towards interesting representations for a particular task and data set. Mental models of the experts play a crucial role in the understanding of the abstract representations and are considered in the design of the visualization system to keep the cognitive load low on the users side. This approach is demonstrated in two distinct fields of placenta research and in silico design of DNA nanostructures. For both fields geometric abstraction facilitates effective visual inspection and modeling. The Adenita toolkit, a software for the design of novel DNA nanostructures, implements the proposed visualization concepts. This toolkit, together with the proposed visualization concepts, is currently deployed to several research groups to help them in nanotechnology research.6

Visuelle Analyse der zerebralen Arterien

Abweichender Titel laut Übersetzung der Verfasserin/des VerfassersZsfassung in dt. SpracheDie vorliegende Arbeit präsentiert eine neuartige Methode zur visuellen Quantifizierung zerebraler Arterien. Der Circle of Willis (CoW) - lat. Circulus arteriosus Willisi - ist eine arterielle Struktur, welche verantwortlich für die Blutversorgung im Gehirn ist. Dysfunktionen dieses arteriellen Kreises können zum Schlaganfall führen. Die Diagnose eines Schlaganfalls ist ein komplexer Vorgang und abhängig vom Expertenwissen des Radiologen sowie von den verwendeten Software-Instrumenten. Diese Instrumente bestehen aus einfachen Darstellungsmethoden volumetrischer Daten, ohne die Unterstützung von State-of-the-art Technologien aus der medizinischen Bildverarbeitung und -visualisierung heranzuziehen. Das Ziel der vorliegenden Diplomarbeit ist die Erarbeitung einer automatisierten Methode für die standardisierte Visualisierung zerebraler Arterien bei Schlaganfall-Patienten. Damit einhergehend sollen visuelle Indikatoren problematischer Bereiche eingeführt, sowie unkomplizierte Vergleiche zwischen verschiedenen Patienten ermöglicht werden. Im Vorfeld der Visualisierung bietet die vorliegende Arbeit einen Lösungsvorschlag für die Extraktion des CoW aus Time-of-Flight Magnetresonanz-Angiographie-Bildern (TOF-MRA). Hierfür wird eine Enumerationsmethode zur Benennung der arteriellen Segmente vorgeschlagen. Des Weiteren wird eine Methode erarbeitet, welche die Detektion der Versorgungsarterien des CoW durch Analyse der koronalen, sagittalen und transversen Bildebenen übernimmt. Die vorliegende Arbeit bietet eine umfassende Darstellung des gesamten Vorgangs zur Extraktion der Arterien des CoW und zur standardisierten Visualisierung derselben. Das zentrale Ziel dieser Diplomarbeit ist es, eine effektive Darstellung der Arterien basierend auf einer radialen Baum-Struktur zu erarbeiten. Die Genauigkeit der visuellen Quantifizierungsmethode wird in einer Studie mit 63 TOF-MRA Bildern erprobt. Die Erkenntnisse der Auswertung der Untersuchungsobjekte mit der vorgestellten Methode werden mit den Befunden von Radiologen verglichen. Die daraus resultierenden Ergebnisse demonstrieren die Effektivität der vorgestellten Techniken bei der Detektion der Arterien im CoW. Schlussendlich wurde der Fokus auf die Identifikation der Hauptarterien gelegt.This thesis presents a novel method for the visual quantification of cerebral arteries. The Circle of Willis (CoW) is an arterial structure that is responsible for the brain's blood supply. Dysfunctions of this arterial circle can lead to strokes. The diagnosis of stroke patients is complex and relies on the radiologist's expertise and the software tools used. These tools consist of very basic display methods of the volumetric data without support of state-of-the-art technologies in medical image processing and visualization. The goal of this thesis is to create an automated method for the standardized visualization of cerebral arteries in stroke patients in order to allow visual indications of problematic areas as well as straightforward inter-patient comparisons. Prior to the visualization, this work offers a solution for the extraction of the CoW from Time-of-Flight Magnetic Resonance Angiography (TOF-MRA) images. An enumeration technique for the labeling of the segments is therefore suggested. Furthermore, it proposes a method for the detection of the CoW's main supplying arteries by analyzing the coronal, sagittal and transverse image planes of the volume. This work gives a comprehensive account of the entire pipeline that is required to extract the arteries in the CoW and to build a model for the standardized visualization. The final goal of this thesis is to create an effective display of the arteries based on a radial tree layout. The feasibility of the visual quantification method is tested in a study of 63 TOF-MRAs. With the proposed methodology applied to the subjects, the results were compared to the findings from radiologists. The obtained results demonstrate that the proposed techniques are effective in detecting the arteries of the CoW. Finally, we focused our methods on the identification of the main arteries.10

Sharp Changes of Crustal Seismic Anisotropy Across the Central Tanlu Fault Zone in East China

Abstract Both seismic and geodetic data suggested that the ∼120‐km long Weifang segment of the Tanlu fault zone, a large‐scale active strike‐slip system at east China, is a seismic gap with no obvious along‐strike shear motion at surface. Measuring crustal deformation around the segment is crucial to constrain stress/strain buildup and potential seismic risk at the fault. We measured crustal and upper mantle seismic anisotropy using P‐to‐S converted waves at the Moho (Pms) and core‐mantle boundary (SKS) recorded by broadband arrays across the Weifang fault segment. The measured crustal anisotropy inside the fault zone shows a fast direction of ∼NNE, parallel to the fault orientation. Right east to the fault zone, the fast axis rotates by almost 90° to ESE. The crustal anisotropy within the fault zone could be caused by aligned microcracks and foliated minerals due to long‐lasting shear motion inside the fault zone