Accelerating Network Traffic Analytics Using Query-DrivenVisualization

Realizing operational analytics solutions where large and complex data must be analyzed in a time-critical fashion entails integrating many different types of technology. This paper focuses on an interdisciplinary combination of scientific data management and visualization/analysis technologies targeted at reducing the time required for data filtering, querying, hypothesis testing and knowledge discovery in the domain of network connection data analysis. We show that use of compressed bitmap indexing can quickly answer queries in an interactive visual data analysis application, and compare its performance with two alternatives for serial and parallel filtering/querying on 2.5 billion records worth of network connection data collected over a period of 42 weeks. Our approach to visual network connection data exploration centers on two primary factors: interactive ad-hoc and multiresolution query formulation and execution over n dimensions and visual display of then-dimensional histogram results. This combination is applied in a case study to detect a distributed network scan and to then identify the set of remote hosts participating in the attack. Our approach is sufficiently general to be applied to a diverse set of data understanding problems as well as used in conjunction with a diverse set of analysis and visualization tools

NERSC 'Visualization Greenbook' Future visualization needs of the DOE computational science community hosted at NERSC

This report presents the findings and recommendations that emerged from a one-day workshop held at Lawrence Berkeley National Laboratory (LBNL) on June 5, 2002, in conjunction with the NERSC User Group (NUG) Meeting. The motivation for this workshop was to solicit direct input from the application science community on the subject of visualization. The workshop speakers and participants included computational scientists from a cross-section of disciplines that use the NERSC facility, as well as visualization researchers from across the country. We asked the workshop contributors how they currently visualize their results, and how they would like to do visualization in the future. We were especially interested in each individual's view of how visualization tools and services could be improved in order to better meet the needs of future computational science projects. The outcome of this workshop is a set of findings and recommendations that are presented in more detail later in this report, and briefly summarized here

NIH-NSF Visualization Research Challenges Report

Engineering and Applied Science

Virtualising visualisation: A distributed service based approach to visualisation on the Grid

Context: Current visualisation systems are not designed to work with the large quantities of data produced by scientists today, they rely on the abilities of a single resource to perform all of the processing and visualisation of data which limits the problem size that they can investigate. Objectives: The objectives of this research are to address the issues encountered by scientists with current visualisation systems and the deficiencies highlighted in current visualisation systems. The research then addresses the question:” How do you design the ideal service oriented architecture for visualisation that meets the needs of scientists?” Method: A new design for a visualisation system based upon a Service Oriented Architecture is proposed to address the issues identified, the architecture is implemented using Java and web service technology. The implementation of the architecture also realised several case study scenarios as demonstrators. Evaluation: Evaluation was performed using case study scenarios of scientific problems and performance data was conducted through experimentation. The scenarios were assessed against the requirements for the architecture and the performance data against a base case simulating a single resource implementation. Conclusion: The virtualised visualisation architecture shows promise for applications where visualisation can be performed in a highly parallel manner and where the problem can be easily sub-divided into chunks for distributed processing