Ferdosi, Bilkis Jamal

English

ARTS repository - University of Groningen

   University of GroningenScalable analysis and visualization of high-dimensional astronomical data setsFerdosi, Bilkis JamalIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2011Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Ferdosi, B. J. (2011). Scalable analysis and visualization of high-dimensional astronomical data sets. s.n.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 01-02-2024Chapter 6Concluding Remarks6.1 Summary and ConclusionCurrent information technology development coupled to modern astronomical instrumentation isleading to accelerated acquisition of large volumes of observational data, fast pipeline processingand enormous on-line archives containing high quality images and multi-dimensional parametervalues of astronomical objects. This is the case for facilities covering a wide spectral range. Afew recent examples are the Sloan Digital Sky Survey (SDSS), the 2 Micron All Sky Survey(2MASS) and the NRAO VLA Sky Survey (NVSS). Each of these provide catalogues of manymillions of objects, with often several tens of parameters characterizing the objects. In the nearfuture these data volumes will increase by an order of magnitude when a new generation ofinstruments comes on-line.In this thesis, we proposed visual and computational paradigms to analyze and extract infor-mation out of this flood of data. To obtain such techniques problems of a twofold nature neededto be overcome: one is the huge size of the datasets and the other is their large dimensionality.Density estimation approaches can be used for handling large size. There exist several tech-niques that can be used to obtain density profiles of the data. However, if we want to use theoutcome of such estimators in later stages of the process, they need to fulfill certain criteria, suchas computational efficiency, correctness, etc.In Chapter 2 we studied the performance of four density estimation techniques: k-nearestneighbors (kNN), adaptive Gaussian kernel density estimation (DEDICA), a special case ofadaptive Epanechnikov kernel density estimation (MBE), and the Delaunay tessellation fieldestimator (DTFE). The adaptive kernel based methods, especially MBE, perform better than theother methods in terms of calculating the density properly, and have stronger predictive powerin astronomical use cases. Moreover, the computation time of these methods is lower than othermethods and they compute the density on grids which can facilitate visualization (as an image in2D and a volume in 3D) and analysis of the data. Using the MBE method we can also achievescalability in terms of number of data points. After the original feature space has been trans-formed into image space, further computation can be done in image space that has constant size,although the size of the dataset can grow very large.104 6.1 Summary and ConclusionThe next step is to extract useful information from such spaces. Clustering is one of thetechniques that can help discovering structures in a dataset. However, full-dimensional clusteringis not so useful since structures may exist in different subspaces that may indicate differentrelations among particular subsets of dimensions. Subspace clustering is an approach that can beapplied for this purpose. Subspace clustering is the process of finding clusters in subspaces of thefull feature space, either directly (Agrawal et al. 1998) or by identifying relevant subspaces for(later) clustering based on some quality criteria (Baumgartner et al. 2004). In Chapter 3 (Ferdosiet al. 2010), we proposed an interactive approach to find relevant subspaces which is strongly tiedto morphological properties of object distributions. We used connected morphological operatorsimplemented using the Max-tree data structure to identify the clusters (high-intensity regionsin the density image). A “quality” of the subspaces was defined depending on their clusteringproperty. We recovered various known relations directly from the data with little or no a prioriassumptions. Therefore, our method can act as a starting point in analyzing large datasets andhelp users to find new relations as well.Using the method described in Chapter 3, we can obtain interesting subspaces of any dimen-sion. However, visualizing high-dimensional structures in a meaningful and user-interpretableway is far from straightforward. Traditionally, low-dimensional representations of high-dimen-sional spaces, obtained by methods such as Principal Component Analysis (PCA), Multi-Dimen-sional Scaling (MDS), etc., are used to perform visualization in a Cartesian coordinate system.However, they pose the problem of interpretation of the visualization, because of the transfor-mation of the original feature space to a new coordinate system. Two widely used methods tovisualize high-dimensional data without transformation are the scatter plot matrix (SPM) and theparallel coordinate plot (PCP). For effective visualization of high-dimensional structures, theyalso requires a proper ordering of the dimensions. In Chapter 4 (Ferdosi and Roerdink 2011),we proposed algorithms for reordering dimensions in PCP and SPM in such a way that high-dimensional structures (if present) become easier to perceive. We used the quality criterion andthe cluster indication capability of the method described in Chapter 3 to present three algorithms:two for finding suitable dimension ordering for PCP and one for SPM.Algorithms presented in this thesis can be very helpful in visualizing and extracting informa-tion from large high-dimensional datasets, not only from astronomy but also from other domainswhere the datasets are of similar nature. Experimental results with synthetic and real life astro-nomical and gene expression datasets proved their potential.In Chapter 5, we discussed several design issues of a visual analytic tool for astronomicaldata using a large touch sensitive display and present a prototype for such a tool. Large touch-sensitive displays provide more screen space, support more intuitive and natural interactions withtouch sensitive inputs, and can make sharing of the analysis process and collaboration with otherspossible. Thus, they can facilitate analysis of astronomical data which are not only large in sizeand dimension but also complex in nature.Concluding Remarks 1056.2 Future Outlook6.2.1 Visual ClusteringThe algorithms proposed in Chapter 3 and Chapter 4 retrieve clustering information withoutdoing the clustering. However, it is possible to perform the actual clustering using the samemethod. In the proposed methods we only keep track of the pixel intensities of the connectedcomponents. It is also possible to identify the positions of the connected components with highintensity using the Max-tree. The pixel positions of the dense regions can be used to providevisual clues about the position of the clusters. Depending on the size of the connected componentit may also possible to draw a circle centering at the pixel position of each cluster center. Furtherrefinement of cluster selection can be done by user interaction. This type of visual clusteringcan be very helpful for datasets such as the Galactic stellar halo dataset used in Chapter 3 wheretraditional clustering methods such as k-means clustering produce unsatisfactory results.6.2.2 Subspace Ranking for Outlier DetectionOutlier detection is one of the tasks that attracts a lot of attention from the astronomers. In agroup of galaxies an outlier may indicate findings of a quasar1. In the literature, most of thetime outliers are identified as a by-product of a clustering task. It is also possible to extend oursubspace ranking for clustering to subspace ranking for outlier detection.6.2.3 Analyzing Astronomical Datasets in Co-located Collaborative Envi-ronmentFigure 6.1. Curved 396” wide touch-screenwith 4900 x 1700 pixel resolution and 100+ si-multaneous touches in the Reality Theatre ofthe University of Groningen.Collaboration, both intra- and inter-disciplinary,plays an important role in scientific research.Co-located collaboration has been practicedfor a long time in scientific communities. Withthe advent of new technologies in communica-tion, remote collaboration became quite com-mon as well. However, increasing size andcomplexity of the datasets and new develop-ments in display technologies, such as wallor tabletop displays, geared up the researchon co-located collaboration using visual aids.To facilitate such collaboration there arises theneed for collaborative visualization of data.In Chapter 5 we discussed different designissues of a visual analytic tool in a two-touchwall display. It has the potential for being transferred into a co-located collaborative environment.1Active galactic nucleus with very high redshift.106 6.2 Future OutlookHowever, in that case we will need a touch-display with more than two-touch sensitivity. TheDonald Smits Center for Information Technology of the University of Groningen owns a curved396” wide touch-screen with a resolution of (4900 ×1700) pixels and 100+ simultaneous touch(see Figure 6.1). For designing a tool for co-located collaboration this screen may offer the bestplatform.There are several studies in the literature trying to understand how co-located collaborationworks in table-top displays. However, there is a lack of studies how such collaboration may workin large wall displays. Before designing tools in such environments it will be necessary to studyhow co-located collaboration may take place in such displays.

Scalable analysis and visualization of high-dimensional astronomical data sets

https://pure.rug.nl/ws/files/14502732/06c6.pdf

Scalable analysis and visualization of high-dimensional astronomical data sets

Abstract

Similar works

Full text

Available Versions

ARTS repository - University of Groningen