Atkinson, Malcolm

Filgueira Vicente, Rosa

Gemuend, Andre

Spinuso, Alessandro

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Visualisation methods for large provenance collections in data-intensive collaborative platformsCitation for published version:Spinuso, A, Filgueira Vicente, R, Atkinson, M & Gemuend, A 2016, Visualisation methods for largeprovenance collections in data-intensive collaborative platforms. in EGU General Assembly 2016 (EGU2016). Geophysical Research Abstracts. vol. 18, EGU General Assembly 2016, Vienna, Austria, 17/04/16.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Publisher's PDF, also known as Version of recordPublished In:EGU General Assembly 2016 (EGU 2016). Geophysical Research AbstractsGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019Geophysical Research AbstractsVol. 18, EGU2016-14793-1, 2016EGU General Assembly 2016© Author(s) 2016. CC Attribution 3.0 License.Visualisation methods for large provenance collections in data-intensivecollaborative platformsAlessandro Spinuso (1), Rosa Fligueira (2), Malcolm Atkinson (2), and Andre Gemuend (3)(1) KNMI, R&D Observation and Data Technology, Utrecht, The Netherlands, (2) University of Edinburgh, School ofInformatics, Edinburgh, UK, (3) Fraunhofer SCAI, Sankt Augustin, GermanyThis work investigates improving the methods of visually representing provenance information in the context ofmodern data-driven scientific research. It explores scenarios where data-intensive workflows systems are servingcommunities of researchers within collaborative environments, supporting the sharing of data and methods, andoffering a variety of computation facilities, including HPC, HTC and Cloud.It focuses on the exploration of big-data visualization techniques aiming at producing comprehensive andinteractive views on top of large and heterogeneous provenance data. The same approach is applicable to control-flow and data-flow workflows or to combinations of the two. This flexibility is achieved using the W3C-PROVrecommendation as a reference model, especially its workflow oriented profiles such as D-PROV (Messier etal. 2013). Our implementation is based on the provenance records produced by the dispel4py data-intensiveprocessing library (Filgueira et al. 2015).dispel4py is an open-source Python framework for describing abstract stream-based workflows for distributeddata-intensive applications, developed during the VERCE project. dispel4py enables scientists to develop theirscientific methods and applications on their laptop and then run them at scale on a wide range of e-Infrastructures(Cloud, Cluster, etc.) without making changes. Users can therefore focus on designing their workflows at anabstract level, describing actions, input and output streams, and how they are connected. The dispel4py systemthen maps these descriptions to the enactment platforms, such as MPI, Storm, multiprocessing. It provides amechanism which allows users to determine the provenance information to be collected and to analyze it at runtime.For this work we consider alternative visualisation methods for provenance data, from infinite lists and lo-calised interactive graphs, to radial-views. The latter technique has been positively explored in many fields, fromtext data visualisation to genomics and social networking analysis. Its adoption for provenance has been presentedin literature (Borkin et al. 2013) in the context of parent-child relationships across processes, constructed fromcontrol-flow information. Computer graphics research has focused on the advantage of this radial distribution ofinterlinked information and on ways to improve the visual efficiency and tunability of such representations, likethe Hierarchical Edge Bundles visualisation method, (Holten et al. 2006), which aims at reducing visual clutter ofhighly connected structures via the generation of bundles.Our approach explores the potential of the combination of these methods. It serves environments where thesize of the provenance collection, coupled with the diversity of the infrastructures and the domain metadata,make the extrapolation of usage trends extremely challenging. Applications of such visualisation systems canengage groups of scientists, data providers and computational engineers, by serving visual snapshots that highlightrelationships between an item and its connected processes.We will present examples of comprehensive views on the distribution of processing and data transfers dur-ing a workflow’s execution in HPC, as well as cross workflows interactions and internal dynamics. The latter inthe context of faceted searches on domain metadata values-range. These are obtained from the analysis of realprovenance data generated by the processing of seismic traces performed through the VERCE platform.

Visualisation methods for large provenance collections in data-intensive collaborative platforms

Filgueira, Rosa

University of St. Andrews - Pure

Heriot Watt Pure

https://www.pure.ed.ac.uk/ws/files/25520607/EGU2016_14793_1_2.pdf

Visualisation methods for large provenance collections in data-intensive collaborative platforms

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer

University of St. Andrews - Pure

Heriot Watt Pure

University of St. Andrews - Pure

Heriot Watt Pure