Haupt, Carina

Kurnatowski, Lynn

Stoffers, Martin

Institute of Transport Research:Publications

Provenance based software dashboardsLynn von KurnatowskiInstitute for Software TechnologyGerman Aerospace Center (DLR)Weßling, Germanyhttps://orcid.org/0000-0001-5144-702XMartin StoffersInstitute for Software TechnologyGerman Aerospace Center (DLR)Cologne, Germanyhttps://orcid.org/0000-0003-2987-4345Carina HauptInstitute for Software TechnologyGerman Aerospace Center (DLR)Berlin, Germanyhttps://orcid.org/0000-0001-6447-1379I. CHALLENGEOver the past years, software conquered a growing numberof application areas. Particularly in research, where softwareis already an important innovation factor and integral part.This statement is substantiated by the results of a surveythat we conducted DLR1-wide in 2018 [1]. It shows thatmany researchers develop software in their daily work. It evenindicates that researchers, who are already involved in softwaredevelopment, are interested in investing even more time insoftware related activities.The increasing role of software in research, in combinationwith the increase of complexity in scientific fields, has led tothe necessity of complex software solutions. Thus intensifyingthe need to ensure the quality, reliability, and trustworthinessof software systems in research. Especially in safety-criticalareas, where small software errors can lead to serious failures.In order to be able to make an assurance about quality,reliability, and trustworthiness of a software product, it isnecessary to have a comprehensive and detailed understandingof a software project. The large amount of heterogeneousdata, which is generated before, during, and after the devel-opment in so-called software repositories can serve as datasources [2]. These repositories already contain informationabout the source code and the software development process.One approach to explore this data is to analyse their prove-nance.As stated by Moreau in 2010 in “The Foundations forProvenance on the Web”:“Provenance [. . .] is becoming an important concernfor several research communities in computer sci-ence, since it offers the means to verify data prod-ucts, to infer their quality, to analyze the processesthat led to them, and to decide whether they can betrusted.” [3, Page 3]Today, provenance [4] is used to verify data products andto analyze processes that led to them. Our work aims tostandardize, generate, and use provenance of software artifactsand provenance of software development processes by defininga general provenance data model for software developmentprocesses. Our goal is to be able to trace and determine theorigin of artifacts such as issues, source code files, build1German Aerospace Center (DLR)results, or documentation, and to understand and get insightsinto the process as a whole using analysis and dashboards.II. OPPORTUNITYSoftware development is a creative and complex processbased on human decisions. Consequently, developing softwareas error-free as possible is a challenging task. To assessthe reliability and trustworthiness of software systems, theprovenance of the development process and the producedsoftware artifacts can be recorded and analyzed.The aforementioned increase of complexity in softwaresolutions results in the need of more extensive developmentprocesses. Thus, resulting in very large and complex prove-nance graphs, which are difficult to manually understand,interpret, and verify.Visual analysis is an opportunity to easily and quickly verifya software project, since the human brain is an amazing patternmatching machine. Thus enabling it to process large amount ofvisual information at once. Our idea is to create an interactivedashboard solution based on a data set of provenance describ-ing software development processes. This allows developersto easily and quickly get an overview about a software statusregarding it’s quality, reliability, and trustworthiness. For this,different metrics are visualised using appropriate visualisa-tion methods, like graph visualizations, metrics visualizations(e.g., bar charts), time-oriented visualizations (e.g., Sankeycharts), task-oriented and work process-oriented visualizations(e.g., Gantt charts), or hierarchy-oriented visualizations (e.g.,treemap charts).As first step, we developed a dashboard prototype thatcontains individual visualizations with interactive controls(Figure 1) [5]. For our prototype we used Python and thelibraries PLOTLY, and DASH2. Before visualizing the data wequery it with the libraryPY2NEO from a Neo4j graph databaseinstance3 and use PANDAS to prepare the data and temporarystore query results.Since not all popular software metrics and information areof equal importance to the user, we are conducting a focusgroup interview to identify what requirements users have forsuch a visualization. As a next step a first version of a dash-board will be developed which takes the user’s requirements2https://dash.plotly.com/3https://neo4j.com/product/neo4j-graph-database/Visualization (Python & Plotly)PROV MetricsData PreparationProvenance GraphGraph DatabaseNeo4jDataFramePandasQUERYCYPHERFilter and clean results storeDashboard (Plotly Dash)Dev. ContributionsNodes TimelineDeveloper ActivityDeveloper Mapping Event TimelineEntities Timeline HierarchyMar 282021Apr 4 Apr 11 Apr 18 Apr 25 May 2 May 9 May 16Added LabelMention In CommitCreate BranchMention In External Merge RequestMention In External IssueChange TitleChange DescriptionCommitMention In IssueIssue CreationNoteAward EmojiTimevisualizeInteractive Web-based Dashboardembedcommits of all userslucaapp/web lucaapp/ioslucaapp/androidlucaapp/security-overviewlucaapp/fdroid-repositorylucaapp/web-cryptolucaapp/badge-generatorInitial commit 2422 files changedchore: release v1.1.3 24 files changedfeat: add dummy traces for notifications endpoint 12 files changedchore: release v1.1.4 10 files changedchore: release v1.1.5 9 files changedchore: release v1.0.3 8 files changedchore: release v1.0.4 8 files changedchore: release v1.0.5 8 files changedchore: release v1.1.0 8 files changed chore: release v1.1.1 8 files changedchore: release v1.1.2 8 files changedRelease 1.4.2 728 files changedRelease 1.5.0 204 files changedRelease 1.6.1 400 files changedMerge branch 'master' into 'dev'  Release 1.7.0  See merge request lucaapp/android!4147 files changedRelease 1.7.0 147 files changedMerge branch 'master' into 'dev'  Release 1.7.1  See merge request lucaapp/android!552 files changedRelease 1.7.1 52 files changedRelease 1.6.6 34 files changedRelease 1.6.7 23 files changedAdd CHANGELOG4 files changed Add LICENSE4 files changedInitial commit 64 files changedInitialized repository 24 files changedInitial commit 32 files changedInitial Commit 22 files changedFeb 142021Feb 28 Mar 14 Mar 28 Apr 11 Apr 25 May 9 May 2305001000150020002500 Event TypesCommit ResourceCommit Resource VersionFileFile VersionIssue ResourceIssue Resource VersionDate (weekly bins)Number of eventsApr 42021Apr 11 Apr 18 Apr 25 May 2 May 9 May 160100020003000Apr 82021Apr 11 Apr 14 Apr 17 Apr 20 Apr 23 Apr 2605001000Apr 182021Apr 25 May 2 May 90100020003000PROV Elements: Entity Agent ActivityNodes Countlucaapp/androidlucaapp/ioslucaapp/webMar 142021Mar 28 Apr 11 Apr 25 May 9Florian SchäferGitHubHannesHannes RantzschPhilipp BergerRalf RottmannRené MeuselSand DornWolfsprohaska1w 1m 6m YTD 1y allTimeAgent Name11 4612 4218 52489985815116630388001740Activity Entity050010001500200025003000lucaapp/web-crypto lucaapp/badge-generator lucaapp/fdroid-repository lucaapp/ioslucaapp/security-overview lucaapp/web lucaapp/androidPROV TypeNode Type Countlucaapp/security-overviewProjectlucaapp/web-cryptoFlorian SchäferAgent NameSand Dornchristoph.absChristoph-Thomas AWolfHannesHannes RantzschFrank DietrichPhilipp BergerRené MeuselsprohaskaGitHubChristopher KrügelsRalf RottmannKonrad KleineFig. 1. Provenance visualization flow: querying the Neo4j database withCYPHER queries, storing query results and filtering and data cleansing withPandas, visualizing with Python and PLOTLY, and displaying in the web-baseddashboard with PLOTLY DASH [5].into consideration. Finally, the resulting dashboard will beevaluated through real-world tests, to ensure that it providesan intuitive way to understand software projects.III. TIMELINESS OR MATURITYDue to the complexity and diversity of today’s software sys-tems, numerous development tools are used to support the thedeveloper in his daily work. A typical tool suite consists of atleast an integrated development environment (IDE), a versioncontrol system, an issue tracker, a continuous integration andcontinuous deployment infrastructure, and a documentationsystem. During the software development process, all thesetools produce a large amount of different data. We use thisheterogeneous data to extract the complete provenance of allprocess steps using W3C PROV Data Model.The W3C specification PROV [6], which among other def-initions defines the Provenance Data Model (PROV-DM) [7],allows to collect and record provenance in a well defined andmachine readable formats.Using the W3C PROV data model, we designed a generalprovenance model to extract retrospective provenance [8]about software development processes [9]. This includesprovenance information of the underlying version control sys-tem as well as provenance of the respective issue trackers andrelease systems. The resulting provenance data then containsall activities (e.g., commits, issues changes, releases), thegenerated or changed entities (e.g., source code files or issues),and involved agents (e.g., developers, testers, or users) alongwith their relations.Using our tool GitLab2Prov [10], [11] this information canalready be collected from the web-based DevOps life-cycletool GitLab4. The tool uses the GitLab API to generate theprovenance information in the form of the text representationPROV-JSON, which is converted to other formats for furtheranalysis and can be imported as a labeled property graph intothe graph database Neo4j5 using the tool prov2neo. Thesetools were already successfully applied to collect provenanceinformation of inner-source and open source project [10], [12].AVAILABILITYGitLab2Prov is available as Open Source software under theMIT license: https://github.com/DLR-SC/gitlab2prov.The tool prov2neo is available under the MIT license athttps://github.com/DLR-SC/prov2neo.REFERENCES[1] L. von Kurnatowski, T. Schlauch, and C. Haupt, “Software developmentat the german aerospace center: Role and status in practice,” inProceedings of the IEEE/ACM 42nd International Conference onSoftware Engineering Workshops, ser. ICSEW’20. New York, NY,USA: Association for Computing Machinery, 2020, p. 157–160.[Online]. Available: https://doi.org/10.1145/3387940.3392244[2] L. von Kurnatowski, M. Stoffers, M. Weigel, M. Meinel, Y. Wasser,K. Rack, and H. Fiedler, “Scientific software engineering: Miningrepositories to gain insights into bacardi,” in 2020 IEEE AerospaceConference, 2020, pp. 1–10.[3] L. Moreau, “The Foundations for Provenance on the Web,” Foundationsand Trends® in Web Science, vol. 2, no. 2-3, pp. 99–241, 2010. [Online].Available: http://www.nowpublishers.com/article/Details/WEB-010[4] L. Moreau, P. Groth, S. Miles, J. Vazquez-Salceda, J. Ibbotson, S. Jiang,S. Munroe, O. Rana, A. Schreiber, V. Tan, and L. Varga, “The prove-nance of electronic data,” Communications of the ACM, vol. 51, no. 4,pp. 52–58, 2008.[5] A. Schreiber, L. Kurnatowski, A. Meinecke, and C. Boer, “An interactivedashboard for visualizing the provenance of software developmentprocesses,” in 2021 Working Conference on Software Visualization(VISSOFT). Los Alamitos, CA, USA: IEEE Computer Society, sep2021, pp. 100–104. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/VISSOFT52517.2021.00019[6] L. Moreau and P. T. Groth, Provenance: An Introduction to PROV,ser. Synthesis Lectures on the Semantic Web: Theory and Technology.Morgan & Claypool Publishers, 2013.[7] L. Moreau and P. M. et al., “Prov-dm: The prov data model,”Online, 2013, retrieved: 03.10.2021. [Online]. Available: https://www.w3.org/TR/prov-dm/[8] T. McPhillips, S. Bowers, K. Belhajjame, and B. Ludäscher, “Retrospec-tive provenance without a runtime provenance recorder,” in Proceedingsof the 7th USENIX Conference on Theory and Practice of Provenance,ser. TaPP’15. USA: USENIX Association, 2015.[9] A. Schreiber and C. de Boer, “Modelling knowledge about softwareprocesses using provenance graphs and its application to git-basedversion control systems,” in Proceedings of the IEEE/ACM 42ndInternational Conference on Software Engineering Workshops, ser.ICSEW’20. New York, NY, USA: Association for ComputingMachinery, 2020, p. 358–359. [Online]. Available: https://doi.org/10.1145/3387940.3392220[10] A. Schreiber, C. de Boer, and L. von Kurnatowski,“Gitlab2prov—provenance of software projects hosted on gitlab,”in 13th International Workshop on Theory and Practice of Provenance(TaPP 2021). USENIX Association, Jul. 2021. [Online]. Available:https://www.usenix.org/conference/tapp2021/presentation/schreiber[11] C. de Boer and A. Schreiber, “Dlr-sc/gitlab2prov: Gitlab2prov 0.5,”Jun. 2021. [Online]. Available: https://doi.org/10.5281/zenodo.5009043[12] A. Schreiber, L. von Kurnatowski, and C. de Boer, “Analyzing softwareengineering processes with provenance-based knowledge graphs,” in2021 IEEE Aerospace Conference (50100), 2021, pp. 1–11.4https://about.gitlab.com/features/5https://neo4j.com/product/neo4j-graph-database/

Provenance based software dashboards

https://elib.dlr.de/147617/1/10332edb5eeb48c6915b4521c39f3db5.pdf

Provenance based software dashboards

Abstract

Similar works

Full text

Available Versions

Institute of Transport Research:Publications