Antoniadis, Anastasios

Dolby, Julian

Grech, Neville

Lagouvardos, Sifis

Smaragdakis, Yannis

English

Dagstuhl Research Online Publication Server

Static Analysis of Shape in TensorFlow Programs(Artifact)Sifis LagouvardosUniversity of Athens, Greecesifis.lag@di.uoa.grJulian DolbyIBM Research, Yorktown Heights, NY, USAdolby@us.ibm.comNeville GrechUniversity of Athens, Greeceme@nevillegrech.comAnastasios AntoniadisUniversity of Athens, Greeceanantoni@di.uoa.grYannis SmaragdakisUniversity of Athens, Greecesmaragd@di.uoa.grAbstractThese instructions are intended for using the ar-tifact for our ECOOP’20 paper entitled “StaticAnalysis of Shape in TensorFlow Programs”. Theycan be used to run Pythia – the tool implementingthe paper’s analysis – on the paper’s evaluation setdemonstrating bug detection in the most preciseconfiguration of our analysis as well as the precisionof the analysis under different configurations.2012 ACM Subject Classification Theory of computation → Program analysis; Software and its engin-eering → Compilers; Software and its engineering → General programming languagesKeywords and phrases Python, TensorFlow, static analysis, Doop, WalaDigital Object Identifier 10.4230/DARTS.6.2.6Funding We gratefully acknowledge funding by the European Research Council, grant 790340 (PARSE),and by the Hellenic Foundation for Research and Innovation (project DEAN-BLOCK).Related Article Sifis Lagouvardos, Julian Dolby, Neville Grech, Anastasios Antoniadis, and YannisSmaragdakis, “Static Analysis of Shape in TensorFlow Programs”, in 34th European Conference onObject-Oriented Programming (ECOOP 2020), LIPIcs, Vol. 166, pp. 15:1–15:29, 2020.https://doi.org/10.4230/LIPIcs.ECOOP.2020.15Related Conference 34th European Conference on Object-Oriented Programming (ECOOP 2020),November 15–17, 2020, Berlin, Germany (Virtual Conference)1 ScopeThe artifact captures code that is also available in a public repository (https://bitbucket.org/yanniss/doop/). The latest version can be obtained online. The artifact snapshot the state ofthe code at the time of paper publication and can be used to reproduce two experiments from ourpaper; with the first one demonstrating precise bug detection and the second one showcasing theeffect different analysis configurations have on precision.© Sifis Lagouvardos, Julian Dolby, Neville Grech, Anastasios Antoniadis, and Yannis Smaragdakis;licensed under Creative Commons Attribution 3.0 Germany (CC BY 3.0 DE)Dagstuhl Artifacts Series, Vol. 6, Issue 2, Artifact No. 6, pp. 6:1–6:3Dagstuhl Artifacts SeriesSchloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany6:2 Static Analysis of Shape in TensorFlow Programs (Artifact)1.1 Bug DetectionThe first experiment includes analyzing our evaluation set using the most precise configuration ofour analysis (1-call-site-sensitive with a context-sensitive heap and full-tensor-precision).Table 2 in our paper summarizes the results for the 14 input programs in our evaluation set.1.2 PrecisionThe second experiment demonstrates the effect different analysis configurations have in theprecision of the analysis, reproducing the experiment shown in section 7.3 (and Figure 2) of ourpaper. We partly reproduce the results of the chart in Figure 2. The reason we cannot fullyreproduce these results is that the false positives of our analysis require manual labor to beidentified. For this reason we report the imprecision metrics that can be automatically producedby our analysis.2 ContentThe artifact contains the Doop program analysis framework that includes Pythia, the dataset of“An empirical study on TensorFlow program bugs”[1] (part of which is our evaluation set), the fullartifact documentation in markdown and PDF formats, and scripts for running the analyses andpost-processing their results.3 Getting the artifactThe artifact endorsed by the Artifact Evaluation Committee is available free of charge on theDagstuhl Research Online Publication Server (DROPS).4 Tested platformsOur artifact is bundled for AMD64 Linux using an Ubuntu 18.04 Docker image. If you do notalready have Docker installed, follow the official installation instructions for your Linux system(can be found at https://docs.docker.com/install/). We’ve successfully tested our artifactusing machines that have 16 GBs of RAM or more. Has been tested using ubuntu 18.04 andWindows 10 using WSL and WSL2.5 LicenseThe artifact is available under The Universal Permissive License (UPL), Version 1.0, Copyright(c)2017 PLAST lab, University of Athens and Martin Bravenboer.6 MD5 sum of the artifact12c251ecc51a5c6dc8bf1fe7ff2e873a7 Size of the artifact1.3 GiBS. Lagouvardos, J. Dolby, N. Grech, A. Antoniadis, and Y. Smaragdakis 6:3References1 Yuhao Zhang, Yifan Chen, Shing-Chi Cheung,Yingfei Xiong, and Lu Zhang. An empirical studyon tensorflow program bugs. In Proceedings of the27th ACM SIGSOFT International Symposium onSoftware Testing and Analysis, ISSTA 2018, page129–140, New York, NY, USA, 2018. Associationfor Computing Machinery. doi:10.1145/3213846.3213866.DARTS

Static Analysis of Shape in TensorFlow Programs (Artifact)

https://drops.dagstuhl.de/opus/volltexte/2020/13203/pdf/DARTS-6-2-6.pdf

Static Analysis of Shape in TensorFlow Programs (Artifact)

Abstract

Similar works

Full text

Available Versions

Dagstuhl Research Online Publication Server