In this paper we present the tool 'DockerConverter', an approach and a software to map a Docker configuration to various matured systems and also to reverse engineer any available Docker image in order to increase the confidence (or trust) into it. We show why a mapping approach is more promising than constructing a Domain Specific Language and why we chose a Docker image instead of the Dockerfile as the source model. Our overall goal is to enable Semantic Web research projects and especially Linked Data enterprise services to be better integrated into enterprise applications and companies

Kastner, Marcus

Meissner, Roy

Qucosa - Publikationsserver der Universität Leipzig

ii“proceedings” — 2017/8/24 — 12:20 — page 1865 — #1865 iiiiiiMaximilian Eibl, Martin Gaedke (Hrsg.): INFORMATIK 2017,Lecture Notes in Informatics (LNI), Gesellschaft für Informatik, Bonn 2017 11A mapping approach for configuration management tools toclose the gap between two worlds and to regain trustOr how to convert from docker to legacy tools (and vice versa)Roy Meissner1 and Marcus Kastner2Abstract: In this paper we present the tool “DockerConverter”, an approach and a software to map aDocker configuration to various matured systems and also to reverse engineer any available Dockerimage in order to increase the confidence (or trust) into it. We show why a mapping approach is morepromising than constructing a Domain Specific Language and why we chose a Docker image insteadof the Dockerfile as the source model. Our overall goal is to enable Semantic Web research projectsand especially Linked Data enterprise services to be better integrated into enterprise applications andcompanies.Keywords: Configuration Management; Orchestration; Trust; Docker; Ansible; Chef; Puppet; ReverseEngineering; DSL1 IntroductionAs [Ar15] showed, containerization is a perfect fit to support the research areas SemanticWeb and Linked Data (and possibly other ones) in order to decouple some of theirenvironment requirements and to integrate complex systems. It also helps other projectsor even companies that want to reuse results of these projects for their own purposes.Docker3 is a containerization technology that exists for about 3 years now and is lately oftenused as a standard technology among research communities, recent projects and startups.Nevertheless and according to [Ri16, page 11 et seqq.], well established enterprises tend touse matured software, approaches and systems and thus have not widely utilized Docker yet,instead using similar, but more matured systems like Chef, Puppet or Ansible. Additionally,these companies are not known for their fast pace in exchanging widely used technologies.This chops a large gap between research projects dealing with state of the art approachesand on the other side matured companies. It’s even larger in case these projects want to (orare forced to) integrate their solutions into established enterprises or public administration.In order to fill this gap, we have developed and now introduce “DockerConverter”4, an1 Institute for Applied Informatics (InfAI) e.V. Hainstraße 11, 04109 Leipzig, Germany meissner@informatik.uni-leipzig.de2 Leipzig University, Augustusplatz 10, 04109 Leipzig, Germany mk97caza@studserv.uni-leipzig.de3 See [Ar15] for an introduction into Docker and its use in the Linked Data Cloud4 Github Repository of DockerConverter: https://github.com/guitarmarx/DockerConvertercbe doi:10.18420/in2017_186aximilian Eibl, artin Gaedke. (Hrsg.): INFOR ATIK 2017,Lecture Notes in Informatics (LNI), Gesellschaft für Informatik, Bonn 2017 1865ii“proceedings” — 2017/8/24 — 12:20 — page 1866 — #1866 iiiiii12 Roy Meissner, Marcus Kastnerapproach and a software to map a Docker configuration to various matured systems andalso to reverse engineer any available Docker image in order to increase the confidence intoit. As the processing of semantic data is getting more and more complex, we are in need ofintegration solutions for complex software systems, that also need to integrate into maturedhosting solutions. DockerConverter is an approach to achieve this goal.We will introduce into related projects and work first in section 2 and dissociate our ownwork from these. In Section 3 we describe the implemented conversion approach, point outdifferent alternatives, as well as their advantages and disadvantages. We describe in section4 how we have implemented and validated the approach and then show how our softwareincreases confidence into Docker images in section 5. This paper ends with a conclusion insection 6, as well as pointing out our future vision.2 Related Work“Image Layers”5 is an open-source tool by CenturyLink Labs that reveals a Docker imageslayers by analysing it. This is similar to a part of our own approach, but will not uncoverany files, is a WebApp and only works for the legacy API of DockerHub6 (and only withDockerHub). Even worse, Image Layers does not provide any APIs in order to add furthersteps or automatically reuse its results and its documentation is sparse. According to theirGithub project7, development has completely stopped in September 2016.Similar to Image Layers is MicroBadger8, that is, in contrast, able to work with the currentDockerHub API. Apart of this, MicroBadger suffers from the same problems that wedescribed for Image Layers and is not an open-source project.“Image2Docker” (Windows only), announced in [Ma16]9, is a recent tool by Trevor Sullivanand enables to convert a virtual machine image (currently only WIM, VHD and VHDXformats and Windows Server 2003, -08, -12 and -16) to a Dockerfile, in order to producean identical Docker image. This is, as compared to us, the opposite approach (kind of the“vice versa”) but is very limited and lacks some major features, like support of establishedconfiguration management tools (CM-tools).Sebastian Günther et. al. describe in [GHS10] an approach for streamlining host setups,software deployments and maintenance approaches into one Domain Specific Language(DSL). They have executed a market analysis and constructed a DSL that is similar tocurrently available tools, like Ansible. In contrast to our approach, they focused on a DSLas the only solution and Docker was not available at this time. Using a DSL has some majordrawbacks that we describe in section 3.5 Image Layers Website: https://imagelayers.io/6 A public hosting service for Docker images, see https://hub.docker.com/7 Image Layers at Github: https://github.com/microscaling/imagelayers8 MicroBadger Website: https://microbadger.com/9 Image2Docker presentation at this years DockerCon: https://www.youtube.com/watch?v=YVfiK72Il5A&t=559s1866 Roy Meissner, Marcus Kastnerii“proceedings” — 2017/8/24 — 12:20 — page 1867 — #1867 iiiiiiA mapping approach for configuration management tools 133 Model Transformation or DSL?There are mainly two possible approaches to convert from Docker to matured CM-tools.One possibility is constructing a DSL or metalanguage, including a generator, parser andinterpreter to convert into the different CM-tool languages. According to [Me12] a DSL hasto include all semantic elements of the sub languages and their specifications to generatefully operational configurations. Arie Van Deursen et al. showed in [DKV00] that becauseof the internal design, a DSL allows validation and optimization at the domain level whichimproves testability and the conversion.The alternative approach is a model transformation, that can be accomplished by creating asemantic mapping between the CM-tool languages. This approach allows the conversionfrom one source model to multiple target models. Every CM-tool language would have itsown mapping with its language specific elements like vocabulary and semantics.Even though a DSL can be very powerful, it has, according to [DKV00, page 2 et seqq.], alarge potential loss of efficiency in comparison with model transformation, due to the costsof designing, implementing and maintaining it. As stated in [MHS05], the extension of aDSL is hard to accomplish because of the internal design. A DSL is optimized to all inputmodels which form the semantic of the DSL in the end. Adding another target model mighteven end in a complete revision process of the semantic and the internal compiler. Apartof this, introducing an artificial language, that acts as a superset of most of the existinglanguages is not advisable as potential operators would need to learn another languagewith a steep training curve and complex syntax because of the included sub languages. Incontrast, we know about some projects that successfully developed and implemented a DSLand got the market to widely adopt it, for instance the LLVM compiler infrastructure.In the end, we decided to use a model transformation approach for the conversion becauseit allows to reuse and convert already existing configurations and will not introducethe mentioned steep learning curve. This is similar to what Trevor Sullivan did in hisImage2Docker approach [Ma16], mentioned in section 2 and is expected to increase themarket adoption of the tool. Nevertheless, the major risk of a model transformation is theinsufficient compatibility of the different models and the loss of information during theconversion process. We explain in section 4 how we handled this risk.4 Implementation & ValidationWe identified two possible alternatives that may act as a proper source model for thetransformation process: the Dockerfile and its syntax or Docker images themselves. Thetransformation between a Dockerfile and a receipt (a collection of commands in a CM-tool)is easier to accomplish because of the similar structure and the same abstraction level. Buta Dockerfile contains only a part of the resulting configuration, as a final image is oftencomposed of several Dockerfiles. To reassemble the full configuration, one would haveA mapping approach for configuration management tools 1867ii“proceedings” — 2017/8/24 — 12:20 — page 1868 — #1868 iiiiii14 Roy Meissner, Marcus Kastnerto trace the composed Dockerfiles all the way up to their root and collect them all. Thisis sometimes impossible, as information in the chain are missing or it is time consuming,because one needs to find also all possibly added files first10.As a Docker image already contains all required information for the transformation, weused Docker images themselves as the source model. Unfortunately an image differs fromCM-tool configurations in various and significant ways. For instance docker splits an imageinto different layers that contain information about executed commands and metadata, aswell as files and their history. Apart of layering an image Docker optimizes commands,variables and files for some internal reasons. In order to resolve these we had to execute thefollowing steps:1. Split the layers of an image apart, to enable further processing2. Match the various layers with the sequence of executed commands3. Transform embedded Docker commands (e.g. ADD, CMD, ...) to valid bash commands4. Extract variables and files within layers by using the information from the previousstepsThus we transformed an image to a lower abstraction level. The different abstraction layersare depicted in figure 1.Fig. 1: Abstraction layers of DockerConverterSpeaking more abstract, a synthesis is executed first, which, according to [CH06], denotes atransformation from a higher to a lower abstraction layer. The subsequent transformationback to the configuration layer is called reverse engineering. We have used a Dockerimage instead of the Dockerfile and the mentioned steps to analyse it, in order to obtaina complete source model. By transforming these information to a lower abstraction layer10 This has also been identified by the projects Image Layers and Microbadger, mentioned in section 21868 Roy Meissner, Marcus Kastnerii“proceedings” — 2017/8/24 — 12:20 — page 1869 — #1869 iiiiiiA mapping approach for configuration management tools 15(bash commands), we were able to accomplish a transformation to other CM-tool languageswithout loss of information, as all CM-tool languages support bash commands.The transformation process may be controlled via a graphical user interface (GUI), which isdepicted in figure 2, or a command line interface (CLI). It is possible to look into differentimage components, like files and commands, before the transformation process is triggeredand also to edit available variables. Additionally, the GUI enables to trigger a fast analysemode that reconstructs the Dockerfile only. Thus, files (e.g. a HTTPD conf file) which mighthave been added to the image as of the Docker build process are not reconstructed. The fastmode analyses images much faster, as shown in figure 3 and might be useful if it is onlyimportant to reconstruct the Dockerfile itself, e.g. to understand an images structure.Fig. 2: Graphical User Interface of DockerConverter with an example analysisWe have tested the execution speed on a workstation (Core i5 3. Gen., SSD, 16GB DDR3)and measured both, available analysis modes (fast and full mode). The graph in figure 3shows the transformation time in relation to the image size for the top ten images fromDockerHub11. The figure shows that the execution time for a full transformation (full11 Top ten images from DockerHub: https://www.ctl.io/developers/blog/post/docker-hub-top-10/A mapping approach for configuration management tools 1869ii“proceedings” — 2017/8/24 — 12:20 — page 1870 — #1870 iiiiii16 Roy Meissner, Marcus Kastneranalysis mode) depends heavily on the image size, as this mode contains the needed stepsfor reconstructing additionally used files, as described in the previous paragraph.Fig. 3: Transformation time graphWe are currently using manual tests to validate the tool. It has been validated by testing itagainst ten images of the German company GIP, five images of the EU research projectSlideWiki, as well as for 5 images of the German research project LEDS (see section 6 formore information about these). We have additionally tested it against the ten most popularimages from DockerHub. In summary, we achieved for all images a transformation to allimplemented CM-tools and were able to reproduce virtual machines that are equivalent tothe used docker images.5 Regain TrustAs explained in section 4, anyone with access to an Docker image is able to reverse engineerit with Docker itself. But because of how Docker structures and stores images (see section 4for an in detail explanation) it ends up to be a time consuming and sometimes complex taskto fully reverse engineer a Docker image. Especially as, for example potential malware isoften hidden in default files, it is important to reconstruct them in general, as well as in theright way.With DockerConverter we have implemented all needed steps as a program and thusautomated them, so anyone is now able to reverse engineer a Docker image within seconds,essentially reproducing the Dockerfile and all needed additional files, that were used tobuild the image. It is even possible to convert this information to one of the more maturedCM-tools in order to analyse it, in case that the person using DockerConverter is more1870 Roy Meissner, Marcus Kastnerii“proceedings” — 2017/8/24 — 12:20 — page 1871 — #1871 iiiiiiA mapping approach for configuration management tools 17familiar with these systems than with Docker itself. So in case other trust or validationmethods failed (either because they threw an error or were not used at all), it is now easilypossible to validate an image manually or to search for bugs or possible intruders. Thisis also useful to simply reverse engineer an image, similar to disassembling a compiledprogram, in order to read and understand it. We have depicted the workflow for in figure 4.Fig. 4: Image and layer analysis and conversion6 Conclusion & PerspectiveWe have implemented an extensible open-source tool for a one way conversion of Dockerimages (and thus also Dockerfiles) to the three most popular (according to [Ri16]) config-uration management tools (CM-tools), that supports a commandline interface, a fast anda full analysis mode as well as a reverse engineering method to reconstruct the completeDockerfile from an Docker image. DockerConverter thus enables Semantic Web researchprojects and especially Linked Data enterprise services to be better integrated into enterpriseapplications, that are more likely to build upon matured CM-tools like Ansible, Chef orPuppet. Therefore we anticipate more possible applications of Semantic Web and LinkedData research results in larger contexts and thus hope to enable better integration of thesesoftware systems at all. Additionally, DockerConverter enables to reverse engineer anyimage in order to gain trust into it, to find out why an already used trust method failed or tosimply understand it. This will hopefully also strengthen the confidence in semantic webprojects. The tool has been validated via in-use testing by the German company GIP, the EUHorizon 2020 research project SlideWiki and the German Federal Ministry for Researchand Educations research project Linked Enterprise Data Services (LEDS) (see section 6 formore information about these), as well as by testing it against the ten most popular Dockerimages at DockerHub. We have additionally outlined possible other ways of implementingor constructing this tool and why we chose a mapping approach over the DSL approach.As the tool is open-source and thus extensible by anyone, it is easily possible to add moremappings in order to support more CM-tools. We have noticed that the mapping of the mostA mapping approach for configuration management tools 1871ii“proceedings” — 2017/8/24 — 12:20 — page 1872 — #1872 iiiiii18 Roy Meissner, Marcus Kastnerbasic commands between those CM-tools is bijective, therefore the mapping is revertible,but currently not implemented as we have focused on the described use case. Accordingthe former paragraph, we have only tested the tool manually. Thus, it is a future goal tocreate an automated test suite for continuous integration to cover unit and integration tests,as well as testing the integration between different CM-tool configurations. Thinking evenmore into the future, it seems to be possible to map whole orchestration configurations fromDocker-Compose and Docker Swarm to the matured systems and vice versa.AcknowledgementsThis work was partly supported by the German company “Gesellschaft für innovativePersonalwirtschaftssysteme mbH” (GIP), located in Leipzig, as well as the following grantsfrom European Union’s Horizon 2020 research and innovation programme for the SlideWikiProject under grant agreement No 688095 and the German Federal Ministry of Educationand Research (BMBF) for the LEDS Project under grant agreement No 03WKCG11C.References[Ar15] Arndt, Natanael; Ackermann, Markus; Brümmer, Martin; Riechert, Thomas: KnowledgeBase Shipping to the Linked Open Data Cloud. In: 11th International Conference onSemantic Systems Proceedings. International Conference on Semantic Systems Proceedings,Vienna, Austria, pp. 73–80, September 2015.[CH06] Czarnecki, Krzysztof; Helsen, Simon: Feature-based survey of model transformationapproaches. IBM Systems Journal, 45(3):621–646, 2006.[DKV00] Deursen, Arie Van; Klint, Paul; Visser, Joost: Domain-Specific Languages: An AnnotatedBibliography. Sigplan Notices, 35(6):26–36, 2000.[GHS10] Günther, Sebastian; Haupt, Maximilian; Splieth, Matthias: Utilizing Internal Domain-Specific Languages for Deployment and Maintenance of IT Infrastructures. March2010. Technical Report, URL: http://www2.cs.uni-magdeburg.de/inf_media/downloads/forschung/technical_reports_und_preprints/2010/TechReport04-EGOTEC-2f2p4di8gee9ocvgnihl5nauh31gad9b.pdf.[Ma16] Marks, Mano: , Image2Docker: A new tool for prototyping Windows VM conversions,September 2016. https://blog.docker.com/2016/09/image2docker-prototyping-windows-vm-conversions/, last access: 05 May 2017.[Me12] Mernik, Marjan, ed. Formal and Practical Aspects of Domain-Specific Languages: RecentDevelopments. IGI Global, September 2012. DOI: 10.4018/978-1-4666-2092-6.[MHS05] Mernik, Marjan; Heering, Jan; Sloane, Anthony M.: When and how to develop domain-specific languages. ACM Computing Surveys, 37(4):316–344, dec 2005.[Ri16] RightScale, Inc.: , State of the Cloud Report: DevOps Trends, January 2016.1872 Roy Meissner, Marcus Kastner

A mapping approach for configuration management tools to close the gap between two worlds and to regain trust: Or how to convert from docker to legacy tools (and vice versa)

https://ul.qucosa.de/api/qucosa%3A32065/attachment/ATT-0/

A mapping approach for configuration management tools to close the gap between two worlds and to regain trust: Or how to convert from docker to legacy tools (and vice versa)

Abstract

Similar works

Full text

Available Versions

Qucosa - Publikationsserver der Universität Leipzig