Lwowski, Brandon

English

IUScholarWorks (Indiana University)

Inter-operable, reproducible and efficient deployment of software andsoftware changes in the Jetstream open stack environmentBRANDON LWOWSKI, University of Texas at San AntonioWith technology on the rise in the education system, the use of applicationsand software for such activities is growing exponentially."Jetstream is the firstproduction cloud funded by the NSF for conducting general-purpose scienceand engineering research as well as an easy-to-use platform for educationactivities. [Fischer et al. [n. d.]]" The ability to deploy and reproduce softwareto multiple cloud platforms with the capabilities of seamless version controlis essential. With a vast range of cloud platforms spreading the populationof users between them, there must be a uniform way to deploy software intoa multi-cloud environment. In order to reach a bigger population, you needyour software readily available across the major cloud platforms includingOpenstack, Oracle, Google, Amazon, and Microsoft. Not only do you needto be able to deploy a multi-cloud environment, you need to be able to pushnew versions onto these cloud platforms as well. In this paper i will discussthe research done during my research experience for undergraduates atIndiana University and working alongside the Jetstream team. I will discussthe implementation of the Spinnaker plug-in, and how i plan to use it inthe Jetstream Openstack Cloud. More specifically i will explain some ofthe successes and failures during this process. At eh end of the 8 weeks, Iplan to give Jetstream the ability to deploy their software not only to theirOpenstack environment and be able to spread their applications and softwareto multiple clouds.Additional Key Words and Phrases: Openstack, Jetstream, VM, REU, Spin-naker, Jenkins, DevstackACM Reference format:Brandon Lwowski. 2017. Inter-operable, reproducible and efficient deploy-ment of software and software changes in the Jetstream open stack environ-ment. 1, 1, Article 1 (July 2017), 2 pages.https://doi.org/0000001.0000001_21 INTRODUCTIONFirst and foremost, coming into this project, I had zero cloud com-puting in my knowledge base. I had limited linux command lineskills and just completed my second year of my bachelor’s degree incomputer science. When reading through the procedure and meth-ods that follow this section, some steps may seem redundant orunnecessary to most, but starting from ground zero and buildingwas necessary. I separated my 8 week block into 3 sections. I gavemyself 3 weeks to learn as much as I could about openstack andJetstream. I read wiki pages, watch hours and hours of openstacksummit videos, built and launched VMs. The goal was to get as fa-miliar with openstack, openstack command line, and the JetstreamAP as possible in a short amount of time. The next 3 weeks, the goalwas to install and configure the Spinnaker plug-in and connect it toThis work is supported by the National Science Foundation, Indiana University, andthe Jetstream Team.Authors’ addresses: Brandon Lwowski, Computer Science Department, University ofTexas at San Antonio, One UTSA Circle, San Antonio, TX 78249, US;.© 2017 Association for Computing Machinery.This is the author’s version of the work. It is posted here for your personal use. Notfor redistribution. The definitive Version of Record was published in , https://doi.org/0000001.0000001_2.the Jetstream openstack environment. The last 2 weeks of my REU Iwanted to test Spinnaker, have a working demo to show the powerof Spinnaker, and present it to the Jetstream team.2 BENEFIT OF MULTI CLOUD SOFTWAREDEPLOYMENTThe major benefit of using the Spinnaker plug-in is the power todeploy software and software changes to multiple cloud platforms.This in itself promotes interoperability and reproducibility. Havingthis capability greatly improves deploying of software and versioncontrol for software developers. For example, being able to releasea bioinformatic software created in the Jetstream Atmosphere, andreproduce that same software, using the Spinnaker pipeline, to thecustomers on the Google cloud just increased the potential audienceand users of your software. Now imagine yourself adding anotherstage into the Spinnaker pipeline to deploy to the Amazon cloud aswell. With a few extra steps you potentially tripled the audience ofyour software.3 PROCEDUREIn the following sections, I will explain the steps I took in order to getSpinnaker connected to the endpoints of Jetstream. I tried 3 differentmethods in order to accomplish the installation and configurationof Spinnaker. I will explain in detail how far I got to completion ineach method and the difficulties and headaches that I came acrossduring these processes.3.1 DevstackWhen installing Spinnaker for the first time, I thought that if Iused the Jetstream Openstack Cloud to implement Spinnaker, and Ihappened to mess up the configuration, then somehow I could messup the entire cloud. This misconception led me to a few weeks offrustration. My original plan to install and implement Spinnakerwas to create my own test cloud using devstack. With this test cloud,I then could run install and run Spinnaker in a test environmentwithout the fear of crashing Jetstream. If could do so correctly then Icould also apply it to the JetstreamOpenstack Cloud. The pro’s of metrying this first is I learned from scratch the different componentsand projects used to get a test cloud up and working. The difficultpart of this method was Spinnaker is very specific on what releaseof openstack you are using and minimum requirements for versionsneeded for each project. After successfully producing a test cloudwith the mitaka release, I still had to upgrade each componentof the services needed. This ended up being a difficult task that Istruggled with and ended up hitting a wall. I could not configure myopenstack test cloud correctly in order to get Spinnaker installed andrunning correctly. After several about a week of trial and error anddoing more research on Spinnaker, I reached out to PHD students,Professors, and the Spinnaker Open source community, searching, Vol. 1, No. 1, Article 1. Publication date: July 2017.1:2 • Brandon Lwowskifor advice on moving forward with this problem. After voicing mythought process, I learned I could just use the Jetstream Openstackendpoints given to me in the Jetstream API to connect Spinnaker tothe openstack cloud. This discovery lead me to my second method.3.2 Different VM’sThis method may have worked for a person with background innetworking and knowledge in json programming. A Lot of theconfiguration needed was manipulation of json files to connectJenkins and Spinnaker together. Once they were connected I couldthen push new code onto my github library which would triggerJenkins to poll for changes and start the Spinnaker pipeline. Thismethod started off excellent. I had Spinnaker linked correctly toJetstreams openstack. I correctly installed Jenkins on the separateVM. I was able to push code and have Jenkins poll for changes.The problem occurred when trying to link the Jenkins master tothe Spinnaker pipeline. I could not figure out how to configure theJenkins settings to broadcast to the Spinnaker host.3.3 Same VMInstalling Jenkins and Spinnaker on the same VM is the method thatgot me the farthest in a working implementation of a Spinnakerpipeline. Having the 2 applications running on a the same VM madethe configuration of connecting Jenkins and Spinnaker a smaller task.There were less steps needed to get Spinnaker and Jenkins talkingto each other. Out of all the methods I tried, this was definitelythe easiest to get started. Once I had all all the pieces installed andconnected, it was time to test the pipeline. This is where thingsbegan to get a little difficult. From my first year in computer scienceand everything up to this point that I had learned about openstack, Ihave never used Jenkins and the Jenkins job builder before. I had tolearn how to create a free style project that Jenkins could build anddeploy to trigger Spinnaker. After reading tutorials about Jenkins, Icame across a "hello world" type project for Jenkins on the Spinnakerwebsite. Now I had a Jenkins project that was able to build and pollfor changes and also be used in my Spinnaker pipeline. It was timeto spin up a instance and bake a VM with the new software fromthe Jenkins build. This final step is where I am currently stuckon. The Jenkins job builds correctly in the Spinnaker pipeline, butonce it’s time to bake the VM and deploy it to a cluster group, thepipeline fails. The error I keep coming across is, Spinnaker has theauthentication to use Jetstreams openstack, but is not receivingthe correct authentication to use the premade jetstream flavors andimages. Spinnaker could not pull this information from the Jetstreamopenstack.4 CONCLUSIONThe Spinnaker plug-in is a powerful "multi-cloud continuous de-livery platform for releasing software changes (SOURCE)". WhileSpinnaker is in its early stages and is used with multiple cloudsincluding Google, Amazon, Microsoft, and Openstack, it is still veryimmature in the openstack environment. Companies like Targethave successfully implemented it into their company’s cloud and Ithink that the Jetstream cloud could benefit greatly from a success-ful implementation of this plug in. If Jetstream had the availableresources to allow their successful educational software to be re-leased to other cloud platforms, it would help spread their resourcesto a bigger audience and help spread the word of the Jetsteam’smission. While difficult to install and configure correctly for a am-ateur computer science student, once this plugin matures for theopenstack environment, I believe it would be extremely useful inpromoting interoperability and reproducibility of applications madein the Jetstream cloud.REFERENCESJeremy Fischer, George Turner, David Y Hancock, Winona Snapp-childs, John MichaelLowe, and Craig A Stewart. [n. d.]. Jetstream : A cloud system enabling learning inhigher education communities. ([n. d.])., Vol. 1, No. 1, Article 1. Publication date: July 2017.

Inter-operable, reproducible and efficient deployment of software and software changes in the Jetstream OpenStack environment

https://scholarworks.iu.edu/dspace/bitstream/2022/21686/1/inter-operable-reproducible.pdf

Inter-operable, reproducible and efficient deployment of software and software changes in the Jetstream OpenStack environment

Abstract

Similar works

Full text

Available Versions

IUScholarWorks (Indiana University)