The composition, execution, and monitoring of challenging scientific applications is often a complex affair. To cope with the issue of workflow management, several tools and frameworks have been designed and put into use. However, the entry barrier to using these tools productively is high, and may hinder the progress of many scientists, or nonexperts, that develop workflows infrequently. As part of our Cyberaide framework we enable workflow definition, execution and monitoring through the Microsoft Project software package. The motivation for this choice is that many scientists are already familiar with Microsoft Project, a project management software package that is perceived to be user friendly. Through our tool we have the ability to seamlessly access Grids, such as the NSF sponsored TeraGrid. Cyberaide abstractions have also the potential to allow integration with other resources, including Microsoft HPC clusters. We test our hypothesis of usability while evaluating the tool as part of several graduate level courses taught in the field of Grid and Cloud computing

Dilmanian, Leor

von Laszewski, Gregor

English

RIT Scholar Works

Accelerating time to scientific discovery with a grid-enhanced microsoft project

Name not available

Rochester Institute of TechnologyRIT Scholar WorksArticles2008Accelerating time to scientific discovery with a grid-enhanced microsoft projectGregor von LaszewskiLeor DilmanianFollow this and additional works at: http://scholarworks.rit.edu/articleThis Article is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in Articles by an authorizedadministrator of RIT Scholar Works. For more information, please contact ritscholarworks@rit.edu.Recommended CitationAvailable at: https://www.academicresourcecenter.net/curriculum/pfv.aspx?ID=7796Accelerating Time to Scientific Discoverywith a Grid-Enhanced Microsoft ProjectGregor von Laszewski and Leor E. DilmanianRochester Institute of Technology, Rochester, NY 14623, Email: laszewski@gmail.comAbstractThe composition, execution, and monitoring of challeng-ing scientific applications is often a complex affair. To copewith the issue of workflow management, several tools andframeworks have been designed and put into use. However,the entry barrier to using these tools productively is high,and may hinder the progress of many scientists, or non-experts, that develop workflows infrequently. As part of ourCyberaide framework we enable workflow definition, execu-tion and monitoring through the Microsoft Project softwarepackage. The motivation for this choice is that many scien-tists are already familiar with Microsoft Project, a projectmanagement software package that is perceived to be userfriendly. Through our tool we have the ability to seamlesslyaccess Grids, such as the NSF sponsored TeraGrid. Cy-beraide abstractions have also the potential to allow in-tegration with other resources, including Microsoft HPCclusters. We test our hypothesis of usability while evaluat-ing the tool as part of several graduate level courses taughtin the field of Grid and Cloud computing.1 IntroductionThe workflow paradigm has proven itself to be an effec-tive paradigm in response to challenges imposed by manyscientific problems. To address these challenges, severaldifferent frameworks have been developed [1] to provideusable tools which simplify the use of executing distributedapplications on a computational Grids. As part of the scien-tific communities, the most demanding computational prob-lems require large-scale resources and orchestration of ac-tivity. To date, numerous Grid workflow management sys-tems have been created. Unfortunately, there is no one-size-fit all workflow management tool or solution. The multitudeof solutions is based on a number of challenges that we out-line next:• Changing standards to integrate Grid concepts: Al-though organizations such as the Workflow Manage-ment Coalition has already defined and implemented astandard set of tools, documents, and reference models,the standards are being redefined to address a whole setof issues unique to the Grid, including the need for a“service-based architecture intended for activities rang-ing from supporting large, distributed virtual organiza-tions for e-science to autonomic and on-demand com-puting for commercial enterprises [2].”• Complexity and unfamiliarity of the user interface: Al-though graphical user interfaces in general can be usedto simplify interaction with a scientific workflow com-position system, they require the learning of yet an-other tool that may in practice be used infrequently. Thelearning curve for a basic workflow remains high.• Integration of middleware is limited: In many cases,the workflow tools developed by industry and also thescientific community are limited and may not provideor allow the integration of other frameworks and mid-dleware. Although Grid protocol standards have beendefined, additional frameworks to enable an advancedcyberinfrastructure which might not have Grid servicesinstalled need to be considered.• Work modalities of the exiting community: In somecommunities, existing tools already cover portions ofthe complex workflow process, but are not integratedwith Grid or Cloud solutions and frameworks.As a direct result, we are in search for a multitude of in-terfaces that support use cases driven by community needs.However, the resulting tool must be easy to learn, adopt,expandable, and integrate with high performance comput-ing resources as part of deployed Grids and Clouds.To identify such a solution, commodity tools that are be-ing used by the community to support increase scientificproductivity need to be identified. One such tool is Mi-crosoft Project [3]. It is often used by scientists to plan,coordinate, and execute various activities; track and displayprogress, and build reports on project related activities withcolleagues and sponsors.The following questions are used to guide our researcheffort:• Is Microsoft Project feasible to support the scientificworkflow that is required as part of Grid and Cloud com-puting activities?• Which types of scientific activities which can be sup-ported using a Microsoft Project based workflow tool?• Which types of middleware solutions that can be inte-grated into Microsoft Project?• What are the advantages and limitations of using suchan approach?12 Use CasesAs obvious from the functionality of Microsoft Project,the use cases can include the execution of dependent jobswith resource constraints. Such constraints are typical forunique and large-scale instruments that need to be sharedamongst a group of scientists. As such resources are scarce.Microsoft Project allows a framework for sharing and col-laboration. Priorities can be used to order the tasks and tocreate a schedule that is suitable to address not only multipletasks by a single scientist but by an entire research group.Such functionality can be made available through both theMicrosoft Project client and server products.Projects using synchrotrons, flow-cytometers, and elec-tron microscopes as resources can benefit greatly from sucha tool. Even without specifying resource constraints as partof the workflow, it allows scientists to organize the experi-ments and enables a rigorous planning of the often complexcalculations while being able to integrate them in the over-all planning of an experiment. Furthermore, it can be usedto guide resource allocations needed to invoke scientific ap-plications required as part of emergency management sit-uations, such as intrusion detection of contaminants in thewater management system of a city.Other use cases are based on the actual graphical user in-terfaces supporting workflow designs. Our framework willallow the import and export of workflows to be executedwith other tools and workflow frameworks common in theGrid community.3 ImplementationAs part of our framework to simplify the use of advancedcyberinfrastructure called Cyberaide we developed a toolthat uses Microsoft Project and is able to steer computa-tions on the Grid. It is extensible as it provides an abstrac-tion layer for the execution of tasks on computational re-sources, which requires activities such as fault managementand data transfer. Figure 1 shows a screenshot of our tool inaction while composing and executing jobs on a variety ofresources on the TeraGrid.Figure 1: Using Microsoft Project to compose and execute gridworkflows on the Teragrid.Microsoft Project provides the appropriate views for thedisplay of resource related information. Such views includeresource graphs, calendars, charts, and usage diagrams. Us-ing both standard and custom fields associated with tasksand resources, the software package can be used to storeand monitor various quality of service parameters, allow-ing built in data analysis to make performance estimation.Resource discovery can be accomplished using built in sup-port from active directory, the address book, or a MicrosoftProject server. Programmatic manipulation of the MicrosoftProject 2007 client in the C# console application was madepossible through the Microsoft Project Primary Interoper-ability Assembly. Besides the manipulation of the objectthrough a graphical user interface, our implementation alsocontains a command line console allowing integration ofscripts and ad-hoc computational steering. Events gener-ated from the Grid middleware are forwarded to the Mi-crosoft Project and allow for dynamic monitoring.4 StatusOur Cyberaide tool demonstrates a feasible workflowframework that uses Microsoft Project as a composition andexecution tool to perform jobs on a Grid such as the Tera-Grid. Our current research efforts includes the creation ofabstractions for Cloud services such as Amazons Cloud Ser-vices and the integration of a workflow enactment enginethat takes advantage of the failure handling, check-pointing,dynamic and distributed workflow features. In addition, weare integrating our Grid and Cloud shell that is under activedevelopment as part of the Cyberaide project.5 EvaluationThe availability of additional Microsoft tools can sup-port planning, specification, execution, publication, sharingnot only of the workflows, but be helpful for the entire sci-entific discovery process. This can be of immense value tomany scientists. Microsoft Project provides for many usersthe necessary abstraction. For them Microsoft Project pro-vides a tool of choice, as our framework has demonstrated.Simplicity and ubiquity are the essential factors for the useradoption.AcknowledgmentWork conducted by Gregor von Laszewski is supported(in part) by NSF CMMI 0540076 and NSF SDCI NMI0721656.References[1] I. Taylor, E. Deelman, D. Gannon, and M. Shields, Eds., Workflowsfor e-Science: Scientific Workflows for Grids. Springer, 2007, iSBN:978-1-84628-519-6.[2] G. Fox and D. Gannon, “Workflow in Grid Systems,” Concurrencyand Computation: Practice and Experience, vol. 18, no. 10, pp. 1009–1019, 2006.[3] MSDN - Project 2007, Microsoft. [Online]. Available: http://msdn.microsoft.com/en-us/library/bb258902.aspx2

https://scholarworks.rit.edu/cgi/viewcontent.cgi?article=1685&amp;context=article

Accelerating time to scientific discovery with a grid-enhanced microsoft project

Abstract

Similar works

Full text

Available Versions

RIT Scholar Works

Name not available