Texts aligned alongside their translation, or Parallel Corpora, are a very widely used resource in Computational Linguistics. Processing these resources, however, is a very intensive, time consuming task, which makes it a suitable case study for High Performance Computing (HPC). HPC underwent several recent changes, with the evolution of Heterogeneous Platforms, where multiple devices with different architectures are able to share workload to increase performance. Several frameworks/toolkits have been under development, in various fields, to aid the programmer in extracting more performance from these platforms. Either by dynamically scheduling the workload across the available resources or by exploring the opportunities for parallelism. However, there is no toolkit targeted at Computational Linguistics, more specifically, Parallel Corpora processing. Parallel Corpora processing can be a very time consuming task, and the field could definitely use a toolkit which aids the programmer in achieving not only better performance, but also a convenient and expressive way of specifying tasks and their dependencies.(undefined)info:eu-repo/semantics/publishedVersio

Almeida, J. J.

Brito, Rui

Texts aligned alongside their translation, or Parallel Corpora, are a very widely used resource in Computational Linguistics. Processing these resources, however, is a very intensive, time consuming task, which makes it a suitable case study for High Performance Computing (HPC).

HPC underwent several recent changes, with the evolution of Heterogeneous Platforms, where multiple devices with different architectures are able to share workload to increase performance.

Several frameworks/toolkits have been under development, in various fields, to aid the programmer in extracting more performance from these platforms. Either by dynamically scheduling the workload across the available resources or by exploring the opportunities for parallelism. However, there is no toolkit targeted at Computational Linguistics, more specifically, Parallel Corpora processing. Parallel Corpora processing can be a very time consuming task, and the field could definitely use a toolkit which aids the programmer in achieving not only better performance, but also a convenient and expressive way of specifying tasks and their dependencies

Dagstuhl Research Online Publication Server

A Workflow Description Language to OrchestrateMulti-Lingual ResourcesRui Brito and José João AlmeidaUniversity of MinhoDepartment of Informatics{ruibrito,jj}@di.uminho.ptAbstractTexts aligned alongside their translation, or Parallel Corpora, are a very widely used resourcein Computational Linguistics. Processing these resources, however, is a very intensive, timeconsuming task, which makes it a suitable case study for High Performance Computing (HPC).HPC underwent several recent changes, with the evolution of Heterogeneous Platforms, wheremultiple devices with different architectures are able to share workload to increase performance.Several frameworks/toolkits have been under development, in various fields, to aid the pro-grammer in extracting more performance from these platforms. Either by dynamically schedulingthe workload across the available resources or by exploring the opportunities for parallelism. How-ever, there is no toolkit targeted at Computational Linguistics, more specifically, Parallel Corporaprocessing. Parallel Corpora processing can be a very time consuming task, and the field coulddefinitely use a toolkit which aids the programmer in achieving not only better performance, butalso a convenient and expressive way of specifying tasks and their dependencies.1998 ACM Subject Classification D.3.4 ProcessorsKeywords and phrases workflow, orchestration, parallelism, domain specific languages, corporaDigital Object Identifier 10.4230/OASIcs.SLATE.2014.771 IntroductionIt is widely accepted that tasks can be very difficult to manage, mainly when they have manysteps, often with time consuming tools, interdependent tasks and huge datasets. As thecomplexity of applications increases, so does the difficulty to efficiently manage the differentavailable resources. Users have to adopt frameworks/toolkits, as a way to improve theirpath to achieve the desired goals. The aim of this paper is to provide a toolkit targetedat Computational Linguistics, which may also benefit other fields that require workflowmanagement. Texts aligned alongside their translation, or parallel corpora, are consideredvery rich resources for machine translation, whose formats usually follow Translation MemoryeXchange (TMX) or Probabilistic Translation Dictionaries (PTD). Tools to aid machinetranslation based on parallel corpora include text aligners (e.g. Moses [6]), morphologicaltaggers (e.g. Apertium [4] and Freeling [8]) and dictionary generators (e.g. NATools [10]).Some of these tools are computationally intensive, taking too long to display useful results.Most data is the result of a complex task chain and ever-changing data, which makes theresults difficult to predict and errors difficult to detect. This type of work is currently doneeither by hand or by very crude scripts, but could be significantly improved if the linguisticscommunity could access a toolkit to create and manage the interrelated parallel corporaprocessing tasks and take advantage of current heterogeneous computing systems. The lackof such a tool is the main motivation to pursue this work, developing a Workflow DescriptionLanguage (WDL), which is a DSL to orchestrate Multi-Lingual resources.© Rui Brito and José João Almeida;licensed under Creative Commons License CC-BY3rd Symposium on Languages, Applications and Technologies (SLATE’14).Editors: Maria João Varanda Pereira, José Paulo Leal, and Alberto Simões; pp. 77–83OpenAccess Series in InformaticsSchloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany78 A Workflow Description Language to Orchestrate Multi-Lingual ResourcesPrevious work of the authors on the Per-Fide [1] project gained insight on the requirementsto specify and develop a toolkit to chain tasks and to handle large data sets with differentdata formats. The task chain usually contains many steps, from different tools, which maychange frequently, making it difficult to keep track of the results being produced. The tasksinvolved are mostly irregular, where memory access patterns may not be predictable, withconsequent penalties on execution time. This brings forth the need to develop a toolkit todescribe tasks, their chaining, their dependencies, their types and with the need to provideenvironment details to better take advantage of current heterogeneous computing systems.Given a set of tools, the user should be able to specify tasks to be executed and visuallyfollow the workflow execution phases and results. This can be achieved with a flow-graph.Some work has been done in this area, particularly with the Makefile::Parallel [9] tool,which, provides a way of describing tasks and workflows to be performed in either a clusterenvironment or a single machine.The expected contribution of the work described in this paper is a toolkit to betterorganize, manage and modify on the fly. The toolkit will also make an efficient use of theavailable computational resources, by dynamically scheduling the workload with high-leveldirectives that abstract efficient implementations.The Section 2 begins by describing the Workflow Description Language and its aims.Section 3 presents some examples of the language, as well as some use case scenarios. Finally,section 4 presents our conclusions on the work already done.2 Workflow Description LanguageThe main goal of this paper is to describe an efficient toolkit to orchestrate computationallinguistics tasks. The user specifies through a WDL how individual tasks are performed. Themain features of the toolkit are as follows:The toolkit will be environment-aware, and thus, take advantage as best as possible ofthe computing units and systems available;The toolkit should also be able to revert back to its most recent stable state (if the taskis aborted);Primitives to generate reports, diagrams and type signature should be provided;There should be a mode in which the user provides heuristics on how to best use theavailable computing units;The user should be able to specify mapReduce [3, 7, 2] functions to achieve certain tasks;A cleaner and more elegant way of specifying tasks;Efficient low-level implementations are hidden from the end user;The WDL should be Algebra based, providing:Familiar operations such as composition, which makes the language clearer;The toolkit should contemplate not only complex cluster systems, but also single usersystems.Some of the generic goals of the toolkit to be developed include:A declarative approach;Separation of elemental/independent tools;Separation of resources/machines;Reuse of existing research;Expressive power.R. Brito and J. J. Almeida 79Figure 1 WDL Description.The general organization of the WDL is described in the Figure 1. There are, essentially,three distinct parts to the language. A Machine Specification part, where all the computingsystems available are described. In its simplest form, this means specifying hosts and usersfor remote machines. In the case of a cluster environment, there are primitives to find outwhich nodes are free, to ask for a full node (or nodes), since most cluster schedulers (e.g.PBS or Maui [5]) never supply the user with a full node, unless asked to. These are calledexecution attributes. There are also primitives to ascertain, for example, if a certain tool isavailable in a specific machine. Primitives of this kind are called existence attributes.Then, there is a Tool Specification part, where all the tool to be used are described, alongwith any eventual execution attributes. Finally, there is an Task Specification part, wherethe Tasks are organized in a workflow.This paper focus mostly on parts 2 and 3.This WDL grammar is being developed using Perl YAPP, a Perl extension for generatingand using LALR parsers. This decision is mainly because of Perl’s expressive power andRegular Expressions, which make it easier to develop the kind of language we’re trying toachieve.As it can be seen from the grammar presented in Table 1, the user begins by specifyingtools, which, in their most distilled form, are a collection of actions paired with an in andout type definition. The action production provides a way to specify how a certain tool isrun. As it can be seen from the grammar, it can be either a bash script, or Perl code whichis preprocessed before execution. The execAttr production’s goal is to provide executiondetails for the tool, such has maximum desired walltime, whether it can run on a GPU or inIntel’s MIC micro-architecture. In the case of a cluster environment, how many cores andhow many nodes are needed for a specific tool. Eventually, more attributes will be added, oreven changed. Once the tools are specified, the user begins specifying tasks. The productionshown here (taskSpec), while not complete, hints at the direction we’re aiming.3 WDL ExamplesListing 1 presents how the syntax expressed in the grammar can be used to specify tools andtasks. First, the tool’s type signature is specified, then a set of execution attributes followedby how to execute the tool. A couple of conventions were adopted, for example, for a givenresource named corpus.pt.en.tmx:$B1 is the name of the resource (corpus);$1 is the name of the resource coupled with its extension (corpus.pt.en.tmx).It is also possible to specify variable filenames, or type extensions, as seen in the tmx2tmxatool definition in listing 1. This way, the user can make variable parts of a filename part ofthe type definition (e.g. language pairs).SLATE 201480 A Workflow Description Language to Orchestrate Multi-Lingual ResourcesTable 1 Workflow Description Language Grammar.wdl → ‘%tools’ toolDefs ‘%tasks’ taskSpecs;toolDefs → toolName ‘:’ signature tool| toolDefs toolName ‘:’ signature tool;signature → type_id ‘→’ type_id;tool → ‘{’ action execAttrs split join ‘}’;action → ‘@BASH’ ‘{’ STRING ‘}’| ‘@PERL’ ‘{’ STRING ‘}’;execAttrs → execAttrs execAttr| εexecAttr → ‘wall=’wall| ‘gpu=’ bool| ‘mic=’ bool| ‘ncores=’ INT| ‘nnodes=’ INT| · · ·;split → ‘split=’ action| ε;join → ‘join=’ action| ε;taskSpecs → taskSpecs task| ε;task → toolName ‘(’ args ‘)’| toolName ‘.’ toolName ‘(’ args ‘)’| ‘#make’ ‘(’ args ‘)’| ‘#span’ ‘(’ args ‘)’;args → ID| args ‘,’ ID;R. Brito and J. J. Almeida 81Listing 1 WDL Example.12 tmx2ptd : tmx -> ptd {3 @BASH {4 nat -create -id $B1.ptd -tmx $15 }6 }78 tmx2tmxa : $l1.$l2.tmx -> tmxa {9 @BASH {10 tmx2tmxa -l $l1 -$l2 -f $1 -o $B1.$l1.$l2.tmxa11 }12 }1314 tmx2ptda : tmxa -> ptda {15 @BASH {16 tmxa -lemmatizer -i $1 -o $B1.tmxpl17 nat -create -id $B1.ptda -tmx $B1.tmxpl18 }19 }2021 tmx2tmxa (corpus.pt.en.tmx)f.tmx f.tmxatmx2tmxa f.ptdatmxa2ptdaFigure 2 Typical serial workflow.An alternative to specifying tasks in a function call fashion is to write something like thefollowing using function composition:1 tmxa2ptda.tmx2tmxa (corpus.pt.en.tmx)The resulting workflow can be seen in Figure 2.For example, the following definition adds mapReduce primitives to the tmxa2ptda tool1.The resulting workflow can be seen in Figure 3.More complex examples, given there are no dependencies, would allow tasks to executein parallel in a Cluster environment, spawning several processes in a single machine, oreven across multiple machines. This would allow for a more efficient use of the availablecomputational resources. Communication is done preferably by rsync, since it is a safer andfaster alternative to scp.There is also ongoing work in modes in which the user writes:1 #span(resource.src_type)2 #make(resource.target_type)The #span construct will traverse the graph generated from the type signatures of thetools and generate all the resources possible from the achievable nodes. The #make construct1 tmxsplit is a script that cuts a tmx when possible after a given number of lines. ptdcat is a script thatconcatenates the various parts of a ptd file.SLATE 201482 A Workflow Description Language to Orchestrate Multi-Lingual ResourcesListing 2 WDL with MapReduce Example.1 tmxa2ptda : tmxa -> ptda {2 @BASH {3 tmxa -lemmatizer -i $1 -o $B1.tmxpl4 nat -create -id $B1.ptda -tmx $B1.tmxpl5 }6 split=@BASH {7 tmxsplit -l 10000 $18 }9 join=@BASH {10 ptdcat *.ptda > result.ptda11 }12 }f.tmx f.tmxatmx2tmxa splitp1.tmxap2.tmxap3.tmxap4.tmxa...pn.tmxaf.ptdareducep1.ptdatmxa2ptdap2.ptdatmxa2ptdap3.ptdatmxa2ptdap4.ptdatmxa2ptda...tmxa2ptdapn.ptdatmxa2ptdaFigure 3 Workflow with MapReduce.will take a resource.target_type, and, by inference, figure out the correct path to generate itfrom an existing resource.src_type. There will also be a built-in Perl preprocessor, so thatthe user can build more complex tasks.4 ConclusionsWorkflow orchestration and management is something very present in today’s computingworld and specially in fields that deal with intensive tools and very large data-sets. The lackof a tool to achieve this end, makes it even more difficult. In this short paper we introduced anew language to orchestrate workflows and to better manage existing computational resources.High level primitives and the direct integration of a scripting language (Perl) means weachieved a versatile format to express complex workflows.Acknowledgments. This work is funded by National Funds through the FCT – Fundaçãopara a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) withinproject PEst-OE/EEI/UI0752/2014R. Brito and J. J. Almeida 83References1 José João Almeida, Sílvia Araújo, Nuno Carvalho, Idalete Dias, Ana Oliveira, André San-tos, and Alberto Simões. The Per-Fide corpus: A new resource for corpus-based termino-logy, contrastive linguistics and translation studies. In Tony Berber Sardinha and TelmaSão-Bento Ferreira, editors, Working with Portuguese Corpora, chapter 9, pages 177–200.Bloomsbury Publishing, April 2014.2 M. Bhandarkar. MapReduce programming with Apache Hadoop. Parallel & DistributedProcessing (IPDPS), 2010 IEEE International Symposium on, 2010.3 Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified Data Processing on LargeClusters. Communications of the ACM, 51(1):1–13, 2008.4 Mikel L. Forcada, Mireia Ginestí-Rosell, Jacob Nordfalk, Jim O’Regan, Sergio Ortiz-Rojas,Juan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez, Gema Ramírez-Sánchez, and Fran-cis M. Tyers. Apertium: a free/open-source platform for rule-based machine translation.Machine Translation, 25(2):127–144, 2011.5 David Jackson, Quinn Snell, and Mark Clement. Core Algorithms of the Maui Scheduler.In Job Scheduling Strategies for Parallel Processing, pages 87–102. Springer-Verlag, 2001.6 Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico,Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer,Ondřej Bojar, Alexandra Constantin, and Evan Herbst. Moses: Open source toolkit forstatistical machine translation. In Proceedings of the 45th Annual Meeting of the ACL onInteractive Poster and Demonstration Sessions, ACL’07, pages 177–180, Stroudsburg, PA,USA, 2007. Association for Computational Linguistics.7 K.H. Lee, Y. J. Lee, H. Choi, Y.D. Chung, and B. Moon. Parallel data processing withMapReduce: a survey. ACM SIGMOD Record, 40(4):11–20, 2012.8 Lluís Padró and Evgeny Stanilovsky. FreeLing 3.0: Towards Wider Multilinguality. LREC,pages 2473–2479, 2012.9 Alberto Simões, Rúben Fonseca, and José João Almeida. Makefile::Parallel DependencySpecification Language. In Anne-Marie Kermarrec, Luc Bougé, and Thierry Priol, editors,Euro-Par 2007, pages 33–41, Rennes, France, August 2007. Springer-Verlag.10 Alberto M. Simões and J. J. Almeida. NATools – a statistical word aligner workbench.Procesamiento del Lenguaje Natural, 31:217–224, Sep. 2003.SLATE 2014

A Workflow Description Language to Orchestrate Multi-Lingual Resources

Universidade do Minho: RepositoriUM

A workflow description language to orchestrate multi-lingual resources

http://drops.dagstuhl.de/opus/volltexte/2014/4560/pdf/8.pdf

A workflow description language to orchestrate multi-lingual resources

Abstract

Similar works

Full text

Available Versions

Dagstuhl Research Online Publication Server

Universidade do Minho: RepositoriUM