Overview:•GENIE Project•Multi-objective Optimisation•Surrogate Modelling•Grid Computing Infrastructure•Parameter Estimation for a new Ocean Mixing Scheme•Conclusion

Cox, Simon J.

Marsh, Robert

Myerscough, Richard J.

Price, Andrew R.

Voutchkov, Ivan I.

English

Southampton (e-Prints Soton)

Multi-objective Optimisation of GENIE Earth System ModelsAndrew R. Price1, Richard J. Myerscough2,Ivan I. Voutchkov1, Robert Marsh2 & Simon J. Cox11 School of Engineering Sciences, University of Southampton, Southampton SO17 1BJ, UK2 National Oceanography Centre, University of Southampton, Southampton SO14 3ZH, UK9 September 2008Overview• GENIE Project• Multi-objective Optimisation• Surrogate Modelling• Grid Computing Infrastructure• Parameter Estimation for a new Ocean Mixing Scheme• Conclusions23Scientific Aims• Orbital parameters affect incident radiation and climate• Biological and geological processes interact with, and feedback upon, the climate (via, for instance, CO2)4ice-age ice-age ice-age ice-ageThe mechanisms that have driven the most fundamental changes of planet Earth are not yet fully understood.Parameter Estimation Problem• Default parameters almost always sub-optimal for newly coupled models or existing models of increased resolution• Non-linear response of a model to its parameters makes “tuning” a difficult task• Often find conflicting design objectives (improvements in atmospheric representation can compromise ocean properties)• Multi-objective design search and optimisation methods to Earth system models found to be effective5GENIE model bathymetry (depth profile) used in the 16-level model. The grid resolution is 64 x 32 x16.Composition of the GENIE model used in this study.Ocean features an increased complexity mixing scheme over the default GOLDSTEIN code.2D atmosphereEMBM (64 x 32)3D oceanIncreased ComplexityMixing Scheme(64 x 32 x 16)2D slab sea-ice2D land surfaceMulti-objective Optimisation• To compose a single objective function a decision maker must provide weighting factors for the individual targets• The optimal or best choice for these weightings is often not known a priori• Multi-objective method seeks Pareto optimal solutions• The C-GOLDSTEIN function is easily split into its Nconstituents6    22 , 1, ,ˆiiiis x Sf x i N Multi-objective Optimisation• Evolutionary programming and Genetic Algorithms are ideal for multi-objective methods– Maintain a population of solutions which “evolve” over generations of the algorithm– Such methods can capture Pareto optimal solutions• Seek designs of high quality that are evenly distributed and widely spread in the objective space• The NSGA-II algorithm is popular in the literature– The goal function used to drive the GA is based on relative ranking and spacing of the designs7Multiobjective Optimisation + Surrogates8GENIE ModelOptimizationOptimal Value~10 hours per iterationGENIE ModelOptimizationOptimal ResultExtensive NSGA search~1 hour for all iterationseSurrogate ModelOptimal ValueCan be extremely expensive(time, computer resources)Parallel updates The use of surrogate models with the OptionsNSGA2 algorithm can reduce, by an order of magnitude, the total number of simulation years required for a high quality result in the calibration of a GENIE model. This approach provides surrogate models of the underlying problem which can be extensively searched at significantly less cost than the true expensive functions.Response Surface Modelling - Kriging• Kriging is a curve-fitting technique that originated in the field of geological surveying• This method has been found to work very well for a wide range of multi-objective problems• However, there is a computational cost to building the Krig models of the underlying functions• The curvature of each Krig is controlled by a set of hyper-parameters that must themselves be tuned (optimised) to provide the best fit of the surface to the sampled data• This is achieved by maximising a concentrated likelihood function (CLF) over a set of sampled data points• The evaluations of the CLF involve the inversion of a matrix of correlation measures (an O(N3) operation) and consequently the tuning of the Krig can incur significant computational expense9Optimisation Workflow1. Initial sampling of the underlying function (LPt)2. Tune the hyper-parameters of the Krigmetamodel for each objective using the best training data available. High Performance Computing resource targeted for thisHigh Performance Computing3. Extensive NSGA-II searches of surrogate models4. Select update points– Points from the Pareto front– Random points (escape from local minima)– Points from a small secondary NSGA-ii– Points of greatest Expected Improvement– Points of greatest RMS error in the Krigs5. Evaluate the update pointsHigh Throughput Computing6. Add the results to the existing data pool7. Choose the best points in terms of closeness to the last Pareto front and separation in objective space8. Rank the pool of function evaluations and extract the Pareto front9. Return to 2CondorHPC ClusterHPC Cluster (concurrent tuning of the hyper-parameters of NOBJ krigs)Submit UpdateEvaluations... GENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIE... ... ......NSGA-II Searchover NOBJ Krigs(Pareto Front)NSGA-II Searchover NOBJ Krigs(Max. Exp. Imp.)NSGA-II Searchover NOBJ Krigs(Max. Error)ConstructKrigTuneKrigConstructKrigTuneKrigBeginDesign ofExperimentsCondor... GENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIEGENIE... ... ...Build DataCacheConverged / Max. Updates?ParetoOptimalResultsNO YESOptionsNSGA2_RSMSoftware• Geodise Compute Toolbox– Grid access from the Desktop– Matlab and Jython interfaces– Globus and Condor support• Geodise Database Toolbox– Associate metadata with data– Programmatic and GUI access• OptionsMatlab– Engineering Design Optimisation– Suite of multi-dimensional optimisation algorithms• OptionsNSGA2– Multi-objective optimisation package– Augmented implementation of NSGA-II– Supplied courtesy of Rolls-Royce, PLC11JavaClient CodeApache AxisCoGJythonFunctionsGlobus ServerGeodise Database ToolboxMetadata DatabaseClient GridSOAPMatlabFunctionsDatabaseWeb ServicesAuthorisationServiceLocationServiceMetadata Archive & QueryServices JythonXML SchemaGridFTPPortalJavaClient CodeApache AxisCoGJythonFunctionsGlobus Server Globus ServerGeodise Database ToolboxMetadata DatabaseClient GridSOAPMatlabFunctionsDatabaseWeb ServicesAuthorisationServiceLocationServiceMetadata Archive & QueryServices DatabaseWeb ServicesAuthorisationServiceLocationServiceMetadata Archive & QueryServices Jython JythonXML SchemaGridFTPPortal PortalGrid Computation12InstitutionalResources (GT2)UK National Grid Service (GT2)Oxford LeedsRAL ManchesterMatlabGeodise Java APICondorNativeGlobusGT2Java CoG.m filesCondor PoolCCS Gateway GRAM GatewayMicrosoft ComputeCluster ServerSSHJython.py filesJava ClientCondorWeb ServiceMatlab PSE Scripting• OptionsNSGA2_RSM requires the user to provide two Matlab function pairs– Submission and post-processing functions for the hyper-parameter tuning process– Submission and post-processing functions for managing the GENIE simulations• Users are free to target the most appropriate resource for their problem13function retrievalID=krigtune_ccs(i,USERDATA)% Configure the location of the CCS clusterGT2HOST='ccsglobusgateway.soton.ac.uk';GT2DIR =['/home/andrew/tuning/',num2str(i)];...try% Create a remote directory and transfer filesgd_makedir(GT2HOST,GT2DIR);gd_putfile(GT2HOST,'tune.zip',[GT2DIR,'/tune.zip'],'binary');...% Write the input training structure to disk and transfer...gd_putfile(GT2HOST,[jobid '/input.dat'],[GT2DIR,'/input.dat']);% Write the RSL string and submit the compute jobrslstr=['&(executable=' GT2DIR '/tuneHP.bat' ')'...'(directory=' GT2DIR ')'...'(stdout=' GT2DIR '/gt2stdout.txt)'...'(stderr=' GT2DIR '/gt2stderr.txt)'...'(count=1)'...'(jobType=single)'...'(maxWallTime=' num2str(60) ')'];handle=gd_jobsubmit(rslstr,[GT2HOST,'/jobmanager-ccs']);% Return job handleretrievalID.handle=handle;catchretrievalID.handle='failed to submit';endfunction eval=krigtune_ccs_parse2(rID)while true,% Poll job statusstatus=gd_jobstatus(rID.handle);% Handle failures...% Process if job completeif status==3,gd_getfile(rID.GT2HOST,[rID.GT2DIR '/hyperDHC.dat'], ...[rID.jobid '/hyperDHC.dat']);% Load the tuned hyper parametershyperDHC=dat2struct([rID.jobid '/hyperDHC.dat']);eval.OBJHYPER=hyperDHC.OBJHYPER;end;pause(checkfrequency);end% Clean up remote resourcegd_rmuniquedir(rID.GT2HOST,rID.GT2DIR);Condor DAGMan• 4,000 simulated model years required ~5 - 7 hours CPU time on the range of resource in the Condor pool• Compute tasks of this duration at high risk of pre-emption, suspension and eviction– Throughput adversely affected• University of Southampton pool exclusively Win32 machines– Native Condor check-pointing not available• Use Condor DAGMan to manage simulations through a linear series of checkpoints and restarts14Condor DAGMan• Matlab scripts auto-generate the Condor Directed Acyclic Graph (DAG) for a given number of checkpoints• Condor DAGMan manages the submission of the DAG of compute tasks to Condor• Pre-processing scripts manage the staging of the output files to the following task• Some fault tolerance is provided through retries of failed tasks.15# Auto-generated DAG fileJob A stageA.condorJob B stageB.condorJob C stageC.condorJob D stageD.condorJob E stageE.condorJob F stageF.condorJob G stageG.condorJob H stageH.condorJob I stageI.condorJob J stageJ.condorScript PRE B PRE_mv_restart.sh BScript PRE C PRE_mv_restart.sh CScript PRE D PRE_mv_restart.sh DScript PRE E PRE_mv_restart.sh EScript PRE F PRE_mv_restart.sh FScript PRE G PRE_mv_restart.sh GScript PRE H PRE_mv_restart.sh HScript PRE I PRE_mv_restart.sh IScript PRE J PRE_mv_restart.sh JPARENT A CHILD BPARENT B CHILD CPARENT C CHILD DPARENT D CHILD EPARENT E CHILD FPARENT F CHILD GPARENT G CHILD HPARENT H CHILD IPARENT I CHILD JRetry A 3Retry B 1Retry C 1Retry D 1Retry E 1Retry F 1Retry G 1Retry H 1Retry I 1Retry J 1Compute TasksDirected Acyclic Graph (DAG)Fault ToleranceTaskPre-ProcessingCondor Pool Usage16University of Southampton Condor pool usage in April 2008. Three OptionsNSGA2 multi-objective optimisation processes were running concurrently from the middle of the month. The management of the short individual compute tasks by Condor DAGMan keeps the optimisations in  phase.Multi-objective Optimisation Results17Results of the multi-objective optimisation. Top right triangle of plots shows the function evaluations projected onto 2D objective space foreach pair of objectives. Two points from the Pareto front are highlighted (A,B) which have a similar score by a “traditional” single objective measure but exhibit significantly different behaviour in the Atlantic  Meridional Overturning Circulation (AMOC).Optimisation Results18Plots of the latitudinally averaged RMS residuals for the ocean temperature and salinity profiles for the default and tuned parameters sets compared to the target observational data.Conclusions• Multi-objective optimisation– Avoids the need for a single weighted composite objective– Surrogate modelling significantly reduces number of expensive objective function evaluations• Grid computing– OptionsNSGA2 implemented in Matlab Problem Solving Environment– Geodise software provides an interface to the Computational Grid– Tailor the demands of the calculation to the most appropriate resource– Concurrent executions of the expensive model code performed using High Throughput Computing– Condor DAGMan used to manage each simulation through a series of checkpoints and restarts– RSM hyper-parameter tuning process targeted at High Performance Computing resource19The GENIE Team20Principal Investigator – GENIEfy (2005-2008): Tim Lenton – UEA NorwichResearch Team and Collaborators:James Annan  – FRSGC, JapanChris Armstrong – ManchesterChris Brockwell  – UEA NorwichDavid Cameron  – CEH EdinburghPeter Cox  – Hadley Centre (UKMO)Neil Edwards  – Open UniversitySudipta Goswami  – UEA NorwichRobin Hankin – NOC SouthamptonJulia Hargreaves  – FRSGC, JapanPhil Harris  – CEH WallingfordZhuoan Jiao – Southampton e-Science CentreMartin Johnson – UEA NorwichEleftheria Katsiri – London e-Science CentreValerie Livina – UEA NorwichDan Lunt  – BristolRichard Myerscough – NOC SouthamptonSofia Panagiotidi  – London e-Science CentreAndrew Price  – Southampton e-Science CentreAndy Ridgwell  – BristolIan Rutt  – BristolGethin Williams – BristolMark Williamson – UEA NorwichGang Xue  – Southampton e-Science CentreAndrew Yool – NOC SouthamptonPrincipal Investigator – GENIE (2002-2005):Paul Valdes  – BristolCo-Investigators / Management team:Peter Challenor – NOC SouthamptonTrevor Cooper-Chadwick  – Southampton e-Science CentreSimon Cox  – Southampton e-Science CentreJohn Darlington  – London e-Science CentreRupert Ford – ManchesterEric Guilyardi – ReadingJohn Gurd – ManchesterRichard Harding  – CEH WallingfordRobert Marsh – NOC SouthamptonTony Payne  – BristolGraham Riley – ManchesterJohn Shepherd  – NOC SouthamptonRachel Warren – UEA NorwichAndrew Watson  – UEA Norwich

Mutli-objective optimisation of GENIE Earth system models

Abstract

Similar works

Full text

Available Versions

Southampton (e-Prints Soton)