Distributed Particle Filters for Data Assimilation in Simulation of Large Scale Spatial Temporal Systems

Assimilating real time sensor into a running simulation model can improve simulation results for simulating large-scale spatial temporal systems such as wildfire, road traffic and flood. Particle filters are important methods to support data assimilation. While particle filters can work effectively with sophisticated simulation models, they have high computation cost due to the large number of particles needed in order to converge to the true system state. This is especially true for large-scale spatial temporal simulation systems that have high dimensional state space and high computation cost by themselves. To address the performance issue of particle filter-based data assimilation, this dissertation developed distributed particle filters and applied them to large-scale spatial temporal systems. We first implemented a particle filter-based data assimilation framework and carried out data assimilation to estimate system state and model parameters based on an application of wildfire spread simulation. We then developed advanced particle routing methods in distributed particle filters to route particles among the Processing Units (PUs) after resampling in effective and efficient manners. In particular, for distributed particle filters with centralized resampling, we developed two routing policies named minimal transfer particle routing policy and maximal balance particle routing policy. For distributed PF with decentralized resampling, we developed a hybrid particle routing approach that combines the global routing with the local routing to take advantage of both. The developed routing policies are evaluated from the aspects of communication cost and data assimilation accuracy based on the application of data assimilation for large-scale wildfire spread simulations. Moreover, as cloud computing is gaining more and more popularity; we developed a parallel and distributed particle filter based on Hadoop & MapReduce to support large-scale data assimilation

2. PHYSICAL PARAMETERIZATIONS IN MODELS 1

OF WGNE/GMPP ACTIVITIES

Eddy-Permitting Ocean Circulation Hindcasts of Past Decades

International audienc

The WWRP Polar Prediction Project (PPP)

Mission statement: “Promote cooperative international research enabling development of improved weather and environmental prediction services for the polar regions, on time scales from hours to seasonal”. Increased economic, transportation and research activities in polar regions are leading to more demands for sustained and improved availability of predictive weather and climate information to support decision-making. However, partly as a result of a strong emphasis of previous international efforts on lower and middle latitudes, many gaps in weather, sub-seasonal and seasonal forecasting in polar regions hamper reliable decision making in the Arctic, Antarctic and possibly the middle latitudes as well. In order to advance polar prediction capabilities, the WWRP Polar Prediction Project (PPP) has been established as one of three THORPEX (THe Observing System Research and Predictability EXperiment) legacy activities. The aim of PPP, a ten year endeavour (2013-2022), is to promote cooperative international research enabling development of improved weather and environmental prediction services for the polar regions, on hourly to seasonal time scales. In order to achieve its goals, PPP will enhance international and interdisciplinary collaboration through the development of strong linkages with related initiatives; strengthen linkages between academia, research institutions and operational forecasting centres; promote interactions and communication between research and stakeholders; and foster education and outreach. Flagship research activities of PPP include sea ice prediction, polar-lower latitude linkages and the Year of Polar Prediction (YOPP) - an intensive observational, coupled modelling, service-oriented research and educational effort in the period mid-2017 to mid-2019

Nowcasting for a high-resolution weather radar network

2010 Fall.Includes bibliographical references.Short-term prediction (nowcasting) of high-impact weather events can lead to significant improvement in warnings and advisories and is of great practical importance. Nowcasting using weather radar reflectivity data has been shown to be particularly useful. The Collaborative Adaptive Sensing of the Atmosphere (CASA) radar network provides high-resolution reflectivity data amenable to producing valuable nowcasts. The high-resolution nature of CASA data requires the use of an efficient nowcasting approach, which necessitated the development of the Dynamic Adaptive Radar Tracking of Storms (DARTS) and sinc kernel-based advection nowcasting methodology. This methodology was implemented operationally in the CASA Distributed Collaborative Adaptive Sensing (DCAS) system in a robust and efficient manner necessitated by the high-resolution nature of CASA data and distributed nature of the environment in which the nowcasting system operates. Nowcasts up to 10 min to support emergency manager decision-making and 1-5 min to steer the CASA radar nodes to better observe the advecting storm patterns for forecasters and researchers are currently provided by this system. Results of nowcasting performance during the 2009 CASA IP experiment are presented. Additionally, currently state-of-the-art scale-based filtering methods were adapted and evaluated for use in the CASA DCAS to provide a scale-based analysis of nowcasting. DARTS was also incorporated in the Weather Support to Deicing Decision Making system to provide more accurate and efficient snow water equivalent nowcasts for aircraft deicing decision support relative to the radar-based nowcasting method currently used in the operational system. Results of an evaluation using data collected from 2007-2008 by the Weather Service Radar-1988 Doppler (WSR-88D) located near Denver, Colorado, and the National Center for Atmospheric Research Marshall Test Site near Boulder, Colorado, are presented. DARTS was also used to study the short-term predictability of precipitation patterns depicted by high-resolution reflectivity data observed at microalpha (0.2-2 km) to mesobeta (20-200 km) scales by the CASA radar network. Additionally, DARTS was used to investigate the performance of nowcasting rainfall fields derived from specific differential phase estimates, which have been shown to provide more accurate and robust rainfall estimates compared to those made from radar reflectivity data

Tropospheric scintillation and attenuation on satellite-to-Earth links at Ka and Q band: modeling, validation and experimental applications

Link budget is a crucial step during the design of every communication system. For this reason it is fundamental to identify and estimate the effects of the atmosphere on the electromagnetic signal along the path from the source to the sink. Troposphere represent the bigger source of attenuation and scintillation for signals in the microwave and upper frequency spectrum. During last years we have participated in the European Space Agency “AlphaSat Aldo Paraboni” experimental campaigns to acquire up to date propagation data at two frequencies of interest for future communication systems. We realized two high performance low-noise receiver located in Rome, one at Ka and one at Q band (19.701 and 39.402 GHz) to detect the two signal beacons sent from the AlphaSat geostationary satellite to a wide area over Europe. Collected data from Rome receiving station have been analysed to measure excess attenuation and scintillation along the path. Such statistics collected in a database together with data from other experimenter will be in the near future a useful instrument, giving professionals updated data for their custom application design. Classical link budget techniques rely on climatological atmospheric statistics based on different time-scales, usually data collected for several years. In the background of the European Space Agency “STEAM” project, we proposed the use of high resolution 3D weather forecast models (up to 166m pixel resolution) for the calculation of excess attenuation and tropospheric scintillation for satellite to earth link. As a result, the estimation of these electromagnetic parameters to use in link budgets could be given no more as a statistical analysis of past events as in the case of Internation Telecommunication Union recommendation but as time-series forecast specific for the selected receiving station and along the slant path of the transmitted signal. Case studies for the use of this technique have been deeply analysed and results compared with data from the AlphaSat measurement campaign for the Rome and Spino d’Adda receiving station, confirming the validity even in different geographical regions. In everyday situations, propagation models based on statistics are often replaced by the use of easier to apply parametric models. Those have the advantage of the simplicity and the need of less input parameter to be applied. In particular, for what concerning the tropospheric scintillation, the Hufnagel-Valley refractive index structure constant (C2n ) parametric model is actually the most used, due to the simplicity and the relative accuracy. We here propose a new Cn2 polynomial parametric model (CPP) based just on the altitude z and a function C2 n0(to,RH0) that allow to calculate the ground refractive index structure constant just using the ground temperature (T0) and the relative humidity (RH0). In this work CPP and Hufnagel-Valley models are applied to different location around the globe to prove their accuracy. The obtained model could be also used in the future to realize a simulator able to generate random C2n vertical profiles specific for the receiver site

Advancements in mesoscale ensemble prediction strategies: Application to Mediterranean high-impact weather

[cat] La predictibilitat d'esdeveniments d'alt impacte a la regi o Mediterr ania ha millorat substancialment al llarg de les darreres d ecades. No obstant aix o, una representaci o precisa d'aspectes dels sistemes convectius rellevants per la societat, tals com el moment en qu e es produeixen, i la seva localitzaci o i intensitat encara suposen un repte. Aquestes febleses de la predicci o a escala convectiva provenen d'imprecisions a l'estimaci o de l'estat atmosf eric inicial, la formulaci o de processos f sics rellevants i la natura ca otica dels sistema associada a la seva no linealitat. En el marc probabil stic imposat per les incerteses intr nseques implicades en la predicci o num erica del temps, l'entitat matem atica que quanti ca la incertesa en l'estat atmosf eric es la funci o densitat de probabilitat. Malgrat aix o, el c alcul de la seva evoluci o temporal es inviable per situacions realistes amb els recursos computacionals disponibles actualment. La modesta aproximaci o habitual per estimar aquesta evoluci o es l' us d'un discret i petit nombre de mostres de l'estat del sistema, que es coneix com a predicci o per conjunts (ensemble forecasting). L'objectiu general d'aquesta Tesi es entendre millor els l mits de la predictibilitat i contribuir a una millora de la predicci o de temps sever a la regi o Mediterr ania. En primer lloc, s'avalua l'evoluci o temporal de les funcions densitat de probabilitat per sistemes de baixa complexitat amb un cert grau de realisme adoptant el formalisme de Liouville. En segon lloc, es dissenya una estrat egia de mostreig per crear pertorbacions a les condicions inicials per abastos de predicci o curts (24-36 h). La t ecnica es basa en el m etode de breeding, que utilitza la din amica completa no lineal per identi car modes de creixement r apid. La modi caci o proposada est a dirigida a ajustar l'escala de les pertorbacions per tal de cobrir l'ample rang d'escales rellevants per la predicci o de curt abast. En tercer lloc, s'investiga el potencial de varis m etodes per tenir en compte la incertesa en el model per a un episodi recent de precipitacions intenses i inundacions que va oc orrer al llarg de la costa Mediterr ania espanyola (12-13 setembre de 2019). S'avaluen m ultiples estrat egies estoc astiques en front l'aproximaci o ordin aria de multif sica en termes de diversitat i habilitat de l'ensemble. Les t ecniques considerades inclouen pertorbacions estoc astiques a les tend encies f siques i pertorbacions a par ametres in uents de l'esquema de microf sica. Finalment, aquestes estrat egies de generaci o d'ensembles s'utilitzen com a for cament meteorol ogic per a un model hidrol ogic per tal d'investigar la predictibilitat 21 22 CONTENTS hidrometeorol ogica de l'episodi del 12-13 setembre de 2019. Les t ecniques desenvolupades, juntament amb l'assimilaci o de dades mitjan cant Ensemble Kalman Filter es comparen amb altres estrat egies populars, tals com el downscaling d'un model global i l'aproximaci o de multif sica. Els resultats d'aquesta Tesi s on rellevants des d'una perspectiva te orica, ja que la soluci o de l'equaci o de Liouville revela estructures complexes per la funci o densitat de probabilitat que podrien comprometre les hip otesis de compacitat i suavitat assumides per la majoria d'eines d'interpretaci o i post proc es d'ensembles. Per altra banda, les estrat egies de generaci o d'ensembles desenvolupades mostren potencial per millorar la predicci o d'esdeveniments d'alt impacte, que es demostra per una major diversitat i habilitat dels ensembles comparades amb les estrat egies de refer encia. Aquests resultats prometedors posen les bases per un sistema avan cat d'alertes a la regi o Mediterr ania per encarar els esdeveniments de temps sever.[spa] La predictibilidad de eventos de alto impacto en la regi on Mediterr anea ha mejorado sustancialmente a lo largo de las ultimas d ecadas. No obstante, una representaci on precisa de aspectos relevantes de los sistemas convectivos relevantes para la sociedad, como el momento en el que se producen, su localizaci on e intensidad a un suponen un reto. Estas debilidades de la predicci on a escala convectiva provienen de imprecisiones en la estimaci on del estado atmosf erico inicial, la formulaci on de los procesos f sicos relevantes y la naturaleza ca otica del sistema asociada a su no linealidad. En el marco probabilista impuesto por las incertidumbres intr nsecas implicadas en la predicci on num erica del tiempo, la entidad matem atica que cuanti ca la incertidumbre en el estado atmosf erico inicial es la funci on densidad de probabilidad. Sin embargo, el c alculo de su evoluci on temporal es inviable para situaciones realistas con los recursos computacionales disponibles actualmente. La modesta aproximaci on habitual para estimar esta evoluci on en el uso de un discreto y peque~no n umero de muestras del estado del sistema, lo que se conoce como predicci on por conjuntos (ensemble forecasting). El objetivo general de esta Tesis es entender mejor los l mites de la predictibilidad y contribuir a una mejora de la predicci on del tiempo severo en la regi on Mediterr anea. En primer lugar, se eval ua la evoluci on temporal de las funciones densidad de probabilidad para sistemas de baja complejidad con un cierto grado de realismo adoptando el formalismo te orico de Liouville. En segundo lugar, se dise~na una estrategia de muestreo para crear perturbaciones en les condiciones iniciales para alcances de predicci on cortos (24-36 h). La t ecnica se basa en el m etodo de breeding, que utiliza la din amica completa no lineal para identi car modos de crecimiento r apido. La modi caci on propuesta est a dirigida a ajustar la escala de las perturbaciones para cubrir el amplio rango de escalas relevantes para la predicci on de corto alcance. En tercer lugar, se investiga el potencial de varios m etodos para tener en cuenta la incertidumbre en el modelo para un episodio reciente de precipitaciones intensas e inundaciones que ocurri o a lo largo de la costa Mediterr anea espa~nola (12-13 de septiembre de 2019). Se eval uan m ultiples estrategias estoc asticas frente a la aproximaci on ordinaria de multif sica en t erminos de diversidad y habilidad del ensemble. Las t ecnicas consideradas incluyen perturbaciones estoc asticas en las tendencias f sicas y perturbaciones en par ametros in uyentes del esquema de microf sica. Finalmente, estas estrategias de generaci on de ensembles se usan como forzamiento meteorol ogico para un modelo hidrol ogico con el n de investigar la predictibilidad hidrometeorol ogica del episodio del 12-13 de septiembre de 2019. Las t ecnicas desarrolladas, junto a la asimilaci on de datos mediante Ensemble Kalman Filter se comparan con otras estrategias populares, como el dowscaling de un modelo global y la aproximaci on de multif sica. Los resultados de esta Tesis son relevantes desde una perspectiva te orica, ya que la soluci on de la ecuaci on de Liouville revela estructuras complejas para la funci on densidad de probabilidad que podr an comprometer las hip otesis de compacidad y suavidad asumidas por la mayor a de herramientas de interpretaci on y pos proceso de ensembles. Por otro lado, las estrategias de generaci on de ensembles desarrolladas muestran potencial para mejorar la predicci on de eventos de alto impacto, que se demuestra por una mayor diversidad y habilidad de los ensembles comparadas con las estrategias de referencia. Estos resultados prometedores sientan las bases para un sistema avanzado de alertas en la regi on Mediterr anea para afrontar los eventos de tiempo severo.[eng] The predictability of meteorological high-impact events in the Mediterranean region has substantially improved over the last decades. Nevertheless, a precise representation of socially relevant aspects of convective systems, such as their timing, location, and intensity is still challenging. These weaknesses of convective-scale forecasting stem from inaccuracies in the estimation of the atmospheric initial state, formulation of relevant physical processes, and the chaotic nature of the system associated with its nonlinearity. In the probabilistic framework imposed by the intrinsic uncertainties involved in numerical weather prediction, the mathematical entity that quanti es the uncertainty in the atmospheric state is the probability density function. However, the computation of its time evolution is unfeasible for realistic situations with the current available computational resources. The usual modest approach to estimate this evolution is the use of a discrete and small number of samples of the state of the system, which is known as ensemble forecasting. The general aim of this Thesis is to better understand the predictability limits and contribute towards the improvement of severe weather forecasting in the Mediterranean region. Firstly, the time evolution of probability density functions for low complexity systems with a certain degree of realism is evaluated by adopting the Liouville formalism. Secondly, a sampling strategy to create initial condition perturbations for the short-range (24-36 h) is designed. The technique is based on the breeding method, which uses the full nonlinear dynamics to identify fast-growing modes. The proposed modi cation is aimed at tailoring the scale of the perturbations in order to cover the wide range of scales relevant for short-range forecasting. Thirdly, the potential of several methods to account for model uncertainty is investigated for a recent heavy precipitation and ash ood episode occurred along the Spanish Mediterranean coast (12-13 September 2019). Multiple stochastic strategies are evaluated against the ordinary multiphysics approach in terms of ensemble diversity and skill. The considered techniques include stochastically perturbed physics tendencies and perturbations to in uential parameters within the microphysics scheme. Finally, these ensemble generation strategies are used as the meteorological forcing for a hydrological model in order to investigate the hydrometeorological predictability of the 12-13 September 2019 episode. The developed techniques, along with data assimilation by means of Ensemble Kalman Filter are compared to other popular strategies, such as the downscaling from a global model and the multiphysics approach. The results of this Thesis are relevant from a theoretical perspective, as the solution of the Liouville equation reveals complex structures for the probability density function that could compromise the hypothesis of compactness and smoothness assumed by most current ensemble interpretation and postprocessing tools. Conversely, the ensemble generation strategies developed show potential to improve the forecasting of high-impact events, proven by higher ensemble diversity and skill compared to the benchmark strategies. These encouraging results lay the foundations for an advanced warning system in the Mediterranean region to deal with severe weather events

Improving meteorological information to air transport

Meteorological information and services supporting the various operations of air transport enable a safe, efficient and cost-effective operating environment for airspace users, air navigation service providers and air traffic management. The continuing pursuit towards an improved quality of observation, forecasting and decision support services is driven by an increasingly weather-sensitive society and growing impacts of hazardous weather events. This thesis provides an overview of the field of aeronautical meteorological research by introducing the organisations involved, global and regional strategies, impacts of weather on air transport, current state of the art in meteorological research and decision support systems serving air transport needs with a view of where the field should evolve next. This thesis is an attempt to highlight key findings and point the reader towards the direction of further research on the given topics. Research supporting air transport operations with the optimal use of weather information is a specialized field where advances are led by the needs of various airspace users. Research institutions for example in the United States have contributed greatly due to the severe weather impacts experienced by the National Airspace System (NAS), the ability of the Federal Aviation Administration (FAA) and the National Oceanic and Atmospheric Administration (NOAA) to direct long-term funding to solve specific aviation-related research questions. The creation and maintenance of long-lived teams of scientists and engineers working together to produce end-to-end solutions that meet the needs of the aviation industry is the key to improving meteorological information to aviation users while university research is typically shorter duration and typical does not result in operational systems. From a global perspective, research is yet to be organised in a way that would contribute to solving aviation issues beyond single research projects and/or programmes. There is a lot more the scientific community could do to develop tailored information to decision support systems used by the aviation sector, but it would require systematic investments and the establishment of research groups focusing on the applied science questions and technology transfer. This thesis provides an overview of recommended decision support system development topics with an outline of potential milestones.Tieto ilmakehän nykyisestä ja tulevasta tilasta sekä tätä tietoa ilmailun tarpeisiin tuottavat palvelut mahdollistavat turvallisen, toimivan sekä kustannustehokkaan toimintaympäristön ilmatilan käyttäjille, ilmailun palveluiden tuottajille sekä ilmatilan hallintaa toteuttaville tahoille. Vaarallisille sääilmiöille herkemmäksi kehittyvä yhteiskunta vaatii havaintojen, ennusteiden sekä päätöksenteon tukijärjestelmien jatkuvaa kehittämistä asiakkaiden tarpeisiin. Tämä lisensiaatintutkielma tarjoaa maailmanlaajuisen yleiskatsauksen ilmailun sääpalveluiden tutkimukseen ja tuotekehitykseen pyrkimyksenään esitellä keskeiset toimijat, alueelliset ja kansalliset kehittämisohjelmat ja strategiat, sään vaikutukset ilmailulle, ilmailun sääpalveluiden nykytila sekä tulevaisuuden toimintaympäristön edellyttämät uudet lentosääpalvelut. Tavoitteena on korostaa ilmailun kannalta tärkeimpiä meteorologisia kehityskohteita ja ohjata lukija jo tehdyn tutkimuksen pariin. Ilmailun toimintoja tukevien sääpalveluiden kehittämiseen tähtäävä tutkimus on hyvin soveltava erikoisala, missä asiakkaiden tarpeet määrittävät tutkimuskohteet. Kehitys on keskittynyt voimakkaasti Yhdysvaltoihin, mihin on syynä kapasiteetin äärirajoilla toimiva ilmatila sekä kyky rahoittaa pitkäkestoisia meteorologisia tutkimushankkeita ilmailun tarpeisiin. Meteorologian tutkijoiden ja insinöörien pitkäkestoinen yhteistyö tuottaa koko arvoketjun kattavia projekteja, joiden lopputuloksena syntyy asiakkaan tarpeisiin räätälöityjä palveluita hyödyntäen yliopistoissa tehtävää tutkimusta sekä tietoteknisten ratkaisujen kehittymistä. Maailmanlaajuisesti katsottuna ilmailun sääpalveluiden tutkimusta ja tuotekehitystä ei ole toistaiseksi järjestetty yhtenäisen strategian tai tavoitteiden alle. Tieteellinen yhteisö pystyisi kasvattamaan merkittävästi panostaan ilmailun turvallisuuden kehittämiseksi, mikäli tuotekehityksen rahoitus organisoitaisiin paremmin ja osaaminen keskitettäisiin soveltavan tutkimuksen ryhmiin. Tämä tutkielma sisältää suosituksia päätöksenteon tukijärjestelmiin integroitavista sääpalveluista, joiden avulla säätilan vaikutus lentotoiminnalle voidaan viedä suoraan päätöksentekotasolle. Tutkielmassa esitettyjen projektiaihioiden tarkoituksena esittää konkreettisia toimenpiteitä, joilla varmistutaan tutkimuksen soveltuvuudesta loppukäyttäjien toimintaan