Interpolation and Extrapolation of Toeplitz Matrices via Optimal Mass Transport

In this work, we propose a novel method for quantifying distances between Toeplitz structured covariance matrices. By exploiting the spectral representation of Toeplitz matrices, the proposed distance measure is defined based on an optimal mass transport problem in the spectral domain. This may then be interpreted in the covariance domain, suggesting a natural way of interpolating and extrapolating Toeplitz matrices, such that the positive semi-definiteness and the Toeplitz structure of these matrices are preserved. The proposed distance measure is also shown to be contractive with respect to both additive and multiplicative noise, and thereby allows for a quantification of the decreased distance between signals when these are corrupted by noise. Finally, we illustrate how this approach can be used for several applications in signal processing. In particular, we consider interpolation and extrapolation of Toeplitz matrices, as well as clustering problems and tracking of slowly varying stochastic processes

Rainfall Nowcasting by Blending of Radar Data and Numerical Weather Prediction

In order to improve conventional rainfall nowcasting, radar extrapolation and high-resolution numerical weather prediction (NWP) were blended to get a 6-h quantitative precipitation forecast (QPF) over the Yangtze River Delta region of China. Modifications and calibrations were done to both the extrapolation and NWP in order to get an integrated result from the two, which mainly included the extension for the extrapolation time and region, intensity and position calibration for the NWP, weighted blending of extrapolation and NWP based on scale and time, and a final real-time Z-R relation conversion. Forecast experiments were done, and results show that the blending technique could effectively extend forecast time compared with conventional radar extrapolation, meanwhile applying a positive calibration to the NWP. The overall CSI score of 0–6 h reflectivity forecast was better than either single forecast

Vision technology/algorithms for space robotics applications

The thrust of automation and robotics for space applications has been proposed for increased productivity, improved reliability, increased flexibility, higher safety, and for the performance of automating time-consuming tasks, increasing productivity/performance of crew-accomplished tasks, and performing tasks beyond the capability of the crew. This paper provides a review of efforts currently in progress in the area of robotic vision. Both systems and algorithms are discussed. The evolution of future vision/sensing is projected to include the fusion of multisensors ranging from microwave to optical with multimode capability to include position, attitude, recognition, and motion parameters. The key feature of the overall system design will be small size and weight, fast signal processing, robust algorithms, and accurate parameter determination. These aspects of vision/sensing are also discussed

RAPIDS: Early Warning System for Urban Flooding and Water Quality Hazards

Machine Learning in Water Systems symposium: part of AISB Annual Convention 2013, University of Exeter, UK, 3-5 April 2013Convention organised by the Society for the Study of Artificial Intelligence and Simulation of Behaviour (AISB), www.aisb.org.uk/This paper describes the application of Artificial Neural Networks (ANNs) as Data Driven Models (DDMs) to predict urban flooding in real-time based on weather radar and/or raingauge rainfall data. A time-lagged ANN is configured for prediction of flooding at sewerage nodes and outfalls based on input parameters including rainfall. In the absence of observed flood data, a hydrodynamic simulator may be used to predict flooding surcharge levels at nodes of interest in sewer networks and thus provide the target data for training and testing the ANN. The model, once trained, acts as a rapid surrogate for the hydrodynamic simulator and can thus be used as part of an urban flooding Early Warning System (EWS). Predicted rainfall over the catchment is required as input, to extend prediction times to operationally useful levels. Both flood-level analogue and flood-severity classification schemes are implemented. An initial case study using Keighley, W Yorks, UK demonstrated proof-of-concept. Three further case studies for UK cities of different sizes explore issues of soil-moisture, early operation of pumps as flood-mitigation/prevention strategy and spatially variable rainfall. We investigate the use of ANNs for nowcasting of rainfall based on the relationship between radar data and recorded rainfall history; a feature extraction scheme is described. This would allow the two ANNs to be cascaded to predict flooding in real-time based on current weather radar Quantitative Precipitation Estimates (QPE). We also briefly describe the extension of this methodology to Bathing Water Quality (BWQ) prediction

Improving the rainfall nowcast for fine temporal and spatial scales suitable for urban hydrology

Accurate Quantitative Precipitation Forecasts (QPF) at high spatial and temporal resolution are crucial for urban flood prediction. Typically, Lagrangian persistence based on radar data is used to nowcast rainfall intensities with up to 3 hours lead time, but nevertheless is not able to deliver reliable QPFs past 20 min lead time (known as well as the predictability limit). Especially, for extreme events causing pluvial floods, accurate QPFs cannot be achieved past 5 min lead time. Furthermore when compared to gauge recordings, the QPFs are not useful at all. There is an essential need to provide better QPFs by improving the rainfall field supplied to the nowcast and by employing non-linear processes for the extrapolation of rainfall into the future. This study is focused on these two main problems, and it investigates different geostatistical and data-driven methods for the improvement of the QPFs at fine scales. The study was conducted within the Hannover radar range where observations between 2000 to 2018 were available. The skill of the nowcast models was assessed on the point (1 km2 and 5 min) and storm scale, based on continuous criteria comparing both radar and gauge observations. A total of 100 gauge measurements inside the study area were as well employed for the assessment. From the period 2000-2012, 93 events of different properties were distinguished and used as a basis for the method development and assessment. Two state-of-the-art nowcast models (HyRaTrac and Lucas-Kanade) were chosen as reference and used as benchmarks for improvement. To improve the rainfall field, a real time merging between radar and gauge data was investigated. Among different merging techniques (mean field bias, quantile bias correction and kriging interpolation), conditional merging (CM) yielded the best rainfall field. When fed to the reference nowcast models, it led to improvements of up to 1 hour of the predictability limit and of the agreement between radar based QPFs and gauge data. To improve the QPF accuracy even further, two different data driven techniques were developed in order to learn non-linear behaviours from past observed rainfall. First, a nearest neighbour approach (k-NN) was developed and employed instead of Lagrangian Persistence on the HyRaTrac nowcast model. The k-NN method accounts for the non-linearity of the storm evolution by consulting k-similar past storms. A deterministic nowcast issued by averaging the behaviours from the 3 most similar storms yielded the best results, extending the predictability limit at the storm scale to 2-3 hours. Second, an ensemble nowcast accounting for the 10 closest neighbours was generated in order to estimate the uncertainty of the QPF. Third, a deep convolution neural network (CNN) was trained on past merged data, in order to learn the non-linearity of the rainfall process. The network based on the last 15 min of observed radar images proved to successfully capture death and decay and partly birth processes, and extended the rainfall predictability limit at the point scale to 3 hours. Lastly, the methods were tested on 17 convective extreme events, extracted from the period 2013-2018, to compare the tested methods for an urban flood nowcast application. The CNN based on merged data outperformed both reference methods as well as the k-NN based nowcast, with the predictability limit reaching 30 – 40 min. The k-NN, although better than the Lagrangian persistence, suffered greatly from the shortcomings of the storm tracking algorithm present under fast moving and extreme storms. To conclude, even though clear improvements were achieved, there is a clear limit to the data-driven methods that cannot be overcome, unless coupled with the convection initialization from Numerical Weather Prediction (NWP) models. Nevertheless, complex relationships learned from past observed data, together with a better rainfall field as input, were proven to be useful in increasing the QPF accuracy and predictability limits for urban hydrology application.Quantitative Niederschlagsvorhersagen (QPF) in hoher räumlicher und zeitlicher Auflösung sind entscheidend für die Prognose urbaner Sturzfluten. Der auf Radardaten basierende Lagrange Ansatz wird typischerweise für Regenintensitätsvorhersagen mit einem Horizont von 3 Stunden verwendet. Zuverlässig ist dieser allerdings nur bis 20 Minuten (bekanntes Prognoselimit). Bei extremen Niederschlagsereignissen, die urbane Sturzfluten verursachen, ist das Limit sogar bereits bei 5 Minuten erreicht. Außerdem kommt es zu deutlichen Abweichungen zwischen der QPF und den Messdaten an Niederschlagsstationen. Eine Verbesserung der QPF ist demnach zwingend erforderlich. Eine solche Verbesserung kann durch die Anpassung des Eingabe-Niederschlagsfeldes und durch die Anwendung nichtlinearer Prozesse für die Extrapolation des Niederschlags erreicht werden. Die vorliegende Studie konzentriert sich auf diese beiden Hauptprobleme und untersucht verschiedene geostatistische und Data-Mining Methoden zur Verbesserung der QPF auf solchen Skalen. Die Studie wurde im Radarbereich von Hannover durchgeführt, wo Beobachtungsdaten von 2000 bis 2018 verfügbar sind. Die Güte der Nowcast-Modelle wurde auf der Punkteskala (1 km2 und 5 min.) anhand kontinuierlicher Kriterien evaluiert und in Relation zu Radar- und Stationsbeobachtungen gesetzt. Hierfür wurden insgesamt 100 Stationsmessungen innerhalb des Untersuchungsgebietes verwendet. Aus dem Zeitraum 2000 bis 2012 wurden 93 Ereignisse mit unterschiedlichen Eigenschaften als Grundlage für die Methodenentwicklung und -beurteilung ausgewertet. Zwei gängige Nowcast-Modelle (HyRaTrac und Lucas-Kanade) wurden als Referenzmodelle ausgewählt und als Maßstab für Verbesserungen eingesetzt. Um das Niederschlagsfeld zu verbessern, wurden Radar- und Stationsdaten in Echtzeit zusammengeführt. Unter den verschiedenen Methoden (Mean Field Bias, Quantile Mapping Bias, Kriging-Interpolation) ergab das Conditional Merging (CM) das optimalste Niederschlagsfeld. Als Input für die beiden Referenzmodelle verwendet, führte das CM zu einer Verlängerung des Prognoselimits auf bis zu eine Stunde. Auch die Übereinstimmung der radargestützten QPF mit den Stationsdaten verbesserte sich. Um das Prognoselimit noch weiter auszudehnen, wurden zwei verschiedene Data-Mining Techniken entwickelt, um die nichtlinearen Verhaltensweisen aus vergangenen Regenfällen zu erlernen: Zunächst wurde ein Nächster-Nachbar-Ansatz (k-NN) entwickelt und anstelle der Lagrange Persistenz im HyRaTrac-Nowcast-Modell eingesetzt. Die k-NN-Methode berücksichtigt die Nichtlinearität der Regensturmentwicklung, indem k-ähnliche vergangene Stürme herangezogen werden. Ein deterministischer Nowcast, der durch Mittelwertbildung der Verhaltensweisen der drei ähnlichsten Stürme erstellt wurde, lieferte die besten Ergebnisse und verlängerte das Prognoselimit auf bis zu zwei-drei Stunden. Ein Ensemble-Nowcast, bei dem die zehn nächsten Nachbarn berücksichtigt wurden, wurde ebenfalls erstellt, um die Unsicherheit des QPF abzuschätzen. Zudem wurde ein künstliches neuronales Netz (CNN) basierend auf vergangenen Daten entwickelt, um die Nichtlinearität des Niederschlagsprozesses zu berücksichtigen. Das neuronale Netz, das mit den beobachteten Radarbildern der letzten 15 Minuten gefüttert wurde, erwies sich als erfolgreich in der Erfassung von Todes-, Zerfalls- und Geburtsprozessen von Stürmen und konnte das Prognoselimit auf bis zu drei Stunden erweitern. Um die Wirksamkeit der entwickelten Methoden für die Vorhersage urbaner Sturzfluten zu untersuchen, wurden sie auf 17 konvektive Extremereignisse aus dem Zeitraum 2013 bis 2018 angewendet. Der k-NN Ansatz war zwar besser als die Lagrange Persistenz, litt aber stark unter den Fehlern des Sturmverfolgungs-Algorithmus bei schnellen und extremen Stürmen. Das CNN übertraf sowohl die Referenzmethoden als auch den k-NN-basierten Nowcast. Das Prognoselimit konnte so von 5 auf 30 bis 40 Minuten erweitert werden. Für eine weitere Verbesserung zeichnete sich letztlich eine klare Grenze ab, die nur mit der Konvektionsinitialisierung aus Numerischen Wettervorhersagemodellen (NWP-Modellen) überwunden werden kann. Im Vergleich mit den ausgewählten Referenzmodellen, können, durch die hier entwickelten Methoden, die Genauigkeit und das Prognoselimit der QPF in der städtischen Hydrologie erheblich verbessert werden