Advanced analysis and visualisation techniques for atmospheric data

Atmospheric science is the study of a large, complex system which is becoming increasingly important to understand. There are many climate models which aim to contribute to that understanding by computational simulation of the atmosphere. To generate these models, and to confirm the accuracy of their outputs, requires the collection of large amounts of data. These data are typically gathered during campaigns lasting a few weeks, during which various sources of measurements are used. Some are ground based, others airborne sondes, but one of the primary sources is from measurement instruments on board aircraft. Flight planning for the numerous sorties is based on pre-determined goals with unpredictable influences, such as weather patterns, and the results of some limited analyses of data from previous sorties. There is little scope for adjusting the flight parameters during the sortie based on the data received due to the large volumes of data and difficulty in processing the data online. The introduction of unmanned aircraft with extended flight durations also requires a team of mission scientists with the added complications of disseminating observations between shifts. Earth’s atmosphere is a non-linear system, whereas the data gathered is sampled at discrete temporal and spatial intervals introducing a source of variance. Clustering data provides a convenient way of grouping similar data while also acknowledging that, for each discrete sample, a minor shift in time and/ or space could produce a range of values which lie within its cluster region. This thesis puts forward a set of requirements to enable the presentation of cluster analyses to the mission scientist in a convenient and functional manner. This will enable in-flight decision making as well as rapid feedback for future flight planning. Current state of the art clustering algorithms are analysed and a solution to all of the proposed requirements is not found. New clustering algorithms are developed to achieve these goals. These novel clustering algorithms are brought together, along with other visualization techniques, into a software package which is used to demonstrate how the analyses can provide information to mission scientists in flight. The ability to carry out offline analyses on historical data, whether to reproduce the online analyses of the current sortie, or to provide comparative analyses from previous missions, is also demonstrated. Methods for offline analyses of historical data prior to continuing the analyses in an online manner are also considered. The original contributions in this thesis are the development of five new clustering algorithms which address key challenges: speed and accuracy for typical hyper-elliptical offline clustering; speed and accuracy for offline arbitrarily shaped clusters; online dynamic and evolving clustering for arbitrary shaped clusters; transitions between offline and online techniques and also the application of these techniques to atmospheric science data analysis

On the dynamics of persistent states and their secular trends in the waveguides of the Southern Hemisphere troposphere

We identify the dynamical drivers of systematic changes in persistent quasi-stationary states (regimes) of the Southern Hemisphere troposphere and their secular trends. We apply a purely data-driven approach, whereby a multiscale approximation to nonstationary dynamical processes is achieved through optimal sequences of locally stationary fast vector autoregressive factor processes, to examine a high resolution atmospheric reanalysis over the period encompassing 1958–2013. This approach identifies regimes and their secular trends in terms of the predictability of the flow and is Granger causal. A comprehensive set of diagnostics on both isentropic and isobaric surfaces is employed to examine teleconnections over the full hemisphere and for a set of regional domains. Composite states for the hemisphere obtained from nonstationary nonparametric cluster analysis reveal patterns consistent with a circumglobal wave 3 (polar)–wave 5 (subtropical) pattern, while regional composites reveal the Pacific South American pattern and blocking modes. The respective roles of potential vorticity sources, stationary Rossby waves and baroclinic instability on the dynamics of these circulation modes are shown to be reflected by the seasonal variations of the waveguides, where Rossby wave sources and baroclinic disturbances are largely contained within the waveguides and with little direct evidence of sustained remote tropical influences on persistent synoptic features. Warm surface temperature anomalies are strongly connected with regions of upper level divergence and anticyclonic Rossby wave sources. The persistent states identified reveal significant variability on interannual to decadal time scales with large secular trends identified in all sectors apart from a region close to South America

Design and validation of novel methods for long-term road traffic forecasting

132 p.Road traffic management is a critical aspect for the design and planning of complex urban transport networks for which vehicle flow forecasting is an essential component. As a testimony of its paramount relevance in transport planning and logistics, thousands of scientific research works have covered the traffic forecasting topic during the last 50 years. In the beginning most approaches relied on autoregressive models and other analysis methods suited for time series data. During the last two decades, the development of new technology, platforms and techniques for massive data processing under the Big Data umbrella, the availability of data from multiple sources fostered by the Open Data philosophy and an ever-growing need of decision makers for accurate traffic predictions have shifted the spotlight to data-driven procedures. Even in this convenient context, with abundance of open data to experiment and advanced techniques to exploit them, most predictive models reported in literature aim for shortterm forecasts, and their performance degrades when the prediction horizon is increased. Long-termforecasting strategies are more scarce, and commonly based on the detection and assignment to patterns. These approaches can perform reasonably well unless an unexpected event provokes non predictable changes, or if the allocation to a pattern is inaccurate.The main core of the work in this Thesis has revolved around datadriven traffic forecasting, ultimately pursuing long-term forecasts. This has broadly entailed a deep analysis and understanding of the state of the art, and dealing with incompleteness of data, among other lesser issues. Besides, the second part of this dissertation presents an application outlook of the developed techniques, providing methods and unexpected insights of the local impact of traffic in pollution. The obtained results reveal that the impact of vehicular emissions on the pollution levels is overshadowe

Design and validation of novel methods for long-term road traffic forecasting

132 p.Road traffic management is a critical aspect for the design and planning of complex urban transport networks for which vehicle flow forecasting is an essential component. As a testimony of its paramount relevance in transport planning and logistics, thousands of scientific research works have covered the traffic forecasting topic during the last 50 years. In the beginning most approaches relied on autoregressive models and other analysis methods suited for time series data. During the last two decades, the development of new technology, platforms and techniques for massive data processing under the Big Data umbrella, the availability of data from multiple sources fostered by the Open Data philosophy and an ever-growing need of decision makers for accurate traffic predictions have shifted the spotlight to data-driven procedures. Even in this convenient context, with abundance of open data to experiment and advanced techniques to exploit them, most predictive models reported in literature aim for shortterm forecasts, and their performance degrades when the prediction horizon is increased. Long-termforecasting strategies are more scarce, and commonly based on the detection and assignment to patterns. These approaches can perform reasonably well unless an unexpected event provokes non predictable changes, or if the allocation to a pattern is inaccurate.The main core of the work in this Thesis has revolved around datadriven traffic forecasting, ultimately pursuing long-term forecasts. This has broadly entailed a deep analysis and understanding of the state of the art, and dealing with incompleteness of data, among other lesser issues. Besides, the second part of this dissertation presents an application outlook of the developed techniques, providing methods and unexpected insights of the local impact of traffic in pollution. The obtained results reveal that the impact of vehicular emissions on the pollution levels is overshadowe

Interdisciplinary application of nonlinear time series methods

This paper reports on the application to field measurements of time series methods developed on the basis of the theory of deterministic chaos. The major difficulties are pointed out that arise when the data cannot be assumed to be purely deterministic and the potential that remains in this situation is discussed. For signals with weakly nonlinear structure, the presence of nonlinearity in a general sense has to be inferred statistically. The paper reviews the relevant methods and discusses the implications for deterministic modeling. Most field measurements yield nonstationary time series, which poses a severe problem for their analysis. Recent progress in the detection and understanding of nonstationarity is reported. If a clear signature of approximate determinism is found, the notions of phase space, attractors, invariant manifolds etc. provide a convenient framework for time series analysis. Although the results have to be interpreted with great care, superior performance can be achieved for typical signal processing tasks. In particular, prediction and filtering of signals are discussed, as well as the classification of system states by means of time series recordings.Comment: 86 pages, 26 figure

Automated OMA and Damage Detection: an Opportunity for Smart SHM Systems

Il monitoraggio strutturale si propone di sviluppare sistemi che siano in grado di monitorare un’opera permettendone l’ispezione ed il rilevamento dei danni con il minimo intervento antropico. Esso rappresenta dunque un processo di implementazione di una strategia di identificazione dei danni attraverso la quale, osservando una struttura con una determinata periodicità, è possibile pervenire alla valutazione di alcune caratteristiche del sistema, in modo tale da definirne il suo stato attuale di salute. La sua prima applicazione ha interessato campi differenti da quello dell’ingegneria civile. Infatti, era una tecnica impiegata prevalentemente in ambito meccanico, aeronautico e nell’ingegneria aerospaziale. Successivamente è risultato evidente come fosse una strategia che, se correttamente adattata, poteva essere di grande aiuto per il controllo di tutte le strutture civili presenti sul territorio. Può essere sviluppata come un sistema autonomo integrato su grandi infrastrutture (come ponti, dighe, ecc.) con l’obiettivo di monitorare la risposta della struttura sotto delle sollecitazioni durante la costruzione per modificare i progetti se necessario. Sempre su questo tipo di strutture, può mantenere costante il controllo durante il suo arco di vita per attuare tempestivamente interventi prima che possano crearsi situazioni irreversibili e pericolose. Inoltre, il monitoraggio strutturale può essere impiegato, come verrà mostrato in questa tesi, per il monitoraggio della salute strutturale di edifici storici, appartenenti al patrimonio culturale. Questo patrimonio è diffuso in Europa ed in particolare in Italia. Si parla di edifici anche di notevole entità (come grandi chiese), costruite in epoche molto lontane, la cui conservazione è oggi il focus di molti ricercatori. Il caso studio oggetto del presente lavoro è la torre campanaria di una chiesa nelle Marche, in Italia, le cui tracce storiche risalgono a circa il 1330, con numerosi interventi e rimaneggiamenti successivi fino ad arrivare ai giorni nostri. Con la tecnica del monitoraggio strutturale è possibile valutare il comportamento dinamico della struttura (monitorata) attraverso l’identificazione dei suoi principali parametri modali a partire dall’analisi dei dati acquisiti. Con l’espressione “identificazione dinamica” di una struttura si intendono tutte quelle tecniche, sia analitiche che sperimentali, attraverso le quali è possibile appunto individuare la risposta dinamica della struttura stessa andando ad estrapolare frequenze naturali, corrispondenti forme modali e coefficienti di smorzamento. Soffermandoci su questo concetto, il lavoro è stato inizialmente impostato implementando un processo automatico per l’elaborazione dei dati in uscita dal monitoraggio e quindi per la definizione dei parametri modali della struttura sotto esame depurandoli dagli effetti delle azioni ambientali quali temperatura, velocità media del vento ed umidità. Rimuovere questi fattori esterni dai risultati, permette di ottenere una previsione dell’evoluzione delle caratteristiche modali. Dal loro confronto con il comportamento reale della struttura si possono evidenziare eventuali anomalie. In riferimento a quanto appena detto, risulta oggigiorno di importanza fondamentale riuscire a riconoscere e prevedere il progressivo deterioramento di una struttura. Questo può avvenire per naturale degrado dei materiali, o dopo aver subito vibrazioni impreviste (terremoti, esplosioni, ecc). Focalizzandosi su questo concetto, l’identificazione del danno basata sulla valutazione delle variazioni dei parametri modali, depurati dell’influenza di agenti esterni, può essere un processo lungo. Al contrario, poter conoscere quasi istantaneamente il "nuovo" comportamento della costruzione, è un aspetto da tenere bene in considerazione sia per salvaguardare la vita delle persone, sia per attuare in maniera precisa e puntuale interventi di miglioramento. Essere tempestivi in caso di situazioni critiche, permette di evitare di arrivare a condizioni tali per cui è necessario interrompere l’operatività della struttura per molto tempo pur di poterla recuperare, se possibile. Nel presente lavoro si sono utilizzati metodi basati sull’elaborazione diretta dei dati acquisiti dal sistema di monitoraggio in continuo. In questo modo l’onere computazionale è stato notevolmente ridotto. Non è stato necessario elaborare i dati acquisiti per estrarre le caratteristiche modali del sistema e si è ottenuto un feedback quasi istantaneo di variazioni nel comportamento dinamico. Tali aspetti, risultano significativi quando l’obiettivo è di sviluppare un "monitoraggio sostenibile" sia da un punto di vista economico che di tempistiche.Structural monitoring aims to develop systems that can monitor a structure by allowing its inspection and detection of damage with minimal human intervention. It therefore represents a process of implementing a damage identification strategy through which, by observing a structure with a certain periodicity, it is possible to arrive at an evaluation of specific characteristics of the system, to define its current state of health. Its first application was in fields other than civil engineering. It later became evident that it was a strategy that could be of great help in the control of all civil structures in the area. It can be developed as an autonomous integrated system on big infrastructures with the aim of monitoring the response of the structure under stress during construction to modify designs if necessary. Also, on these types of structures, it can maintain constant control during its life cycle to implement timely interventions. In addition, structural monitoring can be used to monitor the structural health of historical buildings belonging to cultural heritage. We are talking about buildings even of considerable size, built in very distant epochs, the conservation of which is now the focus of many researchers. The case study that is the subject of the present work is the bell tower of a church in the Marche region, Italy, whose historical traces date back to around 1330, with numerous subsequent interventions and remodelling up to the present day. With the technique of structural monitoring, it is possible to assess the dynamic behaviour of the structure through the identification of its main modal. The term 'dynamic identification' of a structure means all those techniques, both analytical and experimental, through which it is possible to identify the dynamic response of the structure itself by extrapolating natural frequencies, corresponding modal shapes and damping coefficients. The work was initially set up by implementing an automatic procedure for processing the output data from the monitoring and thus defining the modal parameters of the structure under examination purifying them from the effects of environmental actions. By removing these external factors from the results, a prediction of the evolution of the modal characteristics can be obtained. Comparing them with the actual behaviour of the structure, any anomalies can be highlighted. Regarding the above, it is nowadays of fundamental importance, to be able to recognise and predict the progressive failure of a structure. This may occur due to natural degradation of materials, or because of unexpected vibrations. Focusing on this concept, the identification of damage based on the evaluation of changes in modal parameters, purified of the influence of external agents, can be a time-consuming process. On the contrary, being able to know almost instantaneously the "new" behaviour of the construction, is an aspect that must be taken into well consideration both for safeguarding people's lives and for the accurate and timely implementation of improvements. In the present work, methods based on the direct processing of data acquired from the continuous monitoring system were used. In this way, the computational burden was reduced. It was not necessary to elaborate the acquired data to extract the modal characteristics of the system, and almost instantaneous feedback of changes in dynamic behaviour was obtained. These aspects are significant when the purpose is to develop "sustainable monitoring" from both an economic and timing perspective