2,913 research outputs found
Bayesian spline-based hidden Markov models with applications to actimetry data and sleep analysis
B-spline-based hidden Markov models employ B-splines to specify the emission distributions, offering a more flexible modeling approach to data than conventional parametric HMMs. We introduce a Bayesian framework for inference, enabling the simultaneous estimation of all unknown model parameters including the number of states. A parsimonious knot configuration of the B-splines is identified by the use of a trans-dimensional Markov chain sampling algorithm, while model selection regarding the number of states can be performed based on the marginal likelihood within a parallel sampling framework. Using extensive simulation studies, we demonstrate the superiority of our methodology over alternative approaches as well as its robustness and scalability. We illustrate the explorative use of our methods for data on activity in animals, that is whitetip-sharks. The flexibility of our Bayesian approach also facilitates the incorporation of more realistic assumptions and we demonstrate this by developing a novel hierarchical conditional HMM to analyse human activity for circadian and sleep modeling. Supplementary materials for this article are available online
UMSL Bulletin 2023-2024
The 2023-2024 Bulletin and Course Catalog for the University of Missouri St. Louis.https://irl.umsl.edu/bulletin/1088/thumbnail.jp
UMSL Bulletin 2022-2023
The 2022-2023 Bulletin and Course Catalog for the University of Missouri St. Louis.https://irl.umsl.edu/bulletin/1087/thumbnail.jp
Excess risk bound for deep learning under weak dependence
This paper considers deep neural networks for learning weakly dependent
processes in a general framework that includes, for instance, regression
estimation, time series prediction, time series classification. The -weak
dependence structure considered is quite large and covers other conditions such
as mixing, association, Firstly, the approximation of smooth functions
by deep neural networks with a broad class of activation functions is
considered. We derive the required depth, width and sparsity of a deep neural
network to approximate any H\"{o}lder smooth function, defined on any compact
set \mx. Secondly, we establish a bound of the excess risk for the learning
of weakly dependent observations by deep neural networks. When the target
function is sufficiently smooth, this bound is close to the usual
Learning and Control of Dynamical Systems
Despite the remarkable success of machine learning in various domains in recent years, our understanding of its fundamental limitations remains incomplete. This knowledge gap poses a grand challenge when deploying machine learning methods in critical decision-making tasks, where incorrect decisions can have catastrophic consequences. To effectively utilize these learning-based methods in such contexts, it is crucial to explicitly characterize their performance. Over the years, significant research efforts have been dedicated to learning and control of dynamical systems where the underlying dynamics are unknown or only partially known a priori, and must be inferred from collected data. However, much of these classical results have focused on asymptotic guarantees, providing limited insights into the amount of data required to achieve desired control performance while satisfying operational constraints such as safety and stability, especially in the presence of statistical noise.
In this thesis, we study the statistical complexity of learning and control of unknown dynamical systems. By utilizing recent advances in statistical learning theory, high-dimensional statistics, and control theoretic tools, we aim to establish a fundamental understanding of the number of samples required to achieve desired (i) accuracy in learning the unknown dynamics, (ii) performance in the control of the underlying system, and (iii) satisfaction of the operational constraints such as safety and stability. We provide finite-sample guarantees for these objectives and propose efficient learning and control algorithms that achieve the desired performance at these statistical limits in various dynamical systems. Our investigation covers a broad range of dynamical systems, starting from fully observable linear dynamical systems to partially observable linear dynamical systems, and ultimately, nonlinear systems.
We deploy our learning and control algorithms in various adaptive control tasks in real-world control systems and demonstrate their strong empirical performance along with their learning, robustness, and stability guarantees. In particular, we implement one of our proposed methods, Fourier Adaptive Learning and Control (FALCON), on an experimental aerodynamic testbed under extreme turbulent flow dynamics in a wind tunnel. The results show that FALCON achieves state-of-the-art stabilization performance and consistently outperforms conventional and other learning-based methods by at least 37%, despite using 8 times less data. The superior performance of FALCON arises from its physically and theoretically accurate modeling of the underlying nonlinear turbulent dynamics, which yields rigorous finite-sample learning and performance guarantees. These findings underscore the importance of characterizing the statistical complexity of learning and control of unknown dynamical systems.</p
Limit Theory under Network Dependence and Nonstationarity
These lecture notes represent supplementary material for a short course on
time series econometrics and network econometrics. We give emphasis on limit
theory for time series regression models as well as the use of the
local-to-unity parametrization when modeling time series nonstationarity.
Moreover, we present various non-asymptotic theory results for moderate
deviation principles when considering the eigenvalues of covariance matrices as
well as asymptotics for unit root moderate deviations in nonstationary
autoregressive processes. Although not all applications from the literature are
covered we also discuss some open problems in the time series and network
econometrics literature.Comment: arXiv admin note: text overlap with arXiv:1705.08413 by other author
Essays on Portfolio Risk Management and Weather Derivatives
Denne avhandlingen handler om utvikling og praktisk implementering av risikostyringsmetoder for investeringsporteføljer, energiporteføljer, og håndtering av vær- og forurensningsrisiko. Avhandlingen inkluderer tre vitenskapelige artikler som hver tar for seg ulike aspekter av finansiell risikostyring. Den første fokuserer på metoder for aktivaallokering når det eksisterer asymmetrisk avhengighet mellom avkastningene for eiendelene i en investeringsportefølje. Den andre artikkelen omhandler energiprisrisikostyring, og introduserer et åpen kildekodeverktøy for energiporteføljeforvaltning som er utviklet som en del av doktorgradsprosjektet. Den siste artikkelen presenterer et teoretisk rammeverk for håndtering av forurensningsrisiko ved hjelp av finansielle derivatkontrakter, som bygger på den eksisterende teorien om værderivater. Disse arbeidene bidrar alle til det overordnede temaet for avhandlingen, som er utvikling av risikostyringsmetoder for ulike typer porteføljer og utforskingen av rollen til finansielle derivater i håndtering av risiko knyttet til markedspriser, vær og forurensning.
For å sette bidragene inn i en teoretisk kontekst har vi inkludert et kort kapittel som presenterer alternative metoder for avhengighetsmodellering, og hvordan disse kan utnyttes når man forvalter investeringsporteføljer. Ett av disse målene, lokal gaussisk korrelasjon, brukes til å utvide det klassiske mean-variance-rammeverket for aktivaallokering i den første artikkelen. Deretter følger et kort introduksjonskapittel til spot- og forwardmarkeder for energi. Hovedfokuset her er råvareprisrisiko, og hvordan denne kan håndteres med finansielle derivatkontrakter. Vi demonstrerer hvordan forvaltning av energiporteføljer kan gjennomføres med vårt åpen kildekodeverktøy ved bruk av data fra det europeiske kraftmarkedet. Til slutt inkluderes et kapittel om værderivater. Dette inneholder en introduksjon til værrelatert risiko, en kort introduksjon til værmarkedet, vanlige kontraktstyper og alternative metoder for prising.
For å sikre reproduserbarhet har vi også lagt til et kapittel om programkode. Her finnes lenker til Git-repositorier med alle data og R-kode for å gjennomføre analysene som presenteres i avhandlingen.This thesis is concerned with the development and practical implementation of risk management methods for investment portfolios, energy portfolios, and weather and pollution risk. The thesis includes three scientific papers that each address different aspects of financial risk management. The first paper focuses on portfolio allocation in the presence of asymmetric dependence between asset returns. The second paper examines energy price risk management, and introduces an open source toolkit for energy portfolio management which has been developed as a part of the PhD project. The final paper present a theoretical framework for managing pollution risk using financial derivatives contracts, which builds upon the existing theory of weather derivatives. These papers all contribute to the overall theme, which is the development of risk management methods for various types of portfolios and the exploration of the role of financial derivatives in managing risks related to market prices, weather and pollution.
In order to provide a theoretical context, we have included a brief chapter exploring alternative methods for dependence modelling and how these may be utilized when managing investment portfolios. One of these measures, the local Gaussian correlation, is used to extend the classical mean-variance framework for asset allocation in the first paper. Thereafter, a short introduction to spot and forward energy markets is provided. The primary focus here is commodity market price risk, and how this can be managed with financial derivatives contracts. We demonstrate how portfolio management may be performed with our open source toolkit using European energy market data. Finally, we include a chapter on weather derivatives. This contains a introduction to weather related risk, a brief introduction to the weather markets, frequently used contract types and pricing methods.
To ensure reproducibility, we have also added a chapter on computer code, where the interested reader may find links to Git repositories with all data and the R code needed to run the analysis presented in the thesis.Doktorgradsavhandlin
Study of Climate Variability Patterns at Different Scales – A Complex Network Approach
Das Klimasystem der Erde besteht aus zahlreichen interagierenden Teilsystemen, die sich über verschiedene Zeitskalen hinweg verändern, was zu einer äußerst komplizierten räumlich-zeitlichen Klimavariabilität führt. Das Verständnis von Prozessen, die auf verschiedenen räumlichen und zeitlichen Skalen ablaufen, ist ein entscheidender Aspekt bei der numerischen Wettervorhersage. Die Variabilität des Klimas, ein sich selbst konstituierendes System, scheint in Mustern auf großen Skalen organisiert zu sein. Die Verwendung von Klimanetzwerken hat sich als erfolgreicher Ansatz für die Erkennung der räumlichen Ausbreitung dieser großräumigen Muster in der Variabilität des Klimasystems erwiesen.
In dieser Arbeit wird mit Hilfe von Klimanetzwerken gezeigt, dass die Klimavariabilität nicht nur auf größeren Skalen (Asiatischer Sommermonsun, El Niño/Southern Oscillation), sondern auch auf kleineren Skalen, z.B. auf Wetterzeitskalen, in Mustern organisiert ist. Dies findet Anwendung bei der Erkennung einzelner tropischer Wirbelstürme, bei der Charakterisierung binärer Wirbelsturm-Interaktionen, die zu einer vollständigen Verschmelzung führen, und bei der Untersuchung der intrasaisonalen und interannuellen Variabilität des Asiatischen Sommermonsuns.
Schließlich wird die Anwendbarkeit von Klimanetzwerken zur Analyse von Vorhersagefehlern demonstriert, was für die Verbesserung von Vorhersagen von immenser Bedeutung ist. Da korrelierte Fehler durch vorhersagbare Beziehungen zwischen Fehlern verschiedener Regionen aufgrund von zugrunde liegenden systematischen oder zufälligen Prozessen auftreten können, wird gezeigt, dass Fehler-Netzwerke helfen können, die räumlich kohärenten Strukturen von Vorhersagefehlern zu untersuchen. Die Analyse der Fehler-Netzwerk-Topologie von Klimavariablen liefert ein erstes Verständnis der vorherrschenden Fehlerquelle und veranschaulicht das Potenzial von Klimanetzwerken als vielversprechendes Diagnoseinstrument zur Untersuchung von Fehlerkorrelationen.The Earth’s climate system consists of numerous interacting subsystems varying over a multitude of time scales giving rise to highly complicated spatio-temporal climate variability. Understanding processes occurring at different scales, both spatial and temporal, has been a very crucial problem in numerical weather prediction. The variability of climate, a self-constituting system, appears to be organized in patterns on large scales. The climate networks approach has been very successful in detecting the spatial propagation of these large scale patterns of variability in the climate system.
In this thesis, it is demonstrated using climate network approach that climate variability is organized in patterns not only at larger scales (Asian Summer Monsoon, El Niño-Southern Oscillation) but also at shorter scales, e.g., weather time scales. This finds application in detecting individual tropical cyclones, characterizing binary cyclone interaction leading to a complete merger, and studying the intraseasonal and interannual variability of the Asian Summer Monsoon.
Finally, the applicability of the climate network framework to understand forecast error properties is demonstrated, which is crucial for improvement of forecasts. As correlated errors can arise due to the presence of a predictable relationship between errors of different regions because of some underlying systematic or random process, it is shown that error networks can help to analyze the spatially coherent structures of forecast errors. The analysis of the error network topology of a climate variable provides a preliminary understanding of the dominant source of error, which shows the potential of climate networks as a very promising diagnostic tool to study error correlations
A Modified EM Algorithm for Shrinkage Estimation in Multivariate Hidden Markov Models
Τα κρυμμένα Μαρκοβιανά μοντέλα χρησιμοποιούνται σε ένα ευρύ πεδίο εφαρμογών, λόγω της κατασκευής
τους που τα καθιστά μαθηματικώς διαχειρίσιμα και επιτρέπει τη χρήση αποτελεσματικών υπολογιστικών
τεχνικών. ́Εχουν αναπτυχθεί μέθοδοι για την εκτίμηση των παραμέτρων του μοντέλου, όπως ο αλγόριθμος
EM, αλλά και για την εύρεση των κρυμμένων καταστάσεων της Μαρκοβιανής αλυσίδας, όπως ο αλγόριθμος
Viterbi.
Σε εφαρμογές στις οποίες η διάσταση των δεδομένων είναι συγκρίσιμη με το μέγεθος του δέιγματος,
είναι γνωστό πως ο δειγματικός πίνακας συνδιακύμανσης είναι αριθμητικά ασταθής, γεγονός που επηρεάζει
άμεσα το βήμα μεγιστοποίησης (M-step) του αλγορίθμου EM, στο οποίο εμπλέκεται ο υπολογισμός του
αντιστρόφου του. Το πρόβλημα αυτό μπορεί να ενταθεί λόγω ενδεχόμενης ύπαρξης καταστάσεων οι οποίες
εμφανίζονται σπάνια, με αποτέλεσμα το μέγεθος δείγματος για την εκτίμηση των αντίστοιχων παραμέτρων
να είναι μικρό. Επομένως, η άμεση χρήση αυτών των μεθόδων είναι πιθανό να οδηγήσει σε αριθμητικά προβ-
λήματα, όσον αφορά στην εκτίμηση του πίνακα συνδιακύμανσης και του αντιστρόφου του, επηρεάζοντας
επιπλέον την εκτίμηση του πίνακα πιθανοτήτων μετάβασης και την ανακατασκευή της κρυμμένης Μαρκο-
βιανής αλυσίδας.
Στη συγκεκριμένη εργασία μελετάται θεωρητικά και αλγοριθμικά μία τροποποίηση του αλγορίθμου EM,
έτσι ώστε ο εκτιμήτης που προκύπτει για τον πίνακα συνδιακύμανσης, κατά το βήμα μεγιστοποίησης, να
είναι αυτός που απορρέει από τη χρήση της μεθόδου συρρίκνωσης (shrinkage). Για τον σκοπό αυτό, στη
συνάρτηση της λογαριθμικής πιθανοφάνειας ενσωματώνονται κάποιες ποινές, ώστε να κανονικοποιηθεί το
αντίστοιχο πρόβλημα μεγιστοποίησης. Η συνάρτηση αυτή, χρησιμοποιείται και στο βήμα εκτίμησης (E-step).
Επίσης, μελετάται αλγοριθμικά και μία παραλλαγή αυτής της μεθόδου, στην οποία η συνάρτηση με τις ποινές
χρησιμοποιείται μόνο κατά το βήμα μεγιστοποίησης (M-step).Hidden Markov models are used in a wide range of applications due to their construction that
renders them mathematically tractable and allows for the use of efficient computational techniques.
There are methods for the estimation of the model’s parameters, such as the EM algorithm, but also
for the estimation of the hidden states of the underlying Markov chain, such as the Viterbi algorithm.
In applications where the dimension of the data is comparable to the sample size, the sample
covariance matrix is known to be ill-conditioned, which directly affects the maximisation step (M-
step) of the EM algorithm, where its inverse is involved in the computations. This problem might be
amplified if there are rarely visited states resulting in a small sample size for the estimation of the
corresponding parameters. Therefore, the direct implementation of these methods can be proved to
be troublesome, as many computational problems might occur in the estimation of the covariance
matrix and its inverse, further affecting the estimation of the one-step transition probability matrix
and the reconstruction of the hidden Markov chain.
In this paper, a modified version of the EM algorithm is studied, both theoretically and computa-
tionally, in order to obtain the shrinkage estimator of the covariance matrix during the maximisation
step. This is achieved by maximising a penalised log-likelihood function, which is also used in the
estimation step (E-step). A variant of this modified version, where the penalised log-likelihood func-
tion is only used in the maximisation step (M-step), is also studied computationally
Discovering Causal Relations and Equations from Data
Physics is a field of science that has traditionally used the scientific
method to answer questions about why natural phenomena occur and to make
testable models that explain the phenomena. Discovering equations, laws and
principles that are invariant, robust and causal explanations of the world has
been fundamental in physical sciences throughout the centuries. Discoveries
emerge from observing the world and, when possible, performing interventional
studies in the system under study. With the advent of big data and the use of
data-driven methods, causal and equation discovery fields have grown and made
progress in computer science, physics, statistics, philosophy, and many applied
fields. All these domains are intertwined and can be used to discover causal
relations, physical laws, and equations from observational data. This paper
reviews the concepts, methods, and relevant works on causal and equation
discovery in the broad field of Physics and outlines the most important
challenges and promising future lines of research. We also provide a taxonomy
for observational causal and equation discovery, point out connections, and
showcase a complete set of case studies in Earth and climate sciences, fluid
dynamics and mechanics, and the neurosciences. This review demonstrates that
discovering fundamental laws and causal relations by observing natural
phenomena is being revolutionised with the efficient exploitation of
observational data, modern machine learning algorithms and the interaction with
domain knowledge. Exciting times are ahead with many challenges and
opportunities to improve our understanding of complex systems.Comment: 137 page
- …