2,913 research outputs found

    Bayesian spline-based hidden Markov models with applications to actimetry data and sleep analysis

    Get PDF
    B-spline-based hidden Markov models employ B-splines to specify the emission distributions, offering a more flexible modeling approach to data than conventional parametric HMMs. We introduce a Bayesian framework for inference, enabling the simultaneous estimation of all unknown model parameters including the number of states. A parsimonious knot configuration of the B-splines is identified by the use of a trans-dimensional Markov chain sampling algorithm, while model selection regarding the number of states can be performed based on the marginal likelihood within a parallel sampling framework. Using extensive simulation studies, we demonstrate the superiority of our methodology over alternative approaches as well as its robustness and scalability. We illustrate the explorative use of our methods for data on activity in animals, that is whitetip-sharks. The flexibility of our Bayesian approach also facilitates the incorporation of more realistic assumptions and we demonstrate this by developing a novel hierarchical conditional HMM to analyse human activity for circadian and sleep modeling. Supplementary materials for this article are available online

    UMSL Bulletin 2023-2024

    Get PDF
    The 2023-2024 Bulletin and Course Catalog for the University of Missouri St. Louis.https://irl.umsl.edu/bulletin/1088/thumbnail.jp

    UMSL Bulletin 2022-2023

    Get PDF
    The 2022-2023 Bulletin and Course Catalog for the University of Missouri St. Louis.https://irl.umsl.edu/bulletin/1087/thumbnail.jp

    Excess risk bound for deep learning under weak dependence

    Full text link
    This paper considers deep neural networks for learning weakly dependent processes in a general framework that includes, for instance, regression estimation, time series prediction, time series classification. The ψ\psi-weak dependence structure considered is quite large and covers other conditions such as mixing, association,\ldots Firstly, the approximation of smooth functions by deep neural networks with a broad class of activation functions is considered. We derive the required depth, width and sparsity of a deep neural network to approximate any H\"{o}lder smooth function, defined on any compact set \mx. Secondly, we establish a bound of the excess risk for the learning of weakly dependent observations by deep neural networks. When the target function is sufficiently smooth, this bound is close to the usual O(n1/2)\mathcal{O}(n^{-1/2})

    Learning and Control of Dynamical Systems

    Get PDF
    Despite the remarkable success of machine learning in various domains in recent years, our understanding of its fundamental limitations remains incomplete. This knowledge gap poses a grand challenge when deploying machine learning methods in critical decision-making tasks, where incorrect decisions can have catastrophic consequences. To effectively utilize these learning-based methods in such contexts, it is crucial to explicitly characterize their performance. Over the years, significant research efforts have been dedicated to learning and control of dynamical systems where the underlying dynamics are unknown or only partially known a priori, and must be inferred from collected data. However, much of these classical results have focused on asymptotic guarantees, providing limited insights into the amount of data required to achieve desired control performance while satisfying operational constraints such as safety and stability, especially in the presence of statistical noise. In this thesis, we study the statistical complexity of learning and control of unknown dynamical systems. By utilizing recent advances in statistical learning theory, high-dimensional statistics, and control theoretic tools, we aim to establish a fundamental understanding of the number of samples required to achieve desired (i) accuracy in learning the unknown dynamics, (ii) performance in the control of the underlying system, and (iii) satisfaction of the operational constraints such as safety and stability. We provide finite-sample guarantees for these objectives and propose efficient learning and control algorithms that achieve the desired performance at these statistical limits in various dynamical systems. Our investigation covers a broad range of dynamical systems, starting from fully observable linear dynamical systems to partially observable linear dynamical systems, and ultimately, nonlinear systems. We deploy our learning and control algorithms in various adaptive control tasks in real-world control systems and demonstrate their strong empirical performance along with their learning, robustness, and stability guarantees. In particular, we implement one of our proposed methods, Fourier Adaptive Learning and Control (FALCON), on an experimental aerodynamic testbed under extreme turbulent flow dynamics in a wind tunnel. The results show that FALCON achieves state-of-the-art stabilization performance and consistently outperforms conventional and other learning-based methods by at least 37%, despite using 8 times less data. The superior performance of FALCON arises from its physically and theoretically accurate modeling of the underlying nonlinear turbulent dynamics, which yields rigorous finite-sample learning and performance guarantees. These findings underscore the importance of characterizing the statistical complexity of learning and control of unknown dynamical systems.</p

    Limit Theory under Network Dependence and Nonstationarity

    Full text link
    These lecture notes represent supplementary material for a short course on time series econometrics and network econometrics. We give emphasis on limit theory for time series regression models as well as the use of the local-to-unity parametrization when modeling time series nonstationarity. Moreover, we present various non-asymptotic theory results for moderate deviation principles when considering the eigenvalues of covariance matrices as well as asymptotics for unit root moderate deviations in nonstationary autoregressive processes. Although not all applications from the literature are covered we also discuss some open problems in the time series and network econometrics literature.Comment: arXiv admin note: text overlap with arXiv:1705.08413 by other author

    Essays on Portfolio Risk Management and Weather Derivatives

    Get PDF
    Denne avhandlingen handler om utvikling og praktisk implementering av risikostyringsmetoder for investeringsporteføljer, energiporteføljer, og håndtering av vær- og forurensningsrisiko. Avhandlingen inkluderer tre vitenskapelige artikler som hver tar for seg ulike aspekter av finansiell risikostyring. Den første fokuserer på metoder for aktivaallokering når det eksisterer asymmetrisk avhengighet mellom avkastningene for eiendelene i en investeringsportefølje. Den andre artikkelen omhandler energiprisrisikostyring, og introduserer et åpen kildekodeverktøy for energiporteføljeforvaltning som er utviklet som en del av doktorgradsprosjektet. Den siste artikkelen presenterer et teoretisk rammeverk for håndtering av forurensningsrisiko ved hjelp av finansielle derivatkontrakter, som bygger på den eksisterende teorien om værderivater. Disse arbeidene bidrar alle til det overordnede temaet for avhandlingen, som er utvikling av risikostyringsmetoder for ulike typer porteføljer og utforskingen av rollen til finansielle derivater i håndtering av risiko knyttet til markedspriser, vær og forurensning. For å sette bidragene inn i en teoretisk kontekst har vi inkludert et kort kapittel som presenterer alternative metoder for avhengighetsmodellering, og hvordan disse kan utnyttes når man forvalter investeringsporteføljer. Ett av disse målene, lokal gaussisk korrelasjon, brukes til å utvide det klassiske mean-variance-rammeverket for aktivaallokering i den første artikkelen. Deretter følger et kort introduksjonskapittel til spot- og forwardmarkeder for energi. Hovedfokuset her er råvareprisrisiko, og hvordan denne kan håndteres med finansielle derivatkontrakter. Vi demonstrerer hvordan forvaltning av energiporteføljer kan gjennomføres med vårt åpen kildekodeverktøy ved bruk av data fra det europeiske kraftmarkedet. Til slutt inkluderes et kapittel om værderivater. Dette inneholder en introduksjon til værrelatert risiko, en kort introduksjon til værmarkedet, vanlige kontraktstyper og alternative metoder for prising. For å sikre reproduserbarhet har vi også lagt til et kapittel om programkode. Her finnes lenker til Git-repositorier med alle data og R-kode for å gjennomføre analysene som presenteres i avhandlingen.This thesis is concerned with the development and practical implementation of risk management methods for investment portfolios, energy portfolios, and weather and pollution risk. The thesis includes three scientific papers that each address different aspects of financial risk management. The first paper focuses on portfolio allocation in the presence of asymmetric dependence between asset returns. The second paper examines energy price risk management, and introduces an open source toolkit for energy portfolio management which has been developed as a part of the PhD project. The final paper present a theoretical framework for managing pollution risk using financial derivatives contracts, which builds upon the existing theory of weather derivatives. These papers all contribute to the overall theme, which is the development of risk management methods for various types of portfolios and the exploration of the role of financial derivatives in managing risks related to market prices, weather and pollution. In order to provide a theoretical context, we have included a brief chapter exploring alternative methods for dependence modelling and how these may be utilized when managing investment portfolios. One of these measures, the local Gaussian correlation, is used to extend the classical mean-variance framework for asset allocation in the first paper. Thereafter, a short introduction to spot and forward energy markets is provided. The primary focus here is commodity market price risk, and how this can be managed with financial derivatives contracts. We demonstrate how portfolio management may be performed with our open source toolkit using European energy market data. Finally, we include a chapter on weather derivatives. This contains a introduction to weather related risk, a brief introduction to the weather markets, frequently used contract types and pricing methods. To ensure reproducibility, we have also added a chapter on computer code, where the interested reader may find links to Git repositories with all data and the R code needed to run the analysis presented in the thesis.Doktorgradsavhandlin

    Study of Climate Variability Patterns at Different Scales – A Complex Network Approach

    Get PDF
    Das Klimasystem der Erde besteht aus zahlreichen interagierenden Teilsystemen, die sich über verschiedene Zeitskalen hinweg verändern, was zu einer äußerst komplizierten räumlich-zeitlichen Klimavariabilität führt. Das Verständnis von Prozessen, die auf verschiedenen räumlichen und zeitlichen Skalen ablaufen, ist ein entscheidender Aspekt bei der numerischen Wettervorhersage. Die Variabilität des Klimas, ein sich selbst konstituierendes System, scheint in Mustern auf großen Skalen organisiert zu sein. Die Verwendung von Klimanetzwerken hat sich als erfolgreicher Ansatz für die Erkennung der räumlichen Ausbreitung dieser großräumigen Muster in der Variabilität des Klimasystems erwiesen. In dieser Arbeit wird mit Hilfe von Klimanetzwerken gezeigt, dass die Klimavariabilität nicht nur auf größeren Skalen (Asiatischer Sommermonsun, El Niño/Southern Oscillation), sondern auch auf kleineren Skalen, z.B. auf Wetterzeitskalen, in Mustern organisiert ist. Dies findet Anwendung bei der Erkennung einzelner tropischer Wirbelstürme, bei der Charakterisierung binärer Wirbelsturm-Interaktionen, die zu einer vollständigen Verschmelzung führen, und bei der Untersuchung der intrasaisonalen und interannuellen Variabilität des Asiatischen Sommermonsuns. Schließlich wird die Anwendbarkeit von Klimanetzwerken zur Analyse von Vorhersagefehlern demonstriert, was für die Verbesserung von Vorhersagen von immenser Bedeutung ist. Da korrelierte Fehler durch vorhersagbare Beziehungen zwischen Fehlern verschiedener Regionen aufgrund von zugrunde liegenden systematischen oder zufälligen Prozessen auftreten können, wird gezeigt, dass Fehler-Netzwerke helfen können, die räumlich kohärenten Strukturen von Vorhersagefehlern zu untersuchen. Die Analyse der Fehler-Netzwerk-Topologie von Klimavariablen liefert ein erstes Verständnis der vorherrschenden Fehlerquelle und veranschaulicht das Potenzial von Klimanetzwerken als vielversprechendes Diagnoseinstrument zur Untersuchung von Fehlerkorrelationen.The Earth’s climate system consists of numerous interacting subsystems varying over a multitude of time scales giving rise to highly complicated spatio-temporal climate variability. Understanding processes occurring at different scales, both spatial and temporal, has been a very crucial problem in numerical weather prediction. The variability of climate, a self-constituting system, appears to be organized in patterns on large scales. The climate networks approach has been very successful in detecting the spatial propagation of these large scale patterns of variability in the climate system. In this thesis, it is demonstrated using climate network approach that climate variability is organized in patterns not only at larger scales (Asian Summer Monsoon, El Niño-Southern Oscillation) but also at shorter scales, e.g., weather time scales. This finds application in detecting individual tropical cyclones, characterizing binary cyclone interaction leading to a complete merger, and studying the intraseasonal and interannual variability of the Asian Summer Monsoon. Finally, the applicability of the climate network framework to understand forecast error properties is demonstrated, which is crucial for improvement of forecasts. As correlated errors can arise due to the presence of a predictable relationship between errors of different regions because of some underlying systematic or random process, it is shown that error networks can help to analyze the spatially coherent structures of forecast errors. The analysis of the error network topology of a climate variable provides a preliminary understanding of the dominant source of error, which shows the potential of climate networks as a very promising diagnostic tool to study error correlations

    A Modified EM Algorithm for Shrinkage Estimation in Multivariate Hidden Markov Models

    Get PDF
    Τα κρυμμένα Μαρκοβιανά μοντέλα χρησιμοποιούνται σε ένα ευρύ πεδίο εφαρμογών, λόγω της κατασκευής τους που τα καθιστά μαθηματικώς διαχειρίσιμα και επιτρέπει τη χρήση αποτελεσματικών υπολογιστικών τεχνικών. ́Εχουν αναπτυχθεί μέθοδοι για την εκτίμηση των παραμέτρων του μοντέλου, όπως ο αλγόριθμος EM, αλλά και για την εύρεση των κρυμμένων καταστάσεων της Μαρκοβιανής αλυσίδας, όπως ο αλγόριθμος Viterbi. Σε εφαρμογές στις οποίες η διάσταση των δεδομένων είναι συγκρίσιμη με το μέγεθος του δέιγματος, είναι γνωστό πως ο δειγματικός πίνακας συνδιακύμανσης είναι αριθμητικά ασταθής, γεγονός που επηρεάζει άμεσα το βήμα μεγιστοποίησης (M-step) του αλγορίθμου EM, στο οποίο εμπλέκεται ο υπολογισμός του αντιστρόφου του. Το πρόβλημα αυτό μπορεί να ενταθεί λόγω ενδεχόμενης ύπαρξης καταστάσεων οι οποίες εμφανίζονται σπάνια, με αποτέλεσμα το μέγεθος δείγματος για την εκτίμηση των αντίστοιχων παραμέτρων να είναι μικρό. Επομένως, η άμεση χρήση αυτών των μεθόδων είναι πιθανό να οδηγήσει σε αριθμητικά προβ- λήματα, όσον αφορά στην εκτίμηση του πίνακα συνδιακύμανσης και του αντιστρόφου του, επηρεάζοντας επιπλέον την εκτίμηση του πίνακα πιθανοτήτων μετάβασης και την ανακατασκευή της κρυμμένης Μαρκο- βιανής αλυσίδας. Στη συγκεκριμένη εργασία μελετάται θεωρητικά και αλγοριθμικά μία τροποποίηση του αλγορίθμου EM, έτσι ώστε ο εκτιμήτης που προκύπτει για τον πίνακα συνδιακύμανσης, κατά το βήμα μεγιστοποίησης, να είναι αυτός που απορρέει από τη χρήση της μεθόδου συρρίκνωσης (shrinkage). Για τον σκοπό αυτό, στη συνάρτηση της λογαριθμικής πιθανοφάνειας ενσωματώνονται κάποιες ποινές, ώστε να κανονικοποιηθεί το αντίστοιχο πρόβλημα μεγιστοποίησης. Η συνάρτηση αυτή, χρησιμοποιείται και στο βήμα εκτίμησης (E-step). Επίσης, μελετάται αλγοριθμικά και μία παραλλαγή αυτής της μεθόδου, στην οποία η συνάρτηση με τις ποινές χρησιμοποιείται μόνο κατά το βήμα μεγιστοποίησης (M-step).Hidden Markov models are used in a wide range of applications due to their construction that renders them mathematically tractable and allows for the use of efficient computational techniques. There are methods for the estimation of the model’s parameters, such as the EM algorithm, but also for the estimation of the hidden states of the underlying Markov chain, such as the Viterbi algorithm. In applications where the dimension of the data is comparable to the sample size, the sample covariance matrix is known to be ill-conditioned, which directly affects the maximisation step (M- step) of the EM algorithm, where its inverse is involved in the computations. This problem might be amplified if there are rarely visited states resulting in a small sample size for the estimation of the corresponding parameters. Therefore, the direct implementation of these methods can be proved to be troublesome, as many computational problems might occur in the estimation of the covariance matrix and its inverse, further affecting the estimation of the one-step transition probability matrix and the reconstruction of the hidden Markov chain. In this paper, a modified version of the EM algorithm is studied, both theoretically and computa- tionally, in order to obtain the shrinkage estimator of the covariance matrix during the maximisation step. This is achieved by maximising a penalised log-likelihood function, which is also used in the estimation step (E-step). A variant of this modified version, where the penalised log-likelihood func- tion is only used in the maximisation step (M-step), is also studied computationally

    Discovering Causal Relations and Equations from Data

    Full text link
    Physics is a field of science that has traditionally used the scientific method to answer questions about why natural phenomena occur and to make testable models that explain the phenomena. Discovering equations, laws and principles that are invariant, robust and causal explanations of the world has been fundamental in physical sciences throughout the centuries. Discoveries emerge from observing the world and, when possible, performing interventional studies in the system under study. With the advent of big data and the use of data-driven methods, causal and equation discovery fields have grown and made progress in computer science, physics, statistics, philosophy, and many applied fields. All these domains are intertwined and can be used to discover causal relations, physical laws, and equations from observational data. This paper reviews the concepts, methods, and relevant works on causal and equation discovery in the broad field of Physics and outlines the most important challenges and promising future lines of research. We also provide a taxonomy for observational causal and equation discovery, point out connections, and showcase a complete set of case studies in Earth and climate sciences, fluid dynamics and mechanics, and the neurosciences. This review demonstrates that discovering fundamental laws and causal relations by observing natural phenomena is being revolutionised with the efficient exploitation of observational data, modern machine learning algorithms and the interaction with domain knowledge. Exciting times are ahead with many challenges and opportunities to improve our understanding of complex systems.Comment: 137 page
    corecore