Cumulative Distribution Functions As The Foundation For Probabilistic Models

This thesis discusses applications of probabilistic and connectionist models for constructing and training cumulative distribution functions (CDFs). First, it is shown how existing tools from the copula literature can be combined to build probabilistic models. It is found that this simple construction leads to numerical and scalability issues that make training and inference challenging. Next, several innovative ideas, combining neural networks, automatic differentiation and copula functions, introduce how to assemble black-box probabilistic models. The basic building block is a cumulative distribution function that is straightforward to construct, composed of arithmetic operations and nonlinear functions. There is no need to assume any specific parametric probability density function (PDF), making the model flexible and normalisation unnecessary. The only requirement is to design a computational graph that parameterises monotonically non-decreasing functions with a constrained range. Training can be then performed using standard tools from any neural network software library. Finally, factorial hidden Markov models (FHMMs) for sequential data are presented. It is shown how to leverage cumulative distribution functions in the form of the Gaussian copula and amortised stochastic variational method to encode hidden Markov chains coherently. This approach enables efficient learning and inference to model long sequences of high-dimensional data with long-range dependencies. Tackling such complex problems was impossible with the established FHMM approximate inference algorithm. It is empirically verified on several problems that some of the estimators introduced in this work can perform comparably or better than the currently popular models. Especially for tasks requiring tail-area or marginal probabilities that can be read directly from a cumulative distribution function

Penerapan Exponential Smoothing untuk Transformasi Data dalam Meningkatkan Akurasi Neural Network pada Prediksi Harga Emas

Emas menjadi salah satu logam mulia yang paling banyak diminati baik untuk investasi maupun untuk dijadikan perhiasan. Memprediksi harga emas telah menjadi signifikan dan sangat penting bagi investor karena emas merupakan alat yang penting untuk perlindungan nilai resiko serta sebagai jalan investasi. Metode Neural Network merupakan salah satu model yang paling banyak digunakan dalam berbagai bidang penelitian. Neural Network memiliki banyak fitur yang diinginkan yang sangat cocok untuk aplikasi peramalan. Namun sebagai sistem black box, pemodelan Neural Network sepenuhnya tergantung pada input dan output data sehingga kualitas dan distribusi set sampel pembelajaran penting bagi kemampuan generalisasi jaringan. Maka pada penelitian ini, metode Exponential Smoothing digunakan untuk melakukan transformasi data guna meningkatkan kualitas data sehingga dapat meningkatkan akurasi prediksi pada Neural Network. Eksperimen yang dilakukan pada penelitian ini adalah untuk memperoleh arsitektur optimal sehingga menghasilkan prediksi harga emas yang akurat. Penelitian ini menggunakan Neural Network dan Exponential Smoothing dengan 10 kombinasi parameter pada eksperimen yang dilakukan. Kesimpulan yang didapatkan dari eksperimen yang dilakukan adalah bahwa prediksi harga emas menggunakan Neural Network dan Exponential Smoothing lebih akurat dibanding metode individual Neural Network

A contribution to exchange rate forecasting based on machine learning techniques

El propòsit d'aquesta tesi és examinar les aportacions a l'estudi de la predicció de la taxa de canvi basada en l'ús de tècniques d'aprenentatge automàtic. Aquestes aportacions es veuen facilitades i millorades per l'ús de variables econòmiques, indicadors tècnics i variables de tipus ‘business and consumer survey’. Aquesta investigació s’organitza entorn d’una recopilació de quatre articles. L'objectiu de cadascun dels quatre treballs de recerca d'aquesta tesi és el de contribuir a l'avanç del coneixement sobre els efectes i mecanismes mitjançant els quals l'ús de variables econòmiques, indicadors tècnics, variables de tipus ‘business and consumer survey’, i la selecció dels paràmetres de models predictius són capaços de millorar les prediccions de la taxa de canvi. Fent ús d'una tècnica de predicció no lineal, el primer article d'aquesta tesi es centra majoritàriament en l'impacte que tenen l'ús de variables econòmiques i la selecció dels paràmetres dels models en les prediccions de la taxa de canvi per a dos països. L'últim experiment d'aquest primer article fa ús de la taxa de canvi del període anterior i d'indicadors econòmics com a variables d'entrada en els models predictius. El segon article d'aquesta tesi analitza com la combinació de mitjanes mòbils, variables de tipus ‘business and consumer survey’ i la selecció dels paràmetres dels models milloren les prediccions del canvi per a dos països. A diferència del primer article, aquest segon treball de recerca afegeix mitjanes mòbils i variables de tipus ‘business and consumer survey’ com a variables d'entrada en els models predictius, i descarta l'ús de variables econòmiques. Un dels objectius d'aquest segon article és determinar el possible impacte de les variables de tipus ‘business and consumer survey’ en les taxes de canvi. El tercer article d'aquesta tesi té els mateixos objectius que el segon, però amb l'excepció que l'anàlisi abasta les taxes de canvi de set països. El quart article de la tesi compta amb els mateixos objectius que l'article anterior, però amb la diferència que fa ús d'un sol indicador tècnic. En general, l'enfocament d'aquesta tesi pretén examinar diferents alternatives per a millorar les prediccions del tipus de canvi a través de l'ús de màquines de suport vectorial. Una combinació de variables i la selecció dels paràmetres dels models predictius ajudaran a aconseguir aquest propòsit.El propósito de esta tesis es examinar las aportaciones al estudio de la predicción de la tasa de cambio basada en el uso de técnicas de aprendizaje automático. Dichas aportaciones se ven facilitadas y mejoradas por el uso de variables económicas, indicadores técnicos y variables de tipo ‘business and consumer survey’. Esta investigación está organizada en un compendio de cuatro artículos. El objetivo de cada uno de los cuatro trabajos de investigación de esta tesis es el de contribuir al avance del conocimiento sobre los efectos y mecanismos mediante los cuales el uso de variables económicas, indicadores técnicos, variables de tipo ‘business and consumer survey’, y la selección de los parámetros de modelos predictivos son capaces de mejorar las predicciones de la tasa de cambio. Haciendo uso de una técnica de predicción no lineal, el primer artículo de esta tesis se centra mayoritariamente en el impacto que tienen el uso de variables económicas y la selección de los parámetros de los modelos en las predicciones de la tasa de cambio para dos países. El último experimento de este primer artículo hace uso de la tasa de cambio del periodo anterior y de indicadores económicos como variables de entrada en los modelos predictivos. El segundo artículo de esta tesis analiza cómo la combinación de medias móviles, variables de tipo ‘business and consumer survey’ y la selección de los parámetros de los modelos mejoran las predicciones del cambio para dos países. A diferencia del primer artículo, este segundo trabajo de investigación añade medias móviles y variables de tipo ‘business and consumer survey’ como variables de entrada en los modelos predictivos, y descarta el uso de variables económicas. Uno de los objetivos de este segundo artículo es determinar el posible impacto de las variables de tipo ‘business and consumer survey’ en las tasas de cambio. El tercer artículo de esta tesis tiene los mismos objetivos que el segundo, pero con la salvedad de que el análisis abarca las tasas de cambio de siete países. El cuarto artículo de esta tesis cuenta con los mismos objetivos que el artículo anterior, pero con la diferencia de que hace uso de un solo indicador técnico. En general, el enfoque de esta tesis pretende examinar diferentes alternativas para mejorar las predicciones del tipo de cambio a través del uso de máquinas de soporte vectorial. Una combinación de variables y la selección de los parámetros de los modelos predictivos ayudarán a conseguir este propósito.The purpose of this thesis is to examine the contribution made by machine learning techniques on exchange rate forecasting. Such contributions are facilitated and enhanced by the use of fundamental economic variables, technical indicators and business and consumer survey variables as inputs in the forecasting models selected. This research has been organized in a compendium of four articles. The aim of each of these four articles is to contribute to advance our knowledge on the effects and means by which the use of fundamental economic variables, technical indicators, business and consumer surveys, and a model’s free-parameters selection is capable of improving exchange rate predictions. Through the use of a non-linear forecasting technique, one research paper examines the effect of fundamental economic variables and a model’s parameters selection on exchange rate forecasts, whereas the other three articles concentrate on the effect of technical indicators, a model’s parameters selection and business and consumer surveys variables on exchange rate forecasting. The first paper of this thesis has the objective of examining fundamental economic variables and a forecasting model’s parameters in an effort to understand the possible advantages or disadvantages these variables may bring to the exchange rate predictions in terms of forecasting performance and accuracy. The second paper of this thesis analyses how the combination of moving averages, business and consumer surveys and a forecasting model’s parameters improves exchange rate predictions. Compared to the first paper, this second paper adds moving averages and business and consumer surveys variables as inputs to the forecasting model, and disregards the use of fundamental economic variables. One of the goals of this paper is to determine the possible effects of business and consumer surveys on exchange rates. The third paper of this thesis has the same objectives as the second paper, but its analysis is expanded by taking into account the exchange rates of 7 countries. The fourth paper in this thesis takes a similar approach as the second and third papers, but makes use of a single technical indicator. In general, this thesis focuses on the improvement of exchange rate predictions through the use of support vector machines. A combination of variables and a model’s parameters selection enhances the way to achieve this purpose

Techniques to improve forecasting models: applications to energy demand and price

This thesis is a study of three techniques to improve performance of some standard fore-casting models, application to the energy demand and prices. We focus on forecasting demand and price one-day ahead. First, the wavelet transform was used as a pre-processing procedure with two approaches: multicomponent-forecasts and direct-forecasts. We have empirically compared these approaches and found that the former consistently outperformed the latter. Second, adaptive models were introduced to continuously update model parameters in the testing period by combining ?lters with standard forecasting methods. Among these adaptive models, the adaptive LR-GARCH model was proposed for the fi?rst time in the thesis. Third, with regard to noise distributions of the dependent variables in the forecasting models, we used either Gaussian or Student-t distributions. This thesis proposed a novel algorithm to infer parameters of Student-t noise models. The method is an extension of earlier work for models that are linear in parameters to the non-linear multilayer perceptron. Therefore, the proposed method broadens the range of models that can use a Student-t noise distribution. Because these techniques cannot stand alone, they must be combined with prediction models to improve their performance. We combined these techniques with some standard forecasting models: multilayer perceptron, radial basis functions, linear regression, and linear regression with GARCH. These techniques and forecasting models were applied to two datasets from the UK energy markets: daily electricity demand (which is stationary) and gas forward prices (non-stationary). The results showed that these techniques provided good improvement to prediction performance

Advances in machine learning algorithms for financial risk management

In this thesis, three novel machine learning techniques are introduced to address distinct yet interrelated challenges involved in financial risk management tasks. These approaches collectively offer a comprehensive strategy, beginning with the precise classification of credit risks, advancing through the nuanced forecasting of financial asset volatility, and ending with the strategic optimisation of financial asset portfolios. Firstly, a Hybrid Dual-Resampling and Cost-Sensitive technique has been proposed to combat the prevalent issue of class imbalance in financial datasets, particularly in credit risk assessment. The key process involves the creation of heuristically balanced datasets to effectively address the problem. It uses a resampling technique based on Gaussian mixture modelling to generate a synthetic minority class from the minority class data and concurrently uses k-means clustering on the majority class. Feature selection is then performed using the Extra Tree Ensemble technique. Subsequently, a cost-sensitive logistic regression model is then applied to predict the probability of default using the heuristically balanced datasets. The results underscore the effectiveness of our proposed technique, with superior performance observed in comparison to other imbalanced preprocessing approaches. This advancement in credit risk classification lays a solid foundation for understanding individual financial behaviours, a crucial first step in the broader context of financial risk management. Building on this foundation, the thesis then explores the forecasting of financial asset volatility, a critical aspect of understanding market dynamics. A novel model that combines a Triple Discriminator Generative Adversarial Network with a continuous wavelet transform is proposed. The proposed model has the ability to decompose volatility time series into signal-like and noise-like frequency components, to allow the separate detection and monitoring of non-stationary volatility data. The network comprises of a wavelet transform component consisting of continuous wavelet transforms and inverse wavelet transform components, an auto-encoder component made up of encoder and decoder networks, and a Generative Adversarial Network consisting of triple Discriminator and Generator networks. The proposed Generative Adversarial Network employs an ensemble of unsupervised loss derived from the Generative Adversarial Network component during training, supervised loss and reconstruction loss as part of its framework. Data from nine financial assets are employed to demonstrate the effectiveness of the proposed model. This approach not only enhances our understanding of market fluctuations but also bridges the gap between individual credit risk assessment and macro-level market analysis. Finally the thesis ends with a novel proposal of a novel technique or Portfolio optimisation. This involves the use of a model-free reinforcement learning strategy for portfolio optimisation using historical Low, High, and Close prices of assets as input with weights of assets as output. A deep Capsules Network is employed to simulate the investment strategy, which involves the reallocation of the different assets to maximise the expected return on investment based on deep reinforcement learning. To provide more learning stability in an online training process, a Markov Differential Sharpe Ratio reward function has been proposed as the reinforcement learning objective function. Additionally, a Multi-Memory Weight Reservoir has also been introduced to facilitate the learning process and optimisation of computed asset weights, helping to sequentially re-balance the portfolio throughout a specified trading period. The use of the insights gained from volatility forecasting into this strategy shows the interconnected nature of the financial markets. Comparative experiments with other models demonstrated that our proposed technique is capable of achieving superior results based on risk-adjusted reward performance measures. In a nut-shell, this thesis not only addresses individual challenges in financial risk management but it also incorporates them into a comprehensive framework; from enhancing the accuracy of credit risk classification, through the improvement and understanding of market volatility, to optimisation of investment strategies. These methodologies collectively show the potential of the use of machine learning to improve financial risk management