MCMC Learning

The theory of learning under the uniform distribution is rich and deep, with connections to cryptography, computational complexity, and the analysis of boolean functions to name a few areas. This theory however is very limited due to the fact that the uniform distribution and the corresponding Fourier basis are rarely encountered as a statistical model. A family of distributions that vastly generalizes the uniform distribution on the Boolean cube is that of distributions represented by Markov Random Fields (MRF). Markov Random Fields are one of the main tools for modeling high dimensional data in many areas of statistics and machine learning. In this paper we initiate the investigation of extending central ideas, methods and algorithms from the theory of learning under the uniform distribution to the setup of learning concepts given examples from MRF distributions. In particular, our results establish a novel connection between properties of MCMC sampling of MRFs and learning under the MRF distribution.Comment: 28 pages, 1 figur

Permutation Models for Collaborative Ranking

We study the problem of collaborative filtering where ranking information is available. Focusing on the core of the collaborative ranking process, the user and their community, we propose new models for representation of the underlying permutations and prediction of ranks. The first approach is based on the assumption that the user makes successive choice of items in a stage-wise manner. In particular, we extend the Plackett-Luce model in two ways - introducing parameter factoring to account for user-specific contribution, and modelling the latent community in a generative setting. The second approach relies on log-linear parameterisation, which relaxes the discrete-choice assumption, but makes learning and inference much more involved. We propose MCMC-based learning and inference methods and derive linear-time prediction algorithms

A modified weight optimisation for higher-order neural network in time series prediction

Most of time series signals are difficult to predict as consist of non-linear, high complexity (noise) and chaotic processes. The challenges in time series prediction are to provide a technique to better understand a dataset. In line with this, the Cuckoo Search (CS) learning algorithm, a kind of metaheuristics techniques employs high-level techniques for exploration and exploitation of the search space in which its step length is much longer in the long run. Thus, can explicitly being used to address the possibilities of stochastic trends in time series signals. Since its discovery, the CS has been used extensively. However, these methods fixed the parameter values which essential for adjusting the weights. Therefore, a modification was made by the additional step of information exchange between the top eggs, which significantly improve the convergence rate. Hence, motivated by the advantages of those Modified Cuckoo Search (MCS), the improvement of the MCS called Modified Cuckoo Search-Markov chain Monté Carlo (MCS-MCMC) learning algorithm is proposed for weight optimisation. As the Markov chain Monté Carlo can replace the cumbersome in generating the objective functions, it is used to substitute the Lévy flight found in the MCS’s structure to prove that MCS-MCMC is suitable for predictive tasks. The performance of MCS-MCMC learning algorithm was validated with several test functions and compared with those of MCS learning algorithm. The MCS-MCMC results is further benchmarked with the standard Multilayer Perceptron, standard Pi-Sigma Neural Network (PSNN), Pi-Sigma Neural Network-Modified Cuckoo Search, Pi-Sigma Neural Network-Markov chain Monté Carlo, standard Functional Link Neural Network (FLNN), Functional Link Neural Network-Modified Cuckoo Search and Functional Link Neural Network-Markov chain Monté Carlo which emphasis in optimising the accuracy rate. The simulation results proved that MCS-MCMC outperformed in the form of Accuracy with the range of 0.003% to 4.421% when incorporated with standard PSNN and FLNN for three (3) data partitions covering 10 benchmarked time series datasets

Scaling Nonparametric Bayesian Inference via Subsample-Annealing

We describe an adaptation of the simulated annealing algorithm to nonparametric clustering and related probabilistic models. This new algorithm learns nonparametric latent structure over a growing and constantly churning subsample of training data, where the portion of data subsampled can be interpreted as the inverse temperature beta(t) in an annealing schedule. Gibbs sampling at high temperature (i.e., with a very small subsample) can more quickly explore sketches of the final latent state by (a) making longer jumps around latent space (as in block Gibbs) and (b) lowering energy barriers (as in simulated annealing). We prove subsample annealing speeds up mixing time N^2 -> N in a simple clustering model and exp(N) -> N in another class of models, where N is data size. Empirically subsample-annealing outperforms naive Gibbs sampling in accuracy-per-wallclock time, and can scale to larger datasets and deeper hierarchical models. We demonstrate improved inference on million-row subsamples of US Census data and network log data and a 307-row hospital rating dataset, using a Pitman-Yor generalization of the Cross Categorization model.Comment: To appear in AISTATS 201

Bayesian reinforcement learning with MCMC to maximize energy output of vertical axis wind turbine

Optimization of energy output of small scale wind turbines requires a controller which keeps the wind speed to rotor tip speed ratio at the optimum value. An analytic solution can be obtained if the dynamic model of the complete system is known and wind speed can be anticipated. However, not only aging but also errors in modeling and wind speed prediction prevent a straightforward solution. This thesis proposes to apply a reinforcement learning approach designed to optimize dynamic systems with continuous state and action spaces, to the energy output optimization of Vertical Axis Wind Turbines (VAWT). The dynamic modeling and load control of the wind turbine are accomplished in the same process. The proposed algorithm is a model-free Bayesian Reinforcement Learning using Markov Chain Monte Carlo method (MCMC) to obtain the parameters of an optimal policy. The proposed method learns wind speed pro les and system model, therefore, can utilize all system states and observed wind speed pro les to calculate an optimal control signal by using a Radial Basis Function Neural Network (RBFNN). The proposed method is validated by performing simulation studies on a permanent magnet synchronous generator-based VAWT Simulink model to compare with the classical Maximum Power Point Tracking (MPPT). The results show signi cant improvement over the classical method, especially during the wind speed transients, promising a superior energy output in turbulent settings; which coincide with the expected application areas of VAWT