Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases

For big data analysis, high computational cost for Bayesian methods often limits their applications in practice. In recent years, there have been many attempts to improve computational efficiency of Bayesian inference. Here we propose an efficient and scalable computational technique for a state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian Monte Carlo (HMC). The key idea is to explore and exploit the structure and regularity in parameter space for the underlying probabilistic model to construct an effective approximation of its geometric properties. To this end, we build a surrogate function to approximate the target distribution using properly chosen random bases and an efficient optimization process. The resulting method provides a flexible, scalable, and efficient sampling algorithm, which converges to the correct target distribution. We show that by choosing the basis functions and optimization process differently, our method can be related to other approaches for the construction of surrogate functions such as generalized additive models or Gaussian process models. Experiments based on simulated and real data show that our approach leads to substantially more efficient sampling algorithms compared to existing state-of-the art methods

Magnetic Modelling of Synchronous Reluctance and Internal Permanent Magnet Motors Using Radial Basis Function Networks

The general trend toward more intelligent energy-aware ac drives is driving the development of new motor topologies and advanced model-based control techniques. Among the candidates, pure reluctance and anisotropic permanent magnet motors are gaining popularity, despite their complex structure. The availability of accurate mathematical models that describe these motors is essential to the design of any model-based advanced control. This paper focuses on the relations between currents and flux linkages, which are obtained through innovative radial basis function neural networks. These special drive-oriented neural networks take as inputs the motor voltages and currents, returning as output the motor flux linkages, inclusive of any nonlinearity and cross-coupling effect. The theoretical foundations of the radial basis function networks, the design hints, and a commented series of experimental results on a real laboratory prototype are included in this paper. The simple structure of the neural network fits for implementation on standard drives. The online training and tracking will be the next steps in field programmable gate array based control systems

Optical implementations of radial basis classifiers

We describe two optical systems based on the radial basis function approach to pattern classification. An optical-disk-based system for handwritten character recognition is demonstrated. The optical system computes the Euclidean distance between an unknown input and 650 stored patterns at a demonstrated rate of 26,000 pattern comparisons/s. The ultimate performance of this system is limited by optical-disk resolution to 10^11 binary operations/s. An adaptive system is also presented that facilitates on-line learning and provides additional robustness

Evolutionary model type selection for global surrogate modeling

Due to the scale and computational complexity of currently used simulation codes, global surrogate (metamodels) models have become indispensable tools for exploring and understanding the design space. Due to their compact formulation they are cheap to evaluate and thus readily facilitate visualization, design space exploration, rapid prototyping, and sensitivity analysis. They can also be used as accurate building blocks in design packages or larger simulation environments. Consequently, there is great interest in techniques that facilitate the construction of such approximation models while minimizing the computational cost and maximizing model accuracy. Many surrogate model types exist ( Support Vector Machines, Kriging, Neural Networks, etc.) but no type is optimal in all circumstances. Nor is there any hard theory available that can help make this choice. In this paper we present an automatic approach to the model type selection problem. We describe an adaptive global surrogate modeling environment with adaptive sampling, driven by speciated evolution. Different model types are evolved cooperatively using a Genetic Algorithm ( heterogeneous evolution) and compete to approximate the iteratively selected data. In this way the optimal model type and complexity for a given data set or simulation code can be dynamically determined. Its utility and performance is demonstrated on a number of problems where it outperforms traditional sequential execution of each model type

Machine Learning Applications in Estimating Transformer Loss of Life

Transformer life assessment and failure diagnostics have always been important problems for electric utility companies. Ambient temperature and load profile are the main factors which affect aging of the transformer insulation, and consequently, the transformer lifetime. The IEEE Std. C57.911995 provides a model for calculating the transformer loss of life based on ambient temperature and transformer's loading. In this paper, this standard is used to develop a data-driven static model for hourly estimation of the transformer loss of life. Among various machine learning methods for developing this static model, the Adaptive Network-Based Fuzzy Inference System (ANFIS) is selected. Numerical simulations demonstrate the effectiveness and the accuracy of the proposed ANFIS method compared with other relevant machine learning based methods to solve this problem.Comment: IEEE Power and Energy Society General Meeting, 201

Forecasting foreign exchange rates with adaptive neural networks using radial basis functions and particle swarm optimization

The motivation for this paper is to introduce a hybrid Neural Network architecture of Particle Swarm Optimization and Adaptive Radial Basis Function (ARBF-PSO), a time varying leverage trading strategy based on Glosten, Jagannathan and Runkle (GJR) volatility forecasts and a Neural Network fitness function for financial forecasting purposes. This is done by benchmarking the ARBF-PSO results with those of three different Neural Networks architectures, a Nearest Neighbors algorithm (k-NN), an autoregressive moving average model (ARMA), a moving average convergence/divergence model (MACD) plus a naïve strategy. More specifically, the trading and statistical performance of all models is investigated in a forecast simulation of the EUR/USD, EUR/GBP and EUR/JPY ECB exchange rate fixing time series over the period January 1999 to March 2011 using the last two years for out-of-sample testing