Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases

For big data analysis, high computational cost for Bayesian methods often limits their applications in practice. In recent years, there have been many attempts to improve computational efficiency of Bayesian inference. Here we propose an efficient and scalable computational technique for a state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian Monte Carlo (HMC). The key idea is to explore and exploit the structure and regularity in parameter space for the underlying probabilistic model to construct an effective approximation of its geometric properties. To this end, we build a surrogate function to approximate the target distribution using properly chosen random bases and an efficient optimization process. The resulting method provides a flexible, scalable, and efficient sampling algorithm, which converges to the correct target distribution. We show that by choosing the basis functions and optimization process differently, our method can be related to other approaches for the construction of surrogate functions such as generalized additive models or Gaussian process models. Experiments based on simulated and real data show that our approach leads to substantially more efficient sampling algorithms compared to existing state-of-the art methods

Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification

Stacking-based deep neural network (S-DNN) is aggregated with pluralities of basic learning modules, one after another, to synthesize a deep neural network (DNN) alternative for pattern classification. Contrary to the DNNs trained end to end by backpropagation (BP), each S-DNN layer, i.e., a self-learnable module, is to be trained decisively and independently without BP intervention. In this paper, a ridge regression-based S-DNN, dubbed deep analytic network (DAN), along with its kernelization (K-DAN), are devised for multilayer feature re-learning from the pre-extracted baseline features and the structured features. Our theoretical formulation demonstrates that DAN/K-DAN re-learn by perturbing the intra/inter-class variations, apart from diminishing the prediction errors. We scrutinize the DAN/K-DAN performance for pattern classification on datasets of varying domains - faces, handwritten digits, generic objects, to name a few. Unlike the typical BP-optimized DNNs to be trained from gigantic datasets by GPU, we disclose that DAN/K-DAN are trainable using only CPU even for small-scale training sets. Our experimental results disclose that DAN/K-DAN outperform the present S-DNNs and also the BP-trained DNNs, including multiplayer perceptron, deep belief network, etc., without data augmentation applied.Comment: 14 pages, 7 figures, 11 table

Evaluation of the performance of a composite profile at elevated temperatures using finite element and hybrid artificial intelligence techniques

It is very important to keep structures and constructional elements in service during and after exposure to elevated temperatures. Investigation of the structural behaviour of different components and structures at elevated temperatures is an approach to manipulate the serviceability of the structures during heat exposure. Channel connectors are widely used shear connectors not only for their appealing mechanical properties but also for their workability and cost-effective nature. In this study, a finite element (FE) evaluation was performed on an authentic composite model, and the behaviour of the channel shear connector at elevated temperature was examined. Furthermore, a novel hybrid intelligence algorithm based on a feature-selection trait with the incorporation of particle swarm optimization (PSO) and multi-layer perceptron (MLP) algorithms has been developed to predict the slip response of the channel. The hybrid intelligence algorithm that uses artificial neural networks is performed on derived data from the FE study. Finally, the obtained numerical results are compared with extreme learning machine (ELM) and radial basis function (RBF) results. The MLP-PSO represented dramatically accurate results for slip value prediction at elevated temperatures. The results proved the active presence of the channels, especially to improve the stiffness and loading capacity of the composite beam. Although the height enhances the ductility, stiffness is significantly reduced at elevated temperatures. According to the results, temperature, failure load, the height of connector and concrete block strength are the key governing parameters for composite floor design against high temperatures

WOS-ELM-Based Double Redundancy Fault Diagnosis and Reconstruction for Aeroengine Sensor

In order to diagnose sensor fault of aeroengine more quickly and accurately, a double redundancy diagnosis approach based on Weighted Online Sequential Extreme Learning Machine (WOS-ELM) is proposed in this paper. WOS-ELM, which assigns different weights to old and new data, implements weighted dealing with the input data to get more precise training models. The proposed approach contains two series of diagnosis models, that is, spatial model and time model. The application of double redundancy based on spatial and time redundancy can in real time detect the hard fault and soft fault much earlier. The trouble-free or reconstructed time redundancy model can be utilized to update the training model and make it be consistent with the practical operation mode of the aeroengine. Simulation results illustrate the effectiveness and feasibility of the proposed method

A performance evaluation of pruning effects on hybrid neural network

In this paper, we explore the pruning effects on a hybrid mode sequential learning algorithmnamely FuzzyARTMAP-prunable Radial Basis Function (FAM-PRBF) that utilizes FuzzyARTMAP to learn a training dataset and Radial Basis Function Network (RBFN) to performregression and classification. The pruning algorithm is used to optimize the hidden layer ofthe RBFN. The experimental results show that FAM-PRBF has successfully reduced thecomplexity and computation time of the neural network.Keywords: pruning; radial basis function network; fuzzy ARTMAP