A Noise-Robust Fast Sparse Bayesian Learning Model

This paper utilizes the hierarchical model structure from the Bayesian Lasso in the Sparse Bayesian Learning process to develop a new type of probabilistic supervised learning approach. The hierarchical model structure in this Bayesian framework is designed such that the priors do not only penalize the unnecessary complexity of the model but will also be conditioned on the variance of the random noise in the data. The hyperparameters in the model are estimated by the Fast Marginal Likelihood Maximization algorithm which can achieve sparsity, low computational cost and faster learning process. We compare our methodology with two other popular learning models; the Relevance Vector Machine and the Bayesian Lasso. We test our model on examples involving both simulated and empirical data, and the results show that this approach has several performance advantages, such as being fast, sparse and also robust to the variance in random noise. In addition, our method can give out a more stable estimation of variance of random error, compared with the other methods in the study.Comment: 15 page

Bayesian Learning for Earthquake Engineering Applications and Structural Health Monitoring

Parallel to significant advances in sensor hardware, there have been recent developments of sophisticated methods for quantitative assessment of measured data that explicitly deal with all of the involved uncertainties, including inevitable measurement errors. The existence of these uncertainties often causes numerical instabilities in inverse problems that make them ill-conditioned. The Bayesian methodology is known to provide an efficient way to alleviate this illconditioning by incorporating the prior term for regularization of the inverse problem, and to provide probabilistic results which are meaningful for decision making. In this work, the Bayesian methodology is applied to inverse problems in earthquake engineering and especially to structural health monitoring. The proposed methodology of Bayesian learning using automatic relevance determination (ARD) prior, including its kernel version called the Relevance Vector Machine, is presented and applied to earthquake early warning, earthquake ground motion attenuation estimation, and structural health monitoring, using either a Bayesian classification or regression approach. The classification and regression are both performed in three phases: (1) Phase I (feature extraction phase): Determine which features from the data to use in a training dataset; (2) Phase II (training phase): Identify the unknown parameters defining a model by using a training dataset; and (3) Phase III (prediction phase): Predict the results based on the features from new data. This work focuses on the advantages of making probabilistic predictions obtained by Bayesian methods to deal with all uncertainties and the good characteristics of the proposed method in terms of computationally efficient training, and, especially, vi prediction that make it suitable for real-time operation. It is shown that sparseness (using only smaller number of basis function terms) is produced in the regression equations and classification separating boundary by using the ARD prior along with Bayesian model class selection to select the most probable (plausible) model class based on the data. This model class selection procedure automatically produces optimal regularization of the problem at hand, making it well-conditioned. Several applications of the proposed Bayesian learning methodology are presented. First, automatic near-source and far-source classification of incoming ground motion signals is treated and the Bayesian learning method is used to determine which ground motion features are optimal for this classification. Second, a probabilistic earthquake attenuation model for peak ground acceleration is identified using selected optimal features, especially taking a non-linearly involved parameter into consideration. It is shown that the Bayesian learning method can be utilized to estimate not only linear coefficients but also a non-linearly involved parameter to provide an estimate for an unknown parameter in the kernel basis functions for Relevance Vector Machine. Third, the proposed method is extended to a general case of regression problems with vector outputs and applied to structural health monitoring applications. It is concluded that the proposed vector output RVM shows promise for estimating damage locations and their severities from change of modal properties such as natural frequencies and mode shapes

Ensemble Committees for Stock Return Classification and Prediction

This paper considers a portfolio trading strategy formulated by algorithms in the field of machine learning. The profitability of the strategy is measured by the algorithm's capability to consistently and accurately identify stock indices with positive or negative returns, and to generate a preferred portfolio allocation on the basis of a learned model. Stocks are characterized by time series data sets consisting of technical variables that reflect market conditions in a previous time interval, which are utilized produce binary classification decisions in subsequent intervals. The learned model is constructed as a committee of random forest classifiers, a non-linear support vector machine classifier, a relevance vector machine classifier, and a constituent ensemble of k-nearest neighbors classifiers. The Global Industry Classification Standard (GICS) is used to explore the ensemble model's efficacy within the context of various fields of investment including Energy, Materials, Financials, and Information Technology. Data from 2006 to 2012, inclusive, are considered, which are chosen for providing a range of market circumstances for evaluating the model. The model is observed to achieve an accuracy of approximately 70% when predicting stock price returns three months in advance.Comment: 15 pages, 4 figures, Neukom Institute Computational Undergraduate Research prize - second plac

High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso

The goal of supervised feature selection is to find a subset of input features that are responsible for predicting output values. The least absolute shrinkage and selection operator (Lasso) allows computationally efficient feature selection based on linear dependency between input features and output values. In this paper, we consider a feature-wise kernelized Lasso for capturing non-linear input-output dependency. We first show that, with particular choices of kernel functions, non-redundant features with strong statistical dependence on output values can be found in terms of kernel-based independence measures. We then show that the globally optimal solution can be efficiently computed; this makes the approach scalable to high-dimensional problems. The effectiveness of the proposed method is demonstrated through feature selection experiments with thousands of features.Comment: 18 page

Sparse multinomial kernel discriminant analysis (sMKDA)

Dimensionality reduction via canonical variate analysis (CVA) is important for pattern recognition and has been extended variously to permit more flexibility, e.g. by "kernelizing" the formulation. This can lead to over-fitting, usually ameliorated by regularization. Here, a method for sparse, multinomial kernel discriminant analysis (sMKDA) is proposed, using a sparse basis to control complexity. It is based on the connection between CVA and least-squares, and uses forward selection via orthogonal least-squares to approximate a basis, generalizing a similar approach for binomial problems. Classification can be performed directly via minimum Mahalanobis distance in the canonical variates. sMKDA achieves state-of-the-art performance in terms of accuracy and sparseness on 11 benchmark datasets

The application of user log for online business environment using content-based Image retrieval system

Over the past few years, inter-query learning has gained much attention in the research and development of content-based image retrieval (CBIR) systems. This is largely due to the capability of inter-query approach to enable learning from the retrieval patterns of previous query sessions. However, much of the research works in this field have been focusing on analyzing image retrieval patterns stored in the database. This is not suitable for a dynamic environment such as the World Wide Web (WWW) where images are constantly added or removed. A better alternative is to use an image's visual features to capture the knowledge gained from the previous query sessions. Based on the previous work (Chung et al., 2006), the aim of this paper is to propose a framework of inter-query learning for the WWW-CBIR systems. Such framework can be extremely useful for those online companies whose core business involves providing multimedia content-based services and products to their customers