A sparse multinomial probit model for classification

A recent development in penalized probit modelling using a hierarchical Bayesian approach has led to a sparse binomial (two-class) probit classifier that can be trained via an EM algorithm. A key advantage of the formulation is that no tuning of hyperparameters relating to the penalty is needed thus simplifying the model selection process. The resulting model demonstrates excellent classification performance and a high degree of sparsity when used as a kernel machine. It is, however, restricted to the binary classification problem and can only be used in the multinomial situation via a one-against-all or one-against-many strategy. To overcome this, we apply the idea to the multinomial probit model. This leads to a direct multi-classification approach and is shown to give a sparse solution with accuracy and sparsity comparable with the current state-of-the-art. Comparative numerical benchmark examples are used to demonstrate the method

Learning From Labeled And Unlabeled Data: An Empirical Study Across Techniques And Domains

There has been increased interest in devising learning techniques that combine unlabeled data with labeled data ? i.e. semi-supervised learning. However, to the best of our knowledge, no study has been performed across various techniques and different types and amounts of labeled and unlabeled data. Moreover, most of the published work on semi-supervised learning techniques assumes that the labeled and unlabeled data come from the same distribution. It is possible for the labeling process to be associated with a selection bias such that the distributions of data points in the labeled and unlabeled sets are different. Not correcting for such bias can result in biased function approximation with potentially poor performance. In this paper, we present an empirical study of various semi-supervised learning techniques on a variety of datasets. We attempt to answer various questions such as the effect of independence or relevance amongst features, the effect of the size of the labeled and unlabeled sets and the effect of noise. We also investigate the impact of sample-selection bias on the semi-supervised learning techniques under study and implement a bivariate probit technique particularly designed to correct for such bias

Acceptance and profitability modelling for consumer loans

This thesis explores and models the relationships between offers of credit products, credit scores, consumers' acceptance decisions and expected profits generated using data that records actual choices made by customers and their monthly account status after being accepted. Based on Keeney and Oliver's theoretical work, this thesis esti¬ mates the expected profits for the lender at the time of application, draws the iso-profit curves and iso-preference curves, derives optimal policy decisions subject to various constraints and compares the economic benefits after the segmentation analysis.This thesis also addresses other research issues that have emerged during the explo¬ ration into profitability and acceptance. We use a Bivariate Sample Selection model to test the existence of sample selection bias and found that acceptance inference may not be necessary for our data. We compared the predictive performance of Support Vector Machines (SVMs) vs. Logistic Regression (LR) on default data as well as on accep¬ tance data, without finding that SVMs outperform LR. We applied different Survival Analysis models on two events of interest, default and paying back early. Our results favoured semi-parametric PH-Cox models separately estimated for each hazard

Risk prediction of product-harm events using rough sets and multiple classifier fusion:an experimental study of listed companies in China

With the increasing of frequency and destructiveness of product-harm events, study on enterprise crisis management becomes essentially important, but little literature thoroughly explores the risk prediction method of product-harm event. In this study, an initial index system for risk prediction was built based on the analysis of the key drivers of the product-harm event's evolution; ultimately, nine risk-forecasting indexes were obtained using rough set attribute reduction. With the four indexes of cumulative abnormal returns as the input, fuzzy clustering was used to classify the risk level of a product-harm event into four grades. In order to control the uncertainty and instability of single classifiers in risk prediction, multiple classifier fusion was introduced and combined with self-organising data mining (SODM). Further, an SODM-based multiple classifier fusion (SB-MCF) model was presented for the risk prediction related to a product-harm event. The experimental results based on 165 Chinese listed companies indicated that the SB-MCF model improved the average predictive accuracy and reduced variation degree simultaneously. The statistical analysis demonstrated that the SB-MCF model significantly outperformed six widely used single classification models (e.g. neural networks, support vector machine, and case-based reasoning) and other six commonly used multiple classifier fusion methods (e.g. majority voting, Bayesian method, and genetic algorithm)

Visual sense of number vs. sense of magnitude in humans and machines

Numerosity perception is thought to be foundational to mathematical learning, but its computational bases are strongly debated. Some investigators argue that humans are endowed with a specialized system supporting numerical representations; others argue that visual numerosity is estimated using continuous magnitudes, such as density or area, which usually co-vary with number. Here we reconcile these contrasting perspectives by testing deep neural networks on the same numerosity comparison task that was administered to human participants, using a stimulus space that allows the precise measurement of the contribution of non-numerical features. Our model accurately simulates the psychophysics of numerosity perception and the associated developmental changes: discrimination is driven by numerosity, but non-numerical features also have a significant impact, especially early during development. Representational similarity analysis further highlights that both numerosity and continuous magnitudes are spontaneously encoded in deep networks even when no task has to be carried out, suggesting that numerosity is a major, salient property of our visual environment

Multilevel mixed-type data analysis for validating partitions of scrapie isolates

The dissertation arises from a joint study with the Department of Food Safety and Veterinary Public Health of the Istituto Superiore di Sanità. The aim is to investigate and validate the existence of distinct strains of the scrapie disease taking into account the availability of a priori benchmark partition formulated by researchers. Scrapie of small ruminants is caused by prions, which are unconventional infectious agents of proteinaceous nature a ecting humans and animals. Due to the absence of nucleic acids, which precludes direct analysis of strain variation by molecular methods, the presence of di erent sheep scrapie strains is usually investigated by bioassay in laboratory rodents. Data are collected by an experimental study on scrapie conducted at the Istituto Superiore di Sanità by experimental transmission of scrapie isolates to bank voles. We aim to discuss the validation of a given partition in a statistical classification framework using a multi-step procedure. Firstly, we use unsupervised classification to see how alternative clustering results match researchers’ understanding of the heterogeneity of the isolates. We discuss whether and how clustering results can be eventually exploited to extend the preliminary partition elicited by researchers. Then we motivate the subsequent partition validation based on the predictive performance of several supervised classifiers. Our data-driven approach contains two main methodological original contributions. We advocate the use of partition validation measures to investigate a given benchmark partition: firstly we discuss the issue of how the data can be used to evaluate a preliminary benchmark partition and eventually modify it with statistical results to find a conclusive partition that could be used as a “gold standard” in future studies. Moreover, collected data have a multilevel structure and for each lower-level unit, mixed-type data are available. Each step in the procedure is then adapted to deal with multilevel mixed-type data. We extend distance-based clustering algorithms to deal with multilevel mixed-type data. Whereas in supervised classification we propose a two-step approach to classify the higher-level units starting from the lower-level observations. In this framework, we also need to define an ad-hoc cross validation algorithm

Combining artificial intelligence and robotic system in chemical product/process design

Product design for formulations is an active and challenging area of research. The new challenges of a fast-paced market, products of increasing complexity, and practical translation of sustainability paradigms require re-examination the existing theoretical frameworks to include the advantages from business and research digitalization. This thesis is based on the hypotheses that (i) new products with desired properties can be discovered by using a robotic platform combined with an intelligent optimization algorithm, and (ii) we can the connect data-driven optimisation with physico-chemical knowledge generation, which will result in a suitable model for translation of product discovery to production, thus impacting on the process development steps towards industrial applications. This thesis focuses on two complex physicochemical systems as case studies, namely the oil-in-water shampoo system and sunscreen products. Firstly, I report the coupling of a machine-learning classification algorithm with the Thompson-Sampling Efficient Multi-Optimization (TSEMO) for the simultaneous optimization of continuous and discrete outputs. The methodology was successfully applied to the design of a formulated liquid product of commercial interest for which no physical models are available. Experiments were carried out in a semi-automated fashion using robotic platforms triggered by the machine-learning algorithms. The proposed closed-loop optimization framework allowed to find suitable recipes meeting the customer-defined criteria within 15 working days, outperforming human intuition in the target performance of the formulations. The framework was then extended to co-optimization of both formulation and process conditions and ingredients selection. Secondly, I report the methods for the identification of new physical knowledge in a complex system where a prior knowledge is insufficient. The application of feature engineering methods in sun cream protection prediction was discussed. It was found that the concentration of UVA and UVB filters are key features, together with product viscosity, which match with the experts’ domain knowledge in sun cream product design. It was also found that through the combination of feature engineering and machine learning, high-fidelity model could be constructed. Furthermore, a modified mixed-integer nonlinear programming (MINLP) formulation for symbolic regression method was proposed for identification of physical models from noisy experimental data. The globally optimal search was extended to identify physical models and to cope with noise in the experimental data predictor variables.The methodology was proven to be successful in identifying the correct physical models describing the relationship between shear stress and shear rate for both Newtonian and non-Newtonian fluids, and simple kinetic laws of chemical reactions. The work of this thesis shows that machine learning methods, together with automated experimental system, can speed-up the R&D process of formulated product design as well as gain new physical knowledge of the complex systems