3 research outputs found
Generative Imputation and Stochastic Prediction
In many machine learning applications, we are faced with incomplete datasets.
In the literature, missing data imputation techniques have been mostly
concerned with filling missing values. However, the existence of missing values
is synonymous with uncertainties not only over the distribution of missing
values but also over target class assignments that require careful
consideration. In this paper, we propose a simple and effective method for
imputing missing features and estimating the distribution of target assignments
given incomplete data. In order to make imputations, we train a simple and
effective generator network to generate imputations that a discriminator
network is tasked to distinguish. Following this, a predictor network is
trained using the imputed samples from the generator network to capture the
classification uncertainties and make predictions accordingly. The proposed
method is evaluated on CIFAR-10 and MNIST image datasets as well as five
real-world tabular classification datasets, under different missingness rates
and structures. Our experimental results show the effectiveness of the proposed
method in generating imputations as well as providing estimates for the class
uncertainties in a classification task when faced with missing values
Target-Focused Feature Selection Using a Bayesian Approach
In many real-world scenarios where data is high dimensional, test time
acquisition of features is a non-trivial task due to costs associated with
feature acquisition and evaluating feature value. The need for highly confident
models with an extremely frugal acquisition of features can be addressed by
allowing a feature selection method to become target aware. We introduce an
approach to feature selection that is based on Bayesian learning, allowing us
to report target-specific levels of uncertainty, false positive, and false
negative rates. In addition, measuring uncertainty lifts the restriction on
feature selection being target agnostic, allowing for feature acquisition based
on a single target of focus out of many. We show that acquiring features for a
specific target is at least as good as common linear feature selection
approaches for small non-sparse datasets, and surpasses these when faced with
real-world healthcare data that is larger in scale and in sparseness
Group-Connected Multilayer Perceptron Networks
Despite the success of deep learning in domains such as image, voice, and
graphs, there has been little progress in deep representation learning for
domains without a known structure between features. For instance, a tabular
dataset of different demographic and clinical factors where the feature
interactions are not given as a prior. In this paper, we propose
Group-Connected Multilayer Perceptron (GMLP) networks to enable deep
representation learning in these domains. GMLP is based on the idea of learning
expressive feature combinations (groups) and exploiting them to reduce the
network complexity by defining local group-wise operations. During the training
phase, GMLP learns a sparse feature grouping matrix using temperature annealing
softmax with an added entropy loss term to encourage the sparsity. Furthermore,
an architecture is suggested which resembles binary trees, where group-wise
operations are followed by pooling operations to combine information; reducing
the number of groups as the network grows in depth. To evaluate the proposed
method, we conducted experiments on different real-world datasets covering
various application areas. Additionally, we provide visualizations on MNIST and
synthesized data. According to the results, GMLP is able to successfully learn
and exploit expressive feature combinations and achieve state-of-the-art
classification performance on different datasets