4,874 research outputs found
Automated Dependence Plots
In practical applications of machine learning, it is necessary to look beyond
standard metrics such as test accuracy in order to validate various qualitative
properties of a model. Partial dependence plots (PDP), including
instance-specific PDPs (i.e., ICE plots), have been widely used as a visual
tool to understand or validate a model. Yet, current PDPs suffer from two main
drawbacks: (1) a user must manually sort or select interesting plots, and (2)
PDPs are usually limited to plots along a single feature. To address these
drawbacks, we formalize a method for automating the selection of interesting
PDPs and extend PDPs beyond showing single features to show the model response
along arbitrary directions, for example in raw feature space or a latent space
arising from some generative model. We demonstrate the usefulness of our
automated dependence plots (ADP) across multiple use-cases and datasets
including model selection, bias detection, understanding out-of-sample
behavior, and exploring the latent space of a generative model.Comment: In Uncertainty in Artificial Intelligence (UAI 2020). Camera-ready
version. Code is available at https://github.com/davidinouye/ad
Label Noise Filtering Techniques to Improve Monotonic Classification
The monotonic ordinal classification has increased the interest of
researchers and practitioners within machine learning community in the last
years. In real applications, the problems with monotonicity constraints are
very frequent. To construct predictive monotone models from those problems,
many classifiers require as input a data set satisfying the monotonicity
relationships among all samples. Changing the class labels of the data set
(relabelling) is useful for this. Relabelling is assumed to be an important
building block for the construction of monotone classifiers and it is proved
that it can improve the predictive performance.
In this paper, we will address the construction of monotone datasets
considering as noise the cases that do not meet the monotonicity restrictions.
For the first time in the specialized literature, we propose the use of noise
filtering algorithms in a preprocessing stage with a double goal: to increase
both the monotonicity index of the models and the accuracy of the predictions
for different monotonic classifiers. The experiments are performed over 12
datasets coming from classification and regression problems and show that our
scheme improves the prediction capabilities of the monotonic classifiers
instead of being applied to original and relabeled datasets. In addition, we
have included the analysis of noise filtering process in the particular case of
wine quality classification to understand its effect in the predictive models
generated.Comment: This paper is already accepted for publication in Neurocomputin
Integrating Economic Knowledge in Data Mining Algorithms
The assessment of knowledge derived from databases depends on many factors. Decision makers often need to convince others about the correctness and effectiveness of knowledge induced from data.The current data mining techniques do not contribute much to this process of persuasion.Part of this limitation can be removed by integrating knowledge from experts in the field, encoded in some accessible way, with knowledge derived form patterns stored in the database.In this paper we will in particular discuss methods for implementing monotonicity constraints in economic decision problems.This prior knowledge is combined with data mining algorithms based on decision trees and neural networks.The method is illustrated in a hedonic price model.knowledge;neural network;data mining;decision trees
Iterative Machine Teaching
In this paper, we consider the problem of machine teaching, the inverse
problem of machine learning. Different from traditional machine teaching which
views the learners as batch algorithms, we study a new paradigm where the
learner uses an iterative algorithm and a teacher can feed examples
sequentially and intelligently based on the current performance of the learner.
We show that the teaching complexity in the iterative case is very different
from that in the batch case. Instead of constructing a minimal training set for
learners, our iterative machine teaching focuses on achieving fast convergence
in the learner model. Depending on the level of information the teacher has
from the learner model, we design teaching algorithms which can provably reduce
the number of teaching examples and achieve faster convergence than learning
without teachers. We also validate our theoretical findings with extensive
experiments on different data distribution and real image datasets.Comment: Published in ICML 201
HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoning
Large crowdsourced datasets are widely used for training and evaluating
neural models on natural language inference (NLI). Despite these efforts,
neural models have a hard time capturing logical inferences, including those
licensed by phrase replacements, so-called monotonicity reasoning. Since no
large dataset has been developed for monotonicity reasoning, it is still
unclear whether the main obstacle is the size of datasets or the model
architectures themselves. To investigate this issue, we introduce a new
dataset, called HELP, for handling entailments with lexical and logical
phenomena. We add it to training data for the state-of-the-art neural models
and evaluate them on test sets for monotonicity phenomena. The results showed
that our data augmentation improved the overall accuracy. We also find that the
improvement is better on monotonicity inferences with lexical replacements than
on downward inferences with disjunction and modification. This suggests that
some types of inferences can be improved by our data augmentation while others
are immune to it.Comment: 6 pages, 1 figure, accepted as *SEM 201
Can recursive neural tensor networks learn logical reasoning?
Recursive neural network models and their accompanying vector representations
for words have seen success in an array of increasingly semantically
sophisticated tasks, but almost nothing is known about their ability to
accurately capture the aspects of linguistic meaning that are necessary for
interpretation or reasoning. To evaluate this, I train a recursive model on a
new corpus of constructed examples of logical reasoning in short sentences,
like the inference of "some animal walks" from "some dog walks" or "some cat
walks," given that dogs and cats are animals. This model learns representations
that generalize well to new types of reasoning pattern in all but a few cases,
a result which is promising for the ability of learned representation models to
capture logical reasoning.Comment: Submitted for presentation at ICLR 2014. Source code and data:
http://goo.gl/PSyF5
Model-based Pricing for Machine Learning in a Data Marketplace
Data analytics using machine learning (ML) has become ubiquitous in science,
business intelligence, journalism and many other domains. While a lot of work
focuses on reducing the training cost, inference runtime and storage cost of ML
models, little work studies how to reduce the cost of data acquisition, which
potentially leads to a loss of sellers' revenue and buyers' affordability and
efficiency.
In this paper, we propose a model-based pricing (MBP) framework, which
instead of pricing the data, directly prices ML model instances. We first
formally describe the desired properties of the MBP framework, with a focus on
avoiding arbitrage. Next, we show a concrete realization of the MBP framework
via a noise injection approach, which provably satisfies the desired formal
properties. Based on the proposed framework, we then provide algorithmic
solutions on how the seller can assign prices to models under different market
scenarios (such as to maximize revenue). Finally, we conduct extensive
experiments, which validate that the MBP framework can provide high revenue to
the seller, high affordability to the buyer, and also operate on low runtime
cost
SPLINE-Net: Sparse Photometric Stereo through Lighting Interpolation and Normal Estimation Networks
This paper solves the Sparse Photometric stereo through Lighting
Interpolation and Normal Estimation using a generative Network (SPLINE-Net).
SPLINE-Net contains a lighting interpolation network to generate dense lighting
observations given a sparse set of lights as inputs followed by a normal
estimation network to estimate surface normals. Both networks are jointly
constrained by the proposed symmetric and asymmetric loss functions to enforce
isotropic constrain and perform outlier rejection of global illumination
effects. SPLINE-Net is verified to outperform existing methods for photometric
stereo of general BRDFs by using only ten images of different lights instead of
using nearly one hundred images.Comment: Accepted to ICCV 201
ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning
The process of aligning a pair of shapes is a fundamental operation in
computer graphics. Traditional approaches rely heavily on matching
corresponding points or features to guide the alignment, a paradigm that
falters when significant shape portions are missing. These techniques generally
do not incorporate prior knowledge about expected shape characteristics, which
can help compensate for any misleading cues left by inaccuracies exhibited in
the input shapes. We present an approach based on a deep neural network,
leveraging shape datasets to learn a shape-aware prior for source-to-target
alignment that is robust to shape incompleteness. In the absence of ground
truth alignments for supervision, we train a network on the task of shape
alignment using incomplete shapes generated from full shapes for
self-supervision. Our network, called ALIGNet, is trained to warp complete
source shapes to incomplete targets, as if the target shapes were complete,
thus essentially rendering the alignment partial-shape agnostic. We aim for the
network to develop specialized expertise over the common characteristics of the
shapes in each dataset, thereby achieving a higher-level understanding of the
expected shape space to which a local approach would be oblivious. We constrain
ALIGNet through an anisotropic total variation identity regularization to
promote piecewise smooth deformation fields, facilitating both partial-shape
agnosticism and post-deformation applications. We demonstrate that ALIGNet
learns to align geometrically distinct shapes, and is able to infer plausible
mappings even when the target shape is significantly incomplete. We show that
our network learns the common expected characteristics of shape collections,
without over-fitting or memorization, enabling it to produce plausible
deformations on unseen data during test time.Comment: To be presented at SIGGRAPH Asia 201
Coarse Grained Exponential Variational Autoencoders
Variational autoencoders (VAE) often use Gaussian or category distribution to
model the inference process. This puts a limit on variational learning because
this simplified assumption does not match the true posterior distribution,
which is usually much more sophisticated. To break this limitation and apply
arbitrary parametric distribution during inference, this paper derives a
\emph{semi-continuous} latent representation, which approximates a continuous
density up to a prescribed precision, and is much easier to analyze than its
continuous counterpart because it is fundamentally discrete. We showcase the
proposition by applying polynomial exponential family distributions as the
posterior, which are universal probability density function generators. Our
experimental results show consistent improvements over commonly used VAE
models
- …