18,231 research outputs found
Identifying Mislabeled Training Data
This paper presents a new approach to identifying and eliminating mislabeled
training instances for supervised learning. The goal of this approach is to
improve classification accuracies produced by learning algorithms by improving
the quality of the training data. Our approach uses a set of learning
algorithms to create classifiers that serve as noise filters for the training
data. We evaluate single algorithm, majority vote and consensus filters on five
datasets that are prone to labeling errors. Our experiments illustrate that
filtering significantly improves classification accuracy for noise levels up to
30 percent. An analytical and empirical evaluation of the precision of our
approach shows that consensus filters are conservative at throwing away good
data at the expense of retaining bad data and that majority filters are better
at detecting bad data at the expense of throwing away good data. This suggests
that for situations in which there is a paucity of data, consensus filters are
preferable, whereas majority vote filters are preferable for situations with an
abundance of data
Predictive User Modeling with Actionable Attributes
Different machine learning techniques have been proposed and used for
modeling individual and group user needs, interests and preferences. In the
traditional predictive modeling instances are described by observable
variables, called attributes. The goal is to learn a model for predicting the
target variable for unseen instances. For example, for marketing purposes a
company consider profiling a new user based on her observed web browsing
behavior, referral keywords or other relevant information. In many real world
applications the values of some attributes are not only observable, but can be
actively decided by a decision maker. Furthermore, in some of such applications
the decision maker is interested not only to generate accurate predictions, but
to maximize the probability of the desired outcome. For example, a direct
marketing manager can choose which type of a special offer to send to a client
(actionable attribute), hoping that the right choice will result in a positive
response with a higher probability. We study how to learn to choose the value
of an actionable attribute in order to maximize the probability of a desired
outcome in predictive modeling. We emphasize that not all instances are equally
sensitive to changes in actions. Accurate choice of an action is critical for
those instances, which are on the borderline (e.g. users who do not have a
strong opinion one way or the other). We formulate three supervised learning
approaches for learning to select the value of an actionable attribute at an
instance level. We also introduce a focused training procedure which puts more
emphasis on the situations where varying the action is the most likely to take
the effect. The proof of concept experimental validation on two real-world case
studies in web analytics and e-learning domains highlights the potential of the
proposed approaches
Ethical Adversaries: Towards Mitigating Unfairness with Adversarial Machine Learning
Machine learning is being integrated into a growing number of critical
systems with far-reaching impacts on society. Unexpected behaviour and unfair
decision processes are coming under increasing scrutiny due to this widespread
use and its theoretical considerations. Individuals, as well as organisations,
notice, test, and criticize unfair results to hold model designers and
deployers accountable. We offer a framework that assists these groups in
mitigating unfair representations stemming from the training datasets. Our
framework relies on two inter-operating adversaries to improve fairness. First,
a model is trained with the goal of preventing the guessing of protected
attributes' values while limiting utility losses. This first step optimizes the
model's parameters for fairness. Second, the framework leverages evasion
attacks from adversarial machine learning to generate new examples that will be
misclassified. These new examples are then used to retrain and improve the
model in the first step. These two steps are iteratively applied until a
significant improvement in fairness is obtained. We evaluated our framework on
well-studied datasets in the fairness literature -- including COMPAS -- where
it can surpass other approaches concerning demographic parity, equality of
opportunity and also the model's utility. We also illustrate our findings on
the subtle difficulties when mitigating unfairness and highlight how our
framework can assist model designers.Comment: 15 pages, 3 figures, 1 tabl
- …