2,248 research outputs found
Estimating the moments of a random vector with applications
A general result about the quality of approximation of the mean of a distribution by its empirical estimate is proven that does not involve the dimension of the feature space. Using the kernel trick this gives also bounds the quality of approximation of higher order moments. A number of applications are derived of interest in learning theory including a new novelty detection algorithm and rigorous bounds on the Robust Minimax Classification algorithm
Biased Embeddings from Wild Data: Measuring, Understanding and Removing
Many modern Artificial Intelligence (AI) systems make use of data embeddings,
particularly in the domain of Natural Language Processing (NLP). These
embeddings are learnt from data that has been gathered "from the wild" and have
been found to contain unwanted biases. In this paper we make three
contributions towards measuring, understanding and removing this problem. We
present a rigorous way to measure some of these biases, based on the use of
word lists created for social psychology applications; we observe how gender
bias in occupations reflects actual gender bias in the same occupations in the
real world; and finally we demonstrate how a simple projection can
significantly reduce the effects of embedding bias. All this is part of an
ongoing effort to understand how trust can be built into AI systems.Comment: Author's original versio
Social Machinery and Intelligence
Social machines are systems formed by technical and human elements interacting in a
structured manner. The use of digital platforms as mediators allows large numbers of human participants to join such mechanisms, creating systems where interconnected digital and human components operate as a single machine capable of highly sophisticated behaviour. Under certain conditions, such systems can be described as autonomous and goal-driven agents. Many examples of modern Artificial Intelligence (AI) can be regarded as instances of this class of mechanisms. We argue that this type of autonomous social machines has provided a new paradigm for the design of intelligent systems marking a new phase in the field of AI. The consequences of this observation range from methodological, philosophical to ethical. On the one side, it emphasises the role of Human-Computer Interaction in the design of intelligent systems, while on the other side it draws attention to both the risks for a human being and those for a society relying on mechanisms that are not necessarily controllable. The difficulty by companies in regulating the spread of misinformation, as well as those by authorities to protect task-workers managed by a software infrastructure, could be just some of the effects of this technological paradigm
Complexity of pattern classes and Lipschitz property
Rademacher and Gaussian complexities are successfully used in learning theory for measuring the capacity of the class of functions to be learned. One of the most important properties for these complexities is their Lipschitz property: a composition of a class of functions with a fixed Lipschitz function may increase its complexity by at most twice the Lipschitz constant. The proof of this property is non-trivial (in contrast to the other properties) and it is believed that the proof in the Gaussian case is conceptually more difficult then the one for the Rademacher case. In this paper we give a detailed prove of the Lipschitz property for the Rademacher case and generalize the same idea to an arbitrary complexity (including the Gaussian). We also discuss a related topic about the Rademacher complexity of a class consisting of all the Lipschitz functions with a given Lipschitz constant. We show that the complexity is surprisingly low in the one-dimensional case. The question for higher dimensions remains open
Input Fast-Forwarding for Better Deep Learning
This paper introduces a new architectural framework, known as input
fast-forwarding, that can enhance the performance of deep networks. The main
idea is to incorporate a parallel path that sends representations of input
values forward to deeper network layers. This scheme is substantially different
from "deep supervision" in which the loss layer is re-introduced to earlier
layers. The parallel path provided by fast-forwarding enhances the training
process in two ways. First, it enables the individual layers to combine
higher-level information (from the standard processing path) with lower-level
information (from the fast-forward path). Second, this new architecture reduces
the problem of vanishing gradients substantially because the fast-forwarding
path provides a shorter route for gradient backpropagation. In order to
evaluate the utility of the proposed technique, a Fast-Forward Network (FFNet),
with 20 convolutional layers along with parallel fast-forward paths, has been
created and tested. The paper presents empirical results that demonstrate
improved learning capacity of FFNet due to fast-forwarding, as compared to
GoogLeNet (with deep supervision) and CaffeNet, which are 4x and 18x larger in
size, respectively. All of the source code and deep learning models described
in this paper will be made available to the entire research communityComment: Accepted in the 14th International Conference on Image Analysis and
Recognition (ICIAR) 2017, Montreal, Canad
Shortcuts to Artificial Intelligence
The current paradigm of Artiļ¬cial Intelligence emerged as the result of a series of cultural innovations, some technical and some social. Among them are apparently small design decisions, that led to a subtle reframing of the ļ¬eldās original goals, and are by now accepted as standard. They correspond to technical shortcuts, aimed at bypassing problems that were otherwise too complicated or too expensive to solve, while still delivering a viable version of AI. Far from being a series of separate problems, recent cases of unexpected eļ¬ects of AI are the consequences of those very choices that enabled the ļ¬eld to succeed, and this is why it will be diļ¬cult to solve them. In this chapter we review three of these choices, investigating their connection to some of todayās challenges in AI, including those relative to bias, value alignment, privacy and explainability. We introduce the notion of āethical debtā to describe the necessity to undertake expensive rework in the future in order to address ethical problems created by a technical system
History Playground: A Tool for Discovering Temporal Trends in Massive Textual Corpora
Recent studies have shown that macroscopic patterns of continuity and change
over the course of centuries can be detected through the analysis of time
series extracted from massive textual corpora. Similar data-driven approaches
have already revolutionised the natural sciences, and are widely believed to
hold similar potential for the humanities and social sciences, driven by the
mass-digitisation projects that are currently under way, and coupled with the
ever-increasing number of documents which are "born digital". As such, new
interactive tools are required to discover and extract macroscopic patterns
from these vast quantities of textual data. Here we present History Playground,
an interactive web-based tool for discovering trends in massive textual
corpora. The tool makes use of scalable algorithms to first extract trends from
textual corpora, before making them available for real-time search and
discovery, presenting users with an interface to explore the data. Included in
the tool are algorithms for standardization, regression, change-point detection
in the relative frequencies of ngrams, multi-term indices and comparison of
trends across different corpora
- ā¦