161,789 research outputs found
Online Learning with an Almost Perfect Expert
We study the multiclass online learning problem where a forecaster makes a
sequence of predictions using the advice of experts. Our main contribution
is to analyze the regime where the best expert makes at most mistakes and
to show that when , the expected number of mistakes made by
the optimal forecaster is at most . We also describe
an adversary strategy showing that this bound is tight and that the worst case
is attained for binary prediction
Estimating Uncertainty Online Against an Adversary
Assessing uncertainty is an important step towards ensuring the safety and
reliability of machine learning systems. Existing uncertainty estimation
techniques may fail when their modeling assumptions are not met, e.g. when the
data distribution differs from the one seen at training time. Here, we propose
techniques that assess a classification algorithm's uncertainty via calibrated
probabilities (i.e. probabilities that match empirical outcome frequencies in
the long run) and which are guaranteed to be reliable (i.e. accurate and
calibrated) on out-of-distribution input, including input generated by an
adversary. This represents an extension of classical online learning that
handles uncertainty in addition to guaranteeing accuracy under adversarial
assumptions. We establish formal guarantees for our methods, and we validate
them on two real-world problems: question answering and medical diagnosis from
genomic data
Vision of a Visipedia
The web is not perfect: while text is easily
searched and organized, pictures (the vast majority of the bits
that one can find online) are not. In order to see how one could
improve the web and make pictures first-class citizens of the
web, I explore the idea of Visipedia, a visual interface for
Wikipedia that is able to answer visual queries and enables
experts to contribute and organize visual knowledge. Five
distinct groups of humans would interact through Visipedia:
users, experts, editors, visual workers, and machine vision
scientists. The latter would gradually build automata able to
interpret images. I explore some of the technical challenges
involved in making Visipedia happen. I argue that Visipedia will
likely grow organically, combining state-of-the-art machine
vision with human labor
On the organisation of program verification competitions
In this paper, we discuss the challenges that have to be addressed when organising program verification competitions. Our focus is on competitions for verification systems where the participants both formalise an informally stated requirement and (typically) provide some guidance for the tool to show it. The paper draws its insights from our experiences with organising a program verification competition at FoVeOOS 2011. We discuss in particular the following aspects: challenge selection, on-site versus online organisation, team composition and judging. We conclude with a list of recommendations for future competition organisers
Recommended from our members
Self-regulated learning and knowledge sharing in the workplace
This study explores how experts in a global multinational company self-regulate their learning. It investigates experts‟ perceptions of the impact of knowledge sharing on their learning and work. Findings indicate that self-regulated learning (SRL) is a highly social process that is structured by and deeply integrated with work tasks. Experts tend to draw heavily upon their personal networks of trusted colleagues in the process of diagnosing and attaining their learning goals. In contradiction to existing models, SRL in the workplace does not appear to be a clearly delineated, linear process comprised of discrete stages. Further research is needed to understand tacit practices of SRL in the workplace
- …