83 research outputs found
Information theoretic approach to interactive learning
The principles of statistical mechanics and information theory play an
important role in learning and have inspired both theory and the design of
numerous machine learning algorithms. The new aspect in this paper is a focus
on integrating feedback from the learner. A quantitative approach to
interactive learning and adaptive behavior is proposed, integrating model- and
decision-making into one theoretical framework. This paper follows simple
principles by requiring that the observer's world model and action policy
should result in maximal predictive power at minimal complexity. Classes of
optimal action policies and of optimal models are derived from an objective
function that reflects this trade-off between prediction and complexity. The
resulting optimal models then summarize, at different levels of abstraction,
the process's causal organization in the presence of the learner's actions. A
fundamental consequence of the proposed principle is that the learner's optimal
action policies balance exploration and control as an emerging property.
Interestingly, the explorative component is present in the absence of policy
randomness, i.e. in the optimal deterministic behavior. This is a direct result
of requiring maximal predictive power in the presence of feedback.Comment: 6 page
Agnostic Active Learning Without Constraints
We present and analyze an agnostic active learning algorithm that works
without keeping a version space. This is unlike all previous approaches where a
restricted set of candidate hypotheses is maintained throughout learning, and
only hypotheses from this set are ever returned. By avoiding this version space
approach, our algorithm sheds the computational burden and brittleness
associated with maintaining version spaces, yet still allows for substantial
improvements over supervised learning for classification
Multi-View Active Learning in the Non-Realizable Case
The sample complexity of active learning under the realizability assumption
has been well-studied. The realizability assumption, however, rarely holds in
practice. In this paper, we theoretically characterize the sample complexity of
active learning in the non-realizable case under multi-view setting. We prove
that, with unbounded Tsybakov noise, the sample complexity of multi-view active
learning can be , contrasting to
single-view setting where the polynomial improvement is the best possible
achievement. We also prove that in general multi-view setting the sample
complexity of active learning with unbounded Tsybakov noise is
, where the order of is
independent of the parameter in Tsybakov noise, contrasting to previous
polynomial bounds where the order of is related to the parameter
in Tsybakov noise.Comment: 22 pages, 1 figur
SoftwareTesting with Active Learning in a Graph
Motivated by Structural Statistical Software Testing (SSST), this paper
is interested in sampling the feasible execution paths in the
control flow graph of the program being tested. For some complex programs,
the fraction of feasible paths becomes tiny, ranging in
. When relying on the uniform sampling of the
program paths, SSST is thus hindered by
the non-Markovian nature of the ``feasible path\u27\u27 concept, due to the
long-range dependencies between the program nodes.
A divide and generate approach relying on an extended Parikh Map
representation is proposed to address this limitation;
experimental validation on real-world
and artificial problems demonstrates gains of orders of magnitude compared
to the state of the art
Adaptive sensing performance lower bounds for sparse signal detection and support estimation
This paper gives a precise characterization of the fundamental limits of
adaptive sensing for diverse estimation and testing problems concerning sparse
signals. We consider in particular the setting introduced in (IEEE Trans.
Inform. Theory 57 (2011) 6222-6235) and show necessary conditions on the
minimum signal magnitude for both detection and estimation: if is a sparse vector with non-zero components then it
can be reliably detected in noise provided the magnitude of the non-zero
components exceeds . Furthermore, the signal support can be exactly
identified provided the minimum magnitude exceeds . Notably
there is no dependence on , the extrinsic signal dimension. These results
show that the adaptive sensing methodologies proposed previously in the
literature are essentially optimal, and cannot be substantially improved. In
addition, these results provide further insights on the limits of adaptive
compressive sensing.Comment: Published in at http://dx.doi.org/10.3150/13-BEJ555 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
- …