2,864 research outputs found
Lipschitz and Comparator-Norm Adaptivity in Online Learning
We study Online Convex Optimization in the unbounded setting where neither
predictions nor gradient are constrained. The goal is to simultaneously adapt
to both the sequence of gradients and the comparator. We first develop
parameter-free and scale-free algorithms for a simplified setting with hints.
We present two versions: the first adapts to the squared norms of both
comparator and gradients separately using time per round, the second
adapts to their squared inner products (which measure variance only in the
comparator direction) in time per round. We then generalize two prior
reductions to the unbounded setting; one to not need hints, and a second to
deal with the range ratio problem (which already arises in prior work). We
discuss their optimality in light of prior and new lower bounds. We apply our
methods to obtain sharper regret bounds for scale-invariant online prediction
with linear models.Comment: 30 Pages, 1 Figur
Parameter-free Mirror Descent
We develop a modified online mirror descent framework that is suitable for
building adaptive and parameter-free algorithms in unbounded domains. We
leverage this technique to develop the first unconstrained online linear
optimization algorithm achieving an optimal dynamic regret bound, and we
further demonstrate that natural strategies based on
Follow-the-Regularized-Leader are unable to achieve similar results. We also
apply our mirror descent framework to build new parameter-free implicit
updates, as well as a simplified and improved unconstrained scale-free
algorithm.Comment: 52 pages. v3: published at COLT 2022 + fixed typos; v2: improved the
algorithms in sections 3, 5, and 6 (tighter regret, simpler updates and
analysis), corrected minor technical details and fixed typo
Relative Entailment Among Probabilistic Implications
We study a natural variant of the implicational fragment of propositional
logic. Its formulas are pairs of conjunctions of positive literals, related
together by an implicational-like connective; the semantics of this sort of
implication is defined in terms of a threshold on a conditional probability of
the consequent, given the antecedent: we are dealing with what the data
analysis community calls confidence of partial implications or association
rules. Existing studies of redundancy among these partial implications have
characterized so far only entailment from one premise and entailment from two
premises, both in the stand-alone case and in the case of presence of
additional classical implications (this is what we call "relative entailment").
By exploiting a previously noted alternative view of the entailment in terms of
linear programming duality, we characterize exactly the cases of entailment
from arbitrary numbers of premises, again both in the stand-alone case and in
the case of presence of additional classical implications. As a result, we
obtain decision algorithms of better complexity; additionally, for each
potential case of entailment, we identify a critical confidence threshold and
show that it is, actually, intrinsic to each set of premises and antecedent of
the conclusion
Combination Strategies for Semantic Role Labeling
This paper introduces and analyzes a battery of inference models for the
problem of semantic role labeling: one based on constraint satisfaction, and
several strategies that model the inference as a meta-learning problem using
discriminative classifiers. These classifiers are developed with a rich set of
novel features that encode proposition and sentence-level information. To our
knowledge, this is the first work that: (a) performs a thorough analysis of
learning-based inference models for semantic role labeling, and (b) compares
several inference strategies in this context. We evaluate the proposed
inference strategies in the framework of the CoNLL-2005 shared task using only
automatically-generated syntactic information. The extensive experimental
evaluation and analysis indicates that all the proposed inference strategies
are successful -they all outperform the current best results reported in the
CoNLL-2005 evaluation exercise- but each of the proposed approaches has its
advantages and disadvantages. Several important traits of a state-of-the-art
SRL combination strategy emerge from this analysis: (i) individual models
should be combined at the granularity of candidate arguments rather than at the
granularity of complete solutions; (ii) the best combination strategy uses an
inference model based in learning; and (iii) the learning-based inference
benefits from max-margin classifiers and global feedback
- …