862 research outputs found
Compressibility and probabilistic proofs
We consider several examples of probabilistic existence proofs using
compressibility arguments, including some results that involve Lov\'asz local
lemma.Comment: Invited talk for CiE 2017 (full version
Error Bounds for Compressed Sensing Algorithms With Group Sparsity: A Unified Approach
In compressed sensing, in order to recover a sparse or nearly sparse vector
from possibly noisy measurements, the most popular approach is -norm
minimization. Upper bounds for the - norm of the error between the true
and estimated vectors are given in [1] and reviewed in [2], while bounds for
the -norm are given in [3]. When the unknown vector is not
conventionally sparse but is "group sparse" instead, a variety of alternatives
to the -norm have been proposed in the literature, including the group
LASSO, sparse group LASSO, and group LASSO with tree structured overlapping
groups. However, no error bounds are available for any of these modified
objective functions. In the present paper, a unified approach is presented for
deriving upper bounds on the error between the true vector and its
approximation, based on the notion of decomposable and -decomposable
norms. The bounds presented cover all of the norms mentioned above, and also
provide a guideline for choosing norms in future to accommodate alternate forms
of sparsity.Comment: 28 pages, final version of 1401.6623, accepted for publication. arXiv
admin note: substantial text overlap with arXiv:1401.662
Analysis of the implicit upwind finite volume scheme with rough coefficients
We study the implicit upwind finite volume scheme for numerically
approximating the linear continuity equation in the low regularity
DiPerna-Lions setting. That is, we are concerned with advecting velocity fields
that are spatially Sobolev regular and data that are merely integrable. We
prove that on unstructured regular meshes the rate of convergence of
approximate solutions generated by the upwind scheme towards the unique
distributional solution of the continuous model is at least 1/2. The numerical
error is estimated in terms of logarithmic Kantorovich-Rubinstein distances and
provides thus a bound on the rate of weak convergence.Comment: 27 pages. To appear in Numerische Mathemati
Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes
I argue that data becomes temporarily interesting by itself to some
self-improving, but computationally limited, subjective observer once he learns
to predict or compress the data in a better way, thus making it subjectively
simpler and more beautiful. Curiosity is the desire to create or discover more
non-random, non-arbitrary, regular data that is novel and surprising not in the
traditional sense of Boltzmann and Shannon but in the sense that it allows for
compression progress because its regularity was not yet known. This drive
maximizes interestingness, the first derivative of subjective beauty or
compressibility, that is, the steepness of the learning curve. It motivates
exploring infants, pure mathematicians, composers, artists, dancers, comedians,
yourself, and (since 1990) artificial systems.Comment: 35 pages, 3 figures, based on KES 2008 keynote and ALT 2007 / DS 2007
joint invited lectur
POWERPLAY: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem
Most of computer science focuses on automatically solving given computational
problems. I focus on automatically inventing or discovering problems in a way
inspired by the playful behavior of animals and humans, to train a more and
more general problem solver from scratch in an unsupervised fashion. Consider
the infinite set of all computable descriptions of tasks with possibly
computable solutions. The novel algorithmic framework POWERPLAY (2011)
continually searches the space of possible pairs of new tasks and modifications
of the current problem solver, until it finds a more powerful problem solver
that provably solves all previously learned tasks plus the new one, while the
unmodified predecessor does not. Wow-effects are achieved by continually making
previously learned skills more efficient such that they require less time and
space. New skills may (partially) re-use previously learned skills. POWERPLAY's
search orders candidate pairs of tasks and solver modifications by their
conditional computational (time & space) complexity, given the stored
experience so far. The new task and its corresponding task-solving skill are
those first found and validated. The computational costs of validating new
tasks need not grow with task repertoire size. POWERPLAY's ongoing search for
novelty keeps breaking the generalization abilities of its present solver. This
is related to Goedel's sequence of increasingly powerful formal theories based
on adding formerly unprovable statements to the axioms without affecting
previously provable theorems. The continually increasing repertoire of problem
solving procedures can be exploited by a parallel search for solutions to
additional externally posed tasks. POWERPLAY may be viewed as a greedy but
practical implementation of basic principles of creativity. A first
experimental analysis can be found in separate papers [53,54].Comment: 21 pages, additional connections to previous work, references to
first experiments with POWERPLA
Scanning and Sequential Decision Making for Multi-Dimensional Data - Part I: the Noiseless Case
We investigate the problem of scanning and prediction ("scandiction", for
short) of multidimensional data arrays. This problem arises in several aspects
of image and video processing, such as predictive coding, for example, where an
image is compressed by coding the error sequence resulting from scandicting it.
Thus, it is natural to ask what is the optimal method to scan and predict a
given image, what is the resulting minimum prediction loss, and whether there
exist specific scandiction schemes which are universal in some sense.
Specifically, we investigate the following problems: First, modeling the data
array as a random field, we wish to examine whether there exists a scandiction
scheme which is independent of the field's distribution, yet asymptotically
achieves the same performance as if this distribution was known. This question
is answered in the affirmative for the set of all spatially stationary random
fields and under mild conditions on the loss function. We then discuss the
scenario where a non-optimal scanning order is used, yet accompanied by an
optimal predictor, and derive bounds on the excess loss compared to optimal
scanning and prediction.
This paper is the first part of a two-part paper on sequential decision
making for multi-dimensional data. It deals with clean, noiseless data arrays.
The second part deals with noisy data arrays, namely, with the case where the
decision maker observes only a noisy version of the data, yet it is judged with
respect to the original, clean data.Comment: 46 pages, 2 figures. Revised version: title changed, section 1
revised, section 3.1 added, a few minor/technical corrections mad
- …