17,956 research outputs found
Connectionist Theory Refinement: Genetically Searching the Space of Network Topologies
An algorithm that learns from a set of examples should ideally be able to
exploit the available resources of (a) abundant computing power and (b)
domain-specific knowledge to improve its ability to generalize. Connectionist
theory-refinement systems, which use background knowledge to select a neural
network's topology and initial weights, have proven to be effective at
exploiting domain-specific knowledge; however, most do not exploit available
computing power. This weakness occurs because they lack the ability to refine
the topology of the neural networks they produce, thereby limiting
generalization, especially when given impoverished domain theories. We present
the REGENT algorithm which uses (a) domain-specific knowledge to help create an
initial population of knowledge-based neural networks and (b) genetic operators
of crossover and mutation (specifically designed for knowledge-based networks)
to continually search for better network topologies. Experiments on three
real-world domains indicate that our new algorithm is able to significantly
increase generalization compared to a standard connectionist theory-refinement
system, as well as our previous algorithm for growing knowledge-based networks.Comment: See http://www.jair.org/ for any accompanying file
Recent Progress in Image Deblurring
This paper comprehensively reviews the recent development of image
deblurring, including non-blind/blind, spatially invariant/variant deblurring
techniques. Indeed, these techniques share the same objective of inferring a
latent sharp image from one or several corresponding blurry images, while the
blind deblurring techniques are also required to derive an accurate blur
kernel. Considering the critical role of image restoration in modern imaging
systems to provide high-quality images under complex environments such as
motion, undesirable lighting conditions, and imperfect system components, image
deblurring has attracted growing attention in recent years. From the viewpoint
of how to handle the ill-posedness which is a crucial issue in deblurring
tasks, existing methods can be grouped into five categories: Bayesian inference
framework, variational methods, sparse representation-based methods,
homography-based modeling, and region-based methods. In spite of achieving a
certain level of development, image deblurring, especially the blind case, is
limited in its success by complex application conditions which make the blur
kernel hard to obtain and be spatially variant. We provide a holistic
understanding and deep insight into image deblurring in this review. An
analysis of the empirical evidence for representative methods, practical
issues, as well as a discussion of promising future directions are also
presented.Comment: 53 pages, 17 figure
Learning to Learn Kernels with Variational Random Features
In this work, we introduce kernels with random Fourier features in the
meta-learning framework to leverage their strong few-shot learning ability. We
propose meta variational random features (MetaVRF) to learn adaptive kernels
for the base-learner, which is developed in a latent variable model by treating
the random feature basis as the latent variable. We formulate the optimization
of MetaVRF as a variational inference problem by deriving an evidence lower
bound under the meta-learning framework. To incorporate shared knowledge from
related tasks, we propose a context inference of the posterior, which is
established by an LSTM architecture. The LSTM-based inference network can
effectively integrate the context information of previous tasks with
task-specific information, generating informative and adaptive features. The
learned MetaVRF can produce kernels of high representational power with a
relatively low spectral sampling rate and also enables fast adaptation to new
tasks. Experimental results on a variety of few-shot regression and
classification tasks demonstrate that MetaVRF delivers much better, or at least
competitive, performance compared to existing meta-learning alternatives.Comment: ICML'2020; code is available in:
https://github.com/Yingjun-Du/MetaVR
- …