95,327 research outputs found
Structure-preserving deep learning
Over the past few years, deep learning has risen to the foreground as a topic
of massive interest, mainly as a result of successes obtained in solving
large-scale image processing tasks. There are multiple challenging mathematical
problems involved in applying deep learning: most deep learning methods require
the solution of hard optimisation problems, and a good understanding of the
tradeoff between computational effort, amount of data and model complexity is
required to successfully design a deep learning approach for a given problem. A
large amount of progress made in deep learning has been based on heuristic
explorations, but there is a growing effort to mathematically understand the
structure in existing deep learning methods and to systematically design new
deep learning methods to preserve certain types of structure in deep learning.
In this article, we review a number of these directions: some deep neural
networks can be understood as discretisations of dynamical systems, neural
networks can be designed to have desirable properties such as invertibility or
group equivariance, and new algorithmic frameworks based on conformal
Hamiltonian systems and Riemannian manifolds to solve the optimisation problems
have been proposed. We conclude our review of each of these topics by
discussing some open problems that we consider to be interesting directions for
future research
Algorithmic and Statistical Perspectives on Large-Scale Data Analysis
In recent years, ideas from statistics and scientific computing have begun to
interact in increasingly sophisticated and fruitful ways with ideas from
computer science and the theory of algorithms to aid in the development of
improved worst-case algorithms that are useful for large-scale scientific and
Internet data analysis problems. In this chapter, I will describe two recent
examples---one having to do with selecting good columns or features from a (DNA
Single Nucleotide Polymorphism) data matrix, and the other having to do with
selecting good clusters or communities from a data graph (representing a social
or information network)---that drew on ideas from both areas and that may serve
as a model for exploiting complementary algorithmic and statistical
perspectives in order to solve applied large-scale data analysis problems.Comment: 33 pages. To appear in Uwe Naumann and Olaf Schenk, editors,
"Combinatorial Scientific Computing," Chapman and Hall/CRC Press, 201
Approximate Computation and Implicit Regularization for Very Large-scale Data Analysis
Database theory and database practice are typically the domain of computer
scientists who adopt what may be termed an algorithmic perspective on their
data. This perspective is very different than the more statistical perspective
adopted by statisticians, scientific computers, machine learners, and other who
work on what may be broadly termed statistical data analysis. In this article,
I will address fundamental aspects of this algorithmic-statistical disconnect,
with an eye to bridging the gap between these two very different approaches. A
concept that lies at the heart of this disconnect is that of statistical
regularization, a notion that has to do with how robust is the output of an
algorithm to the noise properties of the input data. Although it is nearly
completely absent from computer science, which historically has taken the input
data as given and modeled algorithms discretely, regularization in one form or
another is central to nearly every application domain that applies algorithms
to noisy data. By using several case studies, I will illustrate, both
theoretically and empirically, the nonobvious fact that approximate
computation, in and of itself, can implicitly lead to statistical
regularization. This and other recent work suggests that, by exploiting in a
more principled way the statistical properties implicit in worst-case
algorithms, one can in many cases satisfy the bicriteria of having algorithms
that are scalable to very large-scale databases and that also have good
inferential or predictive properties.Comment: To appear in the Proceedings of the 2012 ACM Symposium on Principles
of Database Systems (PODS 2012
Numerical Investigation of Graph Spectra and Information Interpretability of Eigenvalues
We undertake an extensive numerical investigation of the graph spectra of
thousands regular graphs, a set of random Erd\"os-R\'enyi graphs, the two most
popular types of complex networks and an evolving genetic network by using
novel conceptual and experimental tools. Our objective in so doing is to
contribute to an understanding of the meaning of the Eigenvalues of a graph
relative to its topological and information-theoretic properties. We introduce
a technique for identifying the most informative Eigenvalues of evolving
networks by comparing graph spectra behavior to their algorithmic complexity.
We suggest that extending techniques can be used to further investigate the
behavior of evolving biological networks. In the extended version of this paper
we apply these techniques to seven tissue specific regulatory networks as
static example and network of a na\"ive pluripotent immune cell in the process
of differentiating towards a Th17 cell as evolving example, finding the most
and least informative Eigenvalues at every stage.Comment: Forthcoming in 3rd International Work-Conference on Bioinformatics
and Biomedical Engineering (IWBBIO), Lecture Notes in Bioinformatics, 201
- …