6,455 research outputs found
Optimal Trade-offs in Multi-Processor Approximate Message Passing
We consider large-scale linear inverse problems in Bayesian settings. We
follow a recent line of work that applies the approximate message passing (AMP)
framework to multi-processor (MP) computational systems, where each processor
node stores and processes a subset of rows of the measurement matrix along with
corresponding measurements. In each MP-AMP iteration, nodes of the MP system
and its fusion center exchange lossily compressed messages pertaining to their
estimates of the input. In this setup, we derive the optimal per-iteration
coding rates using dynamic programming. We analyze the excess mean squared
error (EMSE) beyond the minimum mean squared error (MMSE), and prove that, in
the limit of low EMSE, the optimal coding rates increase approximately linearly
per iteration. Additionally, we obtain that the combined cost of computation
and communication scales with the desired estimation quality according to
. Finally, we study trade-offs between the physical
costs of the estimation process including computation time, communication
loads, and the estimation quality as a multi-objective optimization problem,
and characterize the properties of the Pareto optimal surfaces.Comment: 14 pages, 8 figure
LQR based improved discrete PID controller design via optimum selection of weighting matrices using fractional order integral performance index
The continuous and discrete time Linear Quadratic Regulator (LQR) theory has
been used in this paper for the design of optimal analog and discrete PID
controllers respectively. The PID controller gains are formulated as the
optimal state-feedback gains, corresponding to the standard quadratic cost
function involving the state variables and the controller effort. A real coded
Genetic Algorithm (GA) has been used next to optimally find out the weighting
matrices, associated with the respective optimal state-feedback regulator
design while minimizing another time domain integral performance index,
comprising of a weighted sum of Integral of Time multiplied Squared Error
(ITSE) and the controller effort. The proposed methodology is extended for a
new kind of fractional order (FO) integral performance indices. The impact of
fractional order (as any arbitrary real order) cost function on the LQR tuned
PID control loops is highlighted in the present work, along with the achievable
cost of control. Guidelines for the choice of integral order of the performance
index are given depending on the characteristics of the process, to be
controlled.Comment: 22 pages, 12 figure
Non-linear dimensionality reduction: Riemannian metric estimation and the problem of geometric discovery
In recent years, manifold learning has become increasingly popular as a tool
for performing non-linear dimensionality reduction. This has led to the
development of numerous algorithms of varying degrees of complexity that aim to
recover man ifold geometry using either local or global features of the data.
Building on the Laplacian Eigenmap and Diffusionmaps framework, we propose a
new paradigm that offers a guarantee, under reasonable assumptions, that any
manifo ld learning algorithm will preserve the geometry of a data set. Our
approach is based on augmenting the output of embedding algorithms with
geometric informatio n embodied in the Riemannian metric of the manifold. We
provide an algorithm for estimating the Riemannian metric from data and
demonstrate possible application s of our approach in a variety of examples.Comment: 32 page
Persistent-Homology-based Machine Learning and its Applications -- A Survey
A suitable feature representation that can both preserve the data intrinsic
information and reduce data complexity and dimensionality is key to the
performance of machine learning models. Deeply rooted in algebraic topology,
persistent homology (PH) provides a delicate balance between data
simplification and intrinsic structure characterization, and has been applied
to various areas successfully. However, the combination of PH and machine
learning has been hindered greatly by three challenges, namely topological
representation of data, PH-based distance measurements or metrics, and PH-based
feature representation. With the development of topological data analysis,
progresses have been made on all these three problems, but widely scattered in
different literatures. In this paper, we provide a systematical review of PH
and PH-based supervised and unsupervised models from a computational
perspective. Our emphasizes are the recent development of mathematical models
and tools, including PH softwares and PH-based functions, feature
representations, kernels, and similarity models. Essentially, this paper can
work as a roadmap for the practical application of PH-based machine learning
tools. Further, we consider different topological feature representations in
different machine learning models, and investigate their impacts on the protein
secondary structure classification.Comment: 42 pages; 6 figures; 9 table
The Bayesian update: variational formulations and gradient flows
The Bayesian update can be viewed as a variational problem by characterizing
the posterior as the minimizer of a functional. The variational viewpoint is
far from new and is at the heart of popular methods for posterior
approximation. However, some of its consequences seem largely unexplored. We
focus on the following one: defining the posterior as the minimizer of a
functional gives a natural path towards the posterior by moving in the
direction of steepest descent of the functional. This idea is made precise
through the theory of gradient flows, allowing to bring new tools to the study
of Bayesian models and algorithms. Since the posterior may be characterized as
the minimizer of different functionals, several variational formulations may be
considered. We study three of them and their three associated gradient flows.
We show that, in all cases, the rate of convergence of the flows to the
posterior can be bounded by the geodesic convexity of the functional to be
minimized. Each gradient flow naturally suggests a nonlinear diffusion with the
posterior as invariant distribution. These diffusions may be discretized to
build proposals for Markov chain Monte Carlo (MCMC) algorithms. By
construction, the diffusions are guaranteed to satisfy a certain optimality
condition, and rates of convergence are given by the convexity of the
functionals. We use this observation to propose a criterion for the choice of
metric in Riemannian MCMC methods
Orientation-boosted Voxel Nets for 3D Object Recognition
Recent work has shown good recognition results in 3D object recognition using
3D convolutional networks. In this paper, we show that the object orientation
plays an important role in 3D recognition. More specifically, we argue that
objects induce different features in the network under rotation. Thus, we
approach the category-level classification task as a multi-task problem, in
which the network is trained to predict the pose of the object in addition to
the class label as a parallel task. We show that this yields significant
improvements in the classification results. We test our suggested architecture
on several datasets representing various 3D data sources: LiDAR data, CAD
models, and RGB-D images. We report state-of-the-art results on classification
as well as significant improvements in precision and speed over the baseline on
3D detection.Comment: BMVC'17 version. Added some experiments + auto-alignment of
Modelnet4
Latent Skill Embedding for Personalized Lesson Sequence Recommendation
Students in online courses generate large amounts of data that can be used to
personalize the learning process and improve quality of education. In this
paper, we present the Latent Skill Embedding (LSE), a probabilistic model of
students and educational content that can be used to recommend personalized
sequences of lessons with the goal of helping students prepare for specific
assessments. Akin to collaborative filtering for recommender systems, the
algorithm does not require students or content to be described by features, but
it learns a representation using access traces. We formulate this problem as a
regularized maximum-likelihood embedding of students, lessons, and assessments
from historical student-content interactions. An empirical evaluation on
large-scale data from Knewton, an adaptive learning technology company, shows
that this approach predicts assessment results competitively with benchmark
models and is able to discriminate between lesson sequences that lead to
mastery and failure.Comment: Under review by the ACM SIGKDD Conference on Knowledge Discovery and
Data Minin
TATi-Thermodynamic Analytics ToolkIt: TensorFlow-based software for posterior sampling in machine learning applications
With the advent of GPU-assisted hardware and maturing high-efficiency
software platforms such as TensorFlow and PyTorch, Bayesian posterior sampling
for neural networks becomes plausible. In this article we discuss Bayesian
parametrization in machine learning based on Markov Chain Monte Carlo methods,
specifically discretized stochastic differential equations such as Langevin
dynamics and extended system methods in which an ensemble of walkers is
employed to enhance sampling. We provide a glimpse of the potential of the
sampling-intensive approach by studying (and visualizing) the loss landscape of
a neural network applied to the MNIST data set. Moreover, we investigate how
the sampling efficiency itself can be significantly enhanced through an
ensemble quasi-Newton preconditioning method. This article accompanies the
release of a new TensorFlow software package, the Thermodynamic Analytics
ToolkIt, which is used in the computational experiments.Comment: 25 pages: textual improvements with results unchanged, sections on
TATi architecture and software performance removed for size constraints,
extended EQN parts, added MNIST nonlinear perceptron exampl
Deep Learning for Passive Synthetic Aperture Radar
We introduce a deep learning (DL) framework for inverse problems in imaging,
and demonstrate the advantages and applicability of this approach in passive
synthetic aperture radar (SAR) image reconstruction. We interpret image recon-
struction as a machine learning task and utilize deep networks as forward and
inverse solvers for imaging. Specifically, we design a recurrent neural network
(RNN) architecture as an inverse solver based on the iterations of proximal
gradient descent optimization methods. We further adapt the RNN architecture to
image reconstruction problems by transforming the network into a recurrent
auto-encoder, thereby allowing for unsupervised training. Our DL based inverse
solver is particularly suitable for a class of image formation problems in
which the forward model is only partially known. The ability to learn forward
models and hyper parameters combined with unsupervised training approach
establish our recurrent auto-encoder suitable for real world applications. We
demonstrate the performance of our method in passive SAR image reconstruction.
In this regime a source of opportunity, with unknown location and transmitted
waveform, is used to illuminate a scene of interest. We investigate recurrent
auto- encoder architecture based on the 1 and 0 constrained least- squares
problem. We present a projected stochastic gradient descent based training
scheme which incorporates constraints of the unknown model parameters. We
demonstrate through extensive numerical simulations that our DL based approach
out performs conventional sparse coding methods in terms of computation and
reconstructed image quality, specifically, when no information about the
transmitter is available.Comment: Submitted to IEEE Journal of Selected Topics in Signal Processin
A Scala Prototype to Generate Multigrid Solver Implementations for Different Problems and Target Multi-Core Platforms
Many problems in computational science and engineering involve partial
differential equations and thus require the numerical solution of large, sparse
(non)linear systems of equations. Multigrid is known to be one of the most
efficient methods for this purpose. However, the concrete multigrid algorithm
and its implementation highly depend on the underlying problem and hardware.
Therefore, changes in the code or many different variants are necessary to
cover all relevant cases. In this article we provide a prototype implementation
in Scala for a framework that allows abstract descriptions of PDEs, their
discretization, and their numerical solution via multigrid algorithms. From
these, one is able to generate data structures and implementations of multigrid
components required to solve elliptic PDEs on structured grids. Two different
test problems showcase our proposed automatic generation of multigrid solvers
for both CPU and GPU target platforms
- …