1,055 research outputs found
Descent methods for Nonnegative Matrix Factorization
In this paper, we present several descent methods that can be applied to
nonnegative matrix factorization and we analyze a recently developped fast
block coordinate method called Rank-one Residue Iteration (RRI). We also give a
comparison of these different methods and show that the new block coordinate
method has better properties in terms of approximation error and complexity. By
interpreting this method as a rank-one approximation of the residue matrix, we
prove that it \emph{converges} and also extend it to the nonnegative tensor
factorization and introduce some variants of the method by imposing some
additional controllable constraints such as: sparsity, discreteness and
smoothness.Comment: 47 pages. New convergence proof using damped version of RRI. To
appear in Numerical Linear Algebra in Signals, Systems and Control. Accepted.
Illustrating Matlab code is included in the source bundl
Algorithms, applications and systems towards interpretable pattern mining from multi-aspect data
How do humans move around in the urban space and how do they differ when the city undergoes terrorist attacks? How do users behave in Massive Open Online courses~(MOOCs) and how do they differ if some of them achieve certificates while some of them not? What areas in the court elite players, such as Stephen Curry, LeBron James, like to make their shots in the course of the game? How can we uncover the hidden habits that govern our online purchases? Are there unspoken agendas in how different states pass legislation of certain kinds? At the heart of these seemingly unconnected puzzles is this same mystery of multi-aspect mining, i.g., how can we mine and interpret the hidden pattern from a dataset that simultaneously reveals the associations, or changes of the associations, among various aspects of the data (e.g., a shot could be described with three aspects, player, time of the game, and area in the court)? Solving this problem could open gates to a deep understanding of underlying mechanisms for many real-world phenomena. While much of the research in multi-aspect mining contribute broad scope of innovations in the mining part, interpretation of patterns from the perspective of users (or domain experts) is often overlooked. Questions like what do they require for patterns, how good are the patterns, or how to read them, have barely been addressed. Without efficient and effective ways of involving users in the process of multi-aspect mining, the results are likely to lead to something difficult for them to comprehend.
This dissertation proposes the M^3 framework, which consists of multiplex pattern discovery, multifaceted pattern evaluation, and multipurpose pattern presentation, to tackle the challenges of multi-aspect pattern discovery. Based on this framework, we develop algorithms, applications, and analytic systems to enable interpretable pattern discovery from multi-aspect data. Following the concept of meaningful multiplex pattern discovery, we propose PairFac to close the gap between human information needs and naive mining optimization. We demonstrate its effectiveness in the context of impact discovery in the aftermath of urban disasters. We develop iDisc to target the crossing of multiplex pattern discovery with multifaceted pattern evaluation. iDisc meets the specific information need in understanding multi-level, contrastive behavior patterns. As an example, we use iDisc to predict student performance outcomes in Massive Open Online Courses given users' latent behaviors. FacIt is an interactive visual analytic system that sits at the intersection of all three components and enables for interpretable, fine-tunable, and scrutinizable pattern discovery from multi-aspect data. We demonstrate each work's significance and implications in its respective problem context. As a whole, this series of studies is an effort to instantiate the M^3 framework and push the field of multi-aspect mining towards a more human-centric process in real-world applications
LASS: a simple assignment model with Laplacian smoothing
We consider the problem of learning soft assignments of items to
categories given two sources of information: an item-category similarity
matrix, which encourages items to be assigned to categories they are similar to
(and to not be assigned to categories they are dissimilar to), and an item-item
similarity matrix, which encourages similar items to have similar assignments.
We propose a simple quadratic programming model that captures this intuition.
We give necessary conditions for its solution to be unique, define an
out-of-sample mapping, and derive a simple, effective training algorithm based
on the alternating direction method of multipliers. The model predicts
reasonable assignments from even a few similarity values, and can be seen as a
generalization of semisupervised learning. It is particularly useful when items
naturally belong to multiple categories, as for example when annotating
documents with keywords or pictures with tags, with partially tagged items, or
when the categories have complex interrelations (e.g. hierarchical) that are
unknown.Comment: 20 pages, 4 figures. A shorter version appears in AAAI 201
Formation Shape Control Based on Distance Measurements Using Lie Bracket Approximations
We study the problem of distance-based formation control in autonomous
multi-agent systems in which only distance measurements are available. This
means that the target formations as well as the sensed variables are both
determined by distances. We propose a fully distributed distance-only control
law, which requires neither a time synchronization of the agents nor storage of
measured data. The approach is applicable to point agents in the Euclidean
space of arbitrary dimension. Under the assumption of infinitesimal rigidity of
the target formations, we show that the proposed control law induces local
uniform asymptotic stability. Our approach involves sinusoidal perturbations in
order to extract information about the negative gradient direction of each
agent's local potential function. An averaging analysis reveals that the
gradient information originates from an approximation of Lie brackets of
certain vector fields. The method is based on a recently introduced approach to
the problem of extremum seeking control. We discuss the relation in the paper
Quadratic-exponential coherent feedback control of linear quantum stochastic systems
This paper considers a risk-sensitive optimal control problem for a
field-mediated interconnection of a quantum plant with a coherent
(measurement-free) quantum controller. The plant and the controller are
multimode open quantum harmonic oscillators governed by linear quantum
stochastic differential equations, which are coupled to each other and driven
by multichannel quantum Wiener processes modelling the external bosonic fields.
The control objective is to internally stabilize the closed-loop system and
minimize the infinite-horizon asymptotic growth rate of a quadratic-exponential
functional which penalizes the plant variables and the controller output. We
obtain first-order necessary conditions of optimality for this problem by
computing the partial Frechet derivatives of the cost functional with respect
to the energy and coupling matrices of the controller in frequency domain and
state space. An infinitesimal equivalence between the risk-sensitive and
weighted coherent quantum LQG control problems is also established. In addition
to variational methods, we employ spectral factorizations and infinite cascades
of auxiliary classical systems. Their truncations are applicable to numerical
optimization algorithms (such as the gradient descent) for coherent quantum
risk-sensitive feedback synthesis.Comment: 29 pages, 3 figure
New optimization methods in predictive control
This thesis is mainly concerned with the efficient solution of a linear discrete-time
finite horizon optimal control problem (FHOCP) with quadratic cost and linear constraints
on the states and inputs. In predictive control, such a FHOCP needs to be
solved online at each sampling instant. In order to solve such a FHOCP, it is necessary
to solve a quadratic programming (QP) problem. Interior point methods (IPMs) have
proven to be an efficient way of solving quadratic programming problems. A linear system
of equations needs to be solved in each iteration of an IPM. The ill-conditioning
of this linear system in the later iterations of the IPM prevents the use of an iterative
method in solving the linear system due to a very slow rate of convergence; in some cases
the solution never reaches the desired accuracy. A new well-conditioned IPM, which increases
the rate of convergence of the iterative method is proposed. The computational
advantage is obtained by the use of an inexact Newton method along with the use of
novel preconditioners.
A new warm-start strategy is also presented to solve a QP with an interior-point
method whose data is slightly perturbed from the previous QP. The effectiveness of
this warm-start strategy is demonstrated on a number of available online benchmark
problems. Numerical results indicate that the proposed technique depends upon the
size of perturbation and it leads to a reduction of 30-74% in floating point operations
compared to a cold-start interior point method.
Following the main theme of this thesis, which is to improve the computational efficiency
of an algorithm, an efficient algorithm for solving the coupled Sylvester equation
that arises in converting a system of linear differential-algebraic equations (DAEs) to
ordinary differential equations is also presented. A significant computational advantage
is obtained by exploiting the structure of the involved matrices. The proposed algorithm
removes the need to solve a standard Sylvester equation or to invert a matrix. The
improved performance of this new method over existing techniques is demonstrated by
comparing the number of floating-point operations and via numerical examples
- …