249 research outputs found
Dimensionality Reduction for k-Means Clustering and Low Rank Approximation
We show how to approximate a data matrix with a much smaller
sketch that can be used to solve a general class of
constrained k-rank approximation problems to within error.
Importantly, this class of problems includes -means clustering and
unconstrained low rank approximation (i.e. principal component analysis). By
reducing data points to just dimensions, our methods generically
accelerate any exact, approximate, or heuristic algorithm for these ubiquitous
problems.
For -means dimensionality reduction, we provide relative
error results for many common sketching techniques, including random row
projection, column selection, and approximate SVD. For approximate principal
component analysis, we give a simple alternative to known algorithms that has
applications in the streaming setting. Additionally, we extend recent work on
column-based matrix reconstruction, giving column subsets that not only `cover'
a good subspace for \bv{A}, but can be used directly to compute this
subspace.
Finally, for -means clustering, we show how to achieve a
approximation by Johnson-Lindenstrauss projecting data points to just dimensions. This gives the first result that leverages the
specific structure of -means to achieve dimension independent of input size
and sublinear in
Uniform Sampling for Matrix Approximation
Random sampling has become a critical tool in solving massive matrix
problems. For linear regression, a small, manageable set of data rows can be
randomly selected to approximate a tall, skinny data matrix, improving
processing time significantly. For theoretical performance guarantees, each row
must be sampled with probability proportional to its statistical leverage
score. Unfortunately, leverage scores are difficult to compute.
A simple alternative is to sample rows uniformly at random. While this often
works, uniform sampling will eliminate critical row information for many
natural instances. We take a fresh look at uniform sampling by examining what
information it does preserve. Specifically, we show that uniform sampling
yields a matrix that, in some sense, well approximates a large fraction of the
original. While this weak form of approximation is not enough for solving
linear regression directly, it is enough to compute a better approximation.
This observation leads to simple iterative row sampling algorithms for matrix
approximation that run in input-sparsity time and preserve row structure and
sparsity at all intermediate steps. In addition to an improved understanding of
uniform sampling, our main proof introduces a structural result of independent
interest: we show that every matrix can be made to have low coherence by
reweighting a small subset of its rows
Online Row Sampling
Finding a small spectral approximation for a tall matrix is
a fundamental numerical primitive. For a number of reasons, one often seeks an
approximation whose rows are sampled from those of . Row sampling improves
interpretability, saves space when is sparse, and preserves row structure,
which is especially important, for example, when represents a graph.
However, correctly sampling rows from can be costly when the matrix is
large and cannot be stored and processed in memory. Hence, a number of recent
publications focus on row sampling in the streaming setting, using little more
space than what is required to store the outputted approximation [KL13,
KLM+14].
Inspired by a growing body of work on online algorithms for machine learning
and data analysis, we extend this work to a more restrictive online setting: we
read rows of one by one and immediately decide whether each row should be
kept in the spectral approximation or discarded, without ever retracting these
decisions. We present an extremely simple algorithm that approximates up to
multiplicative error and additive error using online samples, with memory overhead
proportional to the cost of storing the spectral approximation. We also present
an algorithm that uses ) memory but only requires
samples, which we show is
optimal.
Our methods are clean and intuitive, allow for lower memory usage than prior
work, and expose new theoretical properties of leverage score based matrix
approximation
Recommended from our members
Online Row Sampling
Finding a small spectral approximation for a tall n X d matrix A is a fundamental numerical primitive. For a number of reasons, one often seeks an approximation whose rows are sampled from those of A. Row sampling improves interpretability, saves space when A is sparse, and preserves structure, which is important, e.g., when A represents a graph.
However, correct sampling rows from A can be costly when the matrix is large and cannot be stored and processed in memory. Hence, a number of recent publications focus on row sampling in the streaming setting, using little more space than what is required to store the returned approximation (Kelner--Levin, Theory Comput. Sys. 2013, Kapralov et al., SIAM J. Comp. 2017).
Inspired by a growing body of work on online algorithms for machine learning and data analysis, we extend this work to a more restrictive online setting: we read rows of A one by one and immediately decide whether each row should be kept in the spectral approximation or discarded, without ever retracting these decisions. We present an extremely simple algorithm that approximates A up to multiplicative-error 1+ϵ and additive-error δ using O(d log d log(ϵ‖A‖22/δ)/ϵ2) online samples, with memory overhead proportional to the cost of storing the spectral approximation. We also present an algorithm that uses O(d2) memory but only requires O(d log d log(ϵ‖A‖22/δ)/ϵ2) samples, which we show is optimal.
Our methods are clean and intuitive, allow for lower memory usage than prior work, and expose new theoretical properties of leverage score based matrix approximation
Critical Phenomena in Neutron Stars I: Linearly Unstable Nonrotating Models
We consider the evolution in full general relativity of a family of linearly
unstable isolated spherical neutron stars under the effects of very small,
perturbations as induced by the truncation error. Using a simple ideal-fluid
equation of state we find that this system exhibits a type-I critical
behaviour, thus confirming the conclusions reached by Liebling et al. [1] for
rotating magnetized stars. Exploiting the relative simplicity of our system, we
are able carry out a more in-depth study providing solid evidences of the
criticality of this phenomenon and also to give a simple interpretation of the
putative critical solution as a spherical solution with the unstable mode being
the fundamental F-mode. Hence for any choice of the polytropic constant, the
critical solution will distinguish the set of subcritical models migrating to
the stable branch of the models of equilibrium from the set of subcritical
models collapsing to a black hole. Finally, we study how the dynamics changes
when the numerically perturbation is replaced by a finite-size, resolution
independent velocity perturbation and show that in such cases a nearly-critical
solution can be changed into either a sub or supercritical. The work reported
here also lays the basis for the analysis carried in a companion paper, where
the critical behaviour in the the head-on collision of two neutron stars is
instead considered [2].Comment: 15 pages, 9 figure
Concern about the spread of the invader seaweed Caulerpa taxifolia var. distichophylla (Chlorophyta: Caulerpales) towards the West Mediterranean
The new Australian alien seaweed Caulerpa taxifolia var. distichophylla, after being established along the Turkish Mediterranean coast in 2006, was recorded in Southern Sicily in 2007. Since then local fishermen claimed support to counteract the effects of entanglement of large amounts of the alien strain wrack in their trammel nets, causing the gear to become ineffective. The further northward and westward spread of the new alien strain is supposed to be limited by winter temperature. We present novel data confirming that the new alien strain is fully naturalized in Central Mediterranean and is expanding its range beyond such limit (i.e. the 15°C February isotherm), thus becoming potentially able to colonize the western basin. By means of a preliminary estimation of effects on native polychaete assemblages, and considering some peculiarities of Sicily (mostly linked to its geographical position in the Mediterranean Sea), the risk linked to the increasing range of distribution of the invasive algae is highlighted
Optimal Sketching Bounds for Sparse Linear Regression
We study oblivious sketching for -sparse linear regression under various
loss functions such as an norm, or from a broad class of hinge-like
loss functions, which includes the logistic and ReLU losses. We show that for
sparse norm regression, there is a distribution over oblivious
sketches with rows, which is tight up to a
constant factor. This extends to loss with an additional additive
term in the upper bound. This
establishes a surprising separation from the related sparse recovery problem,
which is an important special case of sparse regression. For this problem,
under the norm, we observe an upper bound of rows, showing that sparse recovery is
strictly easier to sketch than sparse regression. For sparse regression under
hinge-like loss functions including sparse logistic and sparse ReLU regression,
we give the first known sketching bounds that achieve rows showing that
rows suffice, where
is a natural complexity parameter needed to obtain relative error bounds for
these loss functions. We again show that this dimension is tight, up to lower
order terms and the dependence on . Finally, we show that similar
sketching bounds can be achieved for LASSO regression, a popular convex
relaxation of sparse regression, where one aims to minimize
over . We show that sketching
dimension suffices and that the dependence
on and is tight.Comment: AISTATS 202
In vitro fermentation and chemical characteristics of mediterranean by-products for swine nutrition
The purpose of the study is to determine the nutritional characteristics of some by-products derived from fruit juice and olive oil production to evaluate their use in pig nutrition. Five by-products of citrus fruit (three citrus fruit pulp and two molasses) and three by-products of olive oil (olive cake) obtained by different varieties are analysed for chemical composition. The fermentation characteristics are evaluated in vitro using the gas production technique with swine faecal inoculum. All the citrus by-products are highly fermentable, producing gas and a high amount of short-chain fatty acids. The fermentation kinetics vary when comparing pulps and molasses. Citrus fruit pulps show lower and slower fermentation rates than molasses. The olive oil by-products, compared to citrus fruits ones, are richer in NDF and ADL. These characteristics negatively affect all the fermentation parameters. Therefore, the high concentration of fiber and lipids represents a key aspect in the nutrition of fattening pigs. The preliminary results obtained in this study confirm that the use of by-products in pig nutrition could represent a valid opportunity the reduce the livestock economic cost and environmental impact
Black hole production in tachyonic preheating
We present fully non-linear simulations of a self-interacting scalar field in
the early universe undergoing tachyonic preheating. We find that density
perturbations on sub-horizon scales which are amplified by tachyonic
instability maintain long range correlations even during the succeeding
parametric resonance, in contrast to the standard models of preheating
dominated by parametric resonance. As a result the final spectrum exhibits
memory and is not universal in shape. We find that throughout the subsequent
era of parametric resonance the equation of state of the universe is almost
dust-like, hence the Jeans wavelength is much smaller than the horizon scale.
If our 2D simulations are accurate reflections of the situation in 3D, then
there are wide regions of parameter space ruled out by over-production of black
holes. It is likely however that realistic parameter values, consistent with
COBE/WMAP normalisation, are safetly outside this black hole over-production
region.Comment: 6pages, 7figures, figures correcte
- …