4,892 research outputs found
Structure-Aware Sampling: Flexible and Accurate Summarization
In processing large quantities of data, a fundamental problem is to obtain a
summary which supports approximate query answering. Random sampling yields
flexible summaries which naturally support subset-sum queries with unbiased
estimators and well-understood confidence bounds.
Classic sample-based summaries, however, are designed for arbitrary subset
queries and are oblivious to the structure in the set of keys. The particular
structure, such as hierarchy, order, or product space (multi-dimensional),
makes range queries much more relevant for most analysis of the data.
Dedicated summarization algorithms for range-sum queries have also been
extensively studied. They can outperform existing sampling schemes in terms of
accuracy on range queries per summary size. Their accuracy, however, rapidly
degrades when, as is often the case, the query spans multiple ranges. They are
also less flexible - being targeted for range sum queries alone - and are often
quite costly to build and use.
In this paper we propose and evaluate variance optimal sampling schemes that
are structure-aware. These summaries improve over the accuracy of existing
structure-oblivious sampling schemes on range queries while retaining the
benefits of sample-based summaries: flexible summaries, with high accuracy on
both range queries and arbitrary subset queries
Fast, scalable, Bayesian spike identification for multi-electrode arrays
We present an algorithm to identify individual neural spikes observed on
high-density multi-electrode arrays (MEAs). Our method can distinguish large
numbers of distinct neural units, even when spikes overlap, and accounts for
intrinsic variability of spikes from each unit. As MEAs grow larger, it is
important to find spike-identification methods that are scalable, that is, the
computational cost of spike fitting should scale well with the number of units
observed. Our algorithm accomplishes this goal, and is fast, because it
exploits the spatial locality of each unit and the basic biophysics of
extracellular signal propagation. Human intervention is minimized and
streamlined via a graphical interface. We illustrate our method on data from a
mammalian retina preparation and document its performance on simulated data
consisting of spikes added to experimentally measured background noise. The
algorithm is highly accurate
- …