2,603 research outputs found
Efficient Sketching Algorithm for Sparse Binary Data
Recent advancement of the WWW, IOT, social network, e-commerce, etc. have
generated a large volume of data. These datasets are mostly represented by high
dimensional and sparse datasets. Many fundamental subroutines of common data
analytic tasks such as clustering, classification, ranking, nearest neighbour
search, etc. scale poorly with the dimension of the dataset. In this work, we
address this problem and propose a sketching (alternatively, dimensionality
reduction) algorithm -- \binsketch (Binary Data Sketch) -- for sparse binary
datasets. \binsketch preserves the binary version of the dataset after
sketching and maintains estimates for multiple similarity measures such as
Jaccard, Cosine, Inner-Product similarities, and Hamming distance, on the same
sketch. We present a theoretical analysis of our algorithm and complement it
with extensive experimentation on several real-world datasets. We compare the
performance of our algorithm with the state-of-the-art algorithms on the task
of mean-square-error and ranking. Our proposed algorithm offers a comparable
accuracy while suggesting a significant speedup in the dimensionality reduction
time, with respect to the other candidate algorithms. Our proposal is simple,
easy to implement, and therefore can be adopted in practice
Spatial Random Sampling: A Structure-Preserving Data Sketching Tool
Random column sampling is not guaranteed to yield data sketches that preserve
the underlying structures of the data and may not sample sufficiently from
less-populated data clusters. Also, adaptive sampling can often provide
accurate low rank approximations, yet may fall short of producing descriptive
data sketches, especially when the cluster centers are linearly dependent.
Motivated by that, this paper introduces a novel randomized column sampling
tool dubbed Spatial Random Sampling (SRS), in which data points are sampled
based on their proximity to randomly sampled points on the unit sphere. The
most compelling feature of SRS is that the corresponding probability of
sampling from a given data cluster is proportional to the surface area the
cluster occupies on the unit sphere, independently from the size of the cluster
population. Although it is fully randomized, SRS is shown to provide
descriptive and balanced data representations. The proposed idea addresses a
pressing need in data science and holds potential to inspire many novel
approaches for analysis of big data
Network Sketching: Exploiting Binary Structure in Deep CNNs
Convolutional neural networks (CNNs) with deep architectures have
substantially advanced the state-of-the-art in computer vision tasks. However,
deep networks are typically resource-intensive and thus difficult to be
deployed on mobile devices. Recently, CNNs with binary weights have shown
compelling efficiency to the community, whereas the accuracy of such models is
usually unsatisfactory in practice. In this paper, we introduce network
sketching as a novel technique of pursuing binary-weight CNNs, targeting at
more faithful inference and better trade-off for practical applications. Our
basic idea is to exploit binary structure directly in pre-trained filter banks
and produce binary-weight models via tensor expansion. The whole process can be
treated as a coarse-to-fine model approximation, akin to the pencil drawing
steps of outlining and shading. To further speedup the generated models, namely
the sketches, we also propose an associative implementation of binary tensor
convolutions. Experimental results demonstrate that a proper sketch of AlexNet
(or ResNet) outperforms the existing binary-weight models by large margins on
the ImageNet large scale classification task, while the committed memory for
network parameters only exceeds a little.Comment: To appear in CVPR201
- …