3 research outputs found
Scalar Quantization as Sparse Least Square Optimization
Quantization can be used to form new vectors/matrices with shared values
close to the original. In recent years, the popularity of scalar quantization
for value-sharing applications has been soaring as it has been found huge
utilities in reducing the complexity of neural networks. Existing
clustering-based quantization techniques, while being well-developed, have
multiple drawbacks including the dependency of the random seed, empty or
out-of-the-range clusters, and high time complexity for a large number of
clusters. To overcome these problems, in this paper, the problem of scalar
quantization is examined from a new perspective, namely sparse least square
optimization. Specifically, inspired by the property of sparse least square
regression, several quantization algorithms based on least square are
proposed. In addition, similar schemes with and
regularization are proposed. Furthermore, to compute quantization results with
a given amount of values/clusters, this paper designed an iterative method and
a clustering-based method, and both of them are built on sparse least square.
The paper shows that the latter method is mathematically equivalent to an
improved version of k-means clustering-based quantization algorithm, although
the two algorithms originated from different intuitions. The algorithms
proposed were tested with three types of data and their computational
performances, including information loss, time consumption, and the
distribution of the values of the sparse vectors, were compared and analyzed.
The paper offers a new perspective to probe the area of quantization, and the
algorithms proposed can outperform existing methods especially under some
bit-width reduction scenarios, when the required post-quantization resolution
(number of values) is not significantly lower than the original number
A Fast Clustering Algorithm based on pruning unnecessary distance computations in DBSCAN for High-Dimensional Data
Clustering is an important technique to deal with large scale data which are explosively created in internet. Most data are high-dimensional with a lot of noise, which brings great challenges to retrieval, classification and understanding. No current existing approach is “optimal” for large scale data. For example, DBSCAN requires O(n2) time, Fast-DBSCAN only works well in 2 dimensions, and ρ-Approximate DBSCAN runs in O(n) expected time which needs dimension D to be a relative small constant for the linear running time to hold. However, we prove theoretically and experimentally that ρ-Approximate DBSCAN degenerates to an O(n2) algorithm in very high dimension such that 2D > > n. In this paper, we propose a novel local neighborhood searching technique, and apply it to improve DBSCAN, named as NQ-DBSCAN, such that a large number of unnecessary distance computations can be effectively reduced. Theoretical analysis and experimental results show that NQ-DBSCAN averagely runs in O(n*log(n)) with the help of indexing technique, and the best case is O(n) if proper parameters are used, which makes it suitable for many realtime data