Efficient Randomized Algorithms for the Fixed-Precision Low-Rank Matrix Approximation

Randomized algorithms for low-rank matrix approximation are investigated, with the emphasis on the fixed-precision problem and computational efficiency for handling large matrices. The algorithms are based on the so-called QB factorization, where Q is an orthonormal matrix. Firstly, a mechanism for calculating the approximation error in Frobenius norm is proposed, which enables efficient adaptive rank determination for large and/or sparse matrix. It can be combined with any QB-form factorization algorithm in which B's rows are incrementally generated. Based on the blocked randQB algorithm by P.-G. Martinsson and S. Voronin, this results in an algorithm called randQB EI. Then, we further revise the algorithm to obtain a pass-efficient algorithm, randQB FP, which is mathematically equivalent to the existing randQB algorithms and also suitable for the fixed-precision problem. Especially, randQB FP can serve as a single-pass algorithm for calculating leading singular values, under certain condition. With large and/or sparse test matrices, we have empirically validated the merits of the proposed techniques, which exhibit remarkable speedup and memory saving over the blocked randQB algorithm. We have also demonstrated that the single-pass algorithm derived by randQB FP is much more accurate than an existing single-pass algorithm. And with data from a scenic image and an information retrieval application, we have shown the advantages of the proposed algorithms over the adaptive range finder algorithm for solving the fixed-precision problem.Comment: 21 pages, 10 figure

Triboelectric nanogenerators for self-powered systems

Everything is expected to be informatically connected in the era of the distributed Internet of Things (IoT), artificial intelligence and big data by numerous sensors. Although the power required for a single mobile, wireless and distributed sensor is small, the total amount is gigantic. Batteries are usually applied as their power source, but they cost huge human and financial resources for recharging and replacement, as well as cause environmental pollution. Therefore, self-powered sensors and systems without the needing of batteries are highly desired.Triboelectric nanogenerators (TENGs), based on contact electrification and electrostatic induction effect, shed light on self-powered systems by harvesting mechanical energy from the environment. However, for practical applications, the surface charge density, output and durability still need to be enhanced. In addition, the structure and functionality of TENGs should be designed to be scenario-adaptive. This thesis first realized high-performance TENGs by exploring novel intermediate nanomaterials, including 2D smectite clay nanosheets, 2D mica nanosheets and ordered mesoporous SiO2 nanoparticles as charge donors and storage sites to enhance triboelectric charge density. Then, to improve the durability of TENGs, a nonpolymer-based triboelectric pair composed of diamond-like carbon and glass with excellent durability and triboelectric output is proposed. Finally, to endow specific functionalities of TENGs for novel practical applications, some structural designs of TENGs inspired by nature, including lotus leaf inspired sweat-resistant wearable TENG for movement monitoring during exercise and fitness, Kármán vortex inspired membrane TENG for ultralow-speed wind energy harvesting and flow sensing, and honeybee inspired electrostatic microparticle manipulation system

A Unified Approximation Framework for Compressing and Accelerating Deep Neural Networks

Deep neural networks (DNNs) have achieved significant success in a variety of real world applications, i.e., image classification. However, tons of parameters in the networks restrict the efficiency of neural networks due to the large model size and the intensive computation. To address this issue, various approximation techniques have been investigated, which seek for a light weighted network with little performance degradation in exchange of smaller model size or faster inference. Both low-rankness and sparsity are appealing properties for the network approximation. In this paper we propose a unified framework to compress the convolutional neural networks (CNNs) by combining these two properties, while taking the nonlinear activation into consideration. Each layer in the network is approximated by the sum of a structured sparse component and a low-rank component, which is formulated as an optimization problem. Then, an extended version of alternating direction method of multipliers (ADMM) with guaranteed convergence is presented to solve the relaxed optimization problem. Experiments are carried out on VGG-16, AlexNet and GoogLeNet with large image classification datasets. The results outperform previous work in terms of accuracy degradation, compression rate and speedup ratio. The proposed method is able to remarkably compress the model (with up to 4.9x reduction of parameters) at a cost of little loss or without loss on accuracy.Comment: 8 pages, 5 figures, 6 table