Scalable Compression of Deep Neural Networks

Deep neural networks generally involve some layers with mil- lions of parameters, making them difficult to be deployed and updated on devices with limited resources such as mobile phones and other smart embedded systems. In this paper, we propose a scalable representation of the network parameters, so that different applications can select the most suitable bit rate of the network based on their own storage constraints. Moreover, when a device needs to upgrade to a high-rate network, the existing low-rate network can be reused, and only some incremental data are needed to be downloaded. We first hierarchically quantize the weights of a pre-trained deep neural network to enforce weight sharing. Next, we adaptively select the bits assigned to each layer given the total bit budget. After that, we retrain the network to fine-tune the quantized centroids. Experimental results show that our method can achieve scalable compression with graceful degradation in the performance.Comment: 5 pages, 4 figures, ACM Multimedia 201

Complexity in Prefix-Free Regular Languages

We examine deterministic and nondeterministic state complexities of regular operations on prefix-free languages. We strengthen several results by providing witness languages over smaller alphabets, usually as small as possible. We next provide the tight bounds on state complexity of symmetric difference, and deterministic and nondeterministic state complexity of difference and cyclic shift of prefix-free languages.Comment: In Proceedings DCFS 2010, arXiv:1008.127

Combining All Pairs Shortest Paths and All Pairs Bottleneck Paths Problems

We introduce a new problem that combines the well known All Pairs Shortest Paths (APSP) problem and the All Pairs Bottleneck Paths (APBP) problem to compute the shortest paths for all pairs of vertices for all possible flow amounts. We call this new problem the All Pairs Shortest Paths for All Flows (APSP-AF) problem. We firstly solve the APSP-AF problem on directed graphs with unit edge costs and real edge capacities in $\tilde{O}(\sqrt{t}n^{(\omega+9)/4}) = \tilde{O}(\sqrt{t}n^{2.843})$ time, where $n$ is the number of vertices, $t$ is the number of distinct edge capacities (flow amounts) and $O(n^{\omega}) < O(n^{2.373})$ is the time taken to multiply two $n$-by-$n$ matrices over a ring. Secondly we extend the problem to graphs with positive integer edge costs and present an algorithm with $\tilde{O}(\sqrt{t}c^{(\omega+5)/4}n^{(\omega+9)/4}) = \tilde{O}(\sqrt{t}c^{1.843}n^{2.843})$ worst case time complexity, where $c$ is the upper bound on edge costs

Stokes Parameters as a Minkowskian Four-vector

It is noted that the Jones-matrix formalism for polarization optics is a six-parameter two-by-two representation of the Lorentz group. It is shown that the four independent Stokes parameters form a Minkowskian four-vector, just like the energy-momentum four-vector in special relativity. The optical filters are represented by four-by-four Lorentz-transformation matrices. This four-by-four formalism can deal with partial coherence described by the Stokes parameters. A four-by-four matrix formulation is given for decoherence effects on the Stokes parameters, and a possible experiment is proposed. It is shown also that this Lorentz-group formalism leads to optical filters with a symmetry property corresponding to that of two-dimensional Euclidean transformations.Comment: RevTeX, 22 pages, no figures, submitted to Phys. Rev.

The language of Einstein spoken by optical instruments

Einstein had to learn the mathematics of Lorentz transformations in order to complete his covariant formulation of Maxwell's equations. The mathematics of Lorentz transformations, called the Lorentz group, continues playing its important role in optical sciences. It is the basic mathematical language for coherent and squeezed states. It is noted that the six-parameter Lorentz group can be represented by two-by-two matrices. Since the beam transfer matrices in ray optics is largely based on two-by-two matrices or $ABCD$ matrices, the Lorentz group is bound to be the basic language for ray optics, including polarization optics, interferometers, lens optics, multilayer optics, and the Poincar\'e sphere. Because the group of Lorentz transformations and ray optics are based on the same two-by-two matrix formalism, ray optics can perform mathematical operations which correspond to transformations in special relativity. It is shown, in particular, that one-lens optics provides a mathematical basis for unifying the internal space-time symmetries of massive and massless particles in the Lorentz-covariant world.Comment: LaTex 8 pages, presented at the 10th International Conference on Quantum Optics (Minsk, Belarus, May-June 2004), to be published in the proceeding