ONE-BIT QUANTIZER PARAMETRIZATION FOR ARBITRARY LAPLACIAN SOURCES

In this paper we suggest an exact formula for the total distortion of one-bit quantizer and for the arbitrary Laplacian probability density function (pdf). Suggested formula additionally extends normalized case of zero mean and unit variance, which is the most applied quantization case not only in traditional quantization rather in contemporary solutions that involve quantization. Additionally symmetrical quantizer’s representation levels are calculated from minimal distortion criteria. Note that one-bit quantization is the most sensitive quantization from the standpoint of accuracy degradation and quantization error, thus increasing importance of the suggested parameterization of one-bit quantizer

Projektovanje kvantizera za primenu u obradi signala i neuronskim mrežama

Scalar quantizers are present in many advanced systems for signal processing and transmission, аnd their contribution is particular in the realization of the most important step in digitizing signals: the amplitude discretization. Accordingly, there are justified reasons for the development of innovative solutions, that is, quantizer models which offer reduced complexity, shorter processing time along with performance close to the standard quantizer models. Designing of a quantizer for a certain type of signal is a specific process and several new methods are proposed in the dissertation, which are computationally less intensive compared to the existing ones. Specifically, the design of different types of quantizers with low and high number of levels which apply variable and a fixed length coding, is considered. The dissertation is organized in such a way that it deals with the development of coding solutions for standard telecommunication signals (e.g. speech), as well as other types of signals such as neural network parameters. Many solutions, which belong to the class of waveform encoders, are proposed for speech coding. The developed solutions are characterized by low complexity and are obtained as a result of the implementation of new quantizer models in non-predictive and predictive coding techniques. The target of the proposed solutions is to enhance the performance of some standardized solutions or some advanced solutions with the same/similar complexity. Testing is performed using the speech examples extracted from the well-known databases, while performance evaluation of the proposed coding solutions is done by using the standard objective measures. In order to verify the correctness of the provided solutions, the matching between theoretical and experimental results is examined. In addition to speech coding, in dissertation are proposed some novel solutions based on the scalar quantizers for neural network compression. This is an active research area, whereby the role of quantization in this area is somewhat different than in the speech coding, and consists of providing a compromise between performance and accuracy of the neural network. Dissertation strictly deals with the low-levels (low-resolution) quantizers intended for post-training quantization, since they are more significant regarding compression. The goal is to improve the performance of the quantized neural network by using the novel designing methods for quantizers. The proposed quantizers are applied to several neural network models used for image classification (some benchmark dataset are used), and as performance measure the prediction accuracy along with SQNR is used. In fact, there was an effort to determine the connection between these two measures, which has not been investigated sufficiently so far

Развој кодера таласног облика за потребе неуронских мрежа и обраду сигнала

This doctoral thesis aims to design low-bit scalar quantizers and analyze their application in Neural Networks (NNs) and signal processing. In this thesis, we consider the possibilities and limitations that rest on quantization, as a leading technique for data coding and compression. In particular, we examine the inevitable accuracy loss of signal and data presentation due to quantization in the signal processing area, as well as in many modern solutions, that use quantization. As stated in this thesis, there are a number of qualitative performance indicators, which indicate that appropriate quantizer parameterization can optimize the amount of data transmitted in bits. Quantized Neural Networks (QNNs) is a promising research area, especially important for resource constrained devices. Relying on a plethora of conclusions about scalar quantizers derived for signal processing tasks and taking into account the advantages of scalar quantization, we anticipate that by studying the statistical characteristics of neural network parameters, this thesis will contribute to determining an efficient weights compression solution utilizing new, well-designed scalar quantizers for post-training quantization