Digital Filter Design Using Improved Teaching-Learning-Based Optimization

Digital filters are an important part of digital signal processing systems. Digital filters are divided into finite impulse response (FIR) digital filters and infinite impulse response (IIR) digital filters according to the length of their impulse responses. An FIR digital filter is easier to implement than an IIR digital filter because of its linear phase and stability properties. In terms of the stability of an IIR digital filter, the poles generated in the denominator are subject to stability constraints. In addition, a digital filter can be categorized as one-dimensional or multi-dimensional digital filters according to the dimensions of the signal to be processed. However, for the design of IIR digital filters, traditional design methods have the disadvantages of easy to fall into a local optimum and slow convergence. The Teaching-Learning-Based optimization (TLBO) algorithm has been proven beneficial in a wide range of engineering applications. To this end, this dissertation focusses on using TLBO and its improved algorithms to design five types of digital filters, which include linear phase FIR digital filters, multiobjective general FIR digital filters, multiobjective IIR digital filters, two-dimensional (2-D) linear phase FIR digital filters, and 2-D nonlinear phase FIR digital filters. Among them, linear phase FIR digital filters, 2-D linear phase FIR digital filters, and 2-D nonlinear phase FIR digital filters use single-objective type of TLBO algorithms to optimize; multiobjective general FIR digital filters use multiobjective non-dominated TLBO (MOTLBO) algorithm to optimize; and multiobjective IIR digital filters use MOTLBO with Euclidean distance to optimize. The design results of the five types of filter designs are compared to those obtained by other state-of-the-art design methods. In this dissertation, two major improvements are proposed to enhance the performance of the standard TLBO algorithm. The first improvement is to apply a gradient-based learning to replace the TLBO learner phase to reduce approximation error(s) and CPU time without sacrificing design accuracy for linear phase FIR digital filter design. The second improvement is to incorporate Manhattan distance to simplify the procedure of the multiobjective non-dominated TLBO (MOTLBO) algorithm for general FIR digital filter design. The design results obtained by the two improvements have demonstrated their efficiency and effectiveness

Digital Filter Design Using Improved Artificial Bee Colony Algorithms

Digital filters are often used in digital signal processing applications. The design objective of a digital filter is to find the optimal set of filter coefficients, which satisfies the desired specifications of magnitude and group delay responses. Evolutionary algorithms are population-based meta-heuristic algorithms inspired by the biological behaviors of species. Compared to gradient-based optimization algorithms such as steepest descent and Newton’s like methods, these bio-inspired algorithms have the advantages of not getting stuck at local optima and being independent of the starting point in the solution space. The limitations of evolutionary algorithms include the presence of control parameters, problem specific tuning procedure, premature convergence and slower convergence rate. The artificial bee colony (ABC) algorithm is a swarm-based search meta-heuristic algorithm inspired by the foraging behaviors of honey bee colonies, with the benefit of a relatively fewer control parameters. In its original form, the ABC algorithm has certain limitations such as low convergence rate, and insufficient balance between exploration and exploitation in the search equations. In this dissertation, an ABC-AMR algorithm is proposed by incorporating an adaptive modification rate (AMR) into the original ABC algorithm to increase convergence rate by adjusting the balance between exploration and exploitation in the search equations through an adaptive determination of the number of parameters to be updated in every iteration. A constrained ABC-AMR algorithm is also developed for solving constrained optimization problems.There are many real-world problems requiring simultaneous optimizations of more than one conflicting objectives. Multiobjective (MO) optimization produces a set of feasible solutions called the Pareto front instead of a single optimum solution. For multiobjective optimization, if a decision maker’s preferences can be incorporated during the optimization process, the search process can be confined to the region of interest instead of searching the entire region. In this dissertation, two algorithms are developed for such incorporation. The first one is a reference-point-based MOABC algorithm in which a decision maker’s preferences are included in the optimization process as the reference point. The second one is a physical-programming-based MOABC algorithm in which physical programming is used for setting the region of interest of a decision maker. In this dissertation, the four developed algorithms are applied to solve digital filter design problems. The ABC-AMR algorithm is used to design Types 3 and 4 linear phase FIR differentiators, and the results are compared to those obtained by the original ABC algorithm, three improved ABC algorithms, and the Parks-McClellan algorithm. The constrained ABC-AMR algorithm is applied to the design of sparse Type 1 linear phase FIR filters of filter orders 60, 70 and 80, and the results are compared to three state-of-the-art design methods. The reference-point-based multiobjective ABC algorithm is used to design of asymmetric lowpass, highpass, bandpass and bandstop FIR filters, and the results are compared to those obtained by the preference-based multiobjective differential evolution algorithm. The physical-programming-based multiobjective ABC algorithm is used to design IIR lowpass, highpass and bandpass filters, and the results are compared to three state-of-the-art design methods. Based on the obtained design results, the four design algorithms are shown to be competitive as compared to the state-of-the-art design methods

Performance Analysis of l_0 Norm Constraint Least Mean Square Algorithm

As one of the recently proposed algorithms for sparse system identification, $l_0$ norm constraint Least Mean Square ($l_0$-LMS) algorithm modifies the cost function of the traditional method with a penalty of tap-weight sparsity. The performance of $l_0$-LMS is quite attractive compared with its various precursors. However, there has been no detailed study of its performance. This paper presents all-around and throughout theoretical performance analysis of $l_0$-LMS for white Gaussian input data based on some reasonable assumptions. Expressions for steady-state mean square deviation (MSD) are derived and discussed with respect to algorithm parameters and system sparsity. The parameter selection rule is established for achieving the best performance. Approximated with Taylor series, the instantaneous behavior is also derived. In addition, the relationship between $l_0$-LMS and some previous arts and the sufficient conditions for $l_0$-LMS to accelerate convergence are set up. Finally, all of the theoretical results are compared with simulations and are shown to agree well in a large range of parameter setting.Comment: 31 pages, 8 figure

Solving Weighted Least Squares (WLS) problems on ARM-based architectures

TheWeighted Least Squares algorithm (WLS) is applied to numerous optimization problems, but requires the use of high computational resources, especially when complex arithmetic is involved. This work aims to accelerate the resolution of a WLS problem by reducing the computational cost (relaying on BLAS/LAPACK routines) and the computational precision from double to single. As a test case, we design an IIR filter for a Graphic Equalizer, where the numerical errors due to single precision are easily visualized. In addition, given the importance of low power architectures for this kind of implementations, we evaluate the performance, scalability, and energy efficiency of each method on two different processors implementing the ARMv7 architecture, widely used in current mobile devices with power constraints. Results show that the method that exhibits a high theoretical computational cost overcomes in efficiency other methods with lower theoretical cost in architectures of this type.This work started in spring 2016 when Jose A. Belloch was a visiting postdoctoral researcher at Budapest University of Technology and Economics thanks to the European Network COST Action IC1305 inside the program Short Term Scientific Mission with the following reference: COST-SPASM-ECOST-STSM-IC1305-020416-072431. Dr. Jose A. Belloch is supported by GVA contract APOSTD/2016/069. The researchers from Universitat Jaume I are supported by the CICYT projects TIN2014-53495-R of MINECO and FEDER. The authors from the Universitat Politecnica de Valencia are supported by MINECO Projects TEC2015-67387-C4-1-R, PROMETEOII/2014/003 and CAPAP-H5 network TIN2014-53522-REDT. The researcher from UCM is supported by the EU (FEDER) and the Spanish MINECO, under Grants TIN 2015-65277-R and TIN2012-32180. The work of Balazs Bank was supported by the UNKP-16-4-III New National Excellence Program of the Ministry of Human Capacities, Hungary.Belloch Rodríguez, JA.; Bank, B.; Igual Peña, FD.; Quintana Ortí, ES.; Vidal Maciá, AM. (2017). Solving Weighted Least Squares (WLS) problems on ARM-based architectures. Journal of Supercomputing. 73(1):530-542. https://doi.org/10.1007/s11227-016-1910-9S530542731Smith TM, van de Geijn RA, Smelyanskiy M, Hammond JR, Van Zee FG (2014) Anatomy of high-performance many-threaded matrix multiplication. In: 28th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2014)Burrus CS (2012) Iterative reweighted least squares. OpenStax-CNC document, May 2012, module m45285. http://cnx.org/content/m45285/1.12 . Accessed 2 Nov 2016Khang SW (1972) Best $$L_p$$ L p approximation. Math Comput 26(118):505–508Jackson LB (2008) Frequency-domain Steiglitz-McBride method for least-squares filter design, ARMA modeling, and periodogram smoothing. IEEE Signal Process Lett 15:49–52Bank B (2012) Magnitude-priority filter design for audio applications. In: Proceedings of $$132^{{\rm nd}}$$ 132 nd AES Convention, Preprint No. 8591, Budapest, Hungary, May 2012Daubechies I, Devire R, Fornasier M, Gntrk CS (2010) Iteratively reweighted least squares minimization for sparse recovery. Comput Music J 23(2):52–69Rämö J, Välimäki V, Bank B (2014) High-precision parallel graphic equalizer. IEEE/ACM Trans Audio Speech Lange Proc 22(12):1894–1904Perez Gonzales E, Reiss J (2009) Automatic equalization of multi-channel audio using cross-adaptive methods. In: Proceedings of AES 127th Convention, New York, Oct. 2009Rämö J, Välimäki V (2013) Live sound equalization and attenuation with a headset. In: Proceedings of AES 51st International Conference, Helsinki, Finland, Aug. 2013Mäkivirta A, Antsalo P, Karjalainen M, Välimäki V (2003) Modal equalization of loudspeaker-room responses at low frequencies. J Audio Eng Soc 51(5):324–343Holters M, Zölzer U (2006) Graphic equalizer design using higher-order recursive filters. In: Proceedings of International Conference Digital Audio Effects, Montreal, QC, pp 37–40Tassart S (2013) Graphical equalization using interpolated filter banks. J Audio Eng Soc 61(5):263–279Chen Z, Geng GS, Yin FL, Hao J (2014) A pre-distortion based design method for digital audio graphic equalizer. Digital Signal Process 25:296–302Välimäki V, Reiss J (2016) All about audio equalization: solutions and frontiers. Appl Sci 6(5):129–145Belloch JA, Välimäki V (2016) Efficient target-response interpolation for a graphic equalizer. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), March 2016, pp 564–568Belloch JA, Alventosa FJ, Alonso P, Quintana-Ortí ES, Vidal AM (2016) Accelerating multi-channel filtering of audio signal on arm processors. J Supercomput, pp 1–12. doi: 10.1007/s11227-016-1689-8Belloch JA, Gonzalez A, Igual FD, Mayo R, Quintana-Ortí ES (2015)Vectorization of binaural sound virtualization on the ARM cortex-A15 architecture. In: Proceedings of 23rd European Signal Processing Conference, (EUSIPCO), Nize, France, September 2015Mitra G, Johnston B, Rendell A, McCreath E, Zhou J (2013) Use of simd vector operations to accelerate application code performance on low-powered arm and intel platforms. In: IEEE 27th International Parallel and Distributed Processing Symposium Workshops PhD Forum (IPDPSW), May 2013, pp 1107–1116Tomov S, Dongarra J, Baboulin M (2008) Towards dense linear algebra for hybrid gpu accelerated manycore systems. LAPACK Working Note, Tech. Rep. 210, Oct. 2008. http://www.netlib.org/lapack/lawnspdf/lawn210.pdf . Accessed 2 Nov 2016Dongarra JJ, DuCroz J, Hammarling S, Hanson RJ (1985) A proposal for an extended set of fortran basic linear algebra subprograms. ACM Signum Newsletter, New York, pp 2–18Golub GH, Loan CFV (2013) Matrix Comput, 4th edn. The John Hopkins University Press, BaltimoreAlonso P, Badia RM, Labarta J, Barreda M, Dolz MF, Mayo R, Quintana-Ortí ES, Reyes R (2012) Tools for power-energy modelling and analysis of parallel scientific applications. In: 41st International Conference on Parallel Processing—ICPP, 2012, pp 420–42

Fast Algorithms for ℓp\ell_p-Regression

The $\ell_p$-norm regression problem is a classic problem in optimization with wide ranging applications in machine learning and theoretical computer science. The goal is to compute $x^{\star} =\arg\min_{Ax=b}\|x\|_p^p$, where $x^{\star}\in \mathbb{R}^n, A\in \mathbb{R}^{d\times n},b \in \mathbb{R}^d$ and $d\leq n$. Efficient high-accuracy algorithms for the problem have been challenging both in theory and practice and the state of the art algorithms require $poly(p)\cdot n^{\frac{1}{2}-\frac{1}{p}}$ linear system solves for $p\geq 2$. In this paper, we provide new algorithms for $\ell_p$-regression (and a more general formulation of the problem) that obtain a high-accuracy solution in $O(p n^{\frac{(p-2)}{(3p-2)}})$ linear system solves. We further propose a new inverse maintenance procedure that speeds-up our algorithm to $\widetilde{O}(n^{\omega})$ total runtime, where $O(n^{\omega})$ denotes the running time for multiplying $n \times n$ matrices. Additionally, we give the first Iteratively Reweighted Least Squares (IRLS) algorithm that is guaranteed to converge to an optimum in a few iterations. Our IRLS algorithm has shown exceptional practical performance, beating the currently available implementations in MATLAB/CVX by 10-50x.Comment: This paper is a coherent algorithmic framework that combines and simplifies our previous works: 1. arXiv:1901.06764 2. arXiv:1907.07167 3. arXiv:1910.1057