NATURAL ALGORITHMS IN DIGITAL FILTER DESIGN

Digital filters are an important part of Digital Signal Processing (DSP), which plays vital roles within the modern world, but their design is a complex task requiring a great deal of specialised knowledge. An analysis of this design process is presented, which identifies opportunities for the application of optimisation. The Genetic Algorithm (GA) and Simulated Annealing are problem-independent and increasingly popular optimisation techniques. They do not require detailed prior knowledge of the nature of a problem, and are unaffected by a discontinuous search space, unlike traditional methods such as calculus and hill-climbing. Potential applications of these techniques to the filter design process are discussed, and presented with practical results. Investigations into the design of Frequency Sampling (FS) Finite Impulse Response (FIR) filters using a hybrid GA/hill-climber proved especially successful, improving on published results. An analysis of the search space for FS filters provided useful information on the performance of the optimisation technique. The ability of the GA to trade off a filter's performance with respect to several design criteria simultaneously, without intervention by the designer, is also investigated. Methods of simplifying the design process by using this technique are presented, together with an analysis of the difficulty of the non-linear FIR filter design problem from a GA perspective. This gave an insight into the fundamental nature of the optimisation problem, and also suggested future improvements. The results gained from these investigations allowed the framework for a potential 'intelligent' filter design system to be proposed, in which embedded expert knowledge, Artificial Intelligence techniques and traditional design methods work together. This could deliver a single tool capable of designing a wide range of filters with minimal human intervention, and of proposing solutions to incomplete problems. It could also provide the basis for the development of tools for other areas of DSP system design

Efficient Learning Machines

Computer scienc

Holistic Network Defense: Fusing Host and Network Features for Attack Classification

This work presents a hybrid network-host monitoring strategy, which fuses data from both the network and the host to recognize malware infections. This work focuses on three categories: Normal, Scanning, and Infected. The network-host sensor fusion is accomplished by extracting 248 features from network traffic using the Fullstats Network Feature generator and from the host using text mining, looking at the frequency of the 500 most common strings and analyzing them as word vectors. Improvements to detection performance are made by synergistically fusing network features obtained from IP packet flows and host features, obtained from text mining port, processor, logon information among others. In addition, the work compares three different machine learning algorithms and updates the script required to obtain network features. Hybrid method results outperformed host only classification by 31.7% and network only classification by 25%. The new approach also reduces the number of alerts while remaining accurate compared with the commercial IDS SNORT. These results make it such that even the most typical users could understand alert classification messages

Pruning Deep Convolutional Neural Networks for Fast Inference

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2017. 2. 성원용.Deep learning algorithms have recently achieved human level classification performance on several diverse classification benchmarks including object and speech recognition. However these algorithms are computationally very expensive especially for resource limited portable machines. Several researches have proposed ideas to lower this cost and in this dissertation, we have addressed this problem. We have proposed pruning and fixed-point optimization techniques to reduce the computational complexity of deep neural networks. Pruning is a promising technique where a problem is first approximated with a large sized network followed by removing unimportant parameters. The proposed work induces sparsity in a deep convolutional neural network (CNN) at three levels: feature map, kernels, and intra-kernel. Feature map pruning removes a large number of kernels and directly reduces the width of a layer and does not require any sparse representation. Thus the resulting network is thinner and runs faster than the predecessor unpruned network. However, feature map pruning removes all the incoming and outgoing kernels and thus affects a large number of parameters.We therefore may not achieve higher pruning ratios with feature map pruning. Kernel pruning eliminates kk kernels and is neither too fine nor too coarse. It can change the dense kernel connectivity pattern to a sparse one. Each convolution connection involves WxHxkxk multiply and accumulate (MAC) operations where W, H and k represents the feature map width, height and the kernel size, respectively. Further the sparse representation for kernel pruning is also very simple. A single flag is enough to represent one convolution connection. Intra-kernel pruning removes scalar weights at the finest scale. The conventional pruning techniques induce irregular sparsity at the finest granularity by zeroing scalar weights. This sparsity can be induced in much higher rates but requires sparse representation in order to be translated into computational speedups in VLSI or parallel computer based implementations. Coarse pruning granularities demand very simple sparse representation but higher pruning ratios are comparatively difficult to achieve. On the contrary, fine grained pruning granularities can achieve higher pruning ratios but the sparse representation is more complicated. In this dissertation, we propose pruning techniques at the aforementioned three pruning granularities. We further show that various pruning granularities can be applied in combinations to compress the network size to the maximum limit. The scalar weights inside a kernel is generally pruned in an irregular pattern. In this dissertation, we have proposed intra-kernel strided sparsity (IKSS). The IKSS prunes scalar weights at strided indices. We further impose a condition that all the outgoing kernels from a feature map must have the same stride and offset for IKSS. This has a direct impact on the sizes of matrices when convolutions are unrolled for matrix-matrix multiplications. The sparse representation for the constrained IKSS is only two numbers (stride, offset) per feature map. During pruning, it is important to locate the least adversarial pruning candidates. We have proposed three techniques for pruning candidate selectionparticle filter, selecting the best of N random pruning masks, and activation sum voting for feature map pruning. The dissertation extensively discuss the best of N random masks technique and provide detailed analysis. We obtain more than 80% pruning ratios with various pruning granularities. Moreover, the pruned networks can be further compressed by quantizing the weights and signals. This dissertation discusses our fixed-point optimization algorithm for deep convolutional neural network, where the network weights and signals are represented with 3-8 bits precision. We also discuss the layer-wise sensitivity analysis of deep convolutional neural networks. Thus we reduce the computational complexity of a CNN with pruning and fixed-point optimization. In this dissertation, the proposed pruning techniques fit well to Graphics Processing Units. The IKSS can reduce the size of matrices and GPUs are quite good at multiplying matrices. Secondly, FFT based CNN implementations can benefit from kernel level pruning. For VLSI implementations, the fixed-point optimized techniques reduce the memory requirements and hence the networks can be hosted in the on-chip memory. Thus the proposed techniques can be exploited on a generic set of modern computing platforms.1 Introduction 1 1.1 Convolutional Neural Networks (CNNs) 2 1.2 Computational Complexity of Convolution Layers 5 1.3 Publications Record 8 1.4 Outline of the Dissertation 9 2 Background 10 2.1 Introduction to Pruning 10 2.2 Pruning Candidate Selection 13 2.2.1 Evolutionary Particle Filter 14 2.2.2 Activation Sum Voting 17 2.2.3 Absolute Weight Sum Voting 19 2.2.4 Best of N Random Masks 19 2.2.5 The Effect of Retraining on Pruning Masks 25 2.3 Fixed-Point Optimization 26 3 Structured Pruning 28 3.1 Introduction 29 3.2 Feature Map and Intra-Kernel Pruning 31 3.2.1 Intra Kernel Strided Sparsity 31 3.3 Experimental Results 37 3.3.1 Feature Map Pruning 38 3.3.2 Intra-Kernel Pruning 40 3.3.3 Pruning Granularities Applied in Combinations 42 3.3.4 SVHN Dataset 46 3.3.5 Execution Time Savings 47 3.4 Comparison with the Previous Related Works 49 3.5 Conclusions 50 4 Kernel Pruning 51 4.1 Introduction 51 4.2 Kernel and Feature Map Pruning 52 4.3 Experimental Results 57 4.3.1 CIFAR-10 57 4.3.2 CIFAR-100 62 4.3.3 SVHN 64 4.4 Related Works 65 4.5 Concluding Remarks 66 5 Quantizing the Pruned Networks 68 5.1 Introduction 68 5.2 Retraining Based Quantization 70 5.2.1 L2 Error Minimization and Direct Quantization 70 5.2.2 LayerWise Sensitivity Analysis for Quantization 72 5.2.3 Retraining with the Quantized Weights 73 5.3 Separable Fixed-Point Kernels 74 5.4 Quantizing the Pruned Networks 77 5.4.1 Feature Map Pruned Networks 79 5.4.2 Kernel Pruned Networks 80 5.5 Concluding Remarks 80 6 Conclusion 82 Bibliography 85 Abstract in Korean 93Docto

Industrial applications of the Kalman filter:a review

International audienc

Real-Time Sensor Networks and Systems for the Industrial IoT

The Industrial Internet of Things (Industrial IoT—IIoT) has emerged as the core construct behind the various cyber-physical systems constituting a principal dimension of the fourth Industrial Revolution. While initially born as the concept behind specific industrial applications of generic IoT technologies, for the optimization of operational efficiency in automation and control, it quickly enabled the achievement of the total convergence of Operational (OT) and Information Technologies (IT). The IIoT has now surpassed the traditional borders of automation and control functions in the process and manufacturing industry, shifting towards a wider domain of functions and industries, embraced under the dominant global initiatives and architectural frameworks of Industry 4.0 (or Industrie 4.0) in Germany, Industrial Internet in the US, Society 5.0 in Japan, and Made-in-China 2025 in China. As real-time embedded systems are quickly achieving ubiquity in everyday life and in industrial environments, and many processes already depend on real-time cyber-physical systems and embedded sensors, the integration of IoT with cognitive computing and real-time data exchange is essential for real-time analytics and realization of digital twins in smart environments and services under the various frameworks’ provisions. In this context, real-time sensor networks and systems for the Industrial IoT encompass multiple technologies and raise significant design, optimization, integration and exploitation challenges. The ten articles in this Special Issue describe advances in real-time sensor networks and systems that are significant enablers of the Industrial IoT paradigm. In the relevant landscape, the domain of wireless networking technologies is centrally positioned, as expected

The Applications of Discrete Wavelet Transform in Image Processing: A Review

This paper reviews the newly published works on applying waves to image processing depending on the analysis of multiple solutions. the wavelet transformation reviewed in detail including wavelet function, integrated wavelet transformation, discrete wavelet transformation, rapid wavelet transformation, DWT properties, and DWT advantages. After reviewing the basics of wavelet transformation theory, various applications of wavelet are reviewed and multi-solution analysis, including image compression, image reduction, image optimization, and image watermark. In addition, we present the concept and theory of quadruple waves for the future progress of wavelet transform applications and quadruple solubility applications. The aim of this paper is to provide a wide-ranging review of the survey found able on wavelet-based image processing applications approaches. It will be beneficial for scholars to execute effective image processing applications approaches