Parallel Multistage Wide Neural Network

Deep learning networks have achieved great success in many areas such as in large scale image processing. They usually need large computing resources and time, and process easy and hard samples inefficiently in the same way. Another undesirable problem is that the network generally needs to be retrained to learn new incoming data. Efforts have been made to reduce the computing resources and realize incremental learning by adjusting architectures, such as scalable effort classifiers, multi-grained cascade forest (gc forest), conditional deep learning (CDL), tree CNN, decision tree structure with knowledge transfer (ERDK), forest of decision trees with RBF networks and knowledge transfer (FDRK). In this paper, a parallel multistage wide neural network (PMWNN) is presented. It is composed of multiple stages to classify different parts of data. First, a wide radial basis function (WRBF) network is designed to learn features efficiently in the wide direction. It can work on both vector and image instances, and be trained fast in one epoch using subsampling and least squares (LS). Secondly, successive stages of WRBF networks are combined to make up the PMWNN. Each stage focuses on the misclassified samples of the previous stage. It can stop growing at an early stage, and a stage can be added incrementally when new training data is acquired. Finally, the stages of the PMWNN can be tested in parallel, thus speeding up the testing process. To sum up, the proposed PMWNN network has the advantages of (1) fast training, (2) optimized computing resources, (3) incremental learning, and (4) parallel testing with stages. The experimental results with the MNIST, a number of large hyperspectral remote sensing data, CVL single digits, SVHN datasets, and audio signal datasets show that the WRBF and PMWNN have the competitive accuracy compared to learning models such as stacked auto encoders, deep belief nets, SVM, MLP, LeNet-5, RBF network, recently proposed CDL, broad learning, gc forest etc. In fact, the PMWNN has often the best classification performance

Blockwise Transform Image Coding Enhancement and Edge Detection

The goal of this thesis is high quality image coding, enhancement and edge detection. A unified approach using novel fast transforms is developed to achieve all three objectives. Requirements are low bit rate, low complexity of implementation and parallel processing. The last requirement is achieved by processing the image in small blocks such that all blocks can be processed simultaneously. This is similar to biological vision. A major issue is to minimize the resulting block effects. This is done by using proper transforms and possibly an overlap-save technique. The bit rate in image coding is minimized by developing new results in optimal adaptive multistage transform coding. Newly developed fast trigonometric transforms are also utilized and compared for transform coding, image enhancement and edge detection. Both image enhancement and edge detection involve generalised bandpass filtering wit fast transforms. The algorithms have been developed with special attention to the properties of biological vision systems

Three dimensional DCT based video compression.

by Chan Kwong Wing Raymond.Thesis (M.Phil.)--Chinese University of Hong Kong, 1997.Includes bibliographical references (leaves 115-123).Acknowledgments --- p.iTable of Contents --- p.ii-vList of Tables --- p.viList of Figures --- p.viiAbstract --- p.1Chapter Chapter 1 : --- IntroductionChapter 1.1 --- An Introduction to Video Compression --- p.3Chapter 1.2 --- Overview of Problems --- p.4Chapter 1.2.1 --- Analog Video and Digital Problems --- p.4Chapter 1.2.2 --- Low Bit Rate Application Problems --- p.4Chapter 1.2.3 --- Real Time Video Compression Problems --- p.5Chapter 1.2.4 --- Source Coding and Channel Coding Problems --- p.6Chapter 1.2.5 --- Bit-rate and Quality Problems --- p.7Chapter 1.3 --- Organization of the Thesis --- p.7Chapter Chapter 2 : --- Background and Related WorkChapter 2.1 --- Introduction --- p.9Chapter 2.1.1 --- Analog Video --- p.9Chapter 2.1.2 --- Digital Video --- p.10Chapter 2.1.3 --- Color Theory --- p.10Chapter 2.2 --- Video Coding --- p.12Chapter 2.2.1 --- Predictive Coding --- p.12Chapter 2.2.2 --- Vector Quantization --- p.12Chapter 2.2.3 --- Subband Coding --- p.13Chapter 2.2.4 --- Transform Coding --- p.14Chapter 2.2.5 --- Hybrid Coding --- p.14Chapter 2.3 --- Transform Coding --- p.15Chapter 2.3.1 --- Discrete Cosine Transform --- p.16Chapter 2.3.1.1 --- 1-D Fast Algorithms --- p.16Chapter 2.3.1.2 --- 2-D Fast Algorithms --- p.17Chapter 2.3.1.3 --- Multidimensional DCT Algorithms --- p.17Chapter 2.3.2 --- Quantization --- p.18Chapter 2.3.3 --- Entropy Coding --- p.18Chapter 2.3.3.1 --- Huffman Coding --- p.19Chapter 2.3.3.2 --- Arithmetic Coding --- p.19Chapter Chapter 3 : --- Existing Compression SchemeChapter 3.1 --- Introduction --- p.20Chapter 3.2 --- Motion JPEG --- p.20Chapter 3.3 --- MPEG --- p.20Chapter 3.4 --- H.261 --- p.22Chapter 3.5 --- Other Techniques --- p.23Chapter 3.5.1 --- Fractals --- p.23Chapter 3.5.2 --- Wavelets --- p.23Chapter 3.6 --- Proposed Solution --- p.24Chapter 3.7 --- Summary --- p.25Chapter Chapter 4 : --- Fast 3D-DCT AlgorithmsChapter 4.1 --- Introduction --- p.27Chapter 4.1.1 --- Motivation --- p.27Chapter 4.1.2 --- Potentials of 3D DCT --- p.28Chapter 4.2 --- Three Dimensional Discrete Cosine Transform (3D-DCT) --- p.29Chapter 4.2.1 --- Inverse 3D-DCT --- p.29Chapter 4.2.2 --- Forward 3D-DCT --- p.30Chapter 4.3 --- 3-D FCT (3-D Fast Cosine Transform Algorithm --- p.30Chapter 4.3.1 --- Partitioning and Rearrangement of Data Cube --- p.30Chapter 4.3.1.1 --- Spatio-temporal Data Cube --- p.30Chapter 4.3.1.2 --- Spatio-temporal Transform Domain Cube --- p.31Chapter 4.3.1.3 --- Coefficient Matrices --- p.31Chapter 4.3.2 --- 3-D Inverse Fast Cosine Transform (3-D IFCT) --- p.32Chapter 4.3.2.1 --- Matrix Representations --- p.32Chapter 4.3.2.2 --- Simplification of the calculation steps --- p.33Chapter 4.3.3 --- 3-D Forward Fast Cosine Transform (3-D FCT) --- p.35Chapter 4.3.3.1 --- Decomposition --- p.35Chapter 4.3.3.2 --- Reconstruction --- p.36Chapter 4.4 --- The Fast Algorithm --- p.36Chapter 4.5 --- Example using 4x4x4 IFCT --- p.38Chapter 4.6 --- Complexity Comparison --- p.43Chapter 4.6.1 --- Complexity of Multiplications --- p.43Chapter 4.6.2 --- Complexity of Additions --- p.43Chapter 4.7 --- Implementation Issues --- p.44Chapter 4.8 --- Summary --- p.46Chapter Chapter 5 : --- QuantizationChapter 5.1 --- Introduction --- p.49Chapter 5.2 --- Dynamic Ranges of 3D-DCT Coefficients --- p.49Chapter 5.3 --- Distribution of 3D-DCT AC Coefficients --- p.54Chapter 5.4 --- Quantization Volume --- p.55Chapter 5.4.1 --- Shifted Complement Hyperboloid --- p.55Chapter 5.4.2 --- Quantization Volume --- p.58Chapter 5.5 --- Scan Order for Quantized 3D-DCT Coefficients --- p.59Chapter 5.6 --- Finding Parameter Values --- p.60Chapter 5.7 --- Experimental Results from Using the Proposed Quantization Values --- p.65Chapter 5.8 --- Summary --- p.66Chapter Chapter 6 : --- Entropy CodingChapter 6.1 --- Introduction --- p.69Chapter 6.1.1 --- Huffman Coding --- p.69Chapter 6.1.2 --- Arithmetic Coding --- p.71Chapter 6.2 --- Zero Run-Length Encoding --- p.73Chapter 6.2.1 --- Variable Length Coding in JPEG --- p.74Chapter 6.2.1.1 --- Coding of the DC Coefficients --- p.74Chapter 6.2.1.2 --- Coding of the DC Coefficients --- p.75Chapter 6.2.2 --- Run-Level Encoding of the Quantized 3D-DCT Coefficients --- p.76Chapter 6.3 --- Frequency Analysis of the Run-Length Patterns --- p.76Chapter 6.3.1 --- The Frequency Distributions of the DC Coefficients --- p.77Chapter 6.3.2 --- The Frequency Distributions of the DC Coefficients --- p.77Chapter 6.4 --- Huffman Table Design --- p.84Chapter 6.4.1 --- DC Huffman Table --- p.84Chapter 6.4.2 --- AC Huffman Table --- p.85Chapter 6.5 --- Implementation Issue --- p.85Chapter 6.5.1 --- Get Category --- p.85Chapter 6.5.2 --- Huffman Encode --- p.86Chapter 6.5.3 --- Huffman Decode --- p.86Chapter 6.5.4 --- PutBits --- p.88Chapter 6.5.5 --- GetBits --- p.90Chapter Chapter 7 : --- "Contributions, Concluding Remarks and Future Work"Chapter 7.1 --- Contributions --- p.92Chapter 7.2 --- Concluding Remarks --- p.93Chapter 7.2.1 --- The Advantages of 3D DCT codec --- p.94Chapter 7.2.2 --- Experimental Results --- p.95Chapter 7.1 --- Future Work --- p.95Chapter 7.2.1 --- Integer Discrete Cosine Transform Algorithms --- p.95Chapter 7.2.2 --- Adaptive Quantization Volume --- p.96Chapter 7.2.3 --- Adaptive Huffman Tables --- p.96Appendices:Appendix A : The detailed steps in the simplification of Equation 4.29 --- p.98Appendix B : The program Listing of the Fast DCT Algorithms --- p.101Appendix C : Tables to Illustrate the Reording of the Quantized Coefficients --- p.110Appendix D : Sample Values of the Quantization Volume --- p.111Appendix E : A 16-bit VLC table for AC Run-Level Pairs --- p.113References --- p.11