Soft self-organizing map.

by John Pui-fai Sum.Thesis (M.Phil.)--Chinese University of Hong Kong, 1995.Includes bibliographical references (leaves 99-104).Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Motivation --- p.1Chapter 1.2 --- Idea of SSOM --- p.3Chapter 1.3 --- Other Approaches --- p.3Chapter 1.4 --- Contribution of the Thesis --- p.4Chapter 1.5 --- Outline of Thesis --- p.5Chapter 2 --- Self-Organizing Map --- p.7Chapter 2.1 --- Introduction --- p.7Chapter 2.2 --- Algorithm of SOM --- p.8Chapter 2.3 --- Illustrative Example --- p.10Chapter 2.4 --- Property of SOM --- p.14Chapter 2.4.1 --- Convergence property --- p.14Chapter 2.4.2 --- Topological Order --- p.15Chapter 2.4.3 --- Objective Function of SOM --- p.15Chapter 2.5 --- Conclusion --- p.17Chapter 3 --- Algorithms for Soft Self-Organizing Map --- p.18Chapter 3.1 --- Competitive Learning and Soft Competitive Learning --- p.19Chapter 3.2 --- How does SOM generate ordered map? --- p.21Chapter 3.3 --- Algorithms of Soft SOM --- p.23Chapter 3.4 --- Simulation Results --- p.25Chapter 3.4.1 --- One dimensional map under uniform distribution --- p.25Chapter 3.4.2 --- One dimensional map under Gaussian distribution --- p.27Chapter 3.4.3 --- Two dimensional map in a unit square --- p.28Chapter 3.5 --- Conclusion --- p.30Chapter 4 --- Application to Uncover Vowel Relationship --- p.31Chapter 4.1 --- Experiment Set Up --- p.32Chapter 4.1.1 --- Network structure --- p.32Chapter 4.1.2 --- Training procedure --- p.32Chapter 4.1.3 --- Relationship Construction Scheme --- p.34Chapter 4.2 --- Results --- p.34Chapter 4.2.1 --- Hidden-unit labeling for SSOM2 --- p.34Chapter 4.2.2 --- Hidden-unit labeling for SOM --- p.35Chapter 4.3 --- Conclusion --- p.37Chapter 5 --- Application to vowel data transmission --- p.42Chapter 5.1 --- Introduction --- p.42Chapter 5.2 --- Simulation --- p.45Chapter 5.2.1 --- Setup --- p.45Chapter 5.2.2 --- Noise model and demodulation scheme --- p.46Chapter 5.2.3 --- Performance index --- p.46Chapter 5.2.4 --- Control experiment: random coding scheme --- p.46Chapter 5.3 --- Results --- p.47Chapter 5.3.1 --- Null channel noise (σ = 0) --- p.47Chapter 5.3.2 --- Small channel noise (0 ≤ σ ≤1) --- p.49Chapter 5.3.3 --- Large channel noise (1 ≤σ ≤7) --- p.49Chapter 5.3.4 --- Very large channel noise (σ > 7) --- p.49Chapter 5.4 --- Conclusion --- p.50Chapter 6 --- Convergence Analysis --- p.53Chapter 6.1 --- Kushner and Clark Lemma --- p.53Chapter 6.2 --- Condition for the Convergence of Jou's Algorithm --- p.54Chapter 6.3 --- Alternative Proof on the Convergence of Competitive Learning --- p.56Chapter 6.4 --- Convergence of Soft SOM --- p.58Chapter 6.5 --- Convergence of SOM --- p.60Chapter 7 --- Conclusion --- p.61Chapter 7.1 --- Limitations of SSOM --- p.62Chapter 7.2 --- Further Research --- p.63Chapter A --- Proof of Corollary1 --- p.65Chapter A.l --- Mean Average Update --- p.66Chapter A.2 --- Case 1: Uniform Distribution --- p.68Chapter A.3 --- Case 2: Logconcave Distribution --- p.70Chapter A.4 --- Case 3: Loglinear Distribution --- p.72Chapter B --- Different Senses of neighborhood --- p.79Chapter B.l --- Static neighborhood: Kohonen's sense --- p.79Chapter B.2 --- Dynamic neighborhood --- p.80Chapter B.2.1 --- Mou-Yeung Definition --- p.80Chapter B.2.2 --- Martinetz et al. Definition --- p.81Chapter B.2.3 --- Tsao-Bezdek-Pal Definition --- p.81Chapter B.3 --- Example --- p.82Chapter B.4 --- Discussion --- p.84Chapter C --- Supplementary to Chapter4 --- p.86Chapter D --- Quadrature Amplitude Modulation --- p.92Chapter D.l --- Amplitude Modulation --- p.92Chapter D.2 --- QAM --- p.93Bibliography --- p.9

S-TREE: Self-Organizing Trees for Data Clustering and Online Vector Quantization

This paper introduces S-TREE (Self-Organizing Tree), a family of models that use unsupervised learning to construct hierarchical representations of data and online tree-structured vector quantizers. The S-TREE1 model, which features a new tree-building algorithm, can be implemented with various cost functions. An alternative implementation, S-TREE2, which uses a new double-path search procedure, is also developed. S-TREE2 implements an online procedure that approximates an optimal (unstructured) clustering solution while imposing a tree-structure constraint. The performance of the S-TREE algorithms is illustrated with data clustering and vector quantization examples, including a Gauss-Markov source benchmark and an image compression application. S-TREE performance on these tasks is compared with the standard tree-structured vector quantizer (TSVQ) and the generalized Lloyd algorithm (GLA). The image reconstruction quality with S-TREE2 approaches that of GLA while taking less than 10% of computer time. S-TREE1 and S-TREE2 also compare favorably with the standard TSVQ in both the time needed to create the codebook and the quality of image reconstruction.Office of Naval Research (N00014-95-10409, N00014-95-0G57

Fast embedding for image classification & retrieval and its application to the hostel industry

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonContent-based image classification and retrieval are the automatic processes of taking an unseen image input and extracting its features representing the input image. Then, for the classification task, this mathematically measured input is categorized according to established criteria in the server and consequently shows the output as a result. On the other hand, for the retrieval task, the extracted features of an unseen query image are sent to the server to search for the most visually similar images to a given image and retrieve these images as a result. Despite image features could be represented by classical features, artificial intelligence-based features, Convolutional Neural Networks (CNN) to be precise, have become powerful tools in the field. Nonetheless, the high dimensional CNN features have been a challenge in particular for applications on mobile or Internet of Things devices. Therefore, in this thesis, several fast embeddings are explored and proposed to overcome the constraints of low memory, bandwidth, and power. Furthermore, the first hostel image database is created with three datasets, hostel image dataset containing 13,908 interior and exterior images of hostels across the world, and Hostels-900 dataset and Hostels-2K dataset containing 972 images and 2,380 images, respectively, of 20 London hostel buildings. The results demonstrate that the proposed fast embeddings such as the application of GHM-Rand operator, GHM-Fix operator, and binary feature vectors are able to outperform or give competitive results to those state-of-the-art methods with a lot less computational resource. Additionally, the findings from a ten-year literature review of CBIR study in the tourism industry could picturize the relevant research activities in the past decade which are not only beneficial to the hostel industry or tourism sector but also to the computer science and engineering research communities for the potential real-life applications of the existing and developing technologies in the field

Deep-Learning Inferencing with High-Performance Hardware Accelerators

In order to improve their performance-per-watt capabilities over general-purpose architectures, FPGAs are commonly employed to accelerate applications. With the exponential growth of available data, machine-learning apps have generated greater interest in order to better understand that data and increase autonomous processing. As FPGAs become more readily available through cloud services like Amazon Web Services F1 platform, it is worth studying the performance of accelerating machine-learning apps on FPGAs over traditional fixed-logic devices, like CPUs and GPUs. FPGA frameworks for accelerating convolutional neural networks, which are used in many machine-learning apps, have started emerging for accelerated-application development. This thesis aims to compare the performance of these emerging frameworks on two commonly-used convolutional neural networks, GoogLeNet and AlexNet. Specifically, handwritten Chinese character recognition is benchmarked across multiple currently available FPGA frameworks on Xilinx and Intel FPGAs and compared against multiple CPU and GPU architectures featured on AWS, Google’s Cloud platform, the University of Pittsburgh’s Center for Research Computing (CRC), and Intel’s vLab Academic Cluster. All NVIDIA GPUs have proven to have the best performance over every other device in this study. The Zebra framework available for Xilinx FPGAs showed to have an average 8.3× and 9.3× better performance and efficiency, respectively, over the OpenVINO framework available for Intel FPGAs. Although the Zebra framework on the Xilinx VU9P showed better efficiency than the Pascal-based GPUs, the NVIDIA Tesla V100 proved to be the most efficient device at 125.9 and 47.2 images-per-second-per-Watt for AlexNet and GoogLeNet, respectively. Although currently lacking, FPGA frameworks and devices have the potential to compete with GPUs in terms of performance and efficiency