Investigations on number selection for finite mixture models and clustering analysis.

by Yiu Ming Cheung.Thesis (M.Phil.)--Chinese University of Hong Kong, 1997.Includes bibliographical references (leaves 92-99).Abstract --- p.iAcknowledgement --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Background --- p.1Chapter 1.1.1 --- Bayesian YING-YANG Learning Theory and Number Selec- tion Criterion --- p.5Chapter 1.2 --- General Motivation --- p.6Chapter 1.3 --- Contributions of the Thesis --- p.6Chapter 1.4 --- Other Related Contributions --- p.7Chapter 1.4.1 --- A Fast Number Detection Approach --- p.7Chapter 1.4.2 --- Application of RPCL to Prediction Models for Time Series Forecasting --- p.7Chapter 1.4.3 --- Publications --- p.8Chapter 1.5 --- Outline of the Thesis --- p.8Chapter 2 --- Open Problem: How Many Clusters? --- p.11Chapter 3 --- Bayesian YING-YANG Learning Theory: Review and Experiments --- p.17Chapter 3.1 --- Briefly Review of Bayesian YING-YANG Learning Theory --- p.18Chapter 3.2 --- Number Selection Criterion --- p.20Chapter 3.3 --- Experiments --- p.23Chapter 3.3.1 --- Experimental Purposes and Data Sets --- p.23Chapter 3.3.2 --- Experimental Results --- p.23Chapter 4 --- Conditions of Number Selection Criterion --- p.39Chapter 4.1 --- Alternative Condition of Number Selection Criterion --- p.40Chapter 4.2 --- Conditions of Special Hard-cut Criterion --- p.45Chapter 4.2.1 --- Criterion Conditions in Two-Gaussian Case --- p.45Chapter 4.2.2 --- Criterion Conditions in k*-Gaussian Case --- p.59Chapter 4.3 --- Experimental Results --- p.60Chapter 4.3.1 --- Purpose and Data Sets --- p.60Chapter 4.3.2 --- Experimental Results --- p.63Chapter 4.4 --- Discussion --- p.63Chapter 5 --- Application of Number Selection Criterion to Data Classification --- p.80Chapter 5.1 --- Unsupervised Classification --- p.80Chapter 5.1.1 --- Experiments --- p.81Chapter 5.2 --- Supervised Classification --- p.82Chapter 5.2.1 --- RBF Network --- p.85Chapter 5.2.2 --- Experiments --- p.86Chapter 6 --- Conclusion and Future Work --- p.89Chapter 6.1 --- Conclusion --- p.89Chapter 6.2 --- Future Work --- p.90Bibliography --- p.92Chapter A --- A Number Detection Approach for Equal-and-Isotropic Variance Clusters --- p.100Chapter A.1 --- Number Detection Approach --- p.100Chapter A.2 --- Demonstration Experiments --- p.102Chapter A.3 --- Remarks --- p.105Chapter B --- RBF Network with RPCL Approach --- p.106Chapter B.l --- Introduction --- p.106Chapter B.2 --- Normalized RBF net and Extended Normalized RBF Net --- p.108Chapter B.3 --- Demonstration --- p.110Chapter B.4 --- Remarks --- p.113Chapter C --- Adaptive RPCL-CLP Model for Financial Forecasting --- p.114Chapter C.1 --- Introduction --- p.114Chapter C.2 --- Extraction of Input Patterns and Outputs --- p.115Chapter C.3 --- RPCL-CLP Model --- p.116Chapter C.3.1 --- RPCL-CLP Architecture --- p.116Chapter C.3.2 --- Training Stage of RPCL-CLP --- p.117Chapter C.3.3 --- Prediction Stage of RPCL-CLP --- p.122Chapter C.4 --- Adaptive RPCL-CLP Model --- p.122Chapter C.4.1 --- Data Pre-and-Post Processing --- p.122Chapter C.4.2 --- Architecture and Implementation --- p.122Chapter C.5 --- Computer Experiments --- p.125Chapter C.5.1 --- Data Sets and Experimental Purpose --- p.125Chapter C.5.2 --- Experimental Results --- p.126Chapter C.6 --- Conclusion --- p.134Chapter D --- Publication List --- p.135Chapter D.1 --- Publication List --- p.13

Classification of Occluded Objects using Fast Recurrent Processing

Recurrent neural networks are powerful tools for handling incomplete data problems in computer vision, thanks to their significant generative capabilities. However, the computational demand for these algorithms is too high to work in real time, without specialized hardware or software solutions. In this paper, we propose a framework for augmenting recurrent processing capabilities into a feedforward network without sacrificing much from computational efficiency. We assume a mixture model and generate samples of the last hidden layer according to the class decisions of the output layer, modify the hidden layer activity using the samples, and propagate to lower layers. For visual occlusion problem, the iterative procedure emulates feedforward-feedback loop, filling-in the missing hidden layer activity with meaningful representations. The proposed algorithm is tested on a widely used dataset, and shown to achieve 2$\times$ improvement in classification accuracy for occluded objects. When compared to Restricted Boltzmann Machines, our algorithm shows superior performance for occluded object classification.Comment: arXiv admin note: text overlap with arXiv:1409.8576 by other author

Representation learning for uncertainty-aware clinical decision support

Over the last decade, there has been an increasing trend towards digitalization in healthcare, where a growing amount of patient data is collected and stored electronically. These recorded data are known as electronic health records. They are the basis for state-of-the-art research on clinical decision support so that better patient care can be delivered with the help of advanced analytical techniques like machine learning. Among various technical fields in machine learning, representation learning is about learning good representations from raw data to extract useful information for downstream prediction tasks. Deep learning, a crucial class of methods in representation learning, has achieved great success in many fields such as computer vision and natural language processing. These technical breakthroughs would presumably further advance the research and development of data analytics in healthcare. This thesis addresses clinically relevant research questions by developing algorithms based on state-of-the-art representation learning techniques. When a patient visits the hospital, a physician will suggest a treatment in a deterministic manner. Meanwhile, uncertainty comes into play when the past statistics of treatment decisions from various physicians are analyzed, as they would possibly suggest different treatments, depending on their training and experiences. The uncertainty in clinical decision-making processes is the focus of this thesis. The models developed for supporting these processes will therefore have a probabilistic nature. More specifically, the predictions are predictive distributions in regression tasks and probability distributions over, e.g., different treatment decisions, in classification tasks. The first part of the thesis is concerned with prescriptive analytics to provide treatment recommendations. Apart from patient information and treatment decisions, the outcome after the respective treatment is included in learning treatment suggestions. The problem setting is known as learning individualized treatment rules and is formulated as a contextual bandit problem. A general framework for learning individualized treatment rules using data from observational studies is presented based on state-of-the-art representation learning techniques. From various offline evaluation methods, it is shown that the treatment policy in our proposed framework can demonstrate better performance than both physicians and competitive baselines. Subsequently, the uncertainty-aware regression models in diagnostic and predictive analytics are studied. Uncertainty-aware deep kernel learning models are proposed, which allow the estimation of the predictive uncertainty by a pipeline of neural networks and a sparse Gaussian process. By considering the input data structure, respective models are developed for diagnostic medical image data and sequential electronic health records. Various pre-training methods from representation learning are adapted to investigate their impacts on the proposed models. Through extensive experiments, it is shown that the proposed models delivered better performance than common architectures in most cases. More importantly, uncertainty-awareness of the proposed models is illustrated by systematically expressing higher confidence in more accurate predictions and less confidence in less accurate ones. The last part of the thesis is about missing data imputation in descriptive analytics, which provides essential evidence for subsequent decision-making processes. Rather than traditional mean and median imputation, a more advanced solution based on generative adversarial networks is proposed. The presented method takes the categorical nature of patient features into consideration, which enables the stabilization of the adversarial training. It is shown that the proposed method can better improve the predictive accuracy compared to traditional imputation baselines

Machine Learning for Synthetic Data Generation: A Review

Data plays a crucial role in machine learning. However, in real-world applications, there are several problems with data, e.g., data are of low quality; a limited number of data points lead to under-fitting of the machine learning model; it is hard to access the data due to privacy, safety and regulatory concerns. Synthetic data generation offers a promising new avenue, as it can be shared and used in ways that real-world data cannot. This paper systematically reviews the existing works that leverage machine learning models for synthetic data generation. Specifically, we discuss the synthetic data generation works from several perspectives: (i) applications, including computer vision, speech, natural language, healthcare, and business; (ii) machine learning methods, particularly neural network architectures and deep generative models; (iii) privacy and fairness issue. In addition, we identify the challenges and opportunities in this emerging field and suggest future research directions