Variational Hamiltonian Monte Carlo via Score Matching

Traditionally, the field of computational Bayesian statistics has been divided into two main subfields: variational methods and Markov chain Monte Carlo (MCMC). In recent years, however, several methods have been proposed based on combining variational Bayesian inference and MCMC simulation in order to improve their overall accuracy and computational efficiency. This marriage of fast evaluation and flexible approximation provides a promising means of designing scalable Bayesian inference methods. In this paper, we explore the possibility of incorporating variational approximation into a state-of-the-art MCMC method, Hamiltonian Monte Carlo (HMC), to reduce the required gradient computation in the simulation of Hamiltonian flow, which is the bottleneck for many applications of HMC in big data problems. To this end, we use a {\it free-form} approximation induced by a fast and flexible surrogate function based on single-hidden layer feedforward neural networks. The surrogate provides sufficiently accurate approximation while allowing for fast exploration of parameter space, resulting in an efficient approximate inference algorithm. We demonstrate the advantages of our method on both synthetic and real data problems

Modeling and Optimization of Chemical Mechanical Planarization (Cmp) Using Neural Networks, Anfis and Evolutionary Algorithms

Higher density nano-devices and more metallization layers in microelectronic chips are unceasing goals to the present semiconductor industry. However, topological imperfections (higher non-uniformity) on the wafer surfaces and lower material removal rates (MRR) seriously hamper these pursuing motivations. Since'90, industry has been using chemical mechanical planarization/polishing (CMP) to overcome these obstacles for fabricating integrated circuits (IC) with interconnect geometries of < 0.18 &amp;#956;m. Obviously, the much needed understanding of this new technique is derived basically on the ancient lapping process. Modeling and simulation are critical to transfer CMP from an engineering 'art' to an engineering 'science'. Many efforts in CMP modeling have been made in the last decade, but the available analytical MRR and surface uniformity models cannot precisely describe this highly complicated process, involving simultaneous chemical reactions (and etching), and mechanical abrasion. In this investigation, neural networks (NN), adaptive-based-network fuzzy inference system (ANFIS), and evolutionary algorithms (EA) techniques were applied to successfully overcome the aforementioned modeling and simulation problems. In addition, fine-tuning techniques for re-modifying ANFIS models for sparse-data case using are developed. Furthermore, multi-objective evolutionary algorithms (MOEA) are firstly applied to search for the optimal input settings for CMP process to trade-off the higher MRR and lower non-Uniformity by using the previously constructed models. The results also show the simulation of MOEA optimization can certainly provide accurate guidance to search the optimal input settings for CMP process to produce lower non-uniform wafer surfaces under higher MRR.Mechanical & Aerospace Engineerin

Reasoning with Uncertainty in Deep Learning for Safer Medical Image Computing

Deep learning is now ubiquitous in the research field of medical image computing. As such technologies progress towards clinical translation, the question of safety becomes critical. Once deployed, machine learning systems unavoidably face situations where the correct decision or prediction is ambiguous. However, the current methods disproportionately rely on deterministic algorithms, lacking a mechanism to represent and manipulate uncertainty. In safety-critical applications such as medical imaging, reasoning under uncertainty is crucial for developing a reliable decision making system. Probabilistic machine learning provides a natural framework to quantify the degree of uncertainty over different variables of interest, be it the prediction, the model parameters and structures, or the underlying data (images and labels). Probability distributions are used to represent all the uncertain unobserved quantities in a model and how they relate to the data, and probability theory is used as a language to compute and manipulate these distributions. In this thesis, we explore probabilistic modelling as a framework to integrate uncertainty information into deep learning models, and demonstrate its utility in various high-dimensional medical imaging applications. In the process, we make several fundamental enhancements to current methods. We categorise our contributions into three groups according to the types of uncertainties being modelled: (i) predictive; (ii) structural and (iii) human uncertainty. Firstly, we discuss the importance of quantifying predictive uncertainty and understanding its sources for developing a risk-averse and transparent medical image enhancement application. We demonstrate how a measure of predictive uncertainty can be used as a proxy for the predictive accuracy in the absence of ground-truths. Furthermore, assuming the structure of the model is flexible enough for the task, we introduce a way to decompose the predictive uncertainty into its orthogonal sources i.e. aleatoric and parameter uncertainty. We show the potential utility of such decoupling in providing a quantitative “explanations” into the model performance. Secondly, we introduce our recent attempts at learning model structures directly from data. One work proposes a method based on variational inference to learn a posterior distribution over connectivity structures within a neural network architecture for multi-task learning, and share some preliminary results in the MR-only radiotherapy planning application. Another work explores how the training algorithm of decision trees could be extended to grow the architecture of a neural network to adapt to the given availability of data and the complexity of the task. Lastly, we develop methods to model the “measurement noise” (e.g., biases and skill levels) of human annotators, and integrate this information into the learning process of the neural network classifier. In particular, we show that explicitly modelling the uncertainty involved in the annotation process not only leads to an improvement in robustness to label noise, but also yields useful insights into the patterns of errors that characterise individual experts

NASA patent abstracts bibliography: A continuing bibliography. Section 2: Indexes (supplement 46)

A subject index is provided for over 5600 patents and patent applications for the period May 1969 through December 1994. Additional indexes list personal authors, corporate authors, contract numbers, NASA case numbers, U.S. patent class numbers, U.S. patent numbers, and NASA accession numbers