Efficient Bayesian Model Selection in PARAFAC via Stochastic Thermodynamic Integration

International audienceParallel factor analysis (PARAFAC) is one of the most popular tensor factorization models. Even though it has proven successful in diverse application fields, the performance of PARAFAC usually hinges up on the rank of the factorization, which is typically specified manually by the practitioner. In this study, we develop a novel parallel and distributed Bayesian model selection technique for rank estimation in large-scale PARAFAC models. The proposed approach integrates ideas from the emerging field of stochastic gradient Markov Chain Monte Carlo, statistical physics, and distributed stochastic optimization. As opposed to the existing methods, which are based on some heuristics, our method has a clear mathematical interpretation, and has significantly lower computational requirements, thanks to data subsampling and parallelization. We provide formal theoretical analysis on the bias induced by the proposed approach. Our experiments on synthetic and large-scale real datasets show that our method is able to find the optimal model order while being significantly faster than the state-of-the-art

Development of 2D- and 3D-BTEM for pattern recognition in higher-order spectroscopic and other data arrays

Ph.DDOCTOR OF PHILOSOPH

Bayesian Modelling Approaches for Quantum States -- The Ultimate Gaussian Process States Handbook

Capturing the correlation emerging between constituents of many-body systems accurately is one of the key challenges for the appropriate description of various systems whose properties are underpinned by quantum mechanical fundamentals. This thesis discusses novel tools and techniques for the (classical) modelling of quantum many-body wavefunctions with the ultimate goal to introduce a universal framework for finding accurate representations from which system properties can be extracted efficiently. It is outlined how synergies with standard machine learning approaches can be exploited to enable an automated inference of the most relevant intrinsic characteristics through rigorous Bayesian regression techniques. Based on the probabilistic framework forming the foundation of the introduced ansatz, coined the Gaussian Process State, different compression techniques are explored to extract numerically feasible representations of relevant target states within stochastic schemes. By following intuitively motivated design principles, the resulting model carries a high degree of interpretability and offers an easily applicable tool for the numerical study of quantum systems, including ones which are notoriously difficult to simulate due to a strong intrinsic correlation. The practical applicability of the Gaussian Process States framework is demonstrated within several benchmark applications, in particular, ground state approximations for prototypical quantum lattice models, Fermi-Hubbard models and $J_1-J_2$ models, as well as simple ab-initio quantum chemical systems.Comment: PhD Thesis, King's College London, 202 page

Model based process design for a monoclonal antibody-producing cell line :optimisation using hybrid modelling and an agent based system

PhD ThesisThe biopharmaceutical industry has seen rapid growth over the last 10 years in the area of therapeutic medicines. These include products such as monoclonal antibodies (mAbs) produced using mammalian cell lines such as Chinese Hamster Ovary (CHO). In order to comply with the regulatory authority (FDA) Quality by Design (QbD) and Process Analytical Technology (PAT) requirements, modelling can be used in the development and operation of the bioprocess. A model can assist in both the design, scale up and control of these complex, non-linear processes. A predictive model can be used to identify optimal operating conditions, which is vital for a contract manufacturer. Traditionally industry has approached modelling through the one-unit-at-a-time method, which can fail to capture unit interactions. The research reported in this work addresses this issue by using a whole system approach, which can also capture the interactions between units. Predictive models for each of the process units are combined within an overall framework allowing for the integration of the models, predicting how changes in the output of one unit influence the performance of subsequent units. These predictions can serve as the basis for the modifications to the standard operating procedures to achieve the required performance of the whole process. In this thesis three distinct studies are presented; the first utilises a hybridoma data set and presents a model to predict and characterise the various critical quality attributes (CQAs), such as final product glycosylation profile, and critical process parameters (CPPs) including titre and viable cell count. The second data set concerns the purification of lactoferrin using ion-exchange chromatography as a model system for developing downstream iii processing models. The output of this data set varied widely, and has led to the development of a novel peak isolation methodology, which can ultimately be used to characterise the elution. The final data set contains various CQAs and CPPs for multiple units within one process. This data set has been employed within a proof of concept study to show how an agent based framework can be developed to allow for overall process optimisation. The results showed that it is possible to link process units using a common CPP or CQA. This work shows that using a agent based system of two layers of modelling i.e. individual process unit models connected with a higher level agent model that links via a common measurement allows for the influences between units to be considered. The model presented in this work considers the use of titre, HCP, measure of heterogeneity, and molecular weight as the common measurement. It is shown that it is possible to link the units in this way with the goal of predicting and controlling the glycosylation profile of the Bulk Drug Substance (BDS)

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.Comment: 232 page

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.Comment: 232 page