Modelling and optimisation of post-combustion carbon capture process integrated with coal-fired power plant using computational intelligence techniques

PhD ThesisCoal-fired power plants are the major source of CO2 emission which contributes significantly to global climate change. An effective way to reduce CO2 emission in coal-fired power plants is post-combustion carbon dioxide (CO2) capture (PCC) with chemical absorption. The aim of this project is to carry out some research in model development, process analysis, controller design and process optimization for reliable, optimal design and control of coal-fired supercritical power plant integrated with post-combustion carbon capture plant. In this thesis, three different advanced neural network models are developed: bootstrap aggregated neural networks (BANNs) model, bootstrap aggregated extreme learning machine (BAELM) model and deep belief networks (DBN) model. The bootstrap aggregated model can offer more accurate predictions than a single neural network, as well as provide model prediction confidence bounds. However, both BANNs and BAELM have a shallow architecture, which is limited to represent complex, highly-varying relationship and easy to converge to local optima. To resolve the problem, the DBN model is proposed. The unsupervised training procedure is helpful to get the optimal solution of supervised training. The purpose of developing neural network models is to find a best model which can be used in the optimization of the CO2 capture process precisely. This thesis also presents a comparison of centralized and decentralized control structures for post-combustion CO2 capture plant with chemical absorption. As for centralized configuration, a dynamic multivariate model predictive control (MPC) technique is used to control the post-combustion CO2 capture plant attached to a coal-fired power plant. When consider the decentralized control structures based on multi-loop proportional-integral-derivative (PID) controllers, two different control schemes are designed using relative disturbance gain (RDG) analysis and dynamic relative gain array (DRGA) analysis, respectively. By comparing the two control structures, the MPC structure performs better in terms of closed-loop settling time, integral squared error, and disturbance injection

Survey of deep representation learning for speech emotion recognition

Traditionally, speech emotion recognition (SER) research has relied on manually handcrafted acoustic features using feature engineering. However, the design of handcrafted features for complex SER tasks requires significant manual eort, which impedes generalisability and slows the pace of innovation. This has motivated the adoption of representation learning techniques that can automatically learn an intermediate representation of the input signal without any manual feature engineering. Representation learning has led to improved SER performance and enabled rapid innovation. Its effectiveness has further increased with advances in deep learning (DL), which has facilitated \textit{deep representation learning} where hierarchical representations are automatically learned in a data-driven manner. This paper presents the first comprehensive survey on the important topic of deep representation learning for SER. We highlight various techniques, related challenges and identify important future areas of research. Our survey bridges the gap in the literature since existing surveys either focus on SER with hand-engineered features or representation learning in the general setting without focusing on SER

Machine Learning and Integrative Analysis of Biomedical Big Data.

Recent developments in high-throughput technologies have accelerated the accumulation of massive amounts of omics data from multiple sources: genome, epigenome, transcriptome, proteome, metabolome, etc. Traditionally, data from each source (e.g., genome) is analyzed in isolation using statistical and machine learning (ML) methods. Integrative analysis of multi-omics and clinical data is key to new biomedical discoveries and advancements in precision medicine. However, data integration poses new computational challenges as well as exacerbates the ones associated with single-omics studies. Specialized computational approaches are required to effectively and efficiently perform integrative analysis of biomedical data acquired from diverse modalities. In this review, we discuss state-of-the-art ML-based approaches for tackling five specific computational challenges associated with integrative analysis: curse of dimensionality, data heterogeneity, missing data, class imbalance and scalability issues

Activity recognition in naturalistic environments using body-worn sensors

Phd ThesisThe research presented in this thesis investigates how deep learning and feature learning can address challenges that arise for activity recognition systems in naturalistic, ecologically valid surroundings such as the private home. One of the main aims of ubiquitous computing is the development of automated recognition systems for human activities and behaviour that are sufficiently robust to be deployed in realistic, in-the-wild environments. In most cases, the targeted application scenarios are people’s daily lives, where systems have to abide by practical usability and privacy constraints. We discuss how these constraints impact data collection and analysis and demonstrate how common approaches to the analysis of movement data effectively limit the practical use of activity recognition systems in every-day surroundings. In light of these issues we develop a novel approach to the representation and modelling of movement data based on a data-driven methodology that has applications in activity recognition, behaviour imaging, and skill assessment in ubiquitous computing. A number of case studies illustrate the suitability of the proposed methods and outline how study design can be adapted to maximise the benefit of these techniques, which show promising performance for clinical applications in particular.SiDE research hu

Carbon nanotubes: in situ studies of growth and electromechanical properties

Carbon nanotubes have been found to have extraordinary properties, such as ballistic electrical conductivity, extremely high thermal conductivity and they can be metallic or semiconducting with a wide range of band gaps. There are however several issues that have to be solved before these properties can be fully utilised. One of these issues is that the nanotube growth temperature must be lowered in order to make the synthesis compatible with the fabrication processes used in electronics. The whole environment is heated to temperatures typically higher than 500 °C in the standard growth techniques whereas only a very localised area is heated in the technique developed here. This technique thus provides a way around the temperature issue. In the method developed here, the catalyst is deposited on top of a small metal (molybdenum) wire on the substrate. The high temperature required for nanotube growth is then reached by Joule heating by sending a current through the metal wire. This process eliminates the furnace which is used in conventional chemical vapour deposition and localises the high temperature to a very small and controlled area of the sample. Consequently, this technique is compatible with the semiconductor technology used today. Another advantage of this technique is that, since no furnace is required, a small growth chamber, which fits under a microscope, can be used. This allows in situ studies of the growth by optical microscopy and by Raman spectroscopy. By changing the carbon precursor, single- or multiwalled nanotubes can be grown. This can be important when producing devices since single-walled nanotubes predominantly are semiconducting whereas multi-walled mainly are metallic. The multi-walled nanotubes grow in a rapid and concerted process. This growth was monitored through an optical microscope. It was found that the thickness of the support layer and especially the catalyst are even more crucial parameters for nanotube growth using this local heating technique than in conventional processes. The activation energy could be extracted and was found to be 1.1-1.3 eV. The carbon nanotube growth was investigated by in situ Raman spectroscopy. The growth evolution could be well described by a model using the initial growth rate and the catalyst lifetime as parameters. The process was found to be limited by the mass transport of the carbon precursor. It was found that the molybdenum wire creates an additional pathway for the carbon cycle from gas to nanotube formation. The Raman spectra were studied at elevated temperatures. A decrease in intensity and a shift towards lower wavenumbers with increasing temperature was observed for the Stokes signal. It was found that the laser used for the Raman measurements could heat the nanotubes to high temperatures without any other heat source. Vertically aligned arrays of nanotubes were grown by conventional CVD. These arrays were actuated by applying a DC voltage between them. An effective Young's modulus of the arrays was found to be similar to that of rubber, which is orders of magnitude lower than for individual nanotubes. The capacitance between the arrays was measured to be tens of fF with a tunability of over 20%

Composite load spectra for select space propulsion structural components

A multiyear program is performed with the objective to develop generic load models with multiple levels of progressive sophistication to simulate the composite (combined) load spectra that are induced in space propulsion system components, representative of Space Shuttle Main Engines (SSME), such as transfer ducts, turbine blades, and liquid oxygen (LOX) posts. Progress of the first year's effort includes completion of a sufficient portion of each task -- probabilistic models, code development, validation, and an initial operational code. This code has from its inception an expert system philosophy that could be added to throughout the program and in the future. The initial operational code is only applicable to turbine blade type loadings. The probabilistic model included in the operational code has fitting routines for loads that utilize a modified Discrete Probabilistic Distribution termed RASCAL, a barrier crossing method and a Monte Carlo method. An initial load model was developed by Battelle that is currently used for the slowly varying duty cycle type loading. The intent is to use the model and related codes essentially in the current form for all loads that are based on measured or calculated data that have followed a slowly varying profile

Higher-order interactions in single-cell gene expression: towards a cybergenetic semantics of cell state

Finding and understanding patterns in gene expression guides our understanding of living organisms, their development, and diseases, but is a challenging and high-dimensional problem as there are many molecules involved. One way to learn about the structure of a gene regulatory network is by studying the interdependencies among its constituents in transcriptomic data sets. These interdependencies could be arbitrarily complex, but almost all current models of gene regulation contain pairwise interactions only, despite experimental evidence existing for higher-order regulation that cannot be decomposed into pairwise mechanisms. I set out to capture these higher-order dependencies in single-cell RNA-seq data using two different approaches. First, I fitted maximum entropy (or Ising) models to expression data by training restricted Boltzmann machines (RBMs). On simulated data, RBMs faithfully reproduced both pairwise and third-order interactions. I then trained RBMs on 37 genes from a scRNA-seq data set of 70k astrocytes from an embryonic mouse. While pairwise and third-order interactions were revealed, the estimates contained a strong omitted variable bias, and there was no statistically sound and tractable way to quantify the uncertainty in the estimates. As a result I next adopted a model-free approach. Estimating model-free interactions (MFIs) in single-cell gene expression data required a quasi-causal graph of conditional dependencies among the genes, which I inferred with an MCMC graph-optimisation algorithm on an initial estimate found by the Peter-Clark algorithm. As the estimates are model-free, MFIs can be interpreted either as mechanistic relationships between the genes, or as substructures in the cell population. On simulated data, MFIs revealed synergy and higher-order mechanisms in various logical and causal dynamics more accurately than any correlation- or information-based quantities. I then estimated MFIs among 1,000 genes, at up to seventh-order, in 20k neurons and 20k astrocytes from two different mouse brain scRNA-seq data sets: one developmental, and one adolescent. I found strong evidence for up to fifth-order interactions, and the MFIs mostly disambiguated direct from indirect regulation by preferentially coupling causally connected genes, whereas correlations persisted across causal chains. Validating the predicted interactions against the Pathway Commons database, gene ontology annotations, and semantic similarity, I found that pairwise MFIs contained different but a similar amount of mechanistic information relative to networks based on correlation. Furthermore, third-order interactions provided evidence of combinatorial regulation by transcription factors and immediate early genes. I then switched focus from mechanism to population structure. Each significant MFI can be assigned a set of single cells that most influence its value. Hierarchical clustering of the MFIs by cell assignment revealed substructures in the cell population corresponding to diverse cell states. This offered a new, purely data-driven view on cell states because the inferred states are not required to localise in gene expression space. Across the four data sets, I found 69 significant and biologically interpretable cell states, where only 9 could be obtained by standard approaches. I identified immature neurons among developing astrocytes and radial glial cells, D1 and D2 medium spiny neurons, D1 MSN subtypes, and cell-cycle related states present across four data sets. I further found evidence for states defined by genes associated to neuropeptide signalling, neuronal activity, myelin metabolism, and genomic imprinting. MFIs thus provide a new, statistically sound method to detect substructure in single-cell gene expression data, identifying cell types, subtypes, or states that can be delocalised in gene expression space and whose hierarchical structure provides a new view on the semantics of cell state. The estimation of the quasi-causal graph, the MFIs, and inference of the associated states is implemented as a publicly available Nextflow pipeline called Stator

Online Non-linear Prediction of Financial Time Series Patterns

We consider a mechanistic non-linear machine learning approach to learning signals in financial time series data. A modularised and decoupled algorithm framework is established and is proven on daily sampled closing time-series data for JSE equity markets. The input patterns are based on input data vectors of data windows preprocessed into a sequence of daily, weekly and monthly or quarterly sampled feature measurement changes (log feature fluctuations). The data processing is split into a batch processed step where features are learnt using a Stacked AutoEncoder (SAE) via unsupervised learning, and then both batch and online supervised learning are carried out on Feedforward Neural Networks (FNNs) using these features. The FNN output is a point prediction of measured time-series feature fluctuations (log differenced data) in the future (ex-post). Weight initializations for these networks are implemented with restricted Boltzmann machine pretraining, and variance based initializations. The validity of the FNN backtest results are shown under a rigorous assessment of backtest overfitting using both Combinatorially Symmetrical Cross Validation and Probabilistic and Deflated Sharpe Ratios. Results are further used to develop a view on the phenomenology of financial markets and the value of complex historical data under unstable dynamics

Investigating machine learning methods in recommender systems

This thesis investigates the use of machine learning in improving predictions of the top K* product purchases at a particular a retailer. The data used for this research is a freely-available (for research) sample of the retailer’s transactional data spanning a period of 102 weeks and consisting of several million observations. The thesis consists of four key experiments: 1. Univariate Analysis of the Dataset: The first experiment, which is the univariate analysis of the dataset, sets the background to the following chapters. It provides explanatory insight into the customers’ shopping behaviour and identifies the drivers that connect customers and products. Using various behavioural, descriptive and aggregated features, the training dataset for a group of customers is created to map their future purchasing actions for one specific week. The test dataset is then constructed to predict the purchasing actions for the forthcoming week. This constitutes a univariate analysis and the chapter is an introduction to the features included in the subsequent algorithmic processes. 2. Meta-modelling to predict top K products: The second experiment investigates the improvement in predicting the top K products in terms of precision at K (or precision@K) and Area Under Curve (AUC) through meta-modelling. It compares combining a range of common machine learning algorithms of a supervised nature within a meta-modelling framework (where each generated model will be an input to a secondary model) with any single model involved, field benchmark or simple model combination method. 3. Hybrid method to predict repeated, promotion-driven product purchases in an irregular testing environment: The third experiment demonstrates a hybrid methodology of cross validation, modelling and optimization for improving the accuracy of predicting the products the customers of a retailer will buy after havingbought them at least once with a promotional coupon. This methodology is applied in the context of a train and test environment with limited overlap - the test data includes different coupons, different customers and different time periods. Additionally this chapter uses a real life application and a stress-test of the findings in the feature engineering space from experiment 1. It also borrows ideas from ensemble (or meta) modelling as detailed in experiment 2. 4. The StackNet model: The fourth experiment proposes a framework in the form of a scalable version of [Wolpert 1992] stacked generalization being extended through cross validation methods to many levels resembling in structure a fully connected feedforward neural network where the hidden nodes represent complex functions in the form of machine learning models of any nature. The implementation of the model is made available in the Java programming language. The research contribution of this thesis is to improve the recommendation science used in the grocery and Fast Moving Consumer Goods (FMCG) markets. It seeks to identify methods of increasing the accuracy of predicting what customers are going to buy in the future by leveraging up-to-date innovations in machine learning as well as improving current processes in the areas of feature engineering, data pre-processing and ensemble modelling. For the general scientific community this thesis can be exploited to better understand the type of data available in the grocery market and to gain insights into how to structure similar machine learning and analytical projects. The extensive, computational and algorithmic framework that accompanies this thesis is also available for general use as a prototype to solve similar data challenges. References: Wolpert, D. H. (1992). Stacked generalization. Neural networks, 5(2), 241-259. Yang, X., Steck, H., Guo, Y., & Liu, Y. (2012). On top-k recommendation using social networks. In Proceedings of the sixth ACM conference on Recommender systems (pp. 67-74). ACM