Simulated Annealing

The book contains 15 chapters presenting recent contributions of top researchers working with Simulated Annealing (SA). Although it represents a small sample of the research activity on SA, the book will certainly serve as a valuable tool for researchers interested in getting involved in this multidisciplinary field. In fact, one of the salient features is that the book is highly multidisciplinary in terms of application areas since it assembles experts from the fields of Biology, Telecommunications, Geology, Electronics and Medicine

Neural Networks. A General Framework for Non-Linear Function Approximation

The focus of this paper is on the neural network modelling approach that has gained increasing recognition in GIScience in recent years. The novelty about neural networks lies in their ability to model non-linear processes with few, if any, a priori assumptions about the nature of the data-generating process. The paper discusses some important issues that are central for successful application development. The scope is limited to feedforward neural networks, the leading example of neural networks. It is argued that failures in applications can usually be attributed to inadequate learning and/or inadequate complexity of the network model. Parameter estimation and a suitably chosen number of hidden units are, thus, of crucial importance for the success of real world neural network applications. The paper views network learning as an optimization problem, reviews two alternative approaches to network learning, and provides insights into current best practice to optimize complexity so to perform well on generalization tasks

Theoretical Optimization of Enzymatic Biomass Processes

This dissertation introduces a complete, stochastically-based algorithmic framework Cellulect to study, optimize and predict hydrolysis processes of the structured biomass cellulose. The framework combines a comprehensive geometric model for the cellulosic substrate with microstructured crystalline/amorphous regions distribution, distinctive monomers, polymer chain lengths distribution and free surface area tracking. An efficient tracking algorithm, formulated in a serial fashion, performs the updates of the system. The updates take place reaction-wise. The notion of real time is preserved. Advanced types of enzyme actions (random cuts, reduced/non-reduced end cuts, orientation, and the possibility of a fixed position of active centers) and their modular structure (carbohydrate-binding module with a flexible linker and a catalytic domain) are taken into account within the framework. The concept of state machines is adopted to model enzyme entities. This provides a reliable, powerful and maintainable approach for modelling already known enzyme features and can be extended with additional features not taken into account in the present work. The provided extensive probabilistic catalytic mechanism description further includes adsorption, desorption, competitive inhibition by soluble product polymers, and dynamical bond-breaking reactions with inclusive dependence on monomers and their polymers states within the substrate. All incorporated parameters refer to specific system properties, providing a one to one relationship between degrees of freedom and available features of the model. Finally, time propagation of the system is based on the modified stochastic Gillespie algorithm. It provides an exact stochastic time-reaction propagation algorithm, taking into account the random nature of reaction events as well as its random occurrences. The framework is ready for constrained input parameter estimation with empirical data sets of product concentration profiles by utilizing common optimization routines. Verification of the available data for the most common enzyme kinds (EG, β-G, CBH) in the literature has been accomplished. Sensitivity analysis of estimated model parameters were carried out. Dependency of various experimental input is shown. Optimization behavior in underdetermined conditions is inspected and visualized. Results and predictions for mixtures of optimized enzymes, as well as a practical way to implement and utilize the Cellulect framework are also provided. The obtained results were compared to experimental literature data demonstrate the high flexibility, efficiency and accuracy of the presented framework for the prediction of the cellulose hydrolysis process

Improving individual identification of wolves (Canis lupus) using the fundamental frequency and amplitude of their howls: a new survey method

Many bioacoustic studies have been able to identify individual mammals from variations in the fundamental frequency (F0) of their vocalizations. Other characteristics of vocalization which encode individuality, such as amplitude, are less frequently used because of problems with background noise and recording fidelity over distance. In this thesis, I investigate whether the inclusion of amplitude variables improves the accuracy of individual howl identification in captive Eastern grey wolves (Canis lupus lycaon). I also explore whether the use of a bespoke code to extract the howl features, combined with histogram-derived principal component analysis (PCA) values, can improve current individual wolf howl identification accuracies. From a total of 89 solo howls from six captive individuals, where distances between wolf and observer were short, I achieved 95.5% (+9.0% improvement) individual identification accuracy of captive wolves using discriminant function analysis (DFA) to classify simple scalar variables of F0 and normalized amplitudes. Moreover, this accuracy was increased to 100% when using histogram-derived PCA values of F0 and amplitudes of the first harmonic

On robust and adaptive soft sensors.

In process industries, there is a great demand for additional process information such as the product quality level or the exact process state estimation. At the same time, there is a large amount of process data like temperatures, pressures, etc. measured and stored every moment. This data is mainly measured for process control and monitoring purposes but its potential reaches far beyond these applications. The task of soft sensors is the maximal exploitation of this potential by extracting and transforming the latent information from the data into more useful process knowledge. Theoretically, achieving this goal should be straightforward since the process data as well as the tools for soft sensor development in the form of computational learning methods, are both readily available. However, contrary to this evidence, there are still several obstacles which prevent soft sensors from broader application in the process industry. The identification of the sources of these obstacles and proposing a concept for dealing with them is the general purpose of this work. The proposed solution addressing the issues of current soft sensors is a conceptual architecture for the development of robust and adaptive soft sensing algorithms. The architecture reflects the results of two review studies that were conducted during this project. The first one focuses on the process industry aspects of soft sensor development and application. The main conclusions of this study are that soft sensor development is currently being done in a non-systematic, ad-hoc way which results in a large amount of manual work needed for their development and maintenance. It is also found that a large part of the issues can be related to the process data upon which the soft sensors are built. The second review study dealt with the same topic but this time it was biased towards the machine learning viewpoint. The review focused on the identification of machine learning tools, which support the goals of this work. The machine learning concepts which are considered are: (i) general regression techniques for building of soft sensors; (ii) ensemble methods; (iii) local learning; (iv) meta-learning; and (v) concept drift detection and handling. The proposed architecture arranges the above techniques into a three-level hierarchy, where the actual prediction-making models operate at the bottom level. Their predictions are flexibly merged by applying ensemble methods at the next higher level. Finally from the top level, the underlying algorithm is managed by means of metalearning methods. The architecture has a modular structure that allows new pre-processing, predictive or adaptation methods to be plugged in. Another important property of the architecture is that each of the levels can be equipped with adaptation mechanisms, which aim at prolonging the lifetime of the resulting soft sensors. The relevance of the architecture is demonstrated by means of a complex soft sensing algorithm, which can be seen as its instance. This algorithm provides mechanisms for autonomous selection of data preprocessing and predictive methods and their parameters. It also includes five different adaptation mechanisms, some of which can be applied on a sample-by-sample basis without any requirement to store the on-line data. Other, more complex ones are started only on-demand if the performance of the soft sensor drops below a defined level. The actual soft sensors are built by applying the soft sensing algorithm to three industrial data sets. The different application scenarios aim at the analysis of the fulfilment of the defined goals. It is shown that the soft sensors are able to follow changes in dynamic environment and keep a stable performance level by exploiting the implemented adaptation mechanisms. It is also demonstrated that, although the algorithm is rather complex, it can be applied to develop simple and transparent soft sensors. In another experiment, the soft sensors are built without any manual model selection or parameter tuning, which demonstrates the ability of the algorithm to reduce the effort required for soft sensor development. However, if desirable, the algorithm is at the same time very flexible and provides a number of parameters that can be manually optimised. Evidence of the ability of the algorithm to deploy soft sensors with minimal training data and as such to provide the possibility to save the time consuming and costly training data collection is also given in this work

Learning Transferable Representations

A first contribution of this thesis is to propose causality as a language for problems of distribution shift. First, we consider domain generalisation, where no data from the test distribution are observed during training. What assumptions can be made regarding the relation between train and test distributions for transfer to succeed? We argue that assuming the data in both tasks originate from the same causal graph leads to a natural solution: use only causal features for prediction, as the mechanism mapping causes to effects is invariant to shifts in the probability distributions induced by the causal structure. We provide optimality results when the test task is adversarial, and introduce a method for exploiting all remaining features when data from the test task are observed. We motivate that learning such invariant mechanisms mapping features to outputs leads to machine learning modules robust to transfer. Second, we consider a classification problem where only few examples are available for each label. How should an initial large dataset be leveraged to improve performance in this task? We argue that such a dataset should be used to learn powerful features for batch classification using a neural network. We present a framework which transfers between classes by building a probabilistic model on the weights of the network. Our results suggest that practitioners should use the original dataset for building features whose power can be exploited during few-shot learning. Finally, we extend causal discovery to solve problems such as distinguishing a painting from its counterfeit. Given two such static entities, a proxy random variable introduces the randomness necessary to construct two features of the static entities which preserve their causal footprint, measurable by a standard causal discovery procedure. Experiments on vision and language provide evidence that the causal relation between the static entities can often be identified