Deep-Learning-Based Multivariate Pattern Analysis (dMVPA): A Tutorial and a Toolbox

In recent years, multivariate pattern analysis (MVPA) has been hugely beneficial for cognitive neuroscience by making new experiment designs possible and by increasing the inferential power of functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and other neuroimaging methodologies. In a similar time frame, “deep learning” (a term for the use of artificial neural networks with convolutional, recurrent, or similarly sophisticated architectures) has produced a parallel revolution in the field of machine learning and has been employed across a wide variety of applications. Traditional MVPA also uses a form of machine learning, but most commonly with much simpler techniques based on linear calculations; a number of studies have applied deep learning techniques to neuroimaging data, but we believe that those have barely scratched the surface of the potential deep learning holds for the field. In this paper, we provide a brief introduction to deep learning for those new to the technique, explore the logistical pros and cons of using deep learning to analyze neuroimaging data – which we term “deep MVPA,” or dMVPA – and introduce a new software toolbox (the “Deep Learning In Neuroimaging: Exploration, Analysis, Tools, and Education” package, DeLINEATE for short) intended to facilitate dMVPA for neuroscientists (and indeed, scientists more broadly) everywhere

Facilitating and Enhancing the Performance of Model Selection for Energy Time Series Forecasting in Cluster Computing Environments

Applying Machine Learning (ML) manually to a given problem setting is a tedious and time-consuming process which brings many challenges with it, especially in the context of Big Data. In such a context, gaining insightful information, finding patterns, and extracting knowledge from large datasets are quite complex tasks. Additionally, the configurations of the underlying Big Data infrastructure introduce more complexity for configuring and running ML tasks. With the growing interest in ML the last few years, particularly people without extensive ML expertise have a high demand for frameworks assisting people in applying the right ML algorithm to their problem setting. This is especially true in the field of smart energy system applications where more and more ML algorithms are used e.g. for time series forecasting. Generally, two groups of non-expert users are distinguished to perform energy time series forecasting. The first one includes the users who are familiar with statistics and ML but are not able to write the necessary programming code for training and evaluating ML models using the well-known trial-and-error approach. Such an approach is time consuming and wastes resources for constructing multiple models. The second group is even more inexperienced in programming and not knowledgeable in statistics and ML but wants to apply given ML solutions to their problem settings. The goal of this thesis is to scientifically explore, in the context of more concrete use cases in the energy domain, how such non-expert users can be optimally supported in creating and performing ML tasks in practice on cluster computing environments. To support the first group of non-expert users, an easy-to-use modular extendable microservice-based ML solution for instrumenting and evaluating ML algorithms on top of a Big Data technology stack is conceptualized and evaluated. Our proposed solution facilitates applying trial-and-error approach by hiding the low level complexities from the users and introduces the best conditions to efficiently perform ML tasks in cluster computing environments. To support the second group of non-expert users, the first solution is extended to realize meta learning approaches for automated model selection. We evaluate how meta learning technology can be efficiently applied to the problem space of data analytics for smart energy systems to assist energy system experts which are not data analytics experts in applying the right ML algorithms to their data analytics problems. To enhance the predictive performance of meta learning, an efficient characterization of energy time series datasets is required. To this end, Descriptive Statistics Time based Meta Features (DSTMF), a new kind of meta features, is designed to accurately capture the deep characteristics of energy time series datasets. We find that DSTMF outperforms the other state-of-the-art meta feature sets introduced in the literature to characterize energy time series datasets in terms of the accuracy of meta learning models and the time needed to extract them. Further enhancement in the predictive performance of the meta learning classification model is achieved by training the meta learner on new efficient meta examples. To this end, we proposed two new approaches to generate new energy time series datasets to be used as training meta examples by the meta learner depending on the type of time series dataset (i.e. generation or energy consumption time series). We find that extending the original training sets with new meta examples generated by our approaches outperformed the case in which the original is extended by new simulated energy time series datasets

On the Practice and Application of Context-Free Language Reachability

The Context-Free Language Reachability (CFL-R) formalism relates to some of the most important computational problems facing researchers and industry practitioners. CFL-R is a generalisation of graph reachability and language recognition, such that pairs in a labelled graph are reachable if and only if there is a path between them whose labels, joined together in the order they were encountered, spell a word in a given context-free language. The formalism finds particular use as a vehicle for phrasing and reasoning about program analysis, since complex relationships within the data, logic or structure of computer programs are easily expressed and discovered in CFL-R. Unfortunately, The potential of CFL-R can not be met by state of the art solvers. Current algorithms have scalability and expressibility issues that prevent them from being used on large graph instances or complex grammars. This work outlines our efforts in understanding the practical concerns surrounding CFL-R, and applying this knowledge to improve the performance of CFL-R applications. We examine the major difficulties with solving CFL-R-based analyses at-scale, via a case-study of points-to analysis as a CFL-R problem. Points-to analysis is fundamentally important to many modern research and industry efforts, and is relevant to optimisation, bug-checking and security technologies. Our understanding of the scalability challenge motivates work in developing practical CFL-R techniques. We present improved evaluation algorithms and declarative optimisation techniques for CFL-R, capitalising on the simplicity of CFL-R to creating fully automatic methodologies. The culmination of our work is a general-purpose and high-performance tool called Cauliflower, a solver-generator for CFL-R problems. We describe Cauliflower and evaluate its performance experimentally, showing significant improvement over alternative general techniques

Explicit Building-Block Multiobjective Genetic Algorithms: Theory, Analysis, and Developing

This dissertation research emphasizes explicit Building Block (BB) based MO EAs performance and detailed symbolic representation. An explicit BB-based MOEA for solving constrained and real-world MOPs is developed the Multiobjective Messy Genetic Algorithm II (MOMGA-II) which is designed to validate symbolic BB concepts. The MOMGA-II demonstrates that explicit BB-based MOEAs provide insight into solving difficult MOPs that is generally not realized through the use of implicit BB-based MOEA approaches. This insight is necessary to increase the effectiveness of all MOEA approaches. In order to increase MOEA computational efficiency parallelization of MOEAs is addressed. Communications between processors in a parallel MOEA implementation is extremely important, hence innovative migration and replacement schemes for use in parallel MOEAs are detailed and tested. These parallel concepts support the development of the first explicit BB-based parallel MOEA the pMOMGA-II. MOEA theory is also advanced through the derivation of the first MOEA population sizing theory. The multiobjective population sizing theory presented derives the MOEA population size necessary in order to achieve good results within a specified level of confidence. Just as in the single objective approach the MOEA population sizing theory presents a very conservative sizing estimate. Validated results illustrate insight into building block phenomena good efficiency excellent effectiveness and motivation for future research in the area of explicit BB-based MOEAs. Thus the generic results of this research effort have applicability that aid in solving many different MOPs

A model study of strong correlations in Hund metals

For a long time strong electronic correlations in metals have mainly been associated with Mottness, the proximity to a Mott metal-insulator transition (MIT), where large Coulomb interactions induce the localization of charges. However, triggered by the discovery of the iron-based superconductors about ten years ago, it was realized that multi-orbital materials with only moderate Coulomb but sizeable Hund’s rule interactions – so-called Hund metals – allow for a distinct screening mechanism towards strong correlations: Hundness. Here, Hund’s rule constrains the spin rather than the charge dynamics. This discovery led to a vividly debated fundamental issue in the field of strongly correlated condensed matter systems, which is the main topic of the present thesis: what is the origin of strong correlations in the normal phase of Hund metals, Mottness or Hundness? And what are their decisive fingerprints? The goal of this dissertation is twofold. First, we present and advance our method: the numerical renormalization group (NRG) as viable real-frequency multi-band impurity solver for dynamical mean-field theory (DMFT), a common approach to tackle strongly correlated systems. Second, we apply DMFT+NRG to shed light on the Hund-metal problem raised above. In the first part of this thesis we present our state-of-the-art NRG solver, which offers direct access to data with unprecedented real-frequency spectral resolution at arbitrarily low energies and temperatures in contrast to commonly used Quantum Monte Carlo solvers. It is based on matrix product states and exploits non-abelian symmetries to reduce numerical costs. In the case of orbital symmetry, this allows us to treat multi-band models with more than two bands, and thus to tackle the Hund-metal problem for the first time with NRG. For multi-band models without orbital symmetry, an “interleaved” scheme of NRG (iNRG) was recently developed, dramatically increasing the numerical efficiency. Remarkably, the accuracy of iNRG is comparable to standard NRG, as we reveal in a detailed study. This finding establishes iNRG as a promising DMFT solver for material-specific model simulations. In the second part of this thesis we study a minimal toy model for Hund metals with DMFT+NRG, the orbital-symmetric three-band Hubbard-Hund model (3HHM) close to a lattice filling of 1/3. Our major insight is “spin-orbital separation” (SOS), a Hund’s-ruleinduced two-stage Kondo-type screening process, in which orbital screening occurs at much higher energies than spin screening. In Hund metals, i.e. far from a MIT phase boundary, SOS thus causes large electron masses by strongly reducing the coherence scale below which a Fermi liquid is formed. Further, it opens up a broad incoherent and strongly particle-hole asymmetric intermediate energy regime that reaches up to bare excitation scales. This SOS regime shows fractional power-law behavior and is characterized by resilient “Hund quasiparticles” with itinerant orbital degrees of freedom coupled non-trivially to quasi-free large spins. At zero temperature, the local density of states exhibits a two-tier quasiparticle peak on top of a broad incoherent background. In contrast, in Mott-correlated metals, i.e. close to the MIT phase boundary, the SOS regime becomes negligibly small and the Hubbard bands are well separated. These findings lead to distinct signatures of Hundness and Mottness in the temperature dependence of ARPES spectra, static local susceptibilities, resistivity, thermopower and entropy, many of which were also found in realistic simulations of the archetypal Hund- and Mott-correlated materials, Sr2RuO4 and V2O3. In summary, we provide evidence that and elucidate how Hundness evokes strong correlation effects in Hund metals. This might help to better interpret experimental results and guide superconducting theories

Novel Memetic Computing Structures for Continuous Optimisation

This thesis studies a class of optimisation algorithms, namely Memetic Computing Structures, and proposes a novel set of promising algorithms that move the first step towards an implementation for the automatic generation of optimisation algorithms for continuous domains. This thesis after a thorough review of local search algorithms and popular meta-heuristics, focuses on Memetic Computing in terms of algorithm structures and design philosophy. In particular, most of the design carried out during my doctoral studies is inspired by the lex parsimoniae, aka Ockham’s Razor. It has been shown how simple algorithms, when well implemented can outperform complex implementations. In order to achieve this aim, the design is always carried out by attempting to identify the role of each algorithmic component/operator. In this thesis, on the basis of this logic, a set of variants of a recently proposed algorithms are presented. Subsequently a novel memetic structure, namely Parallel Memetic Structure is proposed and tested against modern algorithms representing the state of the art in optimisation. Furthermore, an initial prototype of an automatic design platform is also included. This prototype performs an analysis on separability of the optimisation problem and, on the basis of the analysis results, designs some parts of the parallel structure. Promising results are included. Finally, an investigation of the correlation among the variables and problem dimensionality has been performed. An extremely interesting finding of this thesis work is that the degree of correlation among the variables decreases when the dimensionality increases. As a direct consequence of this fact, large scale problems are to some extent easier to handle than problems in low dimensionality since, due to the lack of correlation among the variables, they can effectively be tackled by an algorithm that performs moves along the axes

New forms of international investment: a study of alternative strategies to foreign investment

This study is concerned with recent developments in international investment and the theory of the firm. The proposition that markets and hierarchies are alternative governance structures for completing related sets of transactions is less contentious. However, the view that foreign direct investment is the most efficient governance structure, in transaction-cost economizing terms, remains controversial. This research identifies with this contention. The premise of the study is that the governance structure of foreign transactions cannot be confined to or decided within the framework of hierarchy alone. The study presents a number of market mechanisms firms use to accomplish foreign transactions. Termed "New Forms of International Investment", these strategies involve non-equity (i.e. contractual/cooperative) and minority-equity arrangements. Hypotheses concerning the transaction cost nature and the impact of managerial perceptions of several explanatory factors were developed and tested using data gathered from a questionnaire survey of, and interviews with, executives from 66 MNCs and 31 MNBs. The results of the research provide evidence that while firm-specific characteristics offer firms opportunities to evaluate their strengths and weaknesses in relation to given overseas markets, host country-specific characteristics offer a complementary platform for assessing the optimum mode of entry. Also, managerial perceptions of the nature and importance of these factors and their impact on the diversification strategy of the firm were found to be significant in entry mode choices. The greater the perception of distortion propensities in a host country, the more likely resources, insofar as they would be transferred at all, would be transacted via new forms. There was no evidence to support the literature contention that the use of the new forms is a particular phenomenon of developing countries. These findings were reinforced by the interview results