Extending the Modern Synthesis: The evolution of ecosystems

The Modern Evolutionary Synthesis formalizes the role of variation, heredity, differential reproduction and mutation in population genetics. Here we explore a mathematical structure, based on the asymptotic limit theorems of information theory, that instantiates the punctuated dynamic relations of organisms and their embedding environments. The mathematical overhead is considerable, and we conclude that the model must itself be extended even further to allow the possibility of the transfer of heritage information between different classes of organisms. In essence, we provide something of a formal roadmap for the modernization of the Modern Synthesis

Unnatural Selection: A new formal approach to punctuated equilibrium in economic systems

Generalized Darwinian evolutionary theory has emerged as central to the description of economic process (e.g., Aldrich et. al., 2008). Here we demonstrate that, just as Darwinian principles provide necessary, but not sufficient, conditions for understanding the dynamics of social entities, in a similar manner the asymptotic limit theorems of information theory provide another set of necessary conditions that constrain the evolution of socioeconomic process. These latter constraints can, however, easily be formulated as a statistics-like analytic toolbox for the study of empirical data that is consistent with a generalized Darwinism, and this is no small thing

Towards an Information Theoretic Framework for Evolutionary Learning

The vital essence of evolutionary learning consists of information flows between the environment and the entities differentially surviving and reproducing therein. Gain or loss of information in individuals and populations due to evolutionary steps should be considered in evolutionary algorithm theory and practice. Information theory has rarely been applied to evolutionary computation - a lacuna that this dissertation addresses, with an emphasis on objectively and explicitly evaluating the ensemble models implicit in evolutionary learning. Information theoretic functionals can provide objective, justifiable, general, computable, commensurate measures of fitness and diversity. We identify information transmission channels implicit in evolutionary learning. We define information distance metrics and indices for ensembles. We extend Price\u27s Theorem to non-random mating, give it an effective fitness interpretation and decompose it to show the key factors influencing heritability and evolvability. We argue that heritability and evolvability of our information theoretic indicators are high. We illustrate use of our indices for reproductive and survival selection. We develop algorithms to estimate information theoretic quantities on mixed continuous and discrete data via the empirical copula and information dimension. We extend statistical resampling. We present experimental and real world application results: chaotic time series prediction; parity; complex continuous functions; industrial process control; and small sample social science data. We formalize conjectures regarding evolutionary learning and information geometry

Hydrologic prediction using pattern recognition and soft-computing techniques

Several studies indicate that the data-driven models have proven to be potentially useful tools in hydrological modeling. Nevertheless, it is a common perception among researchers and practitioners that the usefulness of the system theoretic models is limited to forecast applications, and they cannot be used as a tool for scientific investigations. Also, the system-theoretic models are believed to be less reliable as they characterize the hydrological processes by learning the input-output patterns embedded in the dataset and not based on strong physical understanding of the system. It is imperative that the above concerns needs to be addressed before the data-driven models can gain wider acceptability by researchers and practitioners.In this research different methods and tools that can be adopted to promote transparency in the data-driven models are probed with the objective of extending the usefulness of data-driven models beyond forecast applications as a tools for scientific investigations, by providing additional insights into the underlying input-output patterns based on which the data-driven models arrive at a decision. In this regard, the utility of self-organizing networks (competitive learning and self-organizing maps) in learning the patterns in the input space is evaluated by developing a novel neural network model called the spiking modular neural networks (SMNNs). The performance of the SMNNs is evaluated based on its ability to characterize streamflows and actual evapotranspiration process. Also the utility of self-organizing algorithms, namely genetic programming (GP), is evaluated with regards to its ability to promote transparency in data-driven models. The robustness of the GP to evolve its own model structure with relevant parameters is illustrated by applying GP to characterize the actual-evapotranspiration process. The results from this research indicate that self-organization in learning, both in terms of self-organizing networks and self-organizing algorithms, could be adopted to promote transparency in data-driven models.In pursuit of improving the reliability of the data-driven models, different methods for incorporating uncertainty estimates as part of the data-driven model building exercise is evaluated in this research. The local-scale models are shown to be more reliable than the global-scale models in characterizing the saturated hydraulic conductivity of soils. In addition, in this research, the importance of model structure uncertainty in geophysical modeling is emphasized by developing a framework to account for the model structure uncertainty in geophysical modeling. The contribution of the model structure uncertainty to the predictive uncertainty of the model is shown to be larger than the uncertainty associated with the model parameters. Also it has been demonstrated that increasing the model complexity may lead to a better fit of the function, but at the cost of an increasing level of uncertainty. It is recommended that the effect of model structure uncertainty should be considered for developing reliable hydrological models

Information theoretic stochastic search

The MAP-i Doctoral Programme in Informatics, of the Universities of Minho, Aveiro and PortoOptimization is the research field that studies the design of algorithms for finding the best solutions to problems we may throw at them. While the whole domain is practically important, the present thesis will focus on the subfield of continuous black-box optimization, presenting a collection of novel, state-of-the-art algorithms for solving problems in that class. In this thesis, we introduce two novel general-purpose stochastic search algorithms for black box optimisation. Stochastic search algorithms aim at repeating the type of mutations that led to fittest search points in a population. We can model those mutations by a stochastic distribution. Typically the stochastic distribution is modelled as a multivariate Gaussian distribution. The key idea is to iteratively change the parameters of the distribution towards higher expected fitness. However we leverage information theoretic trust regions and limit the change of the new distribution. We show how plain maximisation of the fitness expectation without bounding the change of the distribution is destined to fail because of overfitting and the results in premature convergence. Being derived from first principles, the proposed methods can be elegantly extended to contextual learning setting which allows for learning context dependent stochastic distributions that generates optimal individuals for a given context, i.e, instead of learning one task at a time, we can learn multiple related tasks at once. However, the search distribution typically uses a parametric model using some hand-defined context features. Finding good context features is a challenging task, and hence, non-parametric methods are often preferred over their parametric counter-parts. Therefore, we further propose a non-parametric contextual stochastic search algorithm that can learn a non-parametric search distribution for multiple tasks simultaneously.Otimização é área de investigação que estuda o projeto de algoritmos para encontrar as melhores soluções, tendo em conta um conjunto de critérios, para problemas complexos. Embora todo o domínio de otimização tenha grande importância, este trabalho está focado no subcampo da otimização contínua de caixa preta, apresentando uma coleção de novos algoritmos novos de última geração para resolver problemas nessa classe. Nesta tese, apresentamos dois novos algoritmos de pesquisa estocástica de propósito geral para otimização de caixa preta. Os algoritmos de pesquisa estocástica visam repetir o tipo de mutações que levaram aos melhores pontos de pesquisa numa população. Podemos modelar essas mutações por meio de uma distribuição estocástica e, tipicamente, a distribuição estocástica é modelada como uma distribuição Gaussiana multivariada. A ideia chave é mudar iterativamente os parâmetros da distribuição incrementando a avaliação. No entanto, alavancamos as regiões de confiança teóricas de informação e limitamos a mudança de distribuição. Deste modo, demonstra-se como a maximização simples da expectativa de “fitness”, sem limites da mudança da distribuição, está destinada a falhar devido ao “overfitness” e à convergência prematura resultantes. Sendo derivado dos primeiros princípios, as abordagens propostas podem ser ampliadas, de forma elegante, para a configuração de aprendizagem contextual que permite a aprendizagem de distribuições estocásticas dependentes do contexto que geram os indivíduos ideais para um determinado contexto. No entanto, a distribuição de pesquisa geralmente usa um modelo paramétrico linear em algumas das características contextuais definidas manualmente. Encontrar uma contextos bem definidos é uma tarefa desafiadora e, portanto, os métodos não paramétricos são frequentemente preferidos em relação às seus semelhantes paramétricos. Portanto, propomos um algoritmo não paramétrico de pesquisa estocástica contextual que possa aprender uma distribuição de pesquisa não-paramétrica para várias tarefas simultaneamente.FCT - Fundação para a Ciência e a Tecnologia. As well as fundings by European Union’s FP7 under EuRoC grant agreement CP-IP 608849 and by LIACC (UID/CEC/00027/2015) and IEETA (UID/CEC/00127/2015)

Behavioral and morphological traits interact to promote the evolution of alternative reproductive tactics in a lizard

Alternative reproductive tactics (ARTs) are predicted to be the result of disruptive correlational selection on suites of morphological, physiological, and behavioral traits. ARTs are most obvious when they occur in discrete morphs with concomitant behav