Bayesian Brains and the Rényi Divergence

Under the Bayesian brain hypothesis, behavioral variations can be attributed to different priors over generative model parameters. This provides a formal explanation for why individuals exhibit inconsistent behavioral preferences when confronted with similar choices. For example, greedy preferences are a consequence of confident (or precise) beliefs over certain outcomes. Here, we offer an alternative account of behavioral variability using Rényi divergences and their associated variational bounds. Rényi bounds are analogous to the variational free energy (or evidence lower bound) and can be derived under the same assumptions. Importantly, these bounds provide a formal way to establish behavioral differences through an α parameter, given fixed priors. This rests on changes in α that alter the bound (on a continuous scale), inducing different posterior estimates and consequent variations in behavior. Thus, it looks as if individuals have different priors and have reached different conclusions. More specifically, α→0+ optimization constrains the variational posterior to be positive whenever the true posterior is positive. This leads to mass-covering variational estimates and increased variability in choice behavior. Furthermore, α→+∞ optimization constrains the variational posterior to be zero whenever the true posterior is zero. This leads to mass-seeking variational posteriors and greedy preferences. We exemplify this formulation through simulations of the multiarmed bandit task. We note that these α parameterizations may be especially relevant (i.e., shape preferences) when the true posterior is not in the same family of distributions as the assumed (simpler) approximate density, which may be the case in many real-world scenarios. The ensuing departure from vanilla variational inference provides a potentially useful explanation for differences in behavioral preferences of biological (or artificial) agents under the assumption that the brain performs variational Bayesian inference

A configurable deep network for high-dimensional clinical trial data

Clinical studies provide interesting case studies for data mining researchers, given the often high degree of dimensionality and long term nature of these studies. In areas such as dementia, accurate predictions from data scientists provide vital input into the understanding of how certain features (representing lifestyle) can predict outcomes such as dementia. Most research involved has used traditional or shallow data mining approaches which have been shown to offer varying degrees of accuracy in datasets with high dimensionality. In this research, we explore the use of deep learning architectures, as they have been shown to have high predictive capabilities in image and audio datasets. The purpose of our research is to build a framework which allows easy reconfiguration for the performance of experiments across a number of deep learning approaches. In this paper, we present our framework for a configurable deep learning machine and our evaluation and analysis of two shallow approaches: regression and multi-layer perceptron, as a platform to a deep belief network, and using a dataset created over the course of 12 years by researchers in the area of dementia

EvoDeep: A new evolutionary approach for automatic Deep Neural Networks parametrisation

[EN] Deep Neural Networks (DNN) have become a powerful, and extremely popular mechanism, which has been widely used to solve problems of varied complexity, due to their ability to make models fitted to non-linear complex problems. Despite its well-known benefits, DNNs are complex learning models whose parametrisation and architecture are made usually by hand. This paper proposes a new Evolutionary Algorithm, named EvoDeep. devoted to evolve the parameters and the architecture of a DNN in order to maximise its classification accuracy, as well as maintaining a valid sequence of layers. This model is tested against a widely used dataset of handwritten digits images. The experiments performed using this dataset show that the Evolutionary Algorithm is able to select the parameters and the DNN architecture appropriately, achieving a 98.93% accuracy in the best run. (C) 2017 Elsevier Inc. All rights reserved.This work has been co-funded by the next research projects: EphemeCH (TIN2014-56494-C4-4-P) and DeepBio (TIN2017-85727-C4-3-P) Spanish Ministry of Economy and Competitivity and European Regional Development Fund FEDER, Justice Programme of the European Union (2014-2020) 723180 -RiskTrack-JUST-2015-JCOO-AG/JUST-2015-JCOO-AG-1, and by the CAM grant S2013/ICE-3095 (CIBERDINE:Cybersecurity, Data and Risks). The contents of this publication are the sole responsibility of their authors and can in no way be taken to reflect the views of the European Commission.Martín, A.; Lara-Cabrera, R.; Fuentes-Hurtado, FJ.; Naranjo Ornedo, V.; Camacho, D. (2018). EvoDeep: A new evolutionary approach for automatic Deep Neural Networks parametrisation. Journal of Parallel and Distributed Computing. 117:180-191. https://doi.org/10.1016/j.jpdc.2017.09.006S18019111

The Application of Nature-inspired Metaheuristic Methods for Optimising Renewable Energy Problems and the Design of Water Distribution Networks

This work explores the technical challenges that emerge when applying bio-inspired optimisation methods to real-world engineering problems. A number of new heuristic algorithms were proposed and tested to deal with these challenges. The work is divided into three main dimensions: i) One of the most significant industrial optimisation problems is optimising renewable energy systems. Ocean wave energy is a promising technology for helping to meet future growth in global energy demand. However, the current technologies of wave energy converters (WECs) are not fully developed because of technical engineering and design challenges. This work proposes new hybrid heuristics consisting of cooperative coevolutionary frameworks and neuro-surrogate optimisation methods for optimising WECs problem in three domains, including position, control parameters, and geometric parameters. Our problem-specific algorithms perform better than existing approaches in terms of higher quality results and the speed of convergence. ii) The second part applies search methods to the optimization of energy output in wind farms. Wind energy has key advantages in terms of technological maturity, cost, and life-cycle greenhouse gas emissions. However, designing an accurate local wind speed and power prediction is challenging. We propose two models for wind speed and power forecasting for two wind farms located in Sweden and the Baltic Sea by a combination of recurrent neural networks and evolutionary search algorithms. The proposed models are superior to other applied machine learning methods. iii) Finally, we investigate the design of water distribution systems (WDS) as another challenging real-world optimisation problem. WDS optimisation is demanding because it has a high-dimensional discrete search space and complex constraints. A hybrid evolutionary algorithm is suggested for minimising the cost of various water distribution networks and for speeding up the convergence rate of search.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 202

The synergistic effect of operational research and big data analytics in greening container terminal operations: a review and future directions

Container Terminals (CTs) are continuously presented with highly interrelated, complex, and uncertain planning tasks. The ever-increasing intensity of operations at CTs in recent years has also resulted in increasing environmental concerns, and they are experiencing an unprecedented pressure to lower their emissions. Operational Research (OR), as a key player in the optimisation of the complex decision problems that arise from the quay and land side operations at CTs, has been therefore presented with new challenges and opportunities to incorporate environmental considerations into decision making and better utilise the ‘big data’ that is continuously generated from the never-stopping operations at CTs. The state-of-the-art literature on OR's incorporation of environmental considerations and its interplay with Big Data Analytics (BDA) is, however, still very much underdeveloped, fragmented, and divergent, and a guiding framework is completely missing. This paper presents a review of the most relevant developments in the field and sheds light on promising research opportunities for the better exploitation of the synergistic effect of the two disciplines in addressing CT operational problems, while incorporating uncertainty and environmental concerns efficiently. The paper finds that while OR has thus far contributed to improving the environmental performance of CTs (rather implicitly), this can be much further stepped up with more explicit incorporation of environmental considerations and better exploitation of BDA predictive modelling capabilities. New interdisciplinary research at the intersection of conventional CT optimisation problems, energy management and sizing, and net-zero technology and energy vectors adoption is also presented as a prominent line of future research

Advances in Probabilistic Deep Learning

This thesis is concerned with methodological advances in probabilistic inference and their application to core challenges in machine perception and AI. Inferring a posterior distribution over the parameters of a model given some data is a central challenge that occurs in many fields ranging from finance and artificial intelligence to physics. Exact calculation is impossible in all but the simplest cases and a rich field of approximate inference has been developed to tackle this challenge. This thesis develops both an advance in approximate inference and an application of these methods to the problem of speech synthesis. In the first section of this thesis we develop a novel framework for constructing Markov Chain Monte Carlo (MCMC) kernels that can efficiently sample from high dimensional distributions such as the posteriors, that frequently occur in machine perception. We provide a specific instance of this framework and demonstrate that it can match or exceed the performance of Hamiltonian Monte Carlo without requiring gradients of the target distribution. In the second section of the thesis we focus on the application of approximate inference techniques to the task of synthesising human speech from text. By using advances in neural variational inference we are able to construct a state of the art speech synthesis system in which it is possible to control aspects of prosody such as emotional expression from significantly less supervised data than previously existing state of the art methods