A New Framework for Decomposing Multivariate Information

What are the distinct ways in which a set of predictor variables can provide information about a target variable? When does a variable provide unique information, when do variables share redundant information, and when do variables combine synergistically to provide complementary information? The redundancy lattice from the partial information decomposition of Williams and Beer provided a promising glimpse at the answer to these questions. However, this structure was constructed using a much-criticised measure of redundant information, and despite sustained research, no completely satisfactory replacement measure has been proposed. This thesis presents a new framework for information decomposition that is based upon the decomposition of pointwise mutual information rather than mutual information. The framework is derived in two separate ways. The first of these derivations is based upon a modified version of the original axiomatic approach taken by Williams and Beer. However, to overcome the difficulty associated with signed pointwise mutual information, the decomposition is applied separately to the unsigned entropic components of pointwise mutual information which are referred to as the specificity and ambiguity. This yields a separate redundancy lattice for each component. Based upon an operational interpretation of redundancy, measures of redundant specificity and redundant ambiguity are defined which enables one to evaluate the partial information atoms separately for each lattice. These separate atoms can then be recombined to yield the sought-after multivariate information decomposition. This framework is applied to canonical examples from the literature and the results and various properties of the decomposition are discussed. In particular, the pointwise decomposition using specificity and ambiguity is shown to satisfy a chain rule over target variables, which provides new insights into the so-called two-bit-copy example. The second approach begins by considering the distinct ways in which two marginal observers can share their information with the non-observing individual third party. Several novel measures of information content are introduced, namely the union, intersection and unique information contents. Next, the algebraic structure of these new measures of shared marginal information is explored, and it is shown that the structure of shared marginal information is that of a distributive lattice. Furthermore, by using the fundamental theorem of distributive lattices, it is shown that these new measures are isomorphic to a ring of sets. Finally, by combining this structure together with the semi-lattice of joint information, the redundancy lattice form partial information decomposition is found to be embedded within this larger algebraic structure. However, since this structure considers information contents, it is actually equivalent to the specificity lattice from the first derivation of pointwise partial information decomposition. The thesis then closes with a discussion about whether or not one should combine the information contents from the specificity and ambiguity lattices

Pointwise Partial Information Decomposition using the Specificity and Ambiguity Lattices

What are the distinct ways in which a set of predictor variables can provide information about a target variable? When does a variable provide unique information, when do variables share redundant information, and when do variables combine synergistically to provide complementary information? The redundancy lattice from the partial information decomposition of Williams and Beer provided a promising glimpse at the answer to these questions. However, this structure was constructed using a much criticised measure of redundant information, and despite sustained research, no completely satisfactory replacement measure has been proposed. In this paper, we take a different approach, applying the axiomatic derivation of the redundancy lattice to a single realisation from a set of discrete variables. To overcome the difficulty associated with signed pointwise mutual information, we apply this decomposition separately to the unsigned entropic components of pointwise mutual information which we refer to as the specificity and ambiguity. This yields a separate redundancy lattice for each component. Then based upon an operational interpretation of redundancy, we define measures of redundant specificity and ambiguity enabling us to evaluate the partial information atoms in each lattice. These atoms can be recombined to yield the sought-after multivariate information decomposition. We apply this framework to canonical examples from the literature and discuss the results and the various properties of the decomposition. In particular, the pointwise decomposition using specificity and ambiguity satisfies a chain rule over target variables, which provides new insights into the so-called two-bit-copy example.Comment: 31 pages, 10 figures. (v1: preprint; v2: as accepted; v3: title corrected

Generalised Measures of Multivariate Information Content

The entropy of a pair of random variables is commonly depicted using a Venn diagram. This representation is potentially misleading, however, since the multivariate mutual information can be negative. This paper presents new measures of multivariate information content that can be accurately depicted using Venn diagrams for any number of random variables. These measures complement the existing measures of multivariate mutual information and are constructed by considering the algebraic structure of information sharing. It is shown that the distinct ways in which a set of marginal observers can share their information with a non-observing third party corresponds to the elements of a free distributive lattice. The redundancy lattice from partial information decomposition is then subsequently and independently derived by combining the algebraic structures of joint and shared information content.Comment: 31 pages, 11 figure

Probability Mass Exclusions and the Directed Components of Pointwise Mutual Information

This paper examines how an event from one random variable provides pointwise mutual information about an event from another variable via probability mass exclusions. We start by introducing probability mass diagrams, which provide a visual representation of how a prior distribution is transformed to a posterior distribution through exclusions. With the aid of these diagrams, we identify two distinct types of probability mass exclusions---namely informative and misinformative exclusions. Then, motivated by Fano's derivation of the pointwise mutual information, we propose four postulates which aim to decompose the pointwise mutual information into two separate informational components: a non-negative term associated with the informative exclusion and a non-positive term associated with the misinformative exclusions. This yields a novel derivation of a familiar decomposition of the pointwise mutual information into entropic components. We conclude by discussing the relevance of considering information in terms of probability mass exclusions to the ongoing effort to decompose multivariate information.Comment: 6 pages, 7 figure

SherLOCKED: A Detective-Themed Serious Game for Cyber Security Education

Gamification and Serious Games are progressively being used over a host of fields, particularly to support education. Such games provide a new way to engage students with content and can complement more traditional approaches to learning. This article proposes SherLOCKED, a new serious game created in the style of a 2D top-down puzzle adventure. The game is situated in the context of an undergraduate cyber security course, and is used to consolidate students' knowledge of foundational security concepts (e.g. the CIA triad, security threats and attacks and risk management). SherLOCKED was built based on a review of existing serious games and a study of common gamification principles. It was subsequently implemented within an undergraduate course, and evaluated with 112 students. We found the game to be an effective, attractive and fun solution for allowing further engagement with content that students were introduced to during lectures. This research lends additional evidence to the use of serious games in supporting learning about cyber security

The Vibrio parahaemolyticus Type III Secretion Systems manipulate host cell MAPK for critical steps in pathogenesis

<p>Abstract</p> <p>Background</p> <p><it>Vibrio parahaemolyticus </it>is a food-borne pathogen causing inflammation of the gastrointestinal epithelium. Pathogenic strains of this bacterium possess two Type III Secretion Systems (TTSS) that deliver effector proteins into host cells. In order to better understand human host cell responses to <it>V. parahaemolyticus</it>, the modulation of Mitogen Activated Protein Kinase (MAPK) activation in epithelial cells by an O3:K6 clinical isolate, RIMD2210633, was investigated. The importance of MAPK activation for the ability of the bacterium to be cytotoxic and to induce secretion of Interleukin-8 (IL-8) was determined.</p> <p>Results</p> <p><it>V. parahaemolyticus </it>deployed its TTSS1 to induce activation of the JNK, p38 and ERK MAPK in human epithelial cells. VP1680 was identified as the TTSS1 effector protein responsible for MAPK activation in Caco-2 cells and the activation of JNK and ERK by this protein was important in induction of host cell death. <it>V. parahaemolyticus </it>actively induced IL-8 secretion in a response mediated by TTSS1. A role for VP1680 and for the ERK signalling pathway in the stimulation of IL-8 production in epithelial cells by <it>V. parahaemolyticus </it>was established. Interestingly, TTSS2 inhibited IL-8 mRNA transcription at early stages of interaction between the bacterium and the cell.</p> <p>Conclusions</p> <p>This study demonstrated that <it>V. parahaemolyticus </it>activates the three major MAPK signalling pathways in intestinal epithelial cells in a TTSS1-dependent manner that involves the TTSS1 effector VP1680. Furthermore VP1680 and JNK and ERK activation were needed for maximal cytotoxicity of the bacterium. It was shown that <it>V. parahaemolyticus </it>is a strong inducer of IL-8 secretion and that induction reflects a balance between the effects of TTSS1 and TTSS2. Increases in IL-8 secretion were mediated by TTSS1 and VP1680, and augmented by ERK activation. These results shed light on the mechanisms of bacterial pathogenesis mediated by TTSS and suggest significant roles for MAPK signalling during infection with <it>V. parahaemolyticus</it>.</p

An interview based study of pioneering experiences in teaching and learning Complex Systems in Higher Education

Due to the interdisciplinary nature of complex systems as a field, students studying complex systems at University level have diverse disciplinary backgrounds. This brings challenges (e.g. wide range of computer programming skills) but also opportunities (e.g. facilitating interdisciplinary interactions and projects) for the classroom. However, there is little published regarding how these challenges and opportunities are handled in teaching and learning Complex Systems as an explicit subject in higher education, and how this differs in comparison to other subject areas. We seek to explore these particular challenges and opportunities via an interview-based study of pioneering teachers and learners (conducted amongst the authors) regarding their experiences. We compare and contrast those experiences, and analyse them with respect to the educational literature. Our discussions explored: approaches to curriculum design, how theories/models/frameworks of teaching and learning informed decisions and experience, how diversity in student backgrounds was addressed, and assessment task design. We found a striking level of commonality in the issues expressed as well as the strategies to handle them, for example a significant focus on problem-based learning, and the use of major student-led creative projects for both achieving and assessing learning outcomes.Comment: 16 page

Quantifying synergy and redundancy in multiplex networks

Understanding how different networks relate to each other is key for obtaining a greater insight into complex systems. Here, we introduce an intuitive yet powerful framework to characterise the relationship between two networks, comprising the same nodes. We showcase our framework by decomposing the shortest paths between nodes as being contributed uniquely by one or the other source network, or redundantly by either, or synergistically by the two together. Our approach takes into account the networks' full topology, but it also provides insights at multiple levels of resolution: from global statistics, to individual paths of different length. We show that this approach is widely applicable, from brains to the London transport system. In humans and across $123$ other species, we demonstrate that reliance on unique contributions by long-range white matter fibers is a conserved feature of mammalian structural connectomes. Across species, we also find that efficient communication relies on significantly greater synergy between long-range and short-range fibers than expected by chance, and significantly less redundancy. Our framework may find applications to help decide how to trade-off different desiderata when designing network systems, or to evaluate their relative presence in existing systems, whether biological or artificial

Modulation of Lysenin’s Memory by Cu\u3csup\u3e2+\u3c/sup\u3e Ions

Lysenin is a pore-forming protein extracted from the red earthworm E. fetida, which forms voltage-gated channels in artificial and natural lipid membranes. A prominent feature of the channels is their memory, originating in the conductance hysteresis that occurs during the application of slow oscillatory voltages. In this work, we showed this innate memory was strongly influenced by the addition of small amounts of Cu2+ ions. After Cu2+ addition, the lysenin channels previously closed by an applied voltage showed a stronger preference for the closed state, indicative of major changes in kinetics and equilibrium. However, the physiology behind this shift is still obscure. To fill this gap in our knowledge, we employed electrophysiology measurements to identify the changes in the closing and opening rates of lysenin channels exposed to Cu2+ ions and step voltages. We found Cu2+ simultaneously reduced the closing rates and increased the reopening rates, leading to a more prominent hysteretic behavior and improved memory. These findings may constitute the starting point on investigations of the memory of brainless microorganisms, and potential applications to bioelectronics and development of smart biological switches and nano-valves

IDTxl: The Information Dynamics Toolkit xl: a Python package for the efficient analysis of multivariate information dynamics in networks

Producción CientíficaWe present IDTxl (the Information Dynamics Toolkit xl), a new open source Python toolbox for effective network inference from multivariate time series using information theory, available from GitHub (https://github.com/pwollstadt/IDTxl). Information theory (Cover & Thomas, 2006; MacKay, 2003; Shannon, 1948) is the math- ematical theory of information and its transmission over communication channels. In- formation theory provides quantitative measures of the information content of a single random variable (entropy) and of the information shared between two variables (mutual information). The defined measures build on probability theory and solely depend on the probability distributions of the variables involved. As a consequence, the dependence between two variables can be quantified as the information shared between them, without the need to explicitly model a specific type of dependence. Hence, mutual information is a model-free measure of dependence, which makes it a popular choice for the analysis of systems other than communication channels