Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RL

Reinforcement learning agents may sometimes develop habits that are effective only when specific policies are followed. After an initial exploration phase in which agents try out different actions, they eventually converge toward a particular policy. When this occurs, the distribution of state-action trajectories becomes narrower, and agents start experiencing the same transitions again and again. At this point, spurious correlations may arise. Agents may then pick up on these correlations and learn state representations that do not generalize beyond the agent's trajectory distribution. In this paper, we provide a mathematical characterization of this phenomenon, which we refer to as policy confounding, and show, through a series of examples, when and how it occurs in practice

DUET: 2D Structured and Approximately Equivariant Representations

Multiview Self-Supervised Learning (MSSL) is based on learning invariances with respect to a set of input transformations. However, invariance partially or totally removes transformation-related information from the representations, which might harm performance for specific downstream tasks that require such information. We propose 2D strUctured and EquivarianT representations (coined DUET), which are 2d representations organized in a matrix structure, and equivariant with respect to transformations acting on the input data. DUET representations maintain information about an input transformation, while remaining semantically expressive. Compared to SimCLR (Chen et al., 2020) (unstructured and invariant) and ESSL (Dangovski et al., 2022) (unstructured and equivariant), the structured and equivariant nature of DUET representations enables controlled generation with lower reconstruction error, while controllability is not possible with SimCLR or ESSL. DUET also achieves higher accuracy for several discriminative tasks, and improves transfer learning.Comment: Accepted at ICML 202

Enhanced functionality for donor-acceptor oligothiophenes via Inclusion of BODIPY : synthesis, electrochemistry, photophysics and model chemistry

We have synthesized several new push-pull oligothiophenes based on the BODIPY moiety as the electron acceptor and the well known oligothiophenes substituted with N,N-dialkylamino groups to enhance the electron donor ability. A complete characterization of the electronic properties has been done consisting of their photophysical, electrochemical and vibrational properties. The compounds have been studied after chemical treatment with acids and after oxidation. In this regard, they can be termed as NIR dyes and amphoteric redox electroactive molecules. We have described the presence of dual fluorescence in these molecules and fluorescence quenching either by energy transfer or, in the push-pull molecules, by the electron exchange. The combination of electrochemical and proton reversibility combined with the interesting optical properties of the new species offer an interesting platform for sensor and material applications.DGES, MEC (Spain) and Fundação para a Ciência e a Tecnologia (FCT

Air transport liberalisation and airport dependency: developing a composite index

Air transport liberalisation in Europe has produced some major changes to the networks operated by airlines and the services available at airports. Within this context the degree of airport dependency in terms of market, spatial and temporal concentration is important to know from an economic geography and risk management perspective. A composite index called the Airport Dependency Index (ADI) is developed to measure airport dependency based on the concept of the relative Gini coefficient. Liberalisation has had varying impacts depending on the size and type of airport and so a comparison is made of the degree of dependency at a large sample of European airports using the ADI. The ADI has the potential to provide insight on the sustainability and worthiness of financing airport projects, and on whether airports should diversify further their activities by investing in the growth and expansion of their network

Impact of Intestinal Microbiota on Intestinal Luminal Metabolome

Low–molecular-weight metabolites produced by intestinal microbiota play a direct role in health and disease. In this study, we analyzed the colonic luminal metabolome using capillary electrophoresis mass spectrometry with time-of-flight (CE-TOFMS) —a novel technique for analyzing and differentially displaying metabolic profiles— in order to clarify the metabolite profiles in the intestinal lumen. CE-TOFMS identified 179 metabolites from the colonic luminal metabolome and 48 metabolites were present in significantly higher concentrations and/or incidence in the germ-free (GF) mice than in the Ex-GF mice (p < 0.05), 77 metabolites were present in significantly lower concentrations and/or incidence in the GF mice than in the Ex-GF mice (p < 0.05), and 56 metabolites showed no differences in the concentration or incidence between GF and Ex-GF mice. These indicate that intestinal microbiota highly influenced the colonic luminal metabolome and a comprehensive understanding of intestinal luminal metabolome is critical for clarifying host-intestinal bacterial interactions

Evolutionary, ecological and biotechnological perspectives on plasmids resident in the human gut mobile metagenome

Numerous mobile genetic elements (MGE) are associated with the human gut microbiota and collectively referred to as the gut mobile metagenome. The role of this flexible gene pool in development and functioning of the gut microbial community remains largely unexplored, yet recent evidence suggests that at least some MGE comprising this fraction of the gut microbiome reflect the co-evolution of host and microbe in the gastro-intestinal tract. In conjunction, the high level of novel gene content typical of MGE coupled with their predicted high diversity, suggests that the mobile metagenome constitutes an immense and largely unexplored gene-space likely to encode many novel activities with potential biotechnological or pharmaceutical value, as well as being important to the development and functioning of the gut microbiota. Of the various types of MGE that comprise the gut mobile metagenome, plasmids are of particular importance since these elements are often capable of autonomous transfer between disparate bacterial species, and are known to encode accessory functions that increase bacterial fitness in a given environment facilitating bacterial adaptation. In this article current knowledge regarding plasmids resident in the human gut mobile metagenome is reviewed, and available strategies to access and characterize this portion of the gut microbiome are described. The relative merits of these methods and their present as well as prospective impact on our understanding of the human gut microbiota is discussed

Impact of digestive and oropharyngeal decontamination on the intestinal microbiota in ICU patients

Selective digestive microbial decontamination (SDD) is hypothesized to benefit patients in intensive care (ICU) by suppressing Gram-negative potential pathogens from the colon without affecting the anaerobic intestinal microbiota. The purpose of this study was to provide more insight to the effects of digestive tract and oropharyngeal decontamination on the intestinal microbiota by means of a prospective clinical trial in which faecal samples were collected from ICU patients for intestinal microbiota analysis. The faecal samples were collected from ICU patients enrolled in a multicentre trial to study the outcome of SDD and selective oral decontamination (SOD) in comparison with standard care (SC). Fluorescent in situ hybridization (FISH) was used to analyze the faecal microbiota. The numbers of bacteria from different bacterial groups were compared between the three regimens. The total counts of bacteria per gram faeces did not differ between regimens. The F. prausnitzii group of bacteria, representing an important group among intestinal microbiota, was significantly reduced in the SDD regimen compared to the SC and SOD. The Enterobacteriaceae were significantly suppressed during SDD compared to both SOD and SC; enterococci increased in SDD compared to both other regimens. The composition of the intestinal microbiota is importantly affected by SDD. The F. prausnitzii group was significantly suppressed during SDD. This group of microbiota is a predominant producer of butyrate, the main energy source for colonocytes. Reduction of this microbiota is an important trade-off while reducing gram-negative bacteria by SDD