Improving Compositional Generalization Using Iterated Learning and Simplicial Embeddings

Compositional generalization, the ability of an agent to generalize to unseen combinations of latent factors, is easy for humans but hard for deep neural networks. A line of research in cognitive science has hypothesized a process, ``iterated learning,'' to help explain how human language developed this ability; the theory rests on simultaneous pressures towards compressibility (when an ignorant agent learns from an informed one) and expressivity (when it uses the representation for downstream tasks). Inspired by this process, we propose to improve the compositional generalization of deep networks by using iterated learning on models with simplicial embeddings, which can approximately discretize representations. This approach is further motivated by an analysis of compositionality based on Kolmogorov complexity. We show that this combination of changes improves compositional generalization over other approaches, demonstrating these improvements both on vision tasks with well-understood latent factors and on real molecular graph prediction tasks where the latent structure is unknown

Simplicial Embeddings in Self-Supervised Learning and Downstream Classification

We introduce Simplicial Embeddings (SEMs) as a way to constrain the encoded representations of a self-supervised model to $L$ simplices of $V$ dimensions each using a Softmax operation. This procedure imposes a structure on the representations that reduce their expressivity for training downstream classifiers, which helps them generalize better. Specifically, we show that the temperature $\tau$ of the Softmax operation controls for the SEM representation's expressivity, allowing us to derive a tighter downstream classifier generalization bound than that for classifiers using unnormalized representations. We empirically demonstrate that SEMs considerably improve generalization on natural image datasets such as CIFAR-100 and ImageNet. Finally, we also present evidence of the emergence of semantically relevant features in SEMs, a pattern that is absent from baseline self-supervised models.Comment: 22 pages, 5 figures, 5 tables, Preprin

Bigger, Better, Faster: Human-level Atari with human-level efficiency

We introduce a value-based RL agent, which we call BBF, that achieves super-human performance in the Atari 100K benchmark. BBF relies on scaling the neural networks used for value estimation, as well as a number of other design choices that enable this scaling in a sample-efficient manner. We conduct extensive analyses of these design choices and provide insights for future work. We end with a discussion about updating the goalposts for sample-efficient RL research on the ALE. We make our code and data publicly available at https://github.com/google-research/google-research/tree/master/bigger_better_faster.Comment: ICML 2023 Camera Read

Learning and Controlling Silicon Dopant Transitions in Graphene using Scanning Transmission Electron Microscopy

We introduce a machine learning approach to determine the transition dynamics of silicon atoms on a single layer of carbon atoms, when stimulated by the electron beam of a scanning transmission electron microscope (STEM). Our method is data-centric, leveraging data collected on a STEM. The data samples are processed and filtered to produce symbolic representations, which we use to train a neural network to predict transition probabilities. These learned transition dynamics are then leveraged to guide a single silicon atom throughout the lattice to pre-determined target destinations. We present empirical analyses that demonstrate the efficacy and generality of our approach

Bridge to the stars: A mission concept to an interstellar object

Exoplanet discoveries since the mid-1990’s have revealed an astounding diversity of planetary systems. Studying these systems is essential to understanding planetary formation processes, as well as the development of life in the universe. Unfortunately, humanity can only observe limited aspects of exoplanetary systems by telescope, and the significant distances between stars presents a barrier to in situ exploration. In this study, we propose an alternative path to gain insight into exoplanetary systems: Bridge, a mission concept design to fly by an interstellar object as it passes through our solar system. Designed as a New Frontiers-class mission during the National Aeronautics and Space Administration (NASA) Planetary Science Summer School, Bridge would provide a unique opportunity to gain insight into potential physical, chemical, and biological differences between solar systems as well as the possible exchange of planetary materials between them. Bridge employs ultraviolet/visible, near-infrared, and mid-infrared point spectrometers, a visible camera, and a guided impactor. We also provide a quantitative Monte Carlo analysis that estimates wait times for a suitable target, and examines key trades between ground storage and a parking orbit, power sources, inner versus outer solar system encounters, and launch criteria. Due to the fleeting nature of interstellar objects, reaching an interstellar object may require an extended ground storage phase for the spacecraft until a suitable ISO is discovered, followed by a rapid response launch strategy. To enable rapid response missions designed to intercept such unique targets, language would need to be added to future NASA announcements of opportunity such that ground storage and rapid response would be allowable components of a proposed mission

Courville_etal_2022_planetesimal_thermal_models

Planetesimal thermal evolution models used in Courville et al. 2022, Acquisition and Preservation of Remanent Magnetization in Carbonaceous Asteroids

Kenyan nodular lymphocyte predominant Hodgkin lymphoma shares features with western cases and is not associated with EBV

Background: Pulmonary Langerhans cell histiocytosis (PLCH) is a distinctive form of tobacco associated interstitial lung disease. Therapeutic options other than lung transplantation are scarce when a fibrotic phase has been reached. One severe complication is the emergence of pulmonary hypertension that has been related to chronic hypoxia. A few papers describe its morphological characteristics focusing on arterial lesions. Otherwise, BRAF mutations have been recently described in systemic Langerhans cell histiocytosis and only one paper with five cases has been published on PLCH. Methods: We have revised morphological changes associated with pulmonary vasculature and the presence of Langerhans cells by routine and immunohistochemical methods (S100 protein, CD1a and Langerin) in 22 PLCH cases. We have used real time PCR to detect BRAF mutations in Langerhans cells using manual macrodisection in paraffin embedded tissue. Results: Six out of 22 patients (27%) showed some grade of capillary haemangiomatosis-like changes. Aggregates of Langerhans cells were located in interlobular septa in those cases. Two BRAF mutated cases were found, both of them in cellular/mixed lesions. Conclusion: We present six patients with capillary haemangiomatosis-like changes with infrequent located Langerhans cells in pulmonary Langerhans cell histiocytosis. We confirm the occurrence of BRAF mutations in this kind of disease

Beyond Tabula Rasa: Reincarnating Reinforcement Learning

Learning tabula rasa, that is without any prior knowledge, is the prevalent workflow in reinforcement learning (RL) research. However, RL systems, when applied to large-scale settings, rarely operate tabula rasa. Such large-scale systems undergo multiple design or algorithmic changes during their development cycle and use ad hoc approaches for incorporating these changes without re-training from scratch, which would have been prohibitively expensive. Additionally, the inefficiency of deep RL typically excludes researchers without access to industrial-scale resources from tackling computationally-demanding problems. To address these issues, we present reincarnating RL as an alternative workflow, where prior computational work (e.g., learned policies) is reused or transferred between design iterations of an RL agent, or from one RL agent to another. As a step towards enabling reincarnating RL from any agent to any other agent, we focus on the specific setting of efficiently transferring an existing sub-optimal policy to a standalone value-based RL agent. We find that existing approaches fail in this setting and propose a simple algorithm to address their limitations. Equipped with this algorithm, we demonstrate reincarnating RL's gains over tabula rasa RL on Atari 2600 games, a challenging locomotion task, and the real-world problem of navigating stratospheric balloons. Overall, this work argues for an alternative approach to RL research, which we believe could significantly improve real-world RL adoption and help democratize it further