Outcome-Guided Counterfactuals for Reinforcement Learning Agents from a Jointly Trained Generative Latent Space

We present a novel generative method for producing unseen and plausible counterfactual examples for reinforcement learning (RL) agents based upon outcome variables that characterize agent behavior. Our approach uses a variational autoencoder to train a latent space that jointly encodes information about the observations and outcome variables pertaining to an agent's behavior. Counterfactuals are generated using traversals in this latent space, via gradient-driven updates as well as latent interpolations against cases drawn from a pool of examples. These include updates to raise the likelihood of generated examples, which improves the plausibility of generated counterfactuals. From experiments in three RL environments, we show that these methods produce counterfactuals that are more plausible and proximal to their queries compared to purely outcome-driven or case-based baselines. Finally, we show that a latent jointly trained to reconstruct both the input observations and behavioral outcome variables produces higher-quality counterfactuals over latents trained solely to reconstruct the observation inputs

A dual-time-window protocol to reduce acquisition time of dynamic tau PET imaging using [F-18]MK-6240

Background [F-18]MK-6240 is a PET tracer with sub-nanomolar affinity for neurofibrillary tangles. Therefore, tau quantification is possible with [F-18]MK-6240 PET/CT scans, and it can be used for assessment of Alzheimer's disease. However, long acquisition scans are required to provide fully quantitative estimates of pharmacokinetic parameters. Therefore, on the present study, dual-time-window (DTW) acquisitions was simulated to reduce PET/CT acquisition time, while taking into consideration perfusion changes and possible scanning protocol non-compliance. To that end, time activity curves (TACs) representing a 120-min acquisition (TAC(120)) were simulated using a two-tissue compartment model with metabolite corrected arterial input function from 90-min dynamic [F-18]MK-6240 PET scans of three healthy control subjects and five subjects with mild cognitive impairment or Alzheimer's disease. Therefore, TACs corresponding to different levels of specific binding were generated and then various perfusion changes were simulated. Next, DTW acquisitions were simulated consisting of an acquisition starting at tracer injection, a break and a second acquisition starting at 90 min post-injection. Finally, non-compliance with the PET/CT scanning protocol were simulated to assess its impact on quantification. All TACs were quantified using reference Logan's distribution volume ratio (DVR) and standardized uptake value ratio (SUVR90) using the cerebellar cortex as reference region. Results It was found that DVR from a DTW protocol with a 60-min break between two 30-min dynamic scans closely approximates the DVR from the uninterrupted TAC(120), with a regional bias smaller than 2.5%. Moreover, SUVR90 estimates were more susceptible (regional bias</p

Large fluctuations of dissolved oxygen in the Indian and Pacific oceans during Dansgaard-Oeschger oscillations caused by variations of North Atlantic Deep Water Subduction: Paleoceanography

Paleoclimate records from glacial Indian and Pacific oceans sediments document millennial-scale fluctuations of subsurface dissolved oxygen levels and denitrification coherent with North Atlantic temperature oscillations. Yet the mechanism of this teleconnection between the remote ocean basins remains elusive. Here we present model simulations of the oxygen and nitrogen cycles that explain how changes in deepwater subduction in the North Atlantic can cause large and synchronous variations of oxygen minimum zones throughout the Northern Hemisphere of the Indian and Pacific oceans, consistent with the paleoclimate records. Cold periods in the North Atlantic are associated with reduced nutrient delivery to the upper Indo-Pacific oceans, thereby decreasing productivity. Reduced export production diminishes subsurface respiration of organic matter leading to higher oxygen concentrations and less denitrification. This effect of reduced oxygen consumption dominates at low latitudes. At high latitudes in the Southern Ocean and North Pacific, increased mixed layer depths and steepening of isopycnals improve ocean ventilation and oxygen supply to the subsurface. Atmospheric teleconnections through changes in wind-driven ocean circulation modify this basin-scale pattern regionally. These results suggest that changes in the Atlantic Ocean circulation, similar to those projected by climate models to possibly occur in the centuries to come because of anthropogenic climate warming, can have large effects on marine ecosystems and biogeochemical cycles even in remote areas.Keywords: ocean circulation, oxygen, Dansgaard-Oeschger oscillation

Large fluctuations of dissolved oxygen in the Indian and Pacific oceans during Dansgaard-Oeschger oscillations caused by variations of North Atlantic Deep Water subduction

Paleoclimate records from glacial Indian and Pacific oceans sediments document millennial-scale fluctuations of subsurface dissolved oxygen levels and denitrification coherent with North Atlantic temperature oscillations. Yet the mechanism of this teleconnection between the remote ocean basins remains elusive. Here we present model simulations of the oxygen and nitrogen cycles that explain how changes in deepwater subduction in the North Atlantic can cause large and synchronous variations of oxygen minimum zones throughout the Northern Hemisphere of the Indian and Pacific oceans, consistent with the paleoclimate records. Cold periods in the North Atlantic are associated with reduced nutrient delivery to the upper Indo-Pacific oceans, thereby decreasing productivity. Reduced export production diminishes subsurface respiration of organic matter leading to higher oxygen concentrations and less denitrification. This effect of reduced oxygen consumption dominates at low latitudes. At high latitudes in the Southern Ocean and North Pacific, increased mixed layer depths and steepening of isopycnals improve ocean ventilation and oxygen supply to the subsurface. Atmospheric teleconnections through changes in wind-driven ocean circulation modify this basin-scale pattern regionally. These results suggest that changes in the Atlantic Ocean circulation, similar to those projected by climate models to possibly occur in the centuries to come because of anthropogenic climate warming, can have large effects on marine ecosystems and biogeochemical cycles even in remote areas

PET/CT Imaging of Zr-89-N-sucDf-Pembrolizumab in Healthy Cynomolgus Monkeys

PURPOSE: Programmed cell death-1 receptor (PD-1) and its ligand (PD-L1) are the targets for immunotherapy in many cancer types. Although PD-1 blockade has therapeutic effects, the efficacy differs between patients. Factors contributing to this variability are PD-L1 expression levels and immune cells present in tumors. However, it is not well understood how PD-1 expression in the tumor microenvironment impacts immunotherapy response. Thus, imaging of PD-1-expressing immune cells is of interest. This study aims to evaluate the biodistribution of Zirconium-89 (89Zr)-labeled pembrolizumab, a humanized IgG4 kappa monoclonal antibody targeting PD-1, in healthy cynomolgus monkeys as a translational model of tracking PD-1-positive immune cells. PROCEDURES: Pembrolizumab was conjugated with the tetrafluorophenol-N-succinyl desferal-Fe(III) ester (TFP-N-sucDf) and subsequently radiolabeled with 89Zr. Four cynomolgus monkeys with no previous exposure to humanized monoclonal antibodies received tracer only or tracer co-injected with pembrolizumab intravenously over 5 min. Thereafter, a static whole-body positron emission tomography (PET) scan was acquired with 10 min per bed position on days 0, 2, 5, and 7. Image-derived standardized uptake values (SUVmean) were quantified by region of interest (ROI) analysis. RESULTS: 89Zr-N-sucDf-pembrolizumab was synthesized with high radiochemical purity (> 99 %) and acceptable molar activity (> 7 MBq/nmol). In animals dosed with tracer only, 89Zr-N-sucDf-pembrolizumab distribution in lymphoid tissues such as mesenteric lymph nodes, spleen, and tonsils increased over time. Except for the liver, low radiotracer distribution was observed in all non-lymphoid tissue including the lung, muscle, brain, heart, and kidney. When a large excess of pembrolizumab was co-administered with a radiotracer, accumulation in the lymph nodes, spleen, and tonsils was reduced, suggestive of target-mediated accumulation. CONCLUSIONS: 89Zr-N-sucDf-pembrolizumab shows preferential uptake in the lymphoid tissues including the lymph nodes, spleen, and tonsils. 89Zr-N-sucDf-pembrolizumab may be useful in tracking the distribution of a subset of immune cells in non-human primates and humans. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT02760225

Synthesis of colloidal PbSe nanoparticles using a microwave-assisted segmented flow reactor

Colloidal lead selenide nanoparticles (PbSe NPs) were synthesized using a microwave-assisted continuous flow reactor. Rapid heating of precursors was performed in the microwave reaction zone to initiate nucleation and was followed by an oil bath growth zone. In this study we have evaluated the effect of the nucleation temperature on the PbSe NP size distributions, crystallographic structure, particle shape, and particle composition. The PbSe NP size could be varied from 11.2 to 13.9 nm by adjusting the microwave nucleation temperature between 124 and 159 °C. It was found that nucleation of Pb rich species occurred in the microwave reaction zone, while PbSe NPs form in the growth zone.This is an author's peer-reviewed final manuscript, as accepted by the publisher. The published article is copyrighted by Elsevier and can be found at: http://www.sciencedirect.com/science/journal/0167577XKeywords: PbSe, Microwave, Flow synthesis, Nanoparticles, Colloidal processin

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.Comment: To appear in Neural Network

The Complete Genome of Teredinibacter turnerae T7901: An Intracellular Endosymbiont of Marine Wood-Boring Bivalves (Shipworms)

Here we report the complete genome sequence of Teredinibacter turnerae T7901. T. turnerae is a marine gamma proteobacterium that occurs as an intracellular endosymbiont in the gills of wood-boring marine bivalves of the family Teredinidae (shipworms). This species is the sole cultivated member of an endosymbiotic consortium thought to provide the host with enzymes, including cellulases and nitrogenase, critical for digestion of wood and supplementation of the host's nitrogen-deficient diet. T. turnerae is closely related to the free-living marine polysaccharide degrading bacterium Saccharophagus degradans str. 2–40 and to as yet uncultivated endosymbionts with which it coexists in shipworm cells. Like S. degradans, the T. turnerae genome encodes a large number of enzymes predicted to be involved in complex polysaccharide degradation (>100). However, unlike S. degradans, which degrades a broad spectrum (>10 classes) of complex plant, fungal and algal polysaccharides, T. turnerae primarily encodes enzymes associated with deconstruction of terrestrial woody plant material. Also unlike S. degradans and many other eubacteria, T. turnerae dedicates a large proportion of its genome to genes predicted to function in secondary metabolism. Despite its intracellular niche, the T. turnerae genome lacks many features associated with obligate intracellular existence (e.g. reduced genome size, reduced %G+C, loss of genes of core metabolism) and displays evidence of adaptations common to free-living bacteria (e.g. defense against bacteriophage infection). These results suggest that T. turnerae is likely a facultative intracellular ensosymbiont whose niche presently includes, or recently included, free-living existence. As such, the T. turnerae genome provides insights into the range of genomic adaptations associated with intracellular endosymbiosis as well as enzymatic mechanisms relevant to the recycling of plant materials in marine environments and the production of cellulose-derived biofuels

The Complete Genome of \u3cem\u3eTeredinibacter turnerae\u3c/em\u3e T7901: An Intracellular Endosymbiont of Marine Wood-Boring Bivalves (Shipworms)

Here we report the complete genome sequence of Teredinibacter turnerae T7901. T. turnerae is a marine gamma proteobacterium that occurs as an intracellular endosymbiont in the gills of wood-boring marine bivalves of the family Teredinidae (shipworms). This species is the sole cultivated member of an endosymbiotic consortium thought to provide the host with enzymes, including cellulases and nitrogenase, critical for digestion of wood and supplementation of the host\u27s nitrogen-deficient diet. T. turnerae is closely related to the free-living marine polysaccharide degrading bacterium Saccharophagus degradans str. 2–40 and to as yet uncultivated endosymbionts with which it coexists in shipworm cells. Like S. degradans, the T. turnerae genome encodes a large number of enzymes predicted to be involved in complex polysaccharide degradation (\u3e100). However, unlike S. degradans, which degrades a broad spectrum (\u3e10 classes) of complex plant, fungal and algal polysaccharides, T. turnerae primarily encodes enzymes associated with deconstruction of terrestrial woody plant material. Also unlike S. degradans and many other eubacteria, T. turnerae dedicates a large proportion of its genome to genes predicted to function in secondary metabolism. Despite its intracellular niche, the T. turnerae genome lacks many features associated with obligate intracellular existence (e.g. reduced genome size, reduced %G+C, loss of genes of core metabolism) and displays evidence of adaptations common to free-living bacteria (e.g. defense against bacteriophage infection). These results suggest that T. turnerae is likely a facultative intracellular ensosymbiont whose niche presently includes, or recently included, free-living existence. As such, the T. turnerae genome provides insights into the range of genomic adaptations associated with intracellular endosymbiosis as well as enzymatic mechanisms relevant to the recycling of plant materials in marine environments and the production of cellulose-derived biofuels