Outcome-Driven Reinforcement Learning via Variational Inference

While reinforcement learning algorithms provide automated acquisition of optimal policies, practical application of such methods requires a number of design decisions, such as manually designing reward functions that not only define the task, but also provide sufficient shaping to accomplish it. In this paper, we view reinforcement learning as inferring policies that achieve desired outcomes, rather than as a problem of maximizing rewards. To solve this inference problem, we establish a novel variational inference formulation that allows us to derive a well-shaped reward function which can be learned directly from environment interactions. From the corresponding variational objective, we also derive a new probabilistic Bellman backup operator and use it to develop an off-policy algorithm to solve goal-directed tasks. We empirically demonstrate that this method eliminates the need to hand-craft reward functions for a suite of diverse manipulation and locomotion tasks and leads to effective goal-directed behaviors.Comment: Published in Advances in Neural Information Processing Systems 34 (NeurIPS 2021

Unveiling Dust-enshrouded Star Formation in the Early Universe: a Sub-mm Survey of the Hubble Deep Field

The advent of sensitive sub-mm array cameras now allows a proper census of dust-enshrouded massive star-formation in very distant galaxies, previously hidden activity to which even the faintest optical images are insensitive. We present the deepest sub-mm survey of the sky to date, taken with the SCUBA camera on the James Clerk Maxwell Telescope and centred on the Hubble Deep Field. The high source density found in this image implies that the survey is confusion-limited below a flux density of 2 mJy. However, within the central 80 arcsec radius independent analyses yield 5 reproducible sources with S(850um) > 2 mJy which simulations indicate can be ascribed to individual galaxies. We give positions and flux densities for these, and furthermore show using multi-frequency photometric data that the brightest sources in our map lie at redshifts z~3. These results lead to integral source counts which are completely inconsistent with a no-evolution model, and imply that massive star-formation activity continues at redshifts > 2. The combined brightness of the 5 most secure sources in our map is sufficient to account for 30 - 50% of the previously unresolved sub-mm background, and we estimate statistically that the entire background is resolved at about the 0.3 mJy level. Finally we discuss possible optical identifications and redshift estimates for the brightest sources. One source appears to be associated with an extreme starburst galaxy at z~1, whilst the remaining four appear to lie in the redshift range 2 < z < 4. This implies a star-formation density over this redshift range that is at least five times higher than that inferred from the ultraviolet output of HDF galaxies.Comment: 19 pages, 6 figures (to appear as a Nature Article

a trimolecular composition

Photochemical upconversion based on triplet–triplet annihilation (TTA-UC) is employed to enhance the short-circuit currents generated by two varieties of thin-film solar cells, a hydrogenated amorphous silicon (a-Si:H) solar cell and a dye-sensitized solar cell (DSC). TTA-UC is exploited to harvest transmitted sub-bandgap photons, combine their energies and re-radiate upconverted photons back towards the solar cells. In the present study we employ a dual-emitter TTA-UC system which allows for significantly improved UC quantum yields as compared to the previously used single-emitter TTA systems. In doing so we achieve record photo-current enhancement values for both the a-Si:H device and the DSC, surpassing 10−3 mA cm−2 sun−2 for the first time for a TTA-UC system and marking a record for upconversion-enhanced solar cells in general. We discuss pertinent challenges of the TTA-UC technology which need to be addressed in order to achieve its viable device application

Increased upconversion performance for thin film solar cells: A trimolecular composition

Photochemical upconversion based on triplet-triplet annihilation (TTA-UC) is employed to enhance the short-circuit currents generated by two varieties of thin-film solar cells, a hydrogenated amorphous silicon (a-Si:H) solar cell and a dye-sensitized solar cell (DSC). TTA-UC is exploited to harvest transmitted sub-bandgap photons, combine their energies and re-radiate upconverted photons back towards the solar cells. In the present study we employ a dual-emitter TTA-UC system which allows for significantly improved UC quantum yields as compared to the previously used single-emitter TTA systems. In doing so we achieve record photo-current enhancement values for both the a-Si:H device and the DSC, surpassing 10-3 mA cm-2 sun-2 for the first time for a TTA-UC system and marking a record for upconversion-enhanced solar cells in general. We discuss pertinent challenges of the TTA-UC technology which need to be addressed in order to achieve its viable device application

Improving the light-harvesting of amorphous silicon solar cells with photochemical upconversion

Single-threshold solar cells are fundamentally limited by their ability to harvest only those photons above a certain energy. Harvesting below-threshold photons and re-radiating this energy at a shorter wavelength would thus boost the efficiency of such devices. We report an increase in light harvesting efficiency of a hydrogenated amorphous silicon (a-Si:H) thin-film solar cell due to a rear upconvertor based on sensitized triplet–triplet-annihilation in organic molecules. Low energy light in the range 600–750 nm is converted to 550–600 nm light due to the incoherent photochemical process. A peak efficiency enhancement of (1.0 ± 0.2)% at 720 nm is measured under irradiation equivalent to (48 ± 3) suns (AM1.5). We discuss the pathways to be explored in adapting photochemical UC for application in various single threshold devices

IR and UV Galaxies at z=0.6 -- Evolution of Dust Attenuation and Stellar Mass as Revealed by SWIRE and GALEX

We study dust attenuation and stellar mass of $\rm z\sim 0.6$ star-forming galaxies using new SWIRE observations in IR and GALEX observations in UV. Two samples are selected from the SWIRE and GALEX source catalogs in the SWIRE/GALEX field ELAIS-N1-00 ($\Omega = 0.8$ deg$^2$). The UV selected sample has 600 galaxies with photometric redshift (hereafter photo-z) $0.5 \leq z \leq 0.7$ and NUV$\leq 23.5$ (corresponding to \rm L_{FUV} \geq 10^{9.6} L_\sun). The IR selected sample contains 430 galaxies with $f_{24\mu m} \geq 0.2$ mJy (\rm L_{dust} \geq 10^{10.8} L_\sun) in the same photo-z range. It is found that the mean $\rm L_{dust}/L_{FUV}$ ratios of the z=0.6 UV galaxies are consistent with that of their z=0 counterparts of the same $\rm L_{FUV}$. For IR galaxies, the mean $\rm L_{dust}/L_{FUV}$ ratios of the z=0.6 LIRGs (\rm L_{dust} \sim 10^{11} L_\sun) are about a factor of 2 lower than local LIRGs, whereas z=0.6 ULIRGs (\rm L_{dust} \sim 10^{12} L_\sun) have the same mean $\rm L_{dust}/L_{FUV}$ ratios as their local counterparts. This is consistent with the hypothesis that the dominant component of LIRG population has changed from large, gas rich spirals at z$>0.5$ to major-mergers at z=0. The stellar mass of z=0.6 UV galaxies of \rm L_{FUV} \leq 10^{10.2} L_\sun is about a factor 2 less than their local counterparts of the same luminosity, indicating growth of these galaxies. The mass of z=0.6 UV lunmous galaxies (UVLGs: \rm L_{FUV} > 10^{10.2} L_\sun) and IR selected galaxies, which are nearly exclusively LIRGs and ULIRGs, is the same as their local counterparts.Comment: 27 pages, 8 figures, to be published in the Astrophysical Journal Supplement series dedicated to GALEX result

Improving the light-harvesting of second generation solar cells with photochemical upconversion

Photovoltaics (PV) offer a solution for the development of sustainable energy sources, relying on the sheer abundance of sunlight: More sunlight falls on the Earth’s surface in one hour than is required by its inhabitants in a year. However, it is imperative to manage the wide distribution of photon energies available in order to generate more cost efficient PV devices because single threshold PV devices are fundamentally limited to a maximum conversion efficiency, the Shockley-Queisser (SQ) limit. Recent progress has enabled the production of c-Si cells with efficiencies as high as 25%,1 close to the limiting efficiency of ∼30%. But these cells are rather expensive, and ultimately the cost of energy is determined by the ratio of system cost and efficiency of the PV device. A strategy to radically decrease this ratio is to circumvent the SQ limit in cheaper, second generation PV devices. One promising approach is the use of hydrogenated amorphous silicon (a-Si:H), where film thicknesses on the order of several 100nm are sufficient. Unfortunately, the optical threshold of a-Si:H is rather high (1.7-1.8 eV) and the material suffers from light-induced degradation. Thinner absorber layers in a-Si:H devices are generally more stable than thicker films due to the better charge carrier extraction, but at the expense of reduced conversion efficiencies, especially in the red part of the solar spectrum (absorption losses). Hence for higher bandgap materials, which includes a-Si as well as organic and dye-sensitized cells, the major loss mechanism is the inability to harvest low energy photons

Diminazene resistance in Trypanosoma congolense is not caused by reduced transport capacity but associated with reduced mitochondrial membrane potential

Trypanosoma congolense is a principal agent causing livestock trypanosomiasis in Africa, costing developing economies billions of dollars and undermining food security. Only the diamidine diminazene and the phenanthridine isometamidium are regularly used, and resistance is widespread but poorly understood. We induced stable diminazene resistance in T. congolense strain IL3000 in vitro. There was no cross‐resistance with the phenanthridine drugs, melaminophenyl arsenicals, oxaborole trypanocides, or with diamidine trypanocides, except the close analogues DB829 and DB75. Fluorescence microscopy showed that accumulation of DB75 was inhibited by folate. Uptake of [3H]‐diminazene was slow, low affinity and partly but reciprocally inhibited by folate and by competing diamidines. Expression of T. congolense folate transporters in diminazene‐resistant T. b. brucei significantly sensitized the cells to diminazene and DB829, but not to oxaborole AN7973. However, [3H]‐diminazene transport studies, whole genome sequencing and RNA‐seq found no major changes in diminazene uptake, folate transporter sequence or expression. Instead, all resistant clones displayed a moderate reduction in the mitochondrial membrane potential. We conclude that diminazene uptake in T. congolense proceed via multiple low affinity mechanisms including folate transporters; while resistance is associated with a reduction in Ψm it is unclear whether this is the primary cause of the resistance

Effect of a back reflector

Photochemical upconversion is applied to a hydrogenated amorphous silicon solar cell in the presence of a back-scattering layer. A custom-synthesized porphyrin was utilized as the sensitizer species, with rubrene as the emitter. Under a bias of 24 suns, a peak external quantum efficiency (EQE) enhancement of ~2 % was observed at a wavelength of 720 nm. Without the scattering layer, the EQE enhancement was half this value, indicating that the effect of the back-scatterer is to double the efficacy of the upconverting device. The results represent an upconversion figure of merit of 3.5 × 10–4 mA cm–2 sun–2, which is the highest reported to date

The X-Ray Derived Cosmological Star Formation History and the Galaxy X-Ray Luminosity Functions in the Chandra Deep Fields North and South

The cosmological star formation rate in the combined Chandra Deep Fields North and South is derived from our X-Ray Luminosity Function for Galaxies in these Deep Fields. Mild evolution is seen up to redshift order unity with SFR ~ (1 + z)^{2.7}. This is the first directly observed normal star-forming galaxy X-ray luminosity function (XLF) at cosmologically interesting redshifts (z>0). This provides the most direct measure yet of the X-ray derived cosmic star-formation history of the Universe. We make use of Bayesian statistical methods to classify the galaxies and the two types of AGN, finding the most useful discriminators to be the X-ray luminosity, X-ray hardness ratio, and X-ray to optical flux ratio. There is some residual AGN contamination in the sample at the bright end of the luminosity function. Incompleteness slightly flattens the XLF at the faint end of the luminosity function. The XLF has a lognormal distribution and agrees well with the radio and infrared luminosity functions. However, the XLF does not agree with the Schechter luminosity function for the H-alpha LF indicating that additional and different physical processes may be involved in the establishment of the lognormal form of the XLF. The agreement of our star formation history points with the other star formation determinations in different wavebands (IR, Radio, H-alpha) gives an interesting constraint on the IMF, and X-rays may be measuring directly the binary star formation history of the Universe. X-ray studies will continue to be useful for probing the star formation history of the universe by avoiding problems of obscuration. Star formation may therefore be measured in more detail by deep surveys with future x-ray missions.Comment: Accepted for publication in ApJ. 19 pages with 10 figures formatted with emulateapj. Version with B/W only figures available at http://www.pha.jhu.edu/~ptak/paper