Zero-shot compositional reasoning in a reinforcement learning setting

People can easily evoke previously learned concepts, compose them, and apply the result to solve novel tasks on the first attempt. The aim of this paper is to improve our understanding of how people make such zero-shot compositional inferences in a reinforcement learning setting. To achieve this, we introduce an experimental paradigm where people learn two latent reward functions and need to compose them correctly to solve a novel task. We find that people have the capability to engage in zero-shot compositional reinforcement learning but deviate systematically from optimality. However, their mistakes are structured and can be explained by their performance in the sub-tasks leading up to the composition. Through extensive model-based analyses, we found that a meta-learned neural network model that accounts for limited computational resources best captures participants’ behaviour. Moreover, the amount of computational resources this model identified reliably quantifies how good individual participants are at zero-shot compositional reinforcement learning. Taken together, our work takes a considerable step towards studying compositional reasoning in agents – both natural and artificial – with limited computational resources

Hepatoprotection by the farnesoid X receptor agonist GW4064 in rat models of intra- and extrahepatic cholestasis

Farnesoid X receptor (FXR) is a bile acid–activated transcription factor that is a member of the nuclear hormone receptor superfamily. Fxr-null mice exhibit a phenotype similar to Byler disease, an inherited cholestatic liver disorder. In the liver, activation of FXR induces transcription of transporter genes involved in promoting bile acid clearance and represses genes involved in bile acid biosynthesis. We investigated whether the synthetic FXR agonist GW4064 could protect against cholestatic liver damage in rat models of extrahepatic and intrahepatic cholestasis. In the bile duct–ligation and α-naphthylisothiocyanate models of cholestasis, GW4064 treatment resulted in significant reductions in serum alanine aminotransferase, aspartate aminotransferase, and lactate dehydrogenase, as well as other markers of liver damage. Rats that received GW4064 treatment also had decreased incidence and extent of necrosis, decreased inflammatory cell infiltration, and decreased bile duct proliferation. Analysis of gene expression in livers from GW4064-treated cholestatic rats revealed decreased expression of bile acid biosynthetic genes and increased expression of genes involved in bile acid transport, including the phospholipid flippase MDR2. The hepatoprotection seen in these animal models by the synthetic FXR agonist suggests FXR agonists may be useful in the treatment of cholestatic liver disease