Reinforcement Learning with Non-Markovian Rewards

The standard RL world model is that of a Markov Decision Process (MDP). A basic premise of MDPs is that the rewards depend on the last state and action only. Yet, many real-world rewards are non-Markovian. For example, a reward for bringing coffee only if requested earlier and not yet served, is non-Markovian if the state only records current requests and deliveries. Past work considered the problem of modeling and solving MDPs with non-Markovian rewards (NMR), but we know of no principled approaches for RL with NMR. Here, we address the problem of policy learning from experience with such rewards. We describe and evaluate empirically four combinations of the classical RL algorithm Q-learning and R-max with automata learning algorithms to obtain new RL algorithms for domains with NMR. We also prove that some of these variants converge to an optimal policy in the limit.Comment: To Appear in AAAI 202

A practical guide to single-cell RNA-sequencing for biomedical research and clinical applications.

RNA sequencing (RNA-seq) is a genomic approach for the detection and quantitative analysis of messenger RNA molecules in a biological sample and is useful for studying cellular responses. RNA-seq has fueled much discovery and innovation in medicine over recent years. For practical reasons, the technique is usually conducted on samples comprising thousands to millions of cells. However, this has hindered direct assessment of the fundamental unit of biology-the cell. Since the first single-cell RNA-sequencing (scRNA-seq) study was published in 2009, many more have been conducted, mostly by specialist laboratories with unique skills in wet-lab single-cell genomics, bioinformatics, and computation. However, with the increasing commercial availability of scRNA-seq platforms, and the rapid ongoing maturation of bioinformatics approaches, a point has been reached where any biomedical researcher or clinician can use scRNA-seq to make exciting discoveries. In this review, we present a practical guide to help researchers design their first scRNA-seq studies, including introductory information on experimental hardware, protocol choice, quality control, data analysis and biological interpretation

Lessons from non-canonical splicing

Recent improvements in experimental and computational techniques that are used to study the transcriptome have enabled an unprecedented view of RNA processing, revealing many previously unknown non-canonical splicing events. This includes cryptic events located far from the currently annotated exons and unconventional splicing mechanisms that have important roles in regulating gene expression. These non-canonical splicing events are a major source of newly emerging transcripts during evolution, especially when they involve sequences derived from transposable elements. They are therefore under precise regulation and quality control, which minimizes their potential to disrupt gene expression. We explain how non-canonical splicing can lead to aberrant transcripts that cause many diseases, and also how it can be exploited for new therapeutic strategies

Artificial intelligence (AI): multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research and practice

As far back as the industrial revolution, great leaps in technical innovation succeeded in transforming numerous manual tasks and processes that had been in existence for decades where humans had reached the limits of physical capacity. Artificial Intelligence (AI) offers this same transformative potential for the augmentation and potential replacement of human tasks and activities within a wide range of industrial, intellectual and social applications. The pace of change for this new AI technological age is staggering, with new breakthroughs in algorithmic machine learning and autonomous decision making engendering new opportunities for continued innovation. The impact of AI is significant, with industries ranging from: finance, retail, healthcare, manufacturing, supply chain and logistics all set to be disrupted by the onset of AI technologies. The study brings together the collective insight from a number of leading expert contributors to highlight the significant opportunities, challenges and potential research agenda posed by the rapid emergence of AI within a number of domains: technological, business and management, science and technology, government and public sector. The research offers significant and timely insight to AI technology and its impact on the future of industry and society in general

Efficacy and safety of the probiotic Saccharomyces boulardii for the prevention and therapy of gastrointestinal disorders

Several clinical trials and experimental studies strongly suggest a place for Saccharomyces boulardii as a biotherapeutic agent for the prevention and treatment of several gastrointestinal diseases. S. boulardii mediates responses resembling the protective effects of the normal healthy gut flora. The multiple mechanisms of action of S. boulardii and its properties may explain its efficacy and beneficial effects in acute and chronic gastrointestinal diseases that have been confirmed by clinical trials. Caution should be taken in patients with risk factors for adverse events. This review discusses the evidence for efficacy and safety of S. boulardii as a probiotic for the prevention and therapy of gastrointestinal disorders in humans

Dexamethasone reduces acrosin activity of ram spermatozoa

The aim of this study was to investigate the effect of dexamethasone on acrosin activity of spermatozoa in Chios rams during autumn (breeding season for sheep in Greece), in correlation with possible changes in blood testosterone. Dexamethasone was administered in four equal consecutive intramuscular injections, one every four hours (total dose: 3 mg kg(-1)). Total acrosin activity was determined in semen samples collected 48 before and on the 4th and 7th day and thereafter once every week until the 77th day after dexamethasone administration. Blood samples for testosterone radio-immunoassays were collected 24 h before, during dexamethasone administration and on the 4th, 7th 14th and 21st day after administration. Total acrosin activity in spermatozoa was reduced between days 7-28 after dexamethasone administration. Dexamethasone also induced a reduction in mean value and basal level of blood testosterone and inhibited its episodic secretion between 1 and 4 days after administration. As the reduction of acrosin activity appeared relatively soon after dexamethasone administration (7th day), it is likely that the increased amount of dexamethasone did not influence tire synthesis of proacrosin in the late spermatids. As glucocorticoid receptors exist in the epididymis and accessory glands in various species, dexamethasone may have a direct influence on the synthesis and/or release of acrosin inhibitors in epididymal fluid or seminal plasma. These changes in acrosin activity in ovine spermatozoa mediated by dexamethasone may be of importance regarding the role of stress in the reduction of sperm fertilizing ability