Omega-Regular Objectives in Model-Free Reinforcement Learning

We provide the first solution for model-free reinforcement learning of ω-regular objectives for Markov decision processes (MDPs). We present a constructive reduction from the almost-sure satisfaction of ω-regular bjectives to an almost-sure reachability problem, and extend this technique to learning how to control an unknown model so that the chance of satisfying the objective is maximized. We compile ω-regular properties into limit-deterministic B¨uchi automata instead of the traditional Rabin automata; this choice sidesteps difficulties that have marred previous proposals. Our approach allows us to apply model-free, off-the-shelf reinforcement learning algorithms to compute optimal strategies from the observations of the MDP. We present an experimental evaluation of our technique on benchmark learning problems

BRCA1 Regulates Follistatin Function in Ovarian Cancer and Human Ovarian Surface Epithelial Cells

Follistatin (FST), a folliculogenesis regulating protein, is found in relatively high concentrations in female ovarian tissues. FST acts as an antagonist to Activin, which is often elevated in human ovarian carcinoma, and thus may serve as a potential target for therapeutic intervention against ovarian cancer. The breast cancer susceptibility gene 1 (BRCA1) is a known tumor suppressor gene in human breast cancer; however its role in ovarian cancer is not well understood. We performed microarray analysis on human ovarian carcinoma cell line SKOV3 that stably overexpress wild-type BRCA1 and compared with the corresponding empty vector-transfected clones. We found that stable expression of BRCA1 not only stimulates FST secretion but also simultaneously inhibits Activin expression. To determine the physiological importance of this phenomenon, we further investigated the effect of cellular BRCA1 on the FST secretion in immortalized ovarian surface epithelial (IOSE) cells derived from either normal human ovaries or ovaries of an ovarian cancer patient carrying a mutation in BRCA1 gene. Knock-down of BRCA1 in normal IOSE cells demonstrates down-regulation of FST secretion along with the simultaneous up-regulation of Activin expression. Furthermore, knock-down of FST in IOSE cell lines as well as SKOV3 cell line showed significantly reduced cell proliferation and decreased cell migration when compared with the respective controls. Thus, these findings suggest a novel function for BRCA1 as a regulator of FST expression and function in human ovarian cells