Causal Coordinated Concurrent Reinforcement Learning

In this work, we propose a novel algorithmic framework for data sharing and coordinated exploration for the purpose of learning more data-efficient and better performing policies under a concurrent reinforcement learning (CRL) setting. In contrast to other work which make the assumption that all agents act under identical environments, we relax this restriction and instead consider the formulation where each agent acts within an environment which shares a global structure but also exhibits individual variations. Our algorithm leverages a causal inference algorithm in the form of Additive Noise Model - Mixture Model (ANM-MM) in extracting model parameters governing individual differentials via independence enforcement. We propose a new data sharing scheme based on a similarity measure of the extracted model parameters and demonstrate superior learning speeds on a set of autoregressive, pendulum and cart-pole swing-up tasks and finally, we show the effectiveness of diverse action selection between common agents under a sparse reward setting. To the best of our knowledge, this is the first work in considering non-identical environments in CRL and one of the few works which seek to integrate causal inference with reinforcement learning (RL)

Hedgehog Signaling in Hepatocellular Carcinoma

Hepatocellular carcinoma (HCC) is a primary malignancy of the liver and one of the most common and deadly tumors worldwide. Most HCC cases are secondary to cirrhosis of the liver resulting from chronic liver injury states and dysregulated wound healing. While treatment of the cancer often involves resection, high recurrence rates are common, suggesting that cells outside the tumor play an important role in tumorigenesis. While the microenvironment is a key contributor to carcinogensis, the molecular mechanisms underlying their involvement remain unknown. Hedgehog (Hh) signaling has been found to play a critical role in liver regeneration after injury and Hh pathway activity is overexpressed in the pre-neoplastic and injured liver environment, leading us to hypothesize that it plays an essential role in the development and progression of HCC. This dissertation addresses the role of Hh signaling in HCCarcinogenesis and, specifically, the liver tumor microenvironment. Advances in this subject can lead to new therapeutic options for patients suffering from the burden of this disease.Doctor of Philosoph

Helping Students to Build Multicultural and Multidisciplinary Competences: A Pilot of Challenge-Based Collaborative Learning on a Digital Gamified Platform

Global issues such as poverty, hunger, and environmental problems are inextricable and cannot be solved comprehensively by homogeneous groups. With the advance of technology, collaborations with peoples at different geographical locations can be achieved effectively. Higher education in the 21st Century must therefore facilitate students to learn how to eclectically connect their creativity and problem-solving skills with technology, and most importantly to work with heterogeneous groups to solve complex global issues.This paper will elaborate on a pilot study of a project in Hong Kong, titled the CCGame Project, which aims to heighten students’ multicultural and multidisciplinary competences by deploying gamified learning and challenge-based learning. Team-based, self-guided learning is the core of the challenge-based learning approach. To preserve students’ interest in learning and accomplishing the tasks for the team, cloud-based learning platforms have been deployed. In the pilot, the online learning platform collected data for analysis of individual and team behaviour. The pilot demonstrated that students could work in a diverse team to complete a challenge. Evidence-based results supported with data analytics will be presented and the project’s plan of work will also be elucidated in this paper

Poster: Userland Containers for Mobile Systems

Mobile platforms are not rising to their potential as ubiquitous computers, in large part because of the constraints we impose on their apps in the name of security. Mobile operating systems have long struggled with the challenge of isolating untrusted apps. In pursuit of a secure runtime environment, Android and iOS isolate apps inside a gulag of platform-imposed programming languages and runtime libraries, leaving few design decisions to the application developers. These thick layers of custom software eschew app portability and maintainability, as development teams must continually tweak their apps to support modifications to the OS\u27s runtime libraries. Nonstandard and ever-changing interfaces to those APIs invite bugs in the operating system and apps alike. Mobile-only APIs have bifurcated the population of software running on our devices. On one side sits the conventional PC and server programs: compilers, shells, servers, daemons, and many others that use the standard libraries and programming models to interface with the computer and the outside world. On the other side lives the apps: mobile-only and purpose-built, they often serve as user interfaces to some larger cloud-based system. Under the weight of the numerous OS-imposed platform constraints, it is difficult for app developers to innovate: large classes of applications are simply impossible to port to mobile devices because the required APIs are unsupported. To deal with these cross platform dependencies, it is necessary to maintain multiple code bases. In the past, dependency issues have typically been solved through the use of containers. However, deploying containers on mobile systems present unique challenges. To maintain security, mobile operating systems do not give users permission to launch Docker containers. To solve this issue, we consider an older idea known as user-land containerization. Userland containerization allows userland containers to be launched by regular unprivileged users in any Linux or Android based system. Userland containerization works by inserting a modified operating system kernel between the host kernel and the guest processes. We have done an in depth study on the performance of user-mode containers like the user mode linux (UML) kernel [1], repurposing it as a userland hypervisor between the host kernel and the guest processes. We prototype a proof-of-concept usermode kernel with an implementation that is guided by the findings of our empirical study. Our kernel introduces a new technique---similar to paravirtualization---to optimize the syscall interface between the guest process and the usermode kernel to improve its I/O performance. The redesigned syscall interface provides I/O performance that approaches that of conventional virtualization techniques. Our paravirtualization strategies outperform UML by a factor of 3--6X for I/O bound workloads. Furthermore, we achieve 3.5--5X more network throughput and equal disk write speed compared to VMWare Workstation. Although there is still ample opportunity for performance improvements, our approach demonstrates the promise and potential of a usable userland virtualization platform that balances security with performance

Rare Germline Genetic Variants and the Risks of Epithelial Ovarian Cancer.

A family history of ovarian or breast cancer is the strongest risk factor for epithelial ovarian cancer (EOC). Germline deleterious variants in the BRCA1 and BRCA2 genes confer EOC risks by age 80, of 44% and 17% respectively. The mismatch repair genes, particularly MSH2 and MSH6, are also EOC susceptibility genes. Several other DNA repair genes, BRIP1, RAD51C, RAD51D, and PALB2, have been identified as moderate risk EOC genes. EOC has five main histotypes; high-grade serous (HGS), low-grade serous (LGS), clear cell (CCC), endometrioid (END), and mucinous (MUC). This review examines the current understanding of the contribution of rare genetic variants to EOC, focussing on providing frequency data for each histotype. We provide an overview of frequency and risk for pathogenic variants in the known susceptibility genes as well as other proposed genes. We also describe the progress to-date to understand the role of missense variants and the different breast and ovarian cancer risks for each gene. Identification of susceptibility genes have clinical impact by reducing disease-associated mortality through improving risk prediction, with the possibility of prevention strategies, and developing new targeted treatments and these clinical implications are also discussed

Evidence of Spatially Extensive Resistance to PCBs in an Anadromous Fish of the Hudson River

Populations of organisms that are chronically exposed to high levels of chemical contaminants may not suffer the same sublethal or lethal effects as naive populations, a phenomenon called resistance. Atlantic tomcod (Microgadus tomcod) from the Hudson River, New York, are exposed to high concentrations of polycyclic aromatic hydrocarbons (PAHs) and bioaccumulate polychlorinated biphenyls (PCBs), polychlorinated dibenzo-p-dioxins (PCDDs), and polychlorinated dibenzofurans (PCDFs). They have developed resistance to PCBs and PCDDs but not to PAHs. Resistance is largely heritable and manifests at early-life-stage toxic end points and in inducibility of cytochrome P4501A (CYP1A) mRNA expression. Because CYP1A induction is activated by the aryl hydrocarbon receptor (AHR) pathway, as are most toxic responses to these compounds, we sought to determine the geographic extent of resistance to CYP1A mRNA induction by PCBs in the Hudson River tomcod population. Samples of young-of-the-year tomcod were collected from seven locales in the Hudson River, extending from the Battery at river mile 1 (RM 1) to RM 90, and from the Miramichi River, New Brunswick, Canada. Laboratory-reared offspring of tomcod adults from Newark Bay, in the western portion of the Hudson River estuary, were also used in this study. Fish were partially depurated in clean water and intraperitoneally injected with 10 ppm coplanar PCB-77, 10 ppm benzo[a]pyrene (BaP), or corn oil vehicle, and levels of CYP1A mRNA were determined. CYP1A was significantly inducible by treatment with BaP in tomcod from the Miramichi River, from laboratory-spawned offspring of Newark Bay origin, and from all Hudson River sites spanning 90 miles of river. In contrast, only tomcod from the Miramichi River displayed significantly induced CYP1A mRNA expression when treated with PCB-77. Our results suggest that the population of tomcod from throughout the Hudson River estuary has developed resistance to CYP1A inducibility and probably other toxicities mediated by the AHR pathway. Tomcod from the Hudson River may represent the most geographically expansive population of vertebrates with resistance to chemical pollutants that has been characterized

Chinese Social Media Reaction to the MERS-Cov and Avian Influenza A (H7N9) Outbreaks

Background: As internet and social media use have skyrocketed, epidemiologists have begun to use online data such as Google query data and Twitter trends to track the activity levels of influenza and other infectious diseases. In China, Weibo is an extremely popular microblogging site that is equivalent to Twitter. Capitalizing on the wealth of public opinion data contained in posts on Weibo, this study used Weibo as a measure of the Chinese people’s reactions to two different outbreaks: the 2012 Middle East Respiratory Syndrome Coronavirus (MERS-CoV) outbreak, and the 2013 outbreak of human infection of avian influenza A(H7N9) in China. Methods: Keyword searches were performed in Weibo data collected by The University of Hong Kong’s Weiboscope project. Baseline values were determined for each keyword and reaction values per million posts in the days after outbreak information was released to the public. Results: The results show that the Chinese people reacted significantly to both outbreaks online, where their social media reaction was two orders of magnitude stronger to the H7N9 influenza outbreak that happened in China than the MERS-CoV outbreak that was far away from China. Conclusions: These results demonstrate that social media could be a useful measure of public awareness and reaction to disease outbreak information released by health authorities

Design choices and equity implications of community shared solar

What is the best way to deploy solar energy to maximize clean energy growth while equitably sharing benefits? A promising model is community shared solar, which enables energy consumers to purchase shares of electricity generated in an offsite project. Noting how different states and utilities have approached program design, we explore how design decisions affect access to solar and the equity of cost and benefit sharing. We conclude with a set of questions for future research.This work was supported by NSF SRN Grant No. 1444745, the University of Minnesota Center for Urban and Regional Affairs’ Faculty Interactive Research Program, and the University of Minnesota’s Office of the Vice President for Research’s Grant-in-Aid of Research, Artistry, and Scholarship