Massively Scalable Inverse Reinforcement Learning in Google Maps

Optimizing for humans' latent preferences is a grand challenge in route recommendation, where globally-scalable solutions remain an open problem. Although past work created increasingly general solutions for the application of inverse reinforcement learning (IRL), these have not been successfully scaled to world-sized MDPs, large datasets, and highly parameterized models; respectively hundreds of millions of states, trajectories, and parameters. In this work, we surpass previous limitations through a series of advancements focused on graph compression, parallelization, and problem initialization based on dominant eigenvectors. We introduce Receding Horizon Inverse Planning (RHIP), which generalizes existing work and enables control of key performance trade-offs via its planning horizon. Our policy achieves a 16-24% improvement in global route quality, and, to our knowledge, represents the largest instance of IRL in a real-world setting to date. Our results show critical benefits to more sustainable modes of transportation (e.g. two-wheelers), where factors beyond journey time (e.g. route safety) play a substantial role. We conclude with ablations of key components, negative results on state-of-the-art eigenvalue solvers, and identify future opportunities to improve scalability via IRL-specific batching strategies

A probabilistic model for gene content evolution with duplication, loss, and horizontal transfer

We introduce a Markov model for the evolution of a gene family along a phylogeny. The model includes parameters for the rates of horizontal gene transfer, gene duplication, and gene loss, in addition to branch lengths in the phylogeny. The likelihood for the changes in the size of a gene family across different organisms can be calculated in O(N+hM^2) time and O(N+M^2) space, where N is the number of organisms, $h$ is the height of the phylogeny, and M is the sum of family sizes. We apply the model to the evolution of gene content in Preoteobacteria using the gene families in the COG (Clusters of Orthologous Groups) database

Interplay between pleiotropy and secondary selection determines rise and fall of mutators in stress response

Dramatic rise of mutators has been found to accompany adaptation of bacteria in response to many kinds of stress. Two views on the evolutionary origin of this phenomenon emerged: the pleiotropic hypothesis positing that it is a byproduct of environmental stress or other specific stress response mechanisms and the second order selection which states that mutators hitchhike to fixation with unrelated beneficial alleles. Conventional population genetics models could not fully resolve this controversy because they are based on certain assumptions about fitness landscape. Here we address this problem using a microscopic multiscale model, which couples physically realistic molecular descriptions of proteins and their interactions with population genetics of carrier organisms without assuming any a priori fitness landscape. We found that both pleiotropy and second order selection play a crucial role at different stages of adaptation: the supply of mutators is provided through destabilization of error correction complexes or fluctuations of production levels of prototypic mismatch repair proteins (pleiotropic effects), while rise and fixation of mutators occur when there is a sufficient supply of beneficial mutations in replication-controlling genes. This general mechanism assures a robust and reliable adaptation of organisms to unforeseen challenges. This study highlights physical principles underlying physical biological mechanisms of stress response and adaptation

In silico evolution of signaling networks using rule-based models: bistable response dynamics

One of the ultimate goals in biology is to understand the design principles of biological systems. Such principles, if they exist, can help us better understand complex, natural biological systems and guide the engineering of de novo ones. Towards deciphering design principles, in silico evolution of biological systems with proper abstraction is a promising approach. Here, we demonstrate the application of in silico evolution combined with rule-based modelling for exploring design principles of cellular signaling networks. This application is based on a computational platform, called BioJazz, which allows in silico evolution of signaling networks with unbounded complexity. We provide a detailed introduction to BioJazz architecture and implementation and describe how it can be used to evolve and/or design signaling networks with defined dynamics. For the latter, we evolve signaling networks with switch-like response dynamics and demonstrate how BioJazz can result in new biological insights on network structures that can endow bistable response dynamics. This example also demonstrated both the power of BioJazz in evolving and designing signaling networks and its limitations at the current stage of development.Comment: 24 pages, 7 figure

Nicotine Overrides DNA Damage-Induced G1/S Restriction in Lung Cells

As an addictive substance, nicotine has been suggested to facilitate pro-survival activities (such as anchorage-independent growth or angiogenesis) and the establishment of drug resistance to anticancer therapy. Tobacco smoking consists of a variety of carcinogens [such as benzopyrene (BP) and nitrosamine derivatives] that are able to cause DNA double strand breaks. However, the effect of nicotine on DNA damage-induced checkpoint response induced by genotoxins remains unknown. In this study, we investigated the events occurred during G1 arrest induced by γ-radiation or BP in nicotine-treated murine or human lung epithelial cells. DNA synthesis was rapidly inhibited after exposure to γ-radiation or BP treatment, accompanied with the activation of DNA damage checkpoint. When these cells were co-treated with nicotine, the growth restriction was compromised, manifested by upregulation of cyclin D and A, and attenuation of Chk2 phosphorylation. Knockdown of cyclin D or Chk2 by the siRNAs blocked nicotine-mediated effect on DNA damage checkpoint activation. However, nicotine treatment appeared to play no role in nocodazole-induced mitotic checkpoint activation. Overall, our study presented a novel observation, in which nicotine is able to override DNA damage checkpoint activated by tobacco-related carcinogen BP or γ-irradiation. The results not only indicates the potentially important role of nicotine in facilitating the establishment of genetic instability to promote lung tumorigenesis, but also warrants a dismal prognosis for cancer patients who are smokers, heavily exposed second-hand smokers or nicotine users

Does technology and Innovation Management improve Market Position? Empirical Evidence from Innovating Firms in South Africa

There is a growing recognition of the central role of technology and knowledge management for market success of organizations. Little is empirically know, however, about this relationship. Drawing on the South African Innovation Survey, a unique dataset on innovative behavior of South African firms in manufacturing and services, this paper investigates the question to what extent and in which ways do technology and innovation management activities affect firms’ market position. Findings show that conducting technology strategy activities pays out. Moreover, especially a combination of internal and external technology audits seems to be beneficial for organizational performance

The gut microbiota: a major player in the toxicity of environmental pollutants?

Exposure to environmental chemicals has been linked to various health disorders, including obesity, type 2 diabetes, cancer and dysregulation of the immune and reproductive systems, whereas the gastrointestinal microbiota critically contributes to a variety of host metabolic and immune functions. We aimed to evaluate the bidirectional relationship between gut bacteria and environmental pollutants and to assess the toxicological relevance of the bacteria–xenobiotic interplay for the host. We examined studies using isolated bacteria, faecal or caecal suspensions—germ-free or antibiotic-treated animals—as well as animals reassociated with a microbiota exposed to environmental chemicals. The literature indicates that gut microbes have an extensive capacity to metabolise environmental chemicals that can be classified in five core enzymatic families (azoreductases, nitroreductases, β-glucuronidases, sulfatases and β-lyases) unequivocally involved in the metabolism of >30 environmental contaminants. There is clear evidence that bacteria-dependent metabolism of pollutants modulates the toxicity for the host. Conversely, environmental contaminants from various chemical families have been shown to alter the composition and/or the metabolic activity of the gastrointestinal bacteria, which may be an important factor contributing to shape an individual’s microbiotype. The physiological consequences of these alterations have not been studied in details but pollutant-induced alterations of the gut bacteria are likely to contribute to their toxicity. In conclusion, there is a body of evidence suggesting that gut microbiota are a major, yet underestimated element that must be considered to fully evaluate the toxicity of environmental contaminants