PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning

We present PRM-RL, a hierarchical method for long-range navigation task completion that combines sampling based path planning with reinforcement learning (RL). The RL agents learn short-range, point-to-point navigation policies that capture robot dynamics and task constraints without knowledge of the large-scale topology. Next, the sampling-based planners provide roadmaps which connect robot configurations that can be successfully navigated by the RL agent. The same RL agents are used to control the robot under the direction of the planning, enabling long-range navigation. We use the Probabilistic Roadmaps (PRMs) for the sampling-based planner. The RL agents are constructed using feature-based and deep neural net policies in continuous state and action spaces. We evaluate PRM-RL, both in simulation and on-robot, on two navigation tasks with non-trivial robot dynamics: end-to-end differential drive indoor navigation in office environments, and aerial cargo delivery in urban environments with load displacement constraints. Our results show improvement in task completion over both RL agents on their own and traditional sampling-based planners. In the indoor navigation task, PRM-RL successfully completes up to 215 m long trajectories under noisy sensor conditions, and the aerial cargo delivery completes flights over 1000 m without violating the task constraints in an environment 63 million times larger than used in training.Comment: 9 pages, 7 figure

A Cutoff Procedure and Counterterms for Differential Renormalization

Explicit divergences and counterterms do not appear in the differential renormalization method, but they are concealed in the neglected surface terms in the formal partial integration procedure used. A systematic real space cutoff procedure for massless $\phi^4$ theory is therefore studied in order to test the method and its compatibility with unitarity. Through 3-loop order, it is found that cutoff bare amplitudes are equal to the renormalized amplitudes previously obtained using the formal procedure plus singular terms which can be consistently cancelled by adding conventional counterterms to the Lagrangian. Renormalization group functions $\beta (g)$ and $\gamma (g)$ obtained in the cutoff theory also agree with previous results.Comment: 28 pages, CTP#2099-DTP/92/40-UBECMPF/92/13, 2 figures (not included). (Tex problems solved and information about number of pages and figures added

Consistency of Mycobacterium tuberculosis-Specific Interferon-Gamma Responses in HIV-1-Infected Women during Pregnancy and Postpartum

Background. We determined the consistency of positive interferon-gamma (IFN-γ) release assays (IGRAs) to detect latent TB infection (LTBI) over one-year postpartum in HIV-1-infected women. Methods. Women with positive IGRAs during pregnancy had four 3-monthly postpartum IGRAs. Postpartum change in magnitude of IFN-γ response was determined using linear mixed models. Results. Among 18 women with positive pregnancy IGRA, 15 (83%) had a subsequent positive IGRA; 9 (50%) were always positive, 3 (17%) were always negative, and 6 (33%) fluctuated between positive and negative IGRAs. Women with pregnancy IGRA IFN-γ>8 spot forming cells (SFCs)/well were more likely to have consistent postpartum IGRA response (odds ratio: 10.0; 95% confidence interval (CI): 0.9–117.0). Change in IFN-γ response over postpartum was 10.2 SFCs/well (95% CI: −1.5–21.8 SFCs/well). Conclusion. Pregnancy positive IGRAs were often maintained postpartum with increased consistency in women with higher baseline responses. There were modest increases in magnitude of IGRA responses postpartum

Compartmentalization of HIV-1 within the Female Genital Tract Is Due to Monotypic and Low-Diversity Variants Not Distinct Viral Populations

BACKGROUND:Compartmentalization of HIV-1 between the genital tract and blood was noted in half of 57 women included in 12 studies primarily using cell-free virus. To further understand differences between genital tract and blood viruses of women with chronic HIV-1 infection cell-free and cell-associated virus populations were sequenced from these tissues, reasoning that integrated viral DNA includes variants archived from earlier in infection, and provides a greater array of genotypes for comparisons. METHODOLOGY/PRINCIPAL FINDINGS:Multiple sequences from single-genome-amplification of HIV-1 RNA and DNA from the genital tract and blood of each woman were compared in a cross-sectional study. Maximum likelihood phylogenies were evaluated for evidence of compartmentalization using four statistical tests. Genital tract and blood HIV-1 appears compartmentalized in 7/13 women by >/=2 statistical analyses. These subjects' phylograms were characterized by low diversity genital-specific viral clades interspersed between clades containing both genital and blood sequences. Many of the genital-specific clades contained monotypic HIV-1 sequences. In 2/7 women, HIV-1 populations were significantly compartmentalized across all four statistical tests; both had low diversity genital tract-only clades. Collapsing monotypic variants into a single sequence diminished the prevalence and extent of compartmentalization. Viral sequences did not demonstrate tissue-specific signature amino acid residues, differential immune selection, or co-receptor usage. CONCLUSIONS/SIGNIFICANCE:In women with chronic HIV-1 infection multiple identical sequences suggest proliferation of HIV-1-infected cells, and low diversity tissue-specific phylogenetic clades are consistent with bursts of viral replication. These monotypic and tissue-specific viruses provide statistical support for compartmentalization of HIV-1 between the female genital tract and blood. However, the intermingling of these clades with clades comprised of both genital and blood sequences and the absence of tissue-specific genetic features suggests compartmentalization between blood and genital tract may be due to viral replication and proliferation of infected cells, and questions whether HIV-1 in the female genital tract is distinct from blood

Pervasive Synaptic Branch Removal in the Mammalian Neuromuscular System at Birth

SummaryUsing light and serial electron microscopy, we show profound refinements in motor axonal branching and synaptic connectivity before and after birth. Embryonic axons become maximally connected just before birth when they innervate ∼10-fold more muscle fibers than in maturity. In some developing muscles, axons innervate almost every muscle fiber. At birth, each neuromuscular junction is coinnervated by approximately ten highly intermingled axons (versus one in adults). Extensive die off of terminal branches occurs during the first several postnatal days, leading to much sparser arbors that still span the same territory. Despite the extensive pruning, total axoplasm per neuron increases as axons elongate, thicken, and add more synaptic release sites on their remaining targets. Motor axons therefore initially establish weak connections with nearly all available postsynaptic targets but, beginning at birth, massively redistribute synaptic resources, concentrating many more synaptic sites on many fewer muscle fibers. Analogous changes in connectivity may occur in the CNS.Video Abstrac

Oligonucleotide ligation assay detects HIV drug resistance associated with virologic failure among antiretroviral-naive adults in Kenya

Background: Transmitted drug resistance (TDR) is increasing in some areas of Africa. Detection of TDR may predict virologic failure of first-line non-nucleoside reverse-transcriptase inhibitor (NNRTI)-based antiretroviral therapy (ART). We evaluated the utility of a relatively inexpensive oligonucleotide ligation assay (OLA) to detect clinically relevant TDR at time of ART initiation. Methods: Pre-ART plasmas from ART-naive Kenyans initiating an NNRTI-based fixed-dose combination ART in a randomized adherence trial conducted in 2006 were retrospectively analyzed by OLA for mutations conferring resistance to NNRTI (K103N, Y181C, and G190A) and lamivudine (M184V). Post-ART plasmas were analyzed for virologic failure (≥1,000 copies/mL) at 6 month intervals over 18-month follow-up. Pre-ART plasmas of those with virologic failure were evaluated for drug resistance by consensus and 454-pyrosequencing. Results: Among 386 participants, TDR was detected by OLA in 3.89% [95% Confidence Interval (CI), 2.19-6.33], and was associated with a 10-fold higher rate of virologic failure [Hazard Ratio (HR), 10.39; 95% CI, 3.23-32.41; p Conclusions: Detection of TDR by a point mutation assay may prevent use of sub-optimal ART

Increasing HIV-1 pretreatment drug resistance among antiretroviral-naïve adults initiating treatment between 2006 and 2014 in Nairobi, Kenya

Antiretroviral-naïve adults initiating antiretroviral therapy in Nairobi, Kenya were tested for HIV-1 drug resistance at codons K103N, Y181C, G190A, M184V, and K65R using an oligonucleotide ligation assay. Prevalence of pretreatment drug resistance increased from 3.89% in 2006 to 10.93% in 2014 (P \u3c 0.001), and 95% of those with resistance had at least one nonnucleoside reverse transcriptase inhibitor mutation. Resistance to tenofovir (K65R) was found in 2014 but not in 2006

Ultrastructurally-smooth thick partitioning and volume stitching for larger-scale connectomics

FIB-SEM has become an essential tool for studying neural tissue at resolutions below 10×10×10 nm, producing datasets superior for automatic connectome tracing. We present a technical advance, ultrathick sectioning, which reliably subdivides embedded tissue samples into chunks (20 µm thick) optimally sized and mounted for efficient, parallel FIB-SEM imaging. These chunks are imaged separately and then ‘volume stitched’ back together, producing a final 3D dataset suitable for connectome tracing

Research Priorities for Achieving Healthy Marine Ecosystems and Human Communities in a Changing Climate

ABSTRACT: The health of coastal human communities and marine ecosystems are at risk from a host of anthropogenic stressors, in particular, climate change. Because ecological health and human well-being are inextricably connected, effective and positive responses to current risks require multidisciplinary solutions. Yet, the complexity of coupled social-ecological systems has left many potential solutions unidentified or insufficiently explored. The urgent need to achieve positive social and ecological outcomes across local and global scales necessitates rapid and targeted multidisciplinary research to identify solutions that have the greatest chance of promoting benefits for both people and nature. To address these challenges, we conducted a forecasting exercise with a diverse, multidisciplinary team to identify priority research questions needed to promote sustainable and just marine social-ecological systems now and into the future, within the context of climate change and population growth. In contrast to the traditional reactive cycle of science and management, we aimed to generate questions that focus on what we need to know, before we need to know it. Participants were presented with the question, "If we were managing oceans in 2050 and looking back, what research, primary or synthetic, would wish we had invested in today?" We first identified major social and ecological events over the past 60 years that shaped current human relationships with coasts and oceans. We then used a modified Delphi approach to identify nine priority research areas and 46 questions focused on increasing sustainability and well-being in marine social-ecological systems. The research areas we identified include relationships between ecological and human health, access to resources, equity, governance, economics, resilience, and technology. Most questions require increased collaboration across traditionally distinct disciplines and sectors for successful study and implementation. By identifying these questions, we hope to facilitate the discourse, research, and policies needed to rapidly promote healthy marine ecosystems and the human communities that depend upon them

An Energy-Water Corridor Along the US/Mexico Border: Changing the \u27Conversation\u27

Over the last decade, migration has become a divisive issue around the world. A large number of countries have erected barriers along their borders to prevent migration, leading to geopolitical tension. Climate change effects will likely exacerbate migration tensions, which will require bold and creative solutions to this difficult social predicament. Here we detail a plan to construct an energy-water corridor along a border that has been the focus of much attention recently: The U.S.-Mexico border. Our proposed solution helps to alleviate some of the negative effects of climate change, while providing energy and economic stimulus to an area that begs for sustainable development. The energy-water corridor will take advantage of the unique renewable energy resources along the border states and will use state-of-the-art water desalination and treatment systems to provide the resources for economic development in the region