Memory Augmented Control Networks

Planning problems in partially observable environments cannot be solved directly with convolutional networks and require some form of memory. But, even memory networks with sophisticated addressing schemes are unable to learn intelligent reasoning satisfactorily due to the complexity of simultaneously learning to access memory and plan. To mitigate these challenges we introduce the Memory Augmented Control Network (MACN). The proposed network architecture consists of three main parts. The first part uses convolutions to extract features and the second part uses a neural network-based planning module to pre-plan in the environment. The third part uses a network controller that learns to store those specific instances of past information that are necessary for planning. The performance of the network is evaluated in discrete grid world environments for path planning in the presence of simple and complex obstacles. We show that our network learns to plan and can generalize to new environments

End-to-End Navigation in Unknown Environments using Neural Networks

We investigate how a neural network can learn perception actions loops for navigation in unknown environments. Specifically, we consider how to learn to navigate in environments populated with cul-de-sacs that represent convex local minima that the robot could fall into instead of finding a set of feasible actions that take it to the goal. Traditional methods rely on maintaining a global map to solve the problem of over coming a long cul-de-sac. However, due to errors induced from local and global drift, it is highly challenging to maintain such a map for long periods of time. One way to mitigate this problem is by using learning techniques that do not rely on hand engineered map representations and instead output appropriate control policies directly from their sensory input. We first demonstrate that such a problem cannot be solved directly by deep reinforcement learning due to the sparse reward structure of the environment. Further, we demonstrate that deep supervised learning also cannot be used directly to solve this problem. We then investigate network models that offer a combination of reinforcement learning and supervised learning and highlight the significance of adding fully differentiable memory units to such networks. We evaluate our networks on their ability to generalize to new environments and show that adding memory to such networks offers huge jumps in performanceComment: Workshop on Learning Perception and Control for Autonomous Flight: Safety, Memory and Efficiency, Robotics Science and Systems 201

Neural Network Memory Architectures for Autonomous Robot Navigation

This paper highlights the significance of including memory structures in neural networks when the latter are used to learn perception-action loops for autonomous robot navigation. Traditional navigation approaches rely on global maps of the environment to overcome cul-de-sacs and plan feasible motions. Yet, maintaining an accurate global map may be challenging in real-world settings. A possible way to mitigate this limitation is to use learning techniques that forgo hand-engineered map representations and infer appropriate control responses directly from sensed information. An important but unexplored aspect of such approaches is the effect of memory on their performance. This work is a first thorough study of memory structures for deep-neural-network-based robot navigation, and offers novel tools to train such networks from supervision and quantify their ability to generalize to unseen scenarios. We analyze the separation and generalization abilities of feedforward, long short-term memory, and differentiable neural computer networks. We introduce a new method to evaluate the generalization ability by estimating the VC-dimension of networks with a final linear readout layer. We validate that the VC estimates are good predictors of actual test performance. The reported method can be applied to deep learning problems beyond robotics

Beethoven's annotations to Cramer's twenty-one piano studies : context and analysis of performance

The present study focuses on the annotations Beethoven appended to twenty-one piano studies by John Baptist Cramer, when teaching his nephew Karl. Beethoven held Cramer's collection of studies in high esteem and considered them the best preparatory school for his own works. The reading of his annotations reveals a continual preoccupation with issues such as legato (or bindung in Beethoven's own words), accentuation and the application of poetic feet in the music. This study examines the context of these annotations and applies them to Beethoven's piano music. The author's goal is to stimulate interest in Cramer's neglected Studio per il Pianoforte and to use Beethoven's advice on the execution of these studies as a guide for the performance of his own works. The author believes that this study will serve as a valuable tool to the teachers who teach the piano music of Beethoven and his era, the students who study his music as well as the professional performers of Beethoven's piano works

On the Effect of Dust Particles on Global Cloud Condensation Nuclei and Cloud Droplet Number

Aerosol-cloud interaction studies to date consider aerosol with a substantial fraction of soluble material as the sole source of cloud condensation nuclei (CCN). Emerging evidence suggests that mineral dust can act as good CCN through water adsorption onto the surface of particles. This study provides a first assessment of the contribution of insoluble dust to global CCN and cloud droplet number concentration (CDNC). Simulations are carried out with the NASA Global Modeling Initiative chemical transport model with an online aerosol simulation, considering emissions from fossil fuel, biomass burning, marine, and dust sources. CDNC is calculated online and explicitly considers the competition of soluble and insoluble CCN for water vapor. The predicted annual average contribution of insoluble mineral dust to CCN and CDNC in cloud-forming areas is up to 40 and 23.8%, respectively. Sensitivity tests suggest that uncertainties in dust size distribution and water adsorption parameters modulate the contribution of mineral dust to CDNC by 23 and 56%, respectively. Coating of dust by hygroscopic salts during the atmospheric aging causes a twofold enhancement of the dust contribution to CCN; the aged dust, however, can substantially deplete in-cloud supersaturation during the initial stages of cloud formation and can eventually reduce CDNC. Considering the hydrophilicity from adsorption and hygroscopicity from solute is required to comprehensively capture the dust-warm cloud interactions. The framework presented here addresses this need and can be easily integrated in atmospheric models

Bone loss and aggravated autoimmune arthritis in HLA-DRβ1-bearing humanized mice following oral challenge with Porphyromonas gingivalis

BACKGROUND: The linkage between periodontal disease and rheumatoid arthritis is well established. Commonalities among the two are that both are chronic inflammatory diseases characterized by bone loss, an association with the shared epitope susceptibility allele, and anti-citrullinated protein antibodies. METHODS: To explore immune mechanisms that may connect the two seemingly disparate disorders, we measured host immune responses including T-cell phenotype and anti-citrullinated protein antibody production in human leukocyte antigen (HLA)-DR1 humanized C57BL/6 mice following exposure to the Gram-negative anaerobic periodontal disease pathogen Porphyromonas gingivalis. We measured autoimmune arthritis disease expression in mice exposed to P. gingivalis, and also in arthritis-resistant mice by flow cytometry and multiplex cytokine-linked and enzyme-linked immunosorbent assays. We also measured femoral bone density by microcomputed tomography and systemic cytokine production. RESULTS: Exposure of the gingiva of DR1 mice to P. gingivalis results in a transient increase in the percentage of Th17 cells, both in peripheral blood and cervical lymph nodes, a burst of systemic cytokine activity, a loss in femoral bone density, and the generation of anti-citrullinated protein antibodies. Importantly, these antibodies are not produced in response to P. gingivalis treatment of wild-type C57BL/6 mice, and P. gingivalis exposure triggered expression of arthritis in arthritis-resistant mice. CONCLUSIONS: Exposure of gingival tissues to P. gingivalis has systemic effects that can result in disease pathology in tissues that are spatially removed from the initial site of infection, providing evidence for systemic effects of this periodontal pathogen. The elicitation of anti-citrullinated protein antibodies in an HLA-DR1-restricted fashion by mice exposed to P. gingivalis provides support for the role of the shared epitope in both periodontal disease and rheumatoid arthritis. The ability of P. gingivalis to induce disease expression in arthritis-resistant mice provides support for the idea that periodontal infection may be able to trigger autoimmunity if other disease-eliciting factors are already present

Saharan Dust Event Impacts on Cloud Formation and Radiation over Western Europe

We investigated the impact of mineral dust particles on clouds, radiation and atmospheric state during a strong Saharan dust event over Europe in May 2008, applying a comprehensive online-coupled regional model framework that explicitly treats particle-microphysics and chemical composition. Sophisticated parameterizations for aerosol activation and ice nucleation, together with two-moment cloud microphysics are used to calculate the interaction of the different particles with clouds depending on their physical and chemical properties. The impact of dust on cloud droplet number concentration was found to be low, with just a slight increase in cloud droplet number concentration for both uncoated and coated dust. For temperatures lower than the level of homogeneous freezing, no significant impact of dust on the number and mass concentration of ice crystals was found, though the concentration of frozen dust particles reached up to 100 l-1 during the ice nucleation events. Mineral dust particles were found to have the largest impact on clouds in a temperature range between freezing level and the level of homogeneous freezing, where they determined the number concentration of ice crystals due to efficient heterogeneous freezing of the dust particles and modified the glaciation of mixed phase clouds. Our simulations show that during the dust events, ice crystals concentrations were increased twofold in this temperature range (compared to if dust interactions are neglected). This had a significant impact on the cloud optical properties, causing a reduction in the incoming short-wave radiation at the surface up to -75Wm-2. Including the direct interaction of dust with radiation caused an additional reduction in the incoming short-wave radiation by 40 to 80Wm-2, and the incoming long-wave radiation at the surface was increased significantly in the order of +10Wm-2. The strong radiative forcings associated with dust caused a reduction in surface temperature in the order of -0.2 to -0.5K for most parts of France, Germany, and Italy during the dust event. The maximum difference in surface temperature was found in the East of France, the Benelux, and Western Germany with up to -1 K. This magnitude of temperature change was sufficient to explain a systematic bias in numerical weather forecasts during the period of the dust event

Implementation of a comprehensive ice crystal formation parameterization for cirrus and mixed-phase clouds in the EMAC model (based on MESSy 2.53)

A comprehensive ice nucleation parameterization has been implemented in the global chemistry-climate model EMAC to improve the representation of ice crystal number concentrations (ICNCs). The parameterization of Barahona and Nenes (2009, hereafter BN09) allows for the treatment of ice nucleation taking into account the competition for water vapour between homogeneous and heterogeneous nucleation in cirrus clouds. Furthermore, the influence of chemically heterogeneous, polydisperse aerosols is considered by applying one of the multiple ice nucleating particle parameterizations which are included in BN09 to compute the heterogeneously formed ice crystals. BN09 has been modified in order to consider the pre-existing ice crystal effect and implemented to operate both in the cirrus and in the mixed-phase regimes. Compared to the standard EMAC parameterizations, BN09 produces fewer ice crystals in the upper troposphere but higher ICNCs in the middle troposphere, especially in the Northern Hemisphere where ice nucleating mineral dust particles are relatively abundant. Overall, ICNCs agree well with the observations, especially in cold cirrus clouds (at temperatures below 205&thinsp;K), although they are underestimated between 200 and 220&thinsp;K. As BN09 takes into account processes which were previously neglected by the standard version of the model, it is recommended for future EMAC simulations.</p