Visual Rationalizations in Deep Reinforcement Learning for Atari Games

Due to the capability of deep learning to perform well in high dimensional problems, deep reinforcement learning agents perform well in challenging tasks such as Atari 2600 games. However, clearly explaining why a certain action is taken by the agent can be as important as the decision itself. Deep reinforcement learning models, as other deep learning models, tend to be opaque in their decision-making process. In this work, we propose to make deep reinforcement learning more transparent by visualizing the evidence on which the agent bases its decision. In this work, we emphasize the importance of producing a justification for an observed action, which could be applied to a black-box decision agent.Comment: presented as oral talk at BNAIC 201

A buffer gas beam source for short, intense and slow molecular pulses

Experiments with cold molecules usually begin with a molecular source. We describe the construction and characteristics of a cryogenic buff er gas source of CaF molecules. The source emits pulses with a typical duration of 240 μs, a mean speed of about 150 m/s, and a flux of 5x 10¹⁰ molecules per steradian per pulse in a single rotational state

Computational and Mathematical Modelling of the EGF Receptor System

This chapter gives an overview of computational and mathematical modelling of the EGF receptor system. It begins with a survey of motivations for producing such models, then describes the main approaches that are taken to carrying out such modelling, viz. differential equations and individual-based modelling. Finally, a number of projects that applying modelling and simulation techniques to various aspects of the EGF receptor system are described

Impact of facial conformation on canine health: Brachycephalic Obstructive Airway Syndrome

The domestic dog may be the most morphologically diverse terrestrial mammalian species known to man; pedigree dogs are artificially selected for extreme aesthetics dictated by formal Breed Standards, and breed-related disorders linked to conformation are ubiquitous and diverse. Brachycephaly–foreshortening of the facial skeleton–is a discrete mutation that has been selected for in many popular dog breeds e.g. the Bulldog, Pug, and French Bulldog. A chronic, debilitating respiratory syndrome, whereby soft tissue blocks the airways, predominantly affects dogs with this conformation, and thus is labelled Brachycephalic Obstructive Airway Syndrome (BOAS). Despite the name of the syndrome, scientific evidence quantitatively linking brachycephaly with BOAS is lacking, but it could aid efforts to select for healthier conformations. Here we show, in (1) an exploratory study of 700 dogs of diverse breeds and conformations, and (2) a confirmatory study of 154 brachycephalic dogs, that BOAS risk increases sharply in a non-linear manner as relative muzzle length shortens. BOAS only occurred in dogs whose muzzles comprised less than half their cranial lengths. Thicker neck girths also increased BOAS risk in both populations: a risk factor for human sleep apnoea and not previously realised in dogs; and obesity was found to further increase BOAS risk. This study provides evidence that breeding for brachycephaly leads to an increased risk of BOAS in dogs, with risk increasing as the morphology becomes more exaggerated. As such, dog breeders and buyers should be aware of this risk when selecting dogs, and breeding organisations should actively discourage exaggeration of this high-risk conformation in breed standards and the show ring

Recent Salmon Declines: A Result of Lost Feeding Opportunities Due to Bad Timing?

As the timing of spring productivity blooms in near-shore areas advances due to warming trends in global climate, the selection pressures on out-migrating salmon smolts are shifting. Species and stocks that leave natal streams earlier may be favoured over later-migrating fish. The low post-release survival of hatchery fish during recent years may be in part due to static release times that do not take the timing of plankton blooms into account. This study examined the effects of release time on the migratory behaviour and survival of wild and hatchery-reared coho salmon (Oncorhynchus kisutch) using acoustic and coded-wire telemetry. Plankton monitoring and near-shore seining were also conducted to determine which habitat and food sources were favoured. Acoustic tags (n = 140) and coded-wire tags (n = 266,692) were implanted into coho salmon smolts at the Seymour and Quinsam Rivers, in British Columbia, Canada. Differences between wild and hatchery fish, and early and late releases were examined during the entire lifecycle. Physiological sampling was also carried out on 30 fish from each release group. The smolt-to-adult survival of coho salmon released during periods of high marine productivity was 1.5- to 3-fold greater than those released both before and after, and the fish's degree of smoltification affected their downstream migration time and duration of stay in the estuary. Therefore, hatchery managers should consider having smolts fully developed and ready for release during the peak of the near-shore plankton blooms. Monitoring chlorophyll a levels and water temperature early in the spring could provide a forecast of the timing of these blooms, giving hatcheries time to adjust their release schedule

Inducible Ablation of Melanopsin-Expressing Retinal Ganglion Cells Reveals Their Central Role in Non-Image Forming Visual Responses

Rod/cone photoreceptors of the outer retina and the melanopsin-expressing retinal ganglion cells (mRGCs) of the inner retina mediate non-image forming visual responses including entrainment of the circadian clock to the ambient light, the pupillary light reflex (PLR), and light modulation of activity. Targeted deletion of the melanopsin gene attenuates these adaptive responses with no apparent change in the development and morphology of the mRGCs. Comprehensive identification of mRGCs and knowledge of their specific roles in image-forming and non-image forming photoresponses are currently lacking. We used a Cre-dependent GFP expression strategy in mice to genetically label the mRGCs. This revealed that only a subset of mRGCs express enough immunocytochemically detectable levels of melanopsin. We also used a Cre-inducible diphtheria toxin receptor (iDTR) expression approach to express the DTR in mRGCs. mRGCs develop normally, but can be acutely ablated upon diphtheria toxin administration. The mRGC-ablated mice exhibited normal outer retinal function. However, they completely lacked non-image forming visual responses such as circadian photoentrainment, light modulation of activity, and PLR. These results point to the mRGCs as the site of functional integration of the rod/cone and melanopsin phototransduction pathways and as the primary anatomical site for the divergence of image-forming and non-image forming photoresponses in mammals

Impact of early applied upper limb stimulation: The EXPLICIT-stroke programme design

Main claims of the literature are that functional recovery of the paretic upper limb is mainly defined within the first month post stroke and that rehabilitation services should preferably be applied intensively and in a task-oriented way within this particular time window. EXplaining PLastICITy after stroke (acronym EXPLICIT-stroke) aims to explore the underlying mechanisms of post stroke upper limb recovery. Two randomized single blinded trials form the core of the programme, investigating the effects of early modified Constraint-Induced Movement Therapy (modified CIMT) and EMG-triggered Neuro-Muscular Stimulation (EMG-NMS) in patients with respectively a favourable or poor probability for recovery of dexterity.BioMechanical EngineeringMechanical, Maritime and Materials Engineerin

Characterization of primary neurospheres generated from mouse ventral rostral hindbrain

Serotonergic (5-HT) neurons of the reticular formation play a key role in the modulation of behavior, and their dysfunction is associated with severe neurological and psychiatric disorders, such as depression and schizophrenia. However, the molecular mechanisms underlying the differentiation of the progenitor cells and the specification of the 5-HT phenotype are not fully understood. A primary neurosphere cell-culture system from mouse ventral rostral hindbrain at embryonic day 12 was therefore established. The generated primary neurospheres comprised progenitor cells and fully differentiated neurons. Bromodeoxyuridine incorporation experiments in combination with immunocytochemistry for neural markers revealed the proliferation capacity of the neural multipotent hindbrain progenitors within neurospheres and their ability to differentiate toward the neuronal lineage and serotonergic phenotype. Gene expression analysis by reverse transcription with the polymerase chain reaction showed that the neurospheres were regionally specified, as reflected by the expression of the transcription factors Gata2 and Pet1. Treatment of dissociated primary neurospheres with exogenous Shh significantly increased the number of 5-HT-immunopositive cells compared with controls, whereas neutralization of endogenous Shh significantly decreased the number of 5-HT neurons. Thus, the primary neurosphere culture system presented here allows the expansion of hindbrain progenitor cells and the experimental control of their differentiation toward the serotonergic phenotype. This culture system is therefore a useful model for in vitro studies dealing with the development of 5-HT neurons

A neural tracking and motor control approach to improve rehabilitation of upper limb movements

<p>Abstract</p> <p>Background</p> <p>Restoration of upper limb movements in subjects recovering from stroke is an essential keystone in rehabilitative practices. Rehabilitation of arm movements, in fact, is usually a far more difficult one as compared to that of lower extremities. For these reasons, researchers are developing new methods and technologies so that the rehabilitative process could be more accurate, rapid and easily accepted by the patient. This paper introduces the proof of concept for a new non-invasive FES-assisted rehabilitation system for the upper limb, called smartFES (sFES), where the electrical stimulation is controlled by a biologically inspired neural inverse dynamics model, fed by the kinematic information associated with the execution of a planar goal-oriented movement. More specifically, this work details two steps of the proposed system: an <it>ad hoc </it>markerless motion analysis algorithm for the estimation of kinematics, and a neural controller that drives a synthetic arm. The vision of the entire system is to acquire kinematics from the analysis of video sequences during planar arm movements and to use it together with a neural inverse dynamics model able to provide the patient with the electrical stimulation patterns needed to perform the movement with the assisted limb.</p> <p>Methods</p> <p>The markerless motion tracking system aims at localizing and monitoring the arm movement by tracking its silhouette. It uses a specifically designed motion estimation method, that we named Neural Snakes, which predicts the arm contour deformation as a first step for a silhouette extraction algorithm. The starting and ending points of the arm movement feed an Artificial Neural Controller, enclosing the muscular Hill's model, which solves the inverse dynamics to obtain the FES patterns needed to move a simulated arm from the starting point to the desired point. Both position error with respect to the requested arm trajectory and comparison between curvature factors have been calculated in order to determine the accuracy of the system.</p> <p>Results</p> <p>The proposed method has been tested on real data acquired during the execution of planar goal-oriented arm movements. Main results concern the capability of the system to accurately recreate the movement task by providing a synthetic arm model with the stimulation patterns estimated by the inverse dynamics model. In the simulation of movements with a length of ± 20 cm, the model has shown an unbiased angular error, and a mean (absolute) position error of about 1.5 cm, thus confirming the ability of the system to reliably drive the model to the desired targets. Moreover, the curvature factors of the factual human movements and of the reconstructed ones are similar, thus encouraging future developments of the system in terms of reproducibility of the desired movements.</p> <p>Conclusion</p> <p>A novel FES-assisted rehabilitation system for the upper limb is presented and two parts of it have been designed and tested. The system includes a markerless motion estimation algorithm, and a biologically inspired neural controller that drives a biomechanical arm model and provides the stimulation patterns that, in a future development, could be used to drive a smart Functional Electrical Stimulation system (sFES). The system is envisioned to help in the rehabilitation of post stroke hemiparetic patients, by assisting the movement of the paretic upper limb, once trained with a set of movements performed by the therapist or in virtual reality. Future work will include the application and testing of the stimulation patterns in real conditions.</p

Kinesin expands and stabilizes the GDP-microtubule lattice

Kinesin-1 is a nanoscale molecular motor that walks towards the fast-growing (plus) ends of microtubules, hauling molecular cargo to specific reaction sites in cells. Kinesin-driven transport is central to the self-organization of eukaryotic cells and shows great promise as a tool for nano-engineering1. Recent work hints that kinesin may also play a role in modulating the stability of its microtubule track, both in vitro2,3 and in vivo4, but the results are conflicting5,6,7 and the mechanisms are unclear. Here, we report a new dimension to the kinesin–microtubule interaction, whereby strong-binding state (adenosine triphosphate (ATP)-bound and apo) kinesin-1 motor domains inhibit the shrinkage of guanosine diphosphate (GDP) microtubules by up to two orders of magnitude and expand their lattice spacing by ~1.6%. Our data reveal an unexpected mechanism by which the mechanochemical cycles of kinesin and tubulin interlock, and so allow motile kinesins to influence the structure, stability and mechanics of their microtubule track