A system-level neural model of the brain mechanisms underlying instrumental devaluation in rats

Goal-directed behaviours are defined by the presence of two kinds of effect on instrumental learning. First, degrading the contingencies between produced actions and desired outcomes diminishes the number of instrumental responses; second, devaluing a reward results in a lower production of instrumental actions to obtain it. We present a computational model of the neural processes underlying instrumental devaluation in rats. The model reproduces the interaction between the basolateral complex of the amygdala (BLA) and the limbic, associative and somatosensory striato-cortical loops. Firing-rate units are used to abstract the activity features of neural populations. Learning is reproduced through the use of dopamine-dependent simple and differential hebbian rules. Constraints from anatomy of the projections between neural systems are taken into account. The central hypothesis implemented in the model is that pavlovian associations learned within the BLA between manipulanda and rewards modulate goal selection through the activation of the nucleus accumbens core (NaccCo). Selection processes happening in the limbic basal ganglia with the activation of the NaccCo decide which outcome is choosen as a goal within the prelimbic cortex (PL). Connections between the BLA and the NaccCo are learned through hebbian associations mediated by feedbacks from the PL to the NaccCo. Information about selected goals from the limbic striato-cortical loop influences action selection in the sensorimotor loop both through cortico-cortical projections and through a striato-nigro-striatal dopaminergic pathway passing through the associative striato-cortical loop. The model is tested as part of the control system of a simulated rat. Instrumental devaluation tasks are reproduced. Simulated lesions of the BLA, the NaccCo, the PL and the dorsomedial striatum (DMS) both before and after training reproduce the behavioural effect of lesions in real rats. The model provides predictions about the effects of still undocumented lesions

Forward and bidirectional planning based on reinforcement learning and neural networks in a simulated robot.

Building intelligent systems that are capable of learning, acting reactively and planning actions before their execution is a major goal of artificial intelligence. This paper presents two reactive and planning systems that contain important novelties with respect to previous neural-network planners and reinforcement- learning based planners: (a) the introduction of a new component (?matcher?) allows both planners to execute genuine taskable planning (while previous reinforcement-learning based models have used planning only for speeding up learning); (b) the planners show for the first time that trained neural- network models of the world can generate long prediction chains that have an interesting robustness with regards to noise; (c) two novel algorithms that generate chains of predictions in order to plan, and control the flows of information between the systems? different neural components, are presented; (d) one of the planners uses backward ?predictions? to exploit the knowledge of the pursued goal; (e) the two systems presented nicely integrate reactive behavior and planning on the basis of a measure of ?confidence? in action. The soundness and potentialities of the two reactive and planning systems are tested and compared with a simulated robot engaged in a stochastic path-finding task. The paper also presents an extensive literature review on the relevant issues

Self-Organization as Phase Transition in Decentralized Groups of Robots: A Study Based on Boltzmann Entropy

No abstract availabl

Reactor Designs for Safe and Intensified Hydrogenations and Oxidations: From Micro- to Membrane Reactors

The current and pressing environmental challenges are leading towards an important paradigm shift within the chemical industry. Green chemistry can be performed by using selective catalysts, and renewable and environment-friendly feedstock. For this reason, it is essential to provide scientists with platform tools that can allow safe and reliable catalyst testing, screening and studies. At the same time, as the use of green feedstock, such as oxygen and hydrogen, can pose new hazards, the design of intensified reactors can represent an unmissable opportunity to drive this green shift within the safe and scalable production of valuable molecules. This thesis reports reactor design solutions of different scale that have been devised to guarantee safe and intensified catalytic hydrogenations and oxidations for catalyst testing and continuous production purposes. Starting with the aim of studying a catalyst under realistic operating conditions, a silicon microfabricated reactor was designed and tested for the gas phase combustion of methane and carbon monoxide over palladium and platinum catalysts. Owing to its small volume and to its isothermal temperature profile, this microreactor proved to be a safe and effective tool for performing information-rich experiments, while exhibiting a plug-flow behaviour with negligible external and internal mass transfer resistances. Reactions were performed in combination with X-ray absorption and IR spectroscopy, allowed by the detailed microfabrication reactor design, to investigate the catalyst structure-activity relationships in steady-state and transient experiments. Boosting the catalyst activity can be achieved using catalytic nanoparticles, which offer an increased surface area compared to their bulk equivalents and hence an improved reaction rate. However, accessibility of the reactants to supported nanoparticles can be limited by the diffusion phenomena occurring around and inside a catalyst support. A recent trend of supporting nanoparticles onto surfaces modified using polyelectrolyte assemblies has attracted attention owing to the low temperature, ease and environmentally friendly preparation process. Finely tuned ex situ synthesised palladium nanoparticles were adsorbed on the inner surface of a tubular Teflon AF-2400 membrane, which was modified with polyelectrolytes in a layer-by-layer configuration. The membrane was used as a tubular reactor inside an outer tube with pressurised hydrogen, and nanoparticles of different size and shape were tested in the continuous hydrogenation of nitrobenzene to aniline. The observed reactivity depended on the different nanoparticle size and on the palladium oxidation state. The use of a tube-in-tube membrane reactor ensured process safety owing to the small volume of gas stored in the tube annular section and to the continuous processing. Alcohol oxidations using molecular oxygen can be dangerous due to the risk of creating explosive mixtures with the organic substrate. Two slurry loop reactors were developed using the same Teflon AF-2400 membrane in different configurations: a tube-in-tube and a flat membrane configuration for scalable reactions. These were designed and tested to carry out safe aerobic oxidation of alcohols. The membrane separated the oxygen from the organic phase and allowed a controlled dosing of the gaseous reactant. In order to boost the turnover frequency, the catalyst was used in the form of a slurry which was recirculated in a loop where it contacted the membrane saturator and a crossflow filter. This allowed the withdrawal of the liquid products from the loop. The reactors could be operated continuously, and provided improved process safety and comparable catalyst turnover frequency to conventional batch processes. When scaling up reactors, inadequate mixing can occur impacting on process safety and product quality. A Taylor-vortex membrane reactor is presented for the first time, combining the benefits of a flexible baffle structure inside a Taylor-vortex system that can hinder axial dispersion, and a supported tubular membrane for safe gas-liquid reactions. Stable conversion and product selectivity were achieved in the homogeneously catalysed continuous aerobic oxidation of benzyl alcohol. No pervaporation of organics through the membrane was detected during reaction, making this reactor a safe and a scalable tool for continuous gas-liquid reactions

Evolution of optical properties of chromium spinels CdCr2_2O4_4, HgCr2_2S4_4, and ZnCr2_2Se4_4 under high pressure

We report pressure-dependent reflection and transmission measurements on ZnCr$_2$Se$_4$, HgCr$_2$S$_4$, and CdCr$_2$O$_4$ single crystals at room temperature over a broad spectral range 200-24000 cm$^{-1}$. The pressure dependence of the phonon modes and the high-frequency electronic excitations indicates that all three compounds undergo a pressure-induced structural phase transition with the critical pressure 15 GPa, 12 GPa, and 10 GPa for CdCr$_2$O$_4$, HgCr$_2$S$_4$, and ZnCr$_2$Se$_4$, respectively. The eigenfrequencies of the electronic transitions are very close to the expected values for chromium crystal-field transitions. In the case of the chalcogenides pressure induces a red shift of the electronic excitation which indicates a strong hybridization of the Cr d-bands with the chalcogenide bands.Comment: Accepted for publication in Phys. Rev.