Learning obstacle avoidance with an operant behavioral model

Artificial intelligence researchers have been attracted by the idea of having robots learn how to accomplish a task, rather than being told explicitly. Reinforcement learning has been proposed as an appealing framework to be used in controlling mobile agents. Robot learning research, as well as research in biological systems, face many similar problems in order to display high flexibility in performing a variety of tasks. In this work, the controlling of a vehicle in an avoidance task by a previously developed operant learning model (a form of animal learning) is studied. An environment in which a mobile robot with proximity sensors has to minimize the punishment for colliding against obstacles is simulated. The results were compared with the Q-Learning algorithm, and the proposed model had better performance. In this way a new artificial intelligence agent inspired by neurobiology, psychology, and ethology research is proposed.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; Argentin

Cooperation in the iterated prisoner's dilemma is learned by operant conditioning mechanisms

The prisoner's dilemma (PD) is the leading metaphor for the evolution of cooperative behavior in populations of selfish agents. Although cooperation in the iterated prisoner's dilemma (IPD) has been studied for over twenty years, most of this research has been focused on strategies that involve nonlearned behavior. Another approach is to suppose that players' selection of the preferred reply might he enforced in the same way as feeding animals track the best way to feed in changing nonstationary environments. Learning mechanisms such as operant conditioning enable animals to acquire relevant characteristics of their environment in order to get reinforcements and to avoid punishments. In this study, the role of operant conditioning in the learning of cooperation was evaluated in the PD. We found that operant mechanisms allow the learning of IPD play against other strategies. When random moves are allowed in the game, the operant learning model showed low sensitivity. On the basis of this evidence, it is suggested that operant learning might be involved in reciprocal altruism.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentin

Comment on the paper by Y.Komura and Y.Okabe [arXiv:1011.3321]

We point out that the claim of strong universality in the paper J.Phys. A 44, 015002, arXiv:1011.3321 is incorrect, as it contradicts known rigorous results.Comment: submitted to J.Phys.

Co-digestion of macroalgae for biogas production: an LCA-based environmental evaluation

Algae represent a favourable and potentially sustainable source of biomass for bioenergy-based industrial pathways in the future. The study, performed on a real pilot plant implemented in Augusta (Italy) within the frame of the BioWALK4Biofuels project, aims to figure out whether seaweed (macroalgae) cultivated in near-shore open ponds could be considered a beneficial aspect as a source of biomass for biogas production within the co-digestion with local agricultural biological waste. The LCA results confirm that the analysed A and B scenarios (namely the algae-based co-digestion scenario and agricultural mix feedstock scenario) present an environmental performance more favourable than that achieved with conventional non-renewable-based technologies (specifically natural gas - Scenario C). Results show that the use of seaweed (Scenario A) represent a feasible solution in order to replace classical biomass used for biofuel production from a land-based feedstock. The improvement of the environmental performances is quantifiable on 10% respect to Scenario B, and 38 times higher than Scenario

Heavy-to-light form factors: sum rules on the light cone and beyond

We report the first systematic analysis of the off-light-cone effects in sum rules for heavy-to-light form factors. These effects are investigated in a model based on scalar constituents, which allows a technically rather simple analysis but has the essential features of the analogous QCD calculation. The correlator relevant for the extraction of the heavy-to-light form factor is calculated in two different ways: first, by adopting the full Bethe-Salpeter amplitude of the light meson and, second, by performing the expansion of this amplitude near the light cone $x^2=0$. We demonstrate that the contributions to the correlator from the light-cone term $x^2=0$ and the off-light-cone terms $x^2\ne 0$ have the same order in the $1/m_Q$ expansion. The light-cone correlator, corresponding to $x^2=0$, is shown to systematically overestimate the full correlator, the difference being $\sim \Lambda_{\rm QCD}/\delta$, with $\delta$ the continuum subtraction parameter of order 1 GeV. Numerically, this difference is found to be 10-20%.Comment: revtex 14 pages, version to be published in Phys. Rev. D (discussion in Sect. 3 extended, example in Sect. 4 added

A Power-Efficient Methodology for Mapping Applications on Multi-Processor System-on-Chip Architectures

This work introduces an application mapping methodology and case study for multi-processor on-chip architectures. Starting from the description of an application in standard sequential code (e.g. in C), first the application is profiled, parallelized when possible, then its components are moved to hardware implementation when necessary to satisfy performance and power constraints. After mapping, with the use of hardware objects to handle concurrency, the application power consumption can be further optimized by a task-based scheduler for the remaining software part, without the need for operating system support. The key contributions of this work are: a methodology for high-level hardware/software partitioning that allows the designer to use the same code for both hardware and software models for simulation, providing nevertheless preliminary estimations for timing and power consumption; and a task-based scheduling algorithm that does not require operating system support. The methodology has been applied to the co-exploration of an industrial case study: an MPEG4 VGA real-time encoder

Thermodynamic formalism for dissipative quantum walks

We consider the dynamical properties of dissipative continuous-time quantum walks on directed graphs. Using a large-deviation approach we construct a thermodynamic formalism allowing us to define a dynamical order parameter, and to identify transitions between dynamical regimes. For a particular class of dissipative quantum walks we propose a quantum generalization of the the classical PageRank vector, used to rank the importance of nodes in a directed graph. We also provide an example where one can characterize the dynamical transition from an effective classical random walk to a dissipative quantum walk as a thermodynamic crossover between distinct dynamical regimes.Comment: 8 page

Icebergs and sea ice detected with inverted echo sounders

Author Posting. © American Meteorological Society, 2015. This article is posted here by permission of American Meteorological Society for personal use, not for redistribution. The definitive version was published in Journal of Atmospheric and Oceanic Technology 32 (2015): 1042–1057, doi:10.1175/JTECH-D-14-00161.1.A 1-yr experiment using a pressure-sensor-equipped inverted echo sounder (PIES) was conducted in Sermilik Fjord in southeastern Greenland (66°N, 38°E) from August 2011 to September 2012. Based on these high-latitude data, the interpretation of PIESs’ acoustic travel-time records from regions that are periodically ice covered were refined. In addition, new methods using PIESs for detecting icebergs and sea ice and for estimating iceberg drafts and drift speeds were developed and tested. During winter months, the PIES in Sermilik Fjord logged about 300 iceberg detections and recorded a 2-week period in early March of land-fast ice cover over the instrument site, consistent with satellite synthetic aperture radar (SAR) imagery. The deepest icebergs in the fjord were found to have keel depths greater than approximately 350 m. Average and maximum iceberg speeds were approximately 0.2 and 0.5 m s−1, respectively. The maximum tidal range at the site was ±1.8 m and during neap tides the range was ±0.3 m, as shown by the PIES’s pressure record.This work was supported by the National Science Foundation through the Divisions of Ocean Science and Polar Programs under Grant PLR-1332911. A. Silvano was supported as a WHOI guest student through a Gori Fellowship