Learning obstacle avoidance with an operant behavioral model

Artificial intelligence researchers have been attracted by the idea of having robots learn how to accomplish a task, rather than being told explicitly. Reinforcement learning has been proposed as an appealing framework to be used in controlling mobile agents. Robot learning research, as well as research in biological systems, face many similar problems in order to display high flexibility in performing a variety of tasks. In this work, the controlling of a vehicle in an avoidance task by a previously developed operant learning model (a form of animal learning) is studied. An environment in which a mobile robot with proximity sensors has to minimize the punishment for colliding against obstacles is simulated. The results were compared with the Q-Learning algorithm, and the proposed model had better performance. In this way a new artificial intelligence agent inspired by neurobiology, psychology, and ethology research is proposed.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingeniería.Instituto de Ingeniería Biomédica; Argentin

Cooperation in the iterated prisoner's dilemma is learned by operant conditioning mechanisms

The prisoner's dilemma (PD) is the leading metaphor for the evolution of cooperative behavior in populations of selfish agents. Although cooperation in the iterated prisoner's dilemma (IPD) has been studied for over twenty years, most of this research has been focused on strategies that involve nonlearned behavior. Another approach is to suppose that players' selection of the preferred reply might he enforced in the same way as feeding animals track the best way to feed in changing nonstationary environments. Learning mechanisms such as operant conditioning enable animals to acquire relevant characteristics of their environment in order to get reinforcements and to avoid punishments. In this study, the role of operant conditioning in the learning of cooperation was evaluated in the PD. We found that operant mechanisms allow the learning of IPD play against other strategies. When random moves are allowed in the game, the operant learning model showed low sensitivity. On the basis of this evidence, it is suggested that operant learning might be involved in reciprocal altruism.Fil: Gutnisky, D. A.. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; ArgentinaFil: Zanutto, Bonifacio Silvano. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina. Universidad de Buenos Aires. Facultad de Ingenieria. Instituto de Ingeniería Biomédica; Argentin

Incorporating flexibility in the long-term design of water distribution systems using operational variables

This work investigates the effect of operational variables on water distribution system design optimisation. The “Anytown” problem is approached with three formulations of the operational decision variables to examine how different models of such components affect the design solutions and the optimisation process. The formulations that jointly optimise operations and design decision variables can double the energy surplus for the same cost compared to a design-only formulation

Propellantless de orbiting of space debris by bare electrodynamic tethers

A 3-year Project started on November 1 2010, financed by the European Commision within the FP-7 Space Program, and aimed at developing an efficient de-orbit system that could be carried on board by future spacecraft launched into LEO, will be presented. The operational system will deploy a thin uninsulated tape-tether to collect electrons as a giant Langmuir probe, using no propellant/no power supply, and generating power on board. This project will involve free-fall tests, and laboratory hypervelocity-impact and tether-current tests, and design/Manufacturing of subsystems: interface elements, electric control and driving module, electron-ejecting plasma contactor, tether-deployment mechanism/end-mass, and tape samples. Preliminary results to be presented involve: i) devising criteria for sizing the three disparate tape dimensions, affecting mass, resistance, current-collection, magnetic self-field, and survivability against debris itself; ii) assessing the dynamical relevance of tether parameters in implementing control laws to limit oscillations in /off the orbital plane, where passive stability may be marginal; iii) deriving a law for bare-tape current from numerical simulations and chamber tests, taking into account ambient magnetic field, ion ram motion, and adiabatic electron trapping; iv) determining requirements on a year-dormant hollow cathode under long times/broad emission-range operation, and trading-off against use of electron thermal emission; v) determining requirements on magnetic components and power semiconductors for a control module that faces high voltage/power operation under mass/volume limitations; vi) assessing strategies to passively deploy a wide conductive tape that needs no retrieval, while avoiding jamming and ending at minimum libration; vii) evaluating the tape structure as regards conductive and dielectric materials, both lengthwise and in its cross-section, in particular to prevent arcing in triple-point junctions

An universal system to de-orbit satellites at end of life

A 3-year Project financed by the European Commission is aimed at developing a universal system to de-orbit satellites at their end of life, as a fundamental contribution to limit the increase of debris in the Space environment. The operational system involves a conductive tapetether left bare to establish anodic contact with the ambient plasma as a giant Langmuir probe. The Project will size the three disparate dimensions of a tape for a selected de-orbit mission and determine scaling laws to allow system design for a general mission. Starting at the second year, mission selection is carried out while developing numerical codes to implement control laws on tether dynamics in/off the orbital plane; performing numerical simulations and plasma chamber measurements on tether-plasma interaction; and completing design of subsystems: electronejecting plasma contactor, power module, interface elements, deployment mechanism, and tether-tape/end-mass. This will be followed by subsystems manufacturing and by currentcollection, free-fall, and hypervelocity impact tests

Pessimistic Rescaling and Distribution Shift of Boosting Models for Impression-Aware Online Advertising Recommendation

In this paper, we provide an overview of the approach we used as team Gabibboost for the ACM RecSys Challenge 2023, organized by ShareChat and Moj. The challenge focused on predicting user activity in the online advertising setting based on impression data, in particular, predicting whether a user would install an advertised application using a high-dimensional anonymized feature vector. Our proposed solution is based on an ensemble model that combines the strengths of several machine learning sub-models, including CatBoost, LightGBM, HistGradientBoosting, and two hybrid models. Our proposal is able to harness the strengths of our models through a distribution shift postprocessing and fine-Tune the final prediction via a custom build pessimistic rescaling function. The final ensemble model allowed us to rank 1st on the academic leaderboard and 9th overall