A Hierarchical Two-tier Approach to Hyper-parameter Optimization in Reinforcement Learning

Optimization of hyper-parameters in reinforcement learning (RL) algorithms is a key task, because they determine how the agent will learn its policy by interacting with its environment, and thus what data is gathered. In this work, an approach that uses Bayesian optimization to perform a two-step optimization is proposed: rst, categorical RL structure hyper-parameters are taken as binary variables and optimized with an acquisition function tailored for such variables. Then, at a lower level of abstraction, solution-level hyper-parameters are optimized by resorting to the expected improvement acquisition function, while using the best categorical hyper-parameters found in the optimization at the upper-level of abstraction. This two-tier approach is validated in a simulated control task. Results obtained are promising and open the way for more user-independent applications of reinforcement learning.Sociedad Argentina de Informática e Investigación Operativ

A Hierarchical Two-tier Approach to Hyper-parameter Optimization in Reinforcement Learning

Optimization of hyper-parameters in reinforcement learning (RL) algorithms is a key task, because they determine how the agent will learn its policy by interacting with its environment, and thus what data is gathered. In this work, an approach that uses Bayesian optimization to perform a two-step optimization is proposed: rst, categorical RL structure hyper-parameters are taken as binary variables and optimized with an acquisition function tailored for such variables. Then, at a lower level of abstraction, solution-level hyper-parameters are optimized by resorting to the expected improvement acquisition function, while using the best categorical hyper-parameters found in the optimization at the upper-level of abstraction. This two-tier approach is validated in a simulated control task. Results obtained are promising and open the way for more user-independent applications of reinforcement learning.Sociedad Argentina de Informática e Investigación Operativ

Generating rescheduling knowledge using reinforcement learning in a cognitive architecture

In order to reach higher degrees of flexibility, adaptability and autonomy in manufacturing systems, it is essential to develop new rescheduling methodologies which resort to cognitive capabilities, similar to those found in human beings. Artificial cognition is important for designing planning and control systems that generate and represent knowledge about heuristics for repairbased scheduling. Rescheduling knowledge in the form of decision rules is used to deal with unforeseen events and disturbances reactively in real time, and take advantage of the ability to act interactively with the user to counteract the effects of disruptions. In this work, to achieve the aforementioned goals, a novel approach to generate rescheduling knowledge in the form of dynamic first-order logical rules is proposed. The proposed approach is based on the integration of reinforcement learning with artificial cognitive capabilities involving perception and reasoning/learning skills embedded in the Soar cognitive architecture. An industrial example is discussed showing that the approach enables the scheduling system to assess its operational range in an autonomic way, and to acquire experience through intensive simulation while performing repair tasks.Sociedad Argentina de Informática e Investigación Operativa (SADIO

Generating rescheduling knowledge using reinforcement learning in a cognitive architecture

In order to reach higher degrees of flexibility, adaptability and autonomy in manufacturing systems, it is essential to develop new rescheduling methodologies which resort to cognitive capabilities, similar to those found in human beings. Artificial cognition is important for designing planning and control systems that generate and represent knowledge about heuristics for repairbased scheduling. Rescheduling knowledge in the form of decision rules is used to deal with unforeseen events and disturbances reactively in real time, and take advantage of the ability to act interactively with the user to counteract the effects of disruptions. In this work, to achieve the aforementioned goals, a novel approach to generate rescheduling knowledge in the form of dynamic first-order logical rules is proposed. The proposed approach is based on the integration of reinforcement learning with artificial cognitive capabilities involving perception and reasoning/learning skills embedded in the Soar cognitive architecture. An industrial example is discussed showing that the approach enables the scheduling system to assess its operational range in an autonomic way, and to acquire experience through intensive simulation while performing repair tasks.Sociedad Argentina de Informática e Investigación Operativa (SADIO

Reseña del libro “el sujeto según Lacan”. Autor: Guy le Gaufey. Editado por Ediciones Literales. Buenos Aires. 2010- 160 págs.

RESEÑA DEL LIBRO “EL SUJETO SEGÚN LACAN”. AUTOR: GUY LE GAUFEY. EDITADO POR EDICIONES LITERALES. BUENOS AIRES. 2010-160 PÁGS

"El notodo de Lacan. Consistencia lógica, consecuencias clínicas”. Autor: Guy le Gaufey. Buenos aires: El cuenco de plata. Ediciones literales, 2007- 224 pág.

Reseña del libro “El notodo de Lacan. Consistencia lógica, consecuencias clínicas”. Autor: Guy le Gaufey. Buenos aires: El cuenco de plata. Ediciones literales, 2007- 224 pág

A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning

Optimization of hyper-parameters in real-world applications of reinforcement learning (RL) is a key issue, because their settings determine how fast the agent will learn its policy by interacting with its environment due to the information content of data gathered. In this work, an approach that uses Bayesian optimization to perform an autonomous two-tier optimization of both representation decisions and algorithm hyper-parameters is proposed: first, categorical / structural RL hyper-parameters are taken as binary variables and optimized with an acquisition function tailored for such type of variables. Then, at a lower level of abstraction, solution-level hyper-parameters are optimized by resorting to the expected improvement acquisition function, whereas the categorical hyper-parameters found in the optimization at the upperlevelof abstraction are fixed. This two-tier approach is validated with a tabular and neural network setting of the value function, in a classic simulated control task. Results obtained are promising and open the way for more user-independent applications of reinforcement learning.Fil: Barsce, Juan Cruz. Universidad Tecnologica Nacional. Facultad Regional Villa Maria; ArgentinaFil: Palombarini, Jorge Andrés. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Centro de Investigaciones y Transferencia de Villa María. Universidad Nacional de Villa María. Centro de Investigaciones y Transferencia de Villa María; Argentina. Universidad Tecnologica Nacional. Facultad Regional Villa Maria; ArgentinaFil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; Argentin

Avian sibling cannibalism: Hoopoe mothers regularly use their last hatched nestlings to feed older siblings

DATA AVAILABILITY Data used in this paper can be found at https://figshare.com/s/55775bd4f5e31a464fd2ACKNOWLEDGMENTS We thank Carmen Zamora, Manuel Soler Cruz, Juan Moreno Klemming, Anders Pape Møller, and Graham Tebb for discussion and comments on a preliminary version of the manuscript. Alberto Ruiz Moreno and Enrique Cortés Sánchez helped with the recording equipment. Natalia Juárez García Pelayo and Lola Barón helped with fieldwork. The Spanish research group was supported by the Spanish Ministerio de Ciencia, Innovación y Universidades and European (FEDER) funds (CGL2017-83 103-P), and from facilities, including an apartment, provided by the city authorities of Guadix, where a small lab to process samples quickly was installed. Finally, we would like to thank Renate Hengsberger for revising earlier drafts.Sibling cannibalism is relatively common in nature, but its evolution in birds and certain other vertebrates with extended parental care had been discarded. Here, however, we demonstrate its regular occurrence in two European populations of the Eurasian hoopoe (Upupa epops) and explore possible adaptive and non-adaptive explanations. Results showed that sibling cannibalism was more frequently detected in Spain (51.7%) than in Austria (5.9%). In these two populations, the hoopoes laid similar clutch sizes, resulting in similar fledging production, but hatching failures were more frequent in the northern population. Consequently, having more nestlings condemned to die in the southern population may explain the higher incidence of sibling cannibalism. In accordance with this interpretation, hatching span and failure, but not breeding date, explained the probability of sibling cannibalism in the Spanish hoopoes, while all three variables predicted brood reduction intensity. Furthermore, experimental food supply reduced the probability of sibling cannibalism, but not the intensity of brood reduction. Finally, females allocated fewer resources to the smallest nestlings when they were going to starve, but not necessarily when they were going to be used as food for their siblings. These results suggest that hoopoes produce extra eggs that, in the case of reduced hatching failure and food scarcity, produce nestlings that are used to feed older siblings. These findings provide the first evidence that sibling cannibalism occurs regularly in a bird species, thus expanding our evolutionary understanding of clutch size, hatching asynchrony, parent-offspring conflict, infanticide, and sibling cannibalism in the animal kingdom.Spanish Ministerio de Ciencia, Innovación y Universidades and European (FEDER) Funds (CGL2017-83103-P