Reinforcement learning approaches to the analysis of the emergence of goal-directed behaviour

Over recent decades, theoretical neuroscience, helped by computational methods such as Reinforcement Learning (RL), has provided detailed descriptions of the psychology and neurobiology of decision-making. RL has provided many insights into the mechanisms underlying decision-making processes from neuronal to behavioral levels. In this work, we attempt to demonstrate the effectiveness of RL methods in explaining behavior in a normative setting through three main case studies. Evidence from literature shows that, apart from the commonly discussed cognitive search process, that governs the solution procedure of a planning task, there is an online perceptual process that directs the action selection towards moves that appear more ‘natural’ at a given configuration of a task. These two processes can be partially dissociated through developmental studies, with perceptual processes apparently more dominant in the planning of younger children, prior to the maturation of executive functions required for the control of search. Therefore, we present a formalization of planning processes to account for perceptual features of the task, and relate it to human data. Although young children are able to demonstrate their preferences by using physical actions, infants are restricted because of their as-yet-undeveloped motor skills. Eye-tracking methods have been employed to tackle this difficulty. Exploring different model-free RL algorithms and their possible cognitive realizations in decision making, in a second case study, we demonstrate behavioral signatures of decision making processes in eye-movement data and provide a potential framework for integrating eye-movement patterns with behavioral patterns. Finally, in a third project we examine how uncertainty in choices might guide exploration in 10-year-olds, using an abstract RL-based mathematical model. Throughout, aspects of action selection are seen as emerging from the RL computational framework. We, thus, conclude that computational descriptions of the developing decision making functions provide one plausible avenue by which to normatively characterize and define the functions that control action selection

Clay Figurines from Lousoi: Some Thoughts on Local Production: The Ear is Shaky

Papers from the third international seminar on Ancient Arcadia, held at the Norwegian Institute at Athens, 7-10 May 2002The excavations at the sanctuary of Artemis at Lousoi have brought to light a number of Geometric and archaic clay statuettes representing different types and styles, some handmade, others mouldmade. Some of the statuettes can be compared with and attributed to the Corinthian repertoire. Others coincide with known types from various Peloponnesian centers. But there are some types for which it seems difficult to find parallels. Certain characteristic details lead to the hypothesis that they may have been locally produced. In this paper some statuettes are presented and the details leading to this conclusion are discussed. In addition to this, further indications for local production in Hellenistic times are added

Mathematical modeling of the flow field in fish-passes

Εθνικό Μετσόβιο Πολυτεχνείο--Μεταπτυχιακή Εργασία. Διεπιστημονικό-Διατμηματικό Πρόγραμμα Μεταπτυχιακών Σπουδών (Δ.Π.Μ.Σ.) “Υπολογιστική Μηχανική

Protective Effects of Liposomal N-Acetylcysteine against Paraquat-Induced Cytotoxicity and Gene Expression

Paraquat (PQ) is a herbicide that preferentially accumulates in the lung and exerts its cytotoxicity via the generation of reactive oxygen species (ROS). There is no specific treatment for paraquat poisoning. Attempts have been made to increase the antioxidant status in the lung using antioxidants (e.g., superoxide dismutase, vitamin E, N-acetylcysteine) but the outcome from such treatments is limited. Encapsulation of antioxidants in liposomes improves their therapeutic potential against oxidant-induced lung damage because liposomes facilitate intracellular delivery and prolong the retention of entrapped agents inside the cell. In the present study, we compared the effectiveness of conventional N-acetylcysteine (NAC) and liposomal-NAC (L-NAC) against PQ-induced cytotoxicity and examined the mechanism(s) by which these antioxidant formulations conferred cytoprotection. The effects of NAC or L-NAC against PQ-induced cytotoxicity in A549 cells were assessed by measuring cellular PQ uptake, intracellular glutathione content, ROS levels, mitochondrial membrane potential, cellular gene expression, inflammatory cytokine release and cell viability. Pretreatment of cells with L-NAC was significantly more effective than pretreatment with the conventional drug in reducing PQ-induced cytotoxicity, as indicated by the biomarkers used in this study. Our results suggested that the delivery of NAC as a liposomal formulation improves its effectiveness in counteracting PQ-induced cytotoxicity

Protecting Inshore and Demersal Finfish Identification of Critical Habitats for Juvenile Dhufish Workshop Report

A Juvenile Dhufish Workshop was held at the Western Australian Fisheries and Marine Research Laboratories (WAFMRL) on 18th March 2010, which was the first component of a State Natural Resource Management (NRM) funded project entitled “Protecting Inshore and Demersal Finfish - Identification of Critical Habitats for Juvenile Dhufish”. The primary objective of the workshop was to provide a forum for discussion in order to compile all biological, ecological and anecdotal information on the early life history of dhufish, to create hypotheses of habitat requirements for juvenile dhufish and provide recommendations as to which sampling methods should be used to confirm the presence of juveniles at the identified habitats and locations

Model-based analysis of the Tower of London task

The planning process is central to goal-directed behaviour in any task that requires the organization of a series of actions aimed at achieving a goal. Although the planning process has been investigated thoroughly, relatively little is known about how this process emerges and evolves during childhood. In this paper we describe three reinforcement learning models of planning, in the Tower of London (ToL) task, and use Bayesian analysis to fit each model to pre-existing data from 3-4 year-old and 5-6 year-old children performing the task. The models all capture the increased organisation seen in the older children’s performance. It is also shown that, at least for this dataset, the most complex model – that with discounting of future rewards and pruning of highly aversive states – provides no additional explanatory power beyond a simpler discounting-only model. Insights into developmental aspects of the planning process are discussed