Predecessor Features

Any reinforcement learning system must be able to identify which past events contributed to observed outcomes, a problem known as credit assignment. A common solution to this problem is to use an eligibility trace to assign credit to recency-weighted set of experienced events. However, in many realistic tasks, the set of recently experienced events are only one of the many possible action events that could have preceded the current outcome. This suggests that reinforcement learning can be made more efficient by allowing credit assignment to any viable preceding state, rather than only those most recently experienced. Accordingly, we examine ``Predecessor Features'', the fully bootstrapped version of van Hasselt's ``Expected Trace'', an algorithm that achieves this richer form of credit assignment. By maintaining a representation that approximates the expected sum of past occupancies, this algorithm allows temporal difference (TD) errors to be propagated accurately to a larger number of predecessor states than conventional methods, greatly improving learning speed. The algorithm can also be naturally extended from tabular state representation to feature representations allowing for increased performance on a wide range of environments. We demonstrate several use cases for Predecessor Features and compare its performance with other approaches.Comment: Accepted to RLDM 202

RSVP in orbit: Identification of single and dual targets in motion

Three experiments using rapid serial visual presentation (RSVP) tested participants' ability to detect targets in streams that are in motion. These experiments compared the ability to identify moving versus stationary RSVP targets and examined the attentional blink with pairs of targets that were moving or stationary. One condition presented RSVP streams in the center of the screen; a second condition used an RSVP that was orbiting in a circle, with participants instructed to follow the stream with their eyes; and a third condition had participants fixate in the middle while observing a circling RSVP stream. Relative to performance in stationary RSVP streams, participants were not markedly impaired in detecting single targets in RSVP streams that were moving, either with or without instructions to pursue the motion. In streams with two targets, a normal attentional blink effect was observed when participants were instructed to pursue the moving stream. When participants had to maintain central fixation as the RSVP stream moved, the attentional blink was nearly absent even when a trailing mask was added. We suggest that the reduction of the attentional blink for moving RSVP streams may reflect a reduced ability to perceive the temporal boundaries of the individual items

Emergence and reconfiguration of modular structure for synaptic neural networks during continual familiarity detection

While advances in artificial intelligence and neuroscience have enabled the emergence of neural networks capable of learning a wide variety of tasks, our understanding of the temporal dynamics of these networks remains limited. Here, we study the temporal dynamics during learning of Hebbian Feedforward (HebbFF) neural networks in tasks of continual familiarity detection. Drawing inspiration from the field of network neuroscience, we examine the network's dynamic reconfiguration, focusing on how network modules evolve throughout learning. Through a comprehensive assessment involving metrics like network accuracy, modular flexibility, and distribution entropy across diverse learning modes, our approach reveals various previously unknown patterns of network reconfiguration. In particular, we find that the emergence of network modularity is a salient predictor of performance, and that modularization strengthens with increasing flexibility throughout learning. These insights not only elucidate the nuanced interplay of network modularity, accuracy, and learning dynamics but also bridge our understanding of learning in artificial and biological realms

Visual scoping operations for physical assembly

Planning is hard. The use of subgoals can make planning more tractable, but selecting these subgoals is computationally costly. What algorithms might enable us to reap the benefits of planning using subgoals while minimizing the computational overhead of selecting them? We propose visual scoping, a strategy that interleaves planning and acting by alternately defining a spatial region as the next subgoal and selecting actions to achieve it. We evaluated our visual scoping algorithm on a variety of physical assembly problems against two baselines: planning all subgoals in advance and planning without subgoals. We found that visual scoping achieves comparable task performance to the subgoal planner while requiring only a fraction of the total computational cost. Together, these results contribute to our understanding of how humans might make efficient use of cognitive resources to solve complex planning problems

Adaptation decorrelates shape representations

Perception and neural responses are modulated by sensory history. Visual adaptation, an example of such an effect, has been hypothesized to improve stimulus discrimination by decorrelating responses across a set of neural units. While a central theoretical model, behavioral and neural evidence for this theory is limited and inconclusive. Here, we use a parametric 3D shape-space to test whether adaptation decorrelates shape representations in humans. In a behavioral experiment with 20 subjects, we find that adaptation to a shape class improves discrimination of subsequently presented stimuli with similar features. In a BOLD fMRI experiment with 10 subjects, we observe that adaptation to a shape class decorrelates the multivariate representations of subsequently presented stimuli with similar features in object-selective cortex. These results support the long-standing proposal that adaptation improves perceptual discrimination and decorrelates neural representations, offering insights into potential underlying mechanisms

Rickettsioses in Latin America, Caribbean, Spain and Portugal

Data on genus and infectious by Rickettsia were retrospectively compiled from the critical review literature regarding all countries in Latin America, Caribbean islands, Portugal and Spain. We considered all Rickettsia records reported for human and/or animal hosts, and/or invertebrate hosts considered being the vector. In a few cases, when no direct detection of a given Rickettsia group or species was available for a given country, the serologic method was considered. A total of 13 Rickettsia species have been recorded in Latin America and the Caribbean. The species with the largest number of country confirmed records were Rickettsia felis (9 countries), R. prowazekii (7 countries), R. typhi (6 countries), R. rickettsii (6 countries), R. amblyommii (5 countries), and R. parkeri (4 countries). The rickettsial records for the Caribbean islands (West Indies) were grouped in only one geographical area. Both R. bellii, R. akari, and Candidatus ‘R. andeane’ have been recorded in only 2 countries each, whereas R. massiliae, R. rhipicephali, R.monteiroi, and R. africae have each been recorded in a single country (in this case, R. africae has been recorded in nine Caribbean Islands). For El Salvador, Honduras, and Nicaragua, no specific Rickettsia has been reported so far, but there have been serological evidence of human or/and animal infection. The following countries remain without any rickettsial records: Belize, Venezuela, Guyana, Surinam, and Paraguay. In addition, except for a few islands, many Caribbean islands remain without records. A total of 12 Rickettsia species have been reported in Spain and Portugal: R. conorii, R. helvetica, R. monacensis, R. felis, R. slovaca, R. raoultii, R. sibirica, R. aeschlimannii, R. rioja, R. massiliae, R. typhi, and R. prowazekii. Amongst these Rickettsia species reported in Spain and Portugal, only R. prowazekii, R. typhi, R. felis, and R. massiliae have also been reported in Latin America. This study summarizes the current state of art on the rickettsial distribution in Latin America, Caribbean, Spain and Portugal. The data obtained allow a better understanding on rickettsial epidemiology and distribution of vector ecology. Key words: Acari, epidemiology, rocky mountain spotted fever, vector control. (Source: DeCS