Learning Mazes with Aliasing States: An LCS Algorithm with Associative Perception

Learning classifier systems (LCSs) belong to a class of algorithms based on the principle of self-organization and have frequently been applied to the task of solving mazes, an important type of reinforcement learning (RL) problem. Maze problems represent a simplified virtual model of real environments that can be used for developing core algorithms of many real-world applications related to the problem of navigation. However, the best achievements of LCSs in maze problems are still mostly bounded to non-aliasing environments, while LCS complexity seems to obstruct a proper analysis of the reasons of failure. We construct a new LCS agent that has a simpler and more transparent performance mechanism, but that can still solve mazes better than existing algorithms. We use the structure of a predictive LCS model, strip out the evolutionary mechanism, simplify the reinforcement learning procedure and equip the agent with the ability of associative perception, adopted from psychology. To improve our understanding of the nature and structure of maze environments, we analyze mazes used in research for the last two decades, introduce a set of maze complexity characteristics, and develop a set of new maze environments. We then run our new LCS with associative perception through the old and new aliasing mazes, which represent partially observable Markov decision problems (POMDP) and demonstrate that it performs at least as well as, and in some cases better than, other published systems

Genetic basis of sleep in rodents

This chapter explores the progress and various approaches toward identifying genes important for an understanding of sleep and sleep-related traits in rodents. Until the functions of sleep are more clearly defined, it may be difficult to uncover the core genetic basis for sleep and the core molecular events that underlie this fundamental behavior in most, if not all, animal species. However, similar to other areas of medicine, an understanding of the underlying genetics will become increasingly important in sleep medicine. At this time, rodents represent the best animal models for such studies, especially the many genetically modified mouse lines that have been created and the mouse strains that form important genetic reference populations for a wide range of biomedical research. Given the similarities in physiology and genetics across eutherian mammals, it is likely that genes influencing sleep ..

Recent advances in understanding the genetics of sleep.

Sleep is a ubiquitous and complex behavior in both its manifestation and regulation. Despite its essential role in maintaining optimal performance, health, and well-being, the genetic mechanisms underlying sleep remain poorly understood. Here, we review the forward genetic approaches undertaken in the last four years to elucidate the genes and gene pathways affecting sleep and its regulation. Despite an increasing number of studies and mining large databases, a coherent picture on "sleep" genes has yet to emerge. We highlight the results achieved by using unbiased genetic screens mainly in humans, mice, and fruit flies with an emphasis on normal sleep and make reference to lessons learned from the circadian field

Genomic and proteomic approaches towards an understanding of sleep.

The basic functions of sleep are still unclear, however, recent advances in genomics and proteomics have begun to contribute to our understanding of both normal and pathological sleep. In this review, we focus primarily on normal sleep and wake that have been studied in model organisms such as mice. Mice have been especially valuable since many different inbred strains exist that differ in sleep-related traits, and genes can be altered by either mutagenesis or targeted approaches. Advances in QTL (Quantitative Trait Loci) analysis have also helped to identify important sleep related genes, and several other QTLs have been mapped as a first step toward finding the genes that underlie basic sleep traits. In addition to more traditional genetic approaches, the abundance of different mRNAs across sleep and wake can now be studied and compared in different brain regions much more thoroughly using microarray methods. Progress at the protein level has been more difficult, but a few studies have begun to investigate changes in proteins during sleep and wake, and we present some of our own preliminary data in this area. A knowledge of which genes and proteins control or respond to changes in sleep will not only help answer fundamental questions, but may also suggest novel drug targets for improving multiple aspects of sleep and wake

Evaluation of a piezoelectric system as an alternative to electroencephalogram/ electromyogram recordings in mouse sleep studies.

STUDY OBJECTIVES: Traditionally, sleep studies in mammals are performed using electroencephalogram/electromyogram (EEG/EMG) recordings to determine sleep-wake state. In laboratory animals, this requires surgery and recovery time and causes discomfort to the animal. In this study, we evaluated the performance of an alternative, noninvasive approach utilizing piezoelectric films to determine sleep and wakefulness in mice by simultaneous EEG/EMG recordings. The piezoelectric films detect the animal's movements with high sensitivity and the regularity of the piezo output signal, related to the regular breathing movements characteristic of sleep, serves to automatically determine sleep. Although the system is commercially available (Signal Solutions LLC, Lexington, KY), this is the first statistical validation of various aspects of sleep. DESIGN: EEG/EMG and piezo signals were recorded simultaneously during 48 h. SETTING: Mouse sleep laboratory. PARTICIPANTS: Nine male and nine female CFW outbred mice. INTERVENTIONS: EEG/EMG surgery. MEASUREMENTS AND RESULTS: The results showed a high correspondence between EEG/EMG-determined and piezo-determined total sleep time and the distribution of sleep over a 48-h baseline recording with 18 mice. Moreover, the piezo system was capable of assessing sleep quality (i.e., sleep consolidation) and interesting observations at transitions to and from rapid eye movement sleep were made that could be exploited in the future to also distinguish the two sleep states. CONCLUSIONS: The piezo system proved to be a reliable alternative to electroencephalogram/electromyogram recording in the mouse and will be useful for first-pass, large-scale sleep screens for genetic or pharmacological studies. CITATION: Mang GM, Nicod J, Emmenegger Y, Donohue KD, O'Hara BF, Franken P. Evaluation of a piezoelectric system as an alternative to electroencephalogram/electromyogram recordings in mouse sleep studies

Pattern recognition of sleep in rodents using piezoelectric signals generated by gross body movements.

Current research on sleep using experimental animals is limited by the expense and time-consuming nature of traditional EEG/EMG recordings. We present here an alternative, noninvasive approach utilizing piezoelectric films configured as highly sensitive motion detectors. These film strips attached to the floor of the rodent cage produce an electrical output in direct proportion to the distortion of the material. During sleep, movement associated with breathing is the predominant gross body movement and, thus, output from the piezoelectric transducer provided an accurate respiratory trace during sleep. During wake, respiratory movements are masked by other motor activities. An automatic pattern recognition system was developed to identify periods of sleep and wake using the piezoelectric generated signal. Due to the complex and highly variable waveforms that result from subtle postural adjustments in the animals, traditional signal analysis techniques were not sufficient for accurate classification of sleep versus wake. Therefore, a novel pattern recognition algorithm was developed that successfully distinguished sleep from wake in approximately 95% of all epochs. This algorithm may have general utility for a variety of signals in biomedical and engineering applications. This automated system for monitoring sleep is noninvasive, inexpensive, and may be useful for large-scale sleep studies including genetic approaches towards understanding sleep and sleep disorders, and the rapid screening of the efficacy of sleep or wake promoting drugs

Lack of the alanine-serine-cysteine transporter 1 causes tremors, seizures, and early postnatal death in mice.

The Na(+)-independent alanine-serine-cysteine transporter 1 (Asc-1) is exclusively expressed in neuronal structures throughout the central nervous system (CNS). Asc-1 transports small neutral amino acids with high affinity especially for D-serine and glycine (K(i): 8-12 microM), two endogenous glutamate co-agonists that activate N-methyl-D-aspartate (NMDA) receptors through interacting with the strychnine-insensitive glycine binding-site. By regulating D-serine (and possibly glycine) levels in the synaptic cleft, Asc-1 may play an important role in controlling neuronal excitability. We generated asc-1 gene knockout (asc-1(-/-)) mice to test this hypothesis. Behavioral phenotyping combined with electroencephalogram (EEG) recordings revealed that asc-1(-/-) mice developed tremors, ataxia, and seizures that resulted in early postnatal death. Both tremors and seizures were reduced by the NMDA receptor antagonist MK-801. Extracellular recordings from asc-1(-/-) brain slices indicated that the spontaneous seizure activity did not originate in the hippocampus, although, in this region, a relative increase in evoked synaptic responses was observed under nominal Mg(2+)-free conditions. Taken together with the known neurochemistry and neuronal distribution of the Asc-1 transporter, these results indicate that the mechanism underlying the behavioral hyperexcitability in mutant mice is likely due to overactivation of NMDA receptors, presumably resulting from elevated extracellular D-serine. Our study provides the first evidence to support the notion that Asc-1 transporter plays a critical role in regulating neuronal excitability, and indicate that the transporter is vital for normal CNS function and essential to postnatal survival of mice

Molecular analysis of sleep.

Rest or sleep in all animal species constitutes a period of quiescence necessary for recovery from activity. Whether rest and activity observed in all organisms share a similar fundamental molecular basis with sleep and wakefulness in mammals has not yet been established. In addition and in contrast to the circadian system, strong evidence that sleep is regulated at the transcriptional level is lacking. Nevertheless, several studies indicate that single genesmay regulate some specific aspects of sleep. Efforts to better understand or confirm the role of known neurotransmission pathways in sleep-wake regulation using transgenic approaches resulted so far in only limited new insights. Recent gene expression profiling efforts in rats, mice, and fruit flies are promising and suggest that only a few gene categories are differentially regulated by behavioral state. How molecular analysis can help us to understand sleep is the focus of this chapter