Learning Mazes with Aliasing States: An LCS Algorithm with Associative Perception

Learning classifier systems (LCSs) belong to a class of algorithms based on the principle of self-organization and have frequently been applied to the task of solving mazes, an important type of reinforcement learning (RL) problem. Maze problems represent a simplified virtual model of real environments that can be used for developing core algorithms of many real-world applications related to the problem of navigation. However, the best achievements of LCSs in maze problems are still mostly bounded to non-aliasing environments, while LCS complexity seems to obstruct a proper analysis of the reasons of failure. We construct a new LCS agent that has a simpler and more transparent performance mechanism, but that can still solve mazes better than existing algorithms. We use the structure of a predictive LCS model, strip out the evolutionary mechanism, simplify the reinforcement learning procedure and equip the agent with the ability of associative perception, adopted from psychology. To improve our understanding of the nature and structure of maze environments, we analyze mazes used in research for the last two decades, introduce a set of maze complexity characteristics, and develop a set of new maze environments. We then run our new LCS with associative perception through the old and new aliasing mazes, which represent partially observable Markov decision problems (POMDP) and demonstrate that it performs at least as well as, and in some cases better than, other published systems

MILCS: A mutual information learning classifier system

This paper introduces a new variety of learning classifier system (LCS), called MILCS, which utilizes mutual information as fitness feedback. Unlike most LCSs, MILCS is specifically designed for supervised learning. MILCS's design draws on an analogy to the structural learning approach of cascade correlation networks. We present preliminary results, and contrast them to results from XCS. We discuss the explanatory power of the resulting rule sets, and introduce a new technique for visualizing explanatory power. Final comments include future directions for this research, including investigations in neural networks and other systems. Copyright 2007 ACM

A Census of the High-Density Molecular Gas in M82

We present a three-pointing study of the molecular gas in the starburst nucleus of M82 based on 190 - 307 GHz spectra obtained with Z-Spec at the Caltech Submillimeter Observatory. We present intensity measurements, detections and upper limits, for 20 transitions, including several new detections of CS, HNC, C2H, H2CO, and CH3CCH lines. We combine our measurements with previously-published measurements at other frequencies for HCN, HNC, CS, C34S, and HCO+ in a multi-species likelihood analysis constraining gas mass, density and temperature, and the species' relative abundances. We find some 1.7 - 2.7 x 10^8 M_sun of gas with n_H2 between 1 - 6 x 10^4 cm^-3 and T > 50 K. While the mass and temperature are comparable to values inferred from mid-J CO transitions, the thermal pressure is a factor of 10 - 20 greater. The molecular interstellar medium is largely fragmented and is subject to ultraviolet irradiation from the star clusters. It is also likely subject to cosmic rays and mechanical energy input from the supernovae, and is warmer on average than the molecular gas in the massive star formation regions in the Milky Way. The typical conditions in the dense gas in M82's central kpc appear unfavorable for further star formation; if any appreciable stellar populations are currently forming, they are likely biased against low mass stars, producing a top-heavy initial mass function.Comment: 15 pages (using emulateapj.cls), 6 figures, Astrophysical Journal, in pres

A brief history of learning classifier systems: from CS-1 to XCS and its variants

© 2015, Springer-Verlag Berlin Heidelberg. The direction set by Wilson’s XCS is that modern Learning Classifier Systems can be characterized by their use of rule accuracy as the utility metric for the search algorithm(s) discovering useful rules. Such searching typically takes place within the restricted space of co-active rules for efficiency. This paper gives an overview of the evolution of Learning Classifier Systems up to XCS, and then of some of the subsequent developments of Wilson’s algorithm to different types of learning

Optimality-based Analysis of XCSF Compaction in Discrete Reinforcement Learning

Learning classifier systems (LCSs) are population-based predictive systems that were originally envisioned as agents to act in reinforcement learning (RL) environments. These systems can suffer from population bloat and so are amenable to compaction techniques that try to strike a balance between population size and performance. A well-studied LCS architecture is XCSF, which in the RL setting acts as a Q-function approximator. We apply XCSF to a deterministic and stochastic variant of the FrozenLake8x8 environment from OpenAI Gym, with its performance compared in terms of function approximation error and policy accuracy to the optimal Q-functions and policies produced by solving the environments via dynamic programming. We then introduce a novel compaction algorithm (Greedy Niche Mass Compaction - GNMC) and study its operation on XCSF's trained populations. Results show that given a suitable parametrisation, GNMC preserves or even slightly improves function approximation error while yielding a significant reduction in population size. Reasonable preservation of policy accuracy also occurs, and we link this metric to the commonly used steps-to-goal metric in maze-like environments, illustrating how the metrics are complementary rather than competitive

A time series classifier

A time series is a sequence of data measured at successive time intervals. Time series analysis refers to all of the methods employed to understand such data, either with the purpose of explaining the underlying system producing the data or to try to predict future data points in the time series...An evolutionary algorithm is a non-deterministic method of searching a solution space, and modeled after biological evolutionary processes. A learning classifier system (LCS) is a form of evolutionary algorithm that operates on a population of mapping rules. We introduce the time series classifier TSC, a new type of LCS that allows for the modeling and prediction of time series data, derived from Wilson\u27s XCSR, an LCS designed for use with real-valued inputs. Our method works by modifying the makeup of the rules in the LCS so that they are suitable for use on a time series...We tested TSC on real-world historical stock data --Abstract, page iii

Toward Open-Set Text-Independent Speaker Identification in Tactical Communications

Abstract-We present the design and implementation of an open-set textindependent speaker identification system using genetic Learning Classifier Systems (LCS). We examine the use of this system in a real-number problem domain, where there is strong interest in its application to tactical communications. We investigate different encoding methods for representing real-number knowledge and study the efficacy of each method for speaker identification. We also identify several difficulties in solving the speaker identification problems with LCS and introduce new approaches to resolve the difficulties. Experimental results show that our system successfully learns 200 voice features at accuracies of 90% to 100% and 15,000 features to more than 80% for the closed-set problem, which is considered a strong result in the speaker identification community. The open-set capability is also comparable to existing numeric-based methods

学習戦略に基づく学習分類子システムの設計

On Learning Classifier Systems dubbed LCSs a leaning strategy which defines how LCSs cover a state-action space in a problem can be one of the most fundamental options in designing LCSs. There lacks an intensive study of the learning strategy to understand whether and how the learning strategy affects the performance of LCSs. This lack has resulted in the current design methodology of LCS which does not carefully consider the types of learning strategy. The thesis clarifies a need of a design methodology of LCS based on the learning strategy. That is, the thesis shows the learning strategy can be an option that determines the potential performance of LCSs and then claims that LCSs should be designed on the basis of the learning strategy in order to improve the performance of LCSs. First, the thesis empirically claims that the current design methodology of LCS, without the consideration of learning strategy, can be limited to design a proper LCS to solve a problem. This supports the need of design methodology based on the learning strategy. Next, the thesis presents an example of how LCS can be designed on the basis of the learning strategy. The thesis empirically show an adequate learning strategy improving the performance of LCS can be decided depending on a type of problem difficulties such as missing attributes. Then, the thesis draws an inclusive guideline that explains which learning strategy should be used to address which types of problem difficulties. Finally, the thesis further shows, on an application of LCS for a human daily activity recognition problem, the adequate learning strategy according to the guideline effectively improves the performance of the application. The thesis concludes that the learning strategy is the option of the LCS design which determines the potential performance of LCSs. Thus, before designing any type of LCSs including their applications, the learning strategy should be adequately selected at first, because their performance degrades when they employ an inadequate learning strategy to a problem they want to solve. In other words, LCSs should be designed on the basis of the adequate learning strategy.電気通信大学201