A Constructive, Incremental-Learning Network for Mixture Modeling and Classification

Gaussian ARTMAP (GAM) is a supervised-learning adaptive resonance theory (ART) network that uses Gaussian-defined receptive fields. Like other ART networks, GAM incrementally learns and constructs a representation of sufficient complexity to solve a problem it is trained on. GAM's representation is a Gaussian mixture model of the input space, with learned mappings from the mixture components to output classes. We show a close relationship between GAM and the well-known Expectation-Maximization (EM) approach to mixture-modeling. GAM outperforms an EM classification algorithm on a classification benchmark, thereby demonstrating the advantage of the ART match criterion for regulating learning, and the ARTMAP match tracking operation for incorporate environmental feedback in supervised learning situations.Office of Naval Research (N00014-95-1-0409

A Constructive, Incremental-Learning Network for Mixture Modeling and Classification

Gaussian ARTMAP (GAM) is a supervised-learning adaptive resonance theory (ART) network that uses Gaussian-defined receptive fields. Like other ART networks, GAM incrementally learns and constructs a representation of sufficient complexity to solve a problem it is trained on. GAM's representation is a Gaussian mixture model of the input space, with learned mappings from the mixture components to output classes. We show a close relationship between GAM and the well-known Expectation-Maximization (EM) approach to mixture-modeling. GAM outperforms an EM classification algorithm on a classification benchmark, thereby demonstrating the advantage of the ART match criterion for regulating learning, and the ARTMAP match tracking operation for incorporate environmental feedback in supervised learning situations.Office of Naval Research (N00014-95-1-0409

Rerepresenting and Restructuring Domain Theories: A Constructive Induction Approach

Theory revision integrates inductive learning and background knowledge by combining training examples with a coarse domain theory to produce a more accurate theory. There are two challenges that theory revision and other theory-guided systems face. First, a representation language appropriate for the initial theory may be inappropriate for an improved theory. While the original representation may concisely express the initial theory, a more accurate theory forced to use that same representation may be bulky, cumbersome, and difficult to reach. Second, a theory structure suitable for a coarse domain theory may be insufficient for a fine-tuned theory. Systems that produce only small, local changes to a theory have limited value for accomplishing complex structural alterations that may be required. Consequently, advanced theory-guided learning systems require flexible representation and flexible structure. An analysis of various theory revision systems and theory-guided learning systems reveals specific strengths and weaknesses in terms of these two desired properties. Designed to capture the underlying qualities of each system, a new system uses theory-guided constructive induction. Experiments in three domains show improvement over previous theory-guided systems. This leads to a study of the behavior, limitations, and potential of theory-guided constructive induction.Comment: See http://www.jair.org/ for an online appendix and other files accompanying this articl

Neural networks in geophysical applications

Neural networks are increasingly popular in geophysics. Because they are universal approximators, these tools can approximate any continuous function with an arbitrary precision. Hence, they may yield important contributions to finding solutions to a variety of geophysical applications. However, knowledge of many methods and techniques recently developed to increase the performance and to facilitate the use of neural networks does not seem to be widespread in the geophysical community. Therefore, the power of these tools has not yet been explored to their full extent. In this paper, techniques are described for faster training, better overall performance, i.e., generalization,and the automatic estimation of network size and architecture

Incremental construction of LSTM recurrent neural network

Long Short--Term Memory (LSTM) is a recurrent neural network that uses structures called memory blocks to allow the net remember significant events distant in the past input sequence in order to solve long time lag tasks, where other RNN approaches fail. Throughout this work we have performed experiments using LSTM networks extended with growing abilities, which we call GLSTM. Four methods of training growing LSTM has been compared. These methods include cascade and fully connected hidden layers as well as two different levels of freezing previous weights in the cascade case. GLSTM has been applied to a forecasting problem in a biomedical domain, where the input/output behavior of five controllers of the Central Nervous System control has to be modelled. We have compared growing LSTM results against other neural networks approaches, and our work applying conventional LSTM to the task at hand.Postprint (published version

An incremental approach to genetic algorithms based classification

Incremental learning has been widely addressed in the machine learning literature to cope with learning tasks where the learning environment is ever changing or training samples become available over time. However, most research work explores incremental learning with statistical algorithms or neural networks, rather than evolutionary algorithms. The work in this paper employs genetic algorithms (GAs) as basic learning algorithms for incremental learning within one or more classifier agents in a multi-agent environment. Four new approaches with different initialization schemes are proposed. They keep the old solutions and use an “integration” operation to integrate them with new elements to accommodate new attributes, while biased mutation and crossover operations are adopted to further evolve a reinforced solution. The simulation results on benchmark classification data sets show that the proposed approaches can deal with the arrival of new input attributes and integrate them with the original input space. It is also shown that the proposed approaches can be successfully used for incremental learning and improve classification rates as compared to the retraining GA. Possible applications for continuous incremental training and feature selection are also discussed

Analysing road pricing implementation processes in the UK and Norway

Traditional transport policies of road expansion entail a relatively simple system of actors and processes around which expertise, knowledge, and skills which has built up over many decades. Some of the more radical Travel Demand Management measures, including urban road pricing, involve a complicated set of institutions, processes, people and procedures. Road pricing schemes often get delayed or abandoned due to controversy, disagreements, unanticipated problems and a whole host of other delaying factors. If they are implemented, they tend to be diluted and consequently become less effective. Strategic Niche Management (SNM) has previously been used to provide guidelines on the implementation of innovative transport technologies through setting up protected experimental settings (niches) in which actors learn about the design, user needs, social and political acceptability, and other aspects. Here SNM is modified to cover a policy approach through the analysis of road user charging case studies in the UK and Norway. A detailed analysis of the road user charging schemes in Bergen, Oslo, Durham and London is presented. Key factors identified include the role of stakeholder and user networks, the existence of a project champion, understanding the motivations and expectations of stakeholders and users, learning with regards to the regional context, and the change in perceptions associated with acceptance. Comparison between the four cases shows different approaches emerging from each country in implementing and ‘marketing’ of the policies. The paper concentrates on approaches such as: the purpose for introducing the policies, the involvement of users in the planning process and, the use of revenues for either providing alternative transport modes or financing road infrastructure. Key factors identified using the SNM framework include the role of stakeholder and user networks, the existence of a project champion, understanding the motivations and expectations of stakeholders and users, learning with regards to the regional context, and the change in perceptions associated with acceptance. This type of analysis could prove useful for transport planners envisaging the implementation of road pricing projects

Hierarchical incremental class learning with reduced pattern training

Hierarchical Incremental Class Learning (HICL) is a new task decomposition method that addresses the pattern classification problem. HICL is proven to be a good classifier but closer examination reveals areas for potential improvement. This paper proposes a theoretical model to evaluate the performance of HICL and presents an approach to improve the classification accuracy of HICL by applying the concept of Reduced Pattern Training (RPT). The theoretical analysis shows that HICL can achieve better classification accuracy than Output Parallelism [1]. The procedure for RPT is described and compared with the original training procedure. RPT reduces systematically the size of the training data set based on the order of sub-networks built. The results from four benchmark classification problems show much promise for the improved model

Incremental multiple objective genetic algorithms

This paper presents a new genetic algorithm approach to multi-objective optimization problemsIncremental Multiple Objective Genetic Algorithms (IMOGA). Different from conventional MOGA methods, it takes each objective into consideration incrementally. The whole evolution is divided into as many phases as the number of objectives, and one more objective is considered in each phase. Each phase is composed of two stages: first, an independent population is evolved to optimize one specific objective; second, the better-performing individuals from the evolved single-objective population and the multi-objective population evolved in the last phase are joined together by the operation of integration. The resulting population then becomes an initial multi-objective population, to which a multi-objective evolution based on the incremented objective set is applied. The experiment results show that, in most problems, the performance of IMOGA is better than that of three other MOGAs, NSGA-II, SPEA and PAES. IMOGA can find more solutions during the same time span, and the quality of solutions is better

Entering Upon Novelty: Policy and Funding Issues for a New Era in the Arts

The organizational structures and underlying assumptions necessary to thrive in this new development phase for the arts will be quite different from those that served us well -- or that we took for granted -- even in the recent past. Where before we were structured for growth, future success will mean being structured for sustainability; growth capacity as a measure of success will be replaced by "adaptive capacity."This basic change in business assumptions will better reflect the trajectory of contemporary life. Sociologist Zygmunt Bauman suggests we are now living in globalized environments that bypass interdependency and are full of "endemic uncertainty." Living self-determined lives that are independent of the social and cultural norms of the past, people are "looking for engagement, for experiences that they themselves can feel part of creating."We are becoming used to the shift from "proprietary" software to "open-source"; now our organizations have to undergo a similar shift, to accommodate the new "architectures of participation" that Clay Shirky writes about.What all this means is that the ability of an arts organization to adapt its programs, strategies, structures, and systems to address continuous external change and seize fleeting opportunities will become a leading indicator of success and a primary measure of organizational health. In this new era, successful organizations will more deeply recognize and engage with the creativity and artistic potential of the larger community, and the dominant organizational model will change to one that is porous, open, and responsive.This shift will require new forms of strategic thinking, organizational nimbleness, and a commitment to remaining transitory (not to efficiency, specialty, and technical rigidity). Wider definitions of success will center on helping foster "expressive lives" in our communities (a term introduced to arts policy by Bill Ivey), more than on developing a professional cultural community for its own sake. As Samuel Jones wrote recently, "We have moved from a model of provision to one of enabling. The role of the cultural professional has changed.