Adaptive input selection and evolving neural fuzzy networks modeling

This paper suggests an evolving approach to develop neural fuzzy networks for system modeling. The approach uses an incremental learning procedure to simultaneously select the model inputs, to choose the neural network structure, and to update the network weights. Candidate models with larger and smaller number of input variables than the current model are constructed and tested concurrently. The procedure employs a statistical test in each learning step to choose the best model amongst the current and candidate models. Membership functions can be added or deleted to adjust input space granulation and the neural network structure. Granulation and structure adaptation depend of the modeling error. The weights of the neural networks are updated using a gradient-descent algorithm with optimal learning rate. Prediction and nonlinear system identification examples illustrate the usefulness of the approach. Comparisons with state of the art evolving fuzzy modeling alternatives are performed to evaluate performance from the point of view of modeling error. Simulation results show that the evolving adaptive input selection modeling neural network approach achieves as high as, or higher performance than the remaining evolving modeling methods81314CONSELHO NACIONAL DE DESENVOLVIMENTO CIENTÍFICO E TECNOLÓGICO - CNPQFUNDAÇÃO DE AMPARO À PESQUISA DO ESTADO DE MINAS GERAIS - FAPEMIG305906/2014-3não temnão te

Ensembles of Adaptive Model Rules from High-Speed Data Streams

The volume and velocity of data is increasing at astonishing rates. In order to extract knowledge from this huge amount of information there is a need for efficient on-line learning algorithms. Rule-based algorithms produce models that are easy to understand and can be used almost offhand. Ensemble methods combine several predicting models to improve the quality of prediction. In this paper, a new on-line ensemble method that combines a set of rule-based models is proposed to solve regression problems from data streams. Experimental results using synthetic and real time-evolving data streams show the proposed method significantly improves the performance of the single rule-based learner, and outperforms two state-of-the-art regression algorithms for data streams

Learning Behavior Models for Interpreting and Predicting Traffic Situations

In this thesis, we present Bayesian state estimation and machine learning methods for predicting traffic situations. The cognitive ability to assess situations and behaviors of traffic participants, and to anticipate possible developments is an essential requirement for several applications in the traffic domain, especially for self-driving cars. We present a method for learning behavior models from unlabeled traffic observations and develop improved learning methods for decision trees

A Recursive Partitioning Approach to Hospital Case Mix Classification

The present dissertation was focused on the study and development of a clinical data mining methodology for hospital case mix iso-resource classification. Several recursive partitioning methodologies were applied on Emilia-Romagna Region hospital discharge database. Here, the need for developing several alternative iso-resource subgroups was a critical point in the development of case mix classification systems, due to the presence of clinical coherence requirements. Two major classes of trees were assessed: constant-fit trees and model-based trees, with a particular focus on the latter class, which peculiarity is to fit regression models in the nodes of the tree. After an extensive literature review, the traditional regression tree (constant-fit) and four model-based tree algorithms were assessed: two modifications of the Model-Based Recursive Partitioning (MOB) algorithm which were given additional flexibility by performing a within-node model selection step, respectively using count regression and continuous response regression GLMs; a two-step composite algorithm which fits regression trees and models in terminal nodes; quantile-model-based regression trees, by means of the Generalized Unbiased Interaction Detection and Estimation (GUIDE) algorithm. These algorithms were compared under several points of view. Statistical performance, measured via bootstrap out-of-bag performance curves, was in favor of model-based trees, while, among them, competing performances were found. Implications for the design of hospital case mix classification systems were also evaluated, since the two classes of trees can be conceptually linked to different refunding schemes. Moreover, application and advantages of two different ensemble methods were discussed. All the recursive partitioning methods employed resulted in the definition of iso-resource clinically similar subgroups of patients. Different interpretations were given to these alternative subgroups, due to differences in the rationale of the various splitting criteria. In particular, model-based trees identified subgroups with differential effects of patient’s age and clinical severity on resource consumption, here measured with hospital length of stay

Learning Integrated Relational and Continuous Action Models for Continuous Domains.

Long-living autonomous agents must be able to learn to perform competently in novel environments. One important aspect of competence is the ability to plan, which entails the ability to learn models of the agent's own actions and their effects on the environment. This thesis describes an approach to learn action models of environments with continuous-valued spatial states and realistic physics consisting of multiple interacting rigid objects. In such environments, we hypothesize that objects exhibit multiple qualitatively distinct behaviors based on their relationships to each other and how they interact. We call these qualitatively distinct behaviors modes. Our approach models individual modes with linear functions. We extend the standard propositional function representation with learned knowledge about the roles of objects in determining the outcomes of functions. Roles are learned as first-order relations using the FOIL algorithm. This allows the functions modeling individual modes to be "instantiated" with different sets of objects, similar to relational rules such as STRIPS operators. We also use FOIL to learn preconditions for each mode consisting of clauses that test spatial relationships between objects. These relational preconditions naturally capture the interaction dynamics of spatial domains and allow faster learning and generalization of the model. The combination of continuous linear functions, relational roles, and relational mode preconditions effectively capture both continuous and relational regularities prominent in spatial domains. This results in faster and more general action modeling in these domains. We evaluate the algorithm on two domains, one involving pushing stacks of boxes against frictional resistance, and one in which a ball interacts with obstacles in a physics simulator. We show that our algorithm learns more accurate models than locally weighted regression in the physics simulator domain. We also show that relational mode preconditions learned with FOIL are more accurate than continuous classifiers learned with support vector machines and k-nearest-neighbor.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/102383/1/jzxu_1.pd

Recursive Partitioning of Models of a Generalized Linear Model Type

This thesis is concerned with recursive partitioning of models of a generalized linear model type (GLM-type), i.e., maximum likelihood models with a linear predictor for the linked mean, a topic that has received constant interest over the last twenty years. The resulting tree (a ''model tree'') can be seen as an extension of classic trees, to allow for a GLM-type model in the partitions. In this work, the focus lies on applied and computational aspects of model trees with GLM-type node models to work out different areas where application of the combination of parametric models and trees will be beneficial and to build a computational scaffold for future application of model trees. In the first part, model trees are defined and some algorithms for fitting model trees with GLM-type node model are reviewed and compared in terms of their properties of tree induction and node model fitting. Additionally, the design of a particularly versatile algorithm, the MOB algorithm (Zeileis et al. 2008) in R is described and an in-depth discussion of how the functionality offered can be extended to various GLM-type models is provided. This is highlighted by an example of using partitioned negative binomial models for investigating the effect of health care incentives. Part 2 consists of three research articles where model trees are applied to different problems that frequently occur in the social sciences. The first uses trees with GLM-type node models and applies it to a data set of voters, who show a non-monotone relationship between the frequency of attending past elections and the turnout in 2004. Three different type of model tree algorithms are used to investigate this phenomenon and for two the resulting trees can explain the counter-intuitive finding. Here model tress are used to learn a nonlinear relationship between a target model and a big number of candidate variables to provide more insight into a data set. A second application area is also discussed, namely using model trees to detect ill-fitting subsets in the data. The second article uses model trees to model the number of fatalities in Afghanistan war, based on the WikiLeaks Afghanistan war diary. Data pre-processing with a topic model generates predictors that are used as explanatory variables in a model tree for overdispersed count data. Here the combination of model trees and topic models allows to flexibly analyse database data, frequently encountered in data journalism, and provides a coherent description of fatalities in the Afghanistan war. The third paper uses a new framework built around model trees to approach the classic problem of segmentation, frequently encountered in marketing and management science. Here, the framework is used for segmentation of a sample of the US electorate for identifying likely and unlikely voters. It is shown that the framework's model trees enable accurate identification which in turn allows efficient targeted mobilisation of eligible voters. (author's abstract