Adaptive Learning and Mining for Data Streams and Frequent Patterns

Aquesta tesi està dedicada al disseny d'algorismes de mineria de dades per fluxos de dades que evolucionen en el temps i per l'extracció d'arbres freqüents tancats. Primer ens ocupem de cadascuna d'aquestes tasques per separat i, a continuació, ens ocupem d'elles conjuntament, desenvolupant mètodes de classificació de fluxos de dades que contenen elements que són arbres. En el model de flux de dades, les dades arriben a gran velocitat, i els algorismes que els han de processar tenen limitacions estrictes de temps i espai. En la primera part d'aquesta tesi proposem i mostrem un marc per desenvolupar algorismes que aprenen de forma adaptativa dels fluxos de dades que canvien en el temps. Els nostres mètodes es basen en l'ús de mòduls detectors de canvi i estimadors en els llocs correctes. Proposem ADWIN, un algorisme de finestra lliscant adaptativa, per la detecció de canvi i manteniment d'estadístiques actualitzades, i proposem utilitzar-lo com a caixa negra substituint els comptadors en algorismes inicialment no dissenyats per a dades que varien en el temps. Com ADWIN té garanties teòriques de funcionament, això obre la possibilitat d'ampliar aquestes garanties als algorismes d'aprenentatge i de mineria de dades que l'usin. Provem la nostre metodologia amb diversos mètodes d'aprenentatge com el Naïve Bayes, partició, arbres de decisió i conjunt de classificadors. Construïm un marc experimental per fer mineria amb fluxos de dades que varien en el temps, basat en el programari MOA, similar al programari WEKA, de manera que sigui fàcil pels investigadors de realitzar-hi proves experimentals. Els arbres són grafs acíclics connectats i són estudiats com vincles en molts casos. En la segona part d'aquesta tesi, descrivim un estudi formal dels arbres des del punt de vista de mineria de dades basada en tancats. A més, presentem algorismes eficients per fer tests de subarbres i per fer mineria d'arbres freqüents tancats ordenats i no ordenats. S'inclou una anàlisi de l'extracció de regles d'associació de confiança plena dels conjunts d'arbres tancats, on hem trobat un fenomen interessant: les regles que la seva contrapart proposicional és no trivial, són sempre certes en els arbres a causa de la seva peculiar combinatòria. I finalment, usant aquests resultats en fluxos de dades evolutius i la mineria d'arbres tancats freqüents, hem presentat algorismes d'alt rendiment per fer mineria d'arbres freqüents tancats de manera adaptativa en fluxos de dades que evolucionen en el temps. Introduïm una metodologia general per identificar patrons tancats en un flux de dades, utilitzant la Teoria de Reticles de Galois. Usant aquesta metodologia, desenvolupem un algorisme incremental, un basat en finestra lliscant, i finalment un que troba arbres freqüents tancats de manera adaptativa en fluxos de dades. Finalment usem aquests mètodes per a desenvolupar mètodes de classificació per a fluxos de dades d'arbres.This thesis is devoted to the design of data mining algorithms for evolving data streams and for the extraction of closed frequent trees. First, we deal with each of these tasks separately, and then we deal with them together, developing classification methods for data streams containing items that are trees. In the data stream model, data arrive at high speed, and the algorithms that must process them have very strict constraints of space and time. In the first part of this thesis we propose and illustrate a framework for developing algorithms that can adaptively learn from data streams that change over time. Our methods are based on using change detectors and estimator modules at the right places. We propose an adaptive sliding window algorithm ADWIN for detecting change and keeping updated statistics from a data stream, and use it as a black-box in place or counters or accumulators in algorithms initially not designed for drifting data. Since ADWIN has rigorous performance guarantees, this opens the possibility of extending such guarantees to learning and mining algorithms. We test our methodology with several learning methods as Naïve Bayes, clustering, decision trees and ensemble methods. We build an experimental framework for data stream mining with concept drift, based on the MOA framework, similar to WEKA, so that it will be easy for researchers to run experimental data stream benchmarks. Trees are connected acyclic graphs and they are studied as link-based structures in many cases. In the second part of this thesis, we describe a rather formal study of trees from the point of view of closure-based mining. Moreover, we present efficient algorithms for subtree testing and for mining ordered and unordered frequent closed trees. We include an analysis of the extraction of association rules of full confidence out of the closed sets of trees, and we have found there an interesting phenomenon: rules whose propositional counterpart is nontrivial are, however, always implicitly true in trees due to the peculiar combinatorics of the structures. And finally, using these results on evolving data streams mining and closed frequent tree mining, we present high performance algorithms for mining closed unlabeled rooted trees adaptively from data streams that change over time. We introduce a general methodology to identify closed patterns in a data stream, using Galois Lattice Theory. Using this methodology, we then develop an incremental one, a sliding-window based one, and finally one that mines closed trees adaptively from data streams. We use these methods to develop classification methods for tree data streams.Postprint (published version

Preference extraction and reasoning in negotiation dialogues

Modéliser les préférences des utilisateurs est incontournable dans de nombreux problèmes de la vie courante, que ce soit pour la prise de décision individuelle ou collective ou le raisonnement stratégique par exemple. Cependant, il n'est pas facile de travailler avec les préférences. Comme les agents ne connaissent pas complètement leurs préférences à l'avance, nous avons seulement deux moyens de les déterminer pour pouvoir raisonner ensuite : nous pouvons les inférer soit de ce que les agents disent, soit de leurs actions non-linguistiques. Plusieurs méthodes ont été proposées en Intelligence Artificielle pour apprendre les préférences à partir d'actions non-linguistiques mais à notre connaissance très peu de travaux ont étudié comment éliciter efficacement les préférences verbalisées par les utilisateurs grâce à des méthodes de Traitement Automatique des Langues (TAL).Dans ce travail, nous proposons une nouvelle approche pour extraire et raisonner sur les préférences exprimées dans des dialogues de négociation. Après avoir extrait les préférences de chaque tour de dialogue, nous utilisons la structure discursive pour suivre leur évolution au fur et à mesure de la conversation. Nous utilisons les CP-nets, un modèle de représentation des préférences, pour formaliser et raisonner sur ces préférences extraites. Cette méthode est d'abord évaluée sur différents corpus de négociation pour lesquels les résultats montrent que la méthode est prometteuse. Nous l'appliquons ensuite dans sa globalité avec des raisonnements issus de la Théorie des Jeux pour prédire les échanges effectués, ou non, dans le jeu de marchandage Les Colons de Catane. Les résultats obtenus montrent des prédictions significativement meilleures que celles de quatre baselines qui ne gèrent pas correctement le raisonnement stratégique. Cette thèse présente donc une nouvelle approche à la croisée de plusieurs domaines : le Traitement Automatique des Langues (pour l'extraction automatique des préférences et le raisonnement sur leur verbalisation), l'Intelligence Artificielle (pour la modélisation et le raisonnement sur les préférences extraites) et la Théorie des Jeux (pour la prédiction des actions stratégiques dans un jeu de marchandage)Modelling user preferences is crucial in many real-life problems, ranging from individual and collective decision-making to strategic interactions between agents for example. But handling preferences is not easy. Since agents don't come with their preferences transparently given in advance, we have only two means to determine what they are if we wish to exploit them in reasoning: we can infer them from what an agent says or from his nonlinguistic actions. Preference acquisition from nonlinguistic actions has been wildly studied within the Artificial Intelligence community. However, to our knowledge, there has been little work that has so far investigated how preferences can be efficiently elicited from users using Natural Language Processing (NLP) techniques. In this work, we propose a new approach to extract and reason on preferences expressed in negotiation dialogues. After having extracted the preferences expressed in each dialogue turn, we use the discursive structure to follow their evolution as the dialogue progresses. We use CP-nets, a model used for the representation of preferences, to formalize and reason about these extracted preferences. The method is first evaluated on different negotiation corpora for which we obtain promising results. We then apply the end-to-end method with principles from Game Theory to predict trades in the win-lose game The Settlers of Catan. Our method shows good results, beating baselines that don't adequately track or reason about preferences. This work thus presents a new approach at the intersection of several research domains: Natural Language Processing (for the automatic preference extraction and the reasoning on their verbalisation), Artificial Intelligence (for the modelling and reasoning on the extracted preferences) and Game Theory (for strategic action prediction in a bargaining game

First-Order Models for Configuration Analysis

Our world teems with networked devices. Their configuration exerts an ever-expanding influence on our daily lives. Yet correctly configuring systems, networks, and access-control policies is notoriously difficult, even for trained professionals. Automated static analysis techniques provide a way to both verify a configuration\u27s correctness and explore its implications. One such approach is scenario-finding: showing concrete scenarios that illustrate potential (mis-)behavior. Scenarios even have a benefit to users without technical expertise, as concrete examples can both trigger and improve users\u27 intuition about their system. This thesis describes a concerted research effort toward improving scenario-finding tools for configuration analysis. We developed Margrave, a scenario-finding tool with special features designed for security policies and configurations. Margrave is not tied to any one specific policy language; rather, it provides an intermediate input language as expressive as first-order logic. This flexibility allows Margrave to reason about many different types of policy. We show Margrave in action on Cisco IOS, a common language for configuring firewalls, demonstrating that scenario-finding with Margrave is useful for debugging and validating real-world configurations. This thesis also presents a theorem showing that, for a restricted subclass of first-order logic, if a sentence is satisfiable then there must exist a satisfying scenario no larger than a computable bound. For such sentences scenario-finding is complete: one can be certain that no scenarios are missed by the analysis, provided that one checks up to the computed bound. We demonstrate that many common configurations fall into this subclass and give algorithmic tests for both sentence membership and counting. We have implemented both in Margrave. Aluminum is a tool that eliminates superfluous information in scenarios and allows users\u27 goals to guide which scenarios are displayed. We quantitatively show that our methods of scenario-reduction and exploration are effective and quite efficient in practice. Our work on Aluminum is making its way into other scenario-finding tools. Finally, we describe FlowLog, a language for network programming that we created with analysis in mind. We show that FlowLog can express many common network programs, yet demonstrate that automated analysis and bug-finding for FlowLog are both feasible as well as complete