Dynamic Data Mining: Methodology and Algorithms

Supervised data stream mining has become an important and challenging data mining task in modern organizations. The key challenges are threefold: (1) a possibly infinite number of streaming examples and time-critical analysis constraints; (2) concept drift; and (3) skewed data distributions. To address these three challenges, this thesis proposes the novel dynamic data mining (DDM) methodology by effectively applying supervised ensemble models to data stream mining. DDM can be loosely defined as categorization-organization-selection of supervised ensemble models. It is inspired by the idea that although the underlying concepts in a data stream are time-varying, their distinctions can be identified. Therefore, the models trained on the distinct concepts can be dynamically selected in order to classify incoming examples of similar concepts. First, following the general paradigm of DDM, we examine the different concept-drifting stream mining scenarios and propose corresponding effective and efficient data mining algorithms. • To address concept drift caused merely by changes of variable distributions, which we term pseudo concept drift, base models built on categorized streaming data are organized and selected in line with their corresponding variable distribution characteristics. • To address concept drift caused by changes of variable and class joint distributions, which we term true concept drift, an effective data categorization scheme is introduced. A group of working models is dynamically organized and selected for reacting to the drifting concept. Secondly, we introduce an integration stream mining framework, enabling the paradigm advocated by DDM to be widely applicable for other stream mining problems. Therefore, we are able to introduce easily six effective algorithms for mining data streams with skewed class distributions. In addition, we also introduce a new ensemble model approach for batch learning, following the same methodology. Both theoretical and empirical studies demonstrate its effectiveness. Future work would be targeted at improving the effectiveness and efficiency of the proposed algorithms. Meantime, we would explore the possibilities of using the integration framework to solve other open stream mining research problems

Steady state analysis of balanced-allocation routing

We compare the long-term, steady-state performance of a variant of the standard Dynamic Alternative Routing (DAR) technique commonly used in telephone and ATM networks, to the performance of a path-selection algorithm based on the "balanced-allocation" principle; we refer to this new algorithm as the Balanced Dynamic Alternative Routing (BDAR) algorithm. While DAR checks alternative routes sequentially until available bandwidth is found, the BDAR algorithm compares and chooses the best among a small number of alternatives. We show that, at the expense of a minor increase in routing overhead, the BDAR algorithm gives a substantial improvement in network performance, in terms both of network congestion and of bandwidth requirement.Comment: 22 pages, 1 figur

A phase transition for measure-valued SIR epidemic processes

We consider measure-valued processes $X=(X_t)$ that solve the following martingale problem: for a given initial measure $X_0$, and for all smooth, compactly supported test functions $\varphi$, \begin{eqnarray*}X_t(\varphi )=X_0(\varphi)+\frac{1}{2}\int _0^tX_s(\Delta \varphi )\,ds+\theta \int_0^tX_s(\varphi )\,ds\\{}-\int_0^tX_s(L_s\varphi )\,ds+M_t(\varphi ).\end{eqnarray*} Here $L_s(x)$ is the local time density process associated with $X$, and $M_t(\varphi )$ is a martingale with quadratic variation $[M(\varphi )]_t=\int_0^tX_s(\varphi ^2)\,ds$. Such processes arise as scaling limits of SIR epidemic models. We show that there exist critical values $\theta_c(d)\in(0,\infty)$ for dimensions $d=2,3$ such that if $\theta>\theta_c(d)$, then the solution survives forever with positive probability, but if $\theta<\theta_c(d)$, then the solution dies out in finite time with probability 1. For $d=1$ we prove that the solution dies out almost surely for all values of $\theta$. We also show that in dimensions $d=2,3$ the process dies out locally almost surely for any value of $\theta$; that is, for any compact set $K$, the process $X_t(K)=0$ eventually.Comment: Published in at http://dx.doi.org/10.1214/13-AOP846 the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org

Multiscale derivation, analysis and simulation of collective dynamics models: geometrical aspects and applications

This thesis is a contribution to the study of swarming phenomena from the point of view of mathematical kinetic theory. This multiscale approach starts from stochastic individual based (or particle) models and aims at the derivation of partial differential equation models on statistical quantities when the number of particles tends to infinity. This latter class of models is better suited for mathematical analysis in order to reveal and explain large-scale emerging phenomena observed in various biological systems such as flocks of birds or swarms of bacteria. Within this objective, a large part of this thesis is dedicated to the study of a body-attitude coordination model and, through this example, of the influence of geometry on self-organisation. The first part of the thesis deals with the rigorous derivation of partial differential equation models from particle systems with mean-field interactions. After a review of the literature, in particular on the notion of propagation of chaos, a rigorous convergence result is proved for a large class of geometrically enriched piecewise deterministic particle models towards local BGK-type equations. In addition, the method developed is applied to the design and analysis of a new particle-based algorithm for sampling. This first part also addresses the question of the efficient simulation of particle systems using recent GPU routines. The second part of the thesis is devoted to kinetic and fluid models for body-oriented particles. The kinetic model is rigorously derived as the mean-field limit of a particle system. In the spatially homogeneous case, a phase transition phenomenon is investigated which discriminates, depending on the parameters of the model, between a “disordered” dynamics and a self-organised “ordered” dynamics. The fluid (or macroscopic) model was derived as the hydrodynamic limit of the kinetic model a few years ago by Degond et al. The analytical and numerical study of this model reveal the existence of new self-organised phenomena which are confirmed and quantified using particle simulations. Finally a generalisation of this model in arbitrary dimension is presented.Open Acces

Stochastic Models for the 3x+1 and 5x+1 Problems

This paper discusses stochastic models for predicting the long-time behavior of the trajectories of orbits of the 3x+1 problem and, for comparison, the 5x+1 problem. The stochastic models are rigorously analyzable, and yield heuristic predictions (conjectures) for the behavior of 3x+1 orbits and 5x+1 orbits.Comment: 68 pages, 9 figures, 4 table