Decentralized learning with budgeted network load using Gaussian copulas and classifier ensembles

We examine a network of learners which address the same classification task but must learn from different data sets. The learners cannot share data but instead share their models. Models are shared only one time so as to preserve the network load. We introduce DELCO (standing for Decentralized Ensemble Learning with COpulas), a new approach allowing to aggregate the predictions of the classifiers trained by each learner. The proposed method aggregates the base classifiers using a probabilistic model relying on Gaussian copulas. Experiments on logistic regressor ensembles demonstrate competing accuracy and increased robustness in case of dependent classifiers. A companion python implementation can be downloaded at https://github.com/john-klein/DELC

Supervised Classification Using Finite Mixture Copula

Use of copula for statistical classification is recent and gaining popularity. For example, statistical classification using copula has been proposed for automatic character recognition, medical diagnostic and most recently in data mining. Classical discrimination rules assume normality. But in this data age time, this assumption is often questionable. In fact features of data could be a mixture of discrete and continues random variables. In this paper, mixture copula densities are used to model class conditional distributions. Such types of densities are useful when the marginal densities of the vector of features are not normally distributed and are of a mixed kind of variables. Authors have shown that such mixture models are very useful for uncovering hidden structures in the data, and used them for clustering in data mining. Under such mixture models, maximum likelihood estimation methods are not suitable and regular expectation maximization algorithm is inefficient and may not converge. A new estimation method is proposed to estimate such densities and build the classifier based on mixture finite Gaussian densities. Simulations are used to compare the performance of the copula based classifier with classical normal distribution based models, logistic regression based model and independent model cases. The method is also applied to a real data

Detection of Sand Dunes on Mars Using a Regular Vine-based Classification Approach

This paper deals with the problem of detecting sand dunes from remotely sensed images of the surface of Mars. We build on previous approaches that propose methods to extract informative features for the classification of the images. The intricate correlation structure exhibited by these features motivates us to propose the use of probabilistic classifiers based on R-vine distributions to address this problem. R-vines are probabilistic graphical models that combine a set of nested trees with copula functions and are able to model a wide range of pairwise dependencies. We investigate different strategies for building R-vine classifiers and compare them with several state-of-the-art classification algorithms for the identification of Martian dunes. Experimental results show the adequacy of the R-vine-based approach to solve classification problems where the interactions between the variables are of a different nature between classes and play an important role in that the classifier can distinguish the different classes

Contributions to Vine-Copula Modeling

144 p.Regular vine-copula models (R-vines) are a powerful statistical tool for modeling thedependence structure of multivariate distribution functions. In particular, they allow modelingdierent types of dependencies among random variables independently of their marginaldistributions, which is deemed the most valued characteristic of these models. In this thesis, weinvestigate the theoretical properties of R-vines for representing dependencies and extend theiruse to solve supervised classication problems. We focus on three research directions.!In the rst line of research, the relationship between the graphical representations of R-vines!ÁREA LÍNEA1 2 0 3 0 4ÁREA LÍNEA1 2 0 3 1 7ÁREA LÍNEAÁREA LÍNEA!and Bayesian polytree networks is analyzed in terms of how conditional pairwise independence!relationships are represented by both models. In order to do that, we use an extended graphical!representation of R-vines in which the R-vine graph is endowed with further expressiveness,being possible to distinguish between edges representing independence and dependencerelationships. Using this representation, a separation criterion in the R-vine graph, called Rseparation,is dened. The proposed criterion is used in designing methods for building thegraphical structure of polytrees from that of R-vines, and vice versa. Moreover, possiblecorrespondences between the R-vine graph and the associated R-vine copula as well as dierentproperties of R-separation are analyzed. In the second research line, we design methods forlearning the graphical structure of R-vines from dependence lists. The main challenge of thistask lies in the extremely large size of the search space of all possible R-vine structures. Weprovide two strategies to solve the problem of learning R-vines that represent the largestnumber of dependencies in a list. The rst approach is a 0 -1 linear programming formulation forbuilding truncated R-vines with only two trees. The second approach is an evolutionaryalgorithm, which is able to learn complete and truncated R-vines. Experimental results show thesuccess of this strategy in solving the optimization problem posed. In the third research line, weintroduce a supervised classication approach where the dependence structure of the problemfeatures is modeled through R-vines. The ecacy of these classiers is validated in a mentaldecoding problem and in an image recognition task. While Rvines have been extensivelyapplied in elds such as economics, nance and statistics, only recently have they found theirplace in classication tasks. This contribution represents a step forward in understanding R-vinesand the prospect of extending their use to other machine learning tasks

Supervised Classification Using Copula and Mixture Copula

Statistical classification is a field of study that has developed significantly after 1960\u27s. This research has a vast area of applications. For example, pattern recognition has been proposed for automatic character recognition, medical diagnostic and most recently in data mining. Classical discrimination rule assumes normality. However in many situations, this assumption is often questionable. In fact for some data, the pattern vector is a mixture of discrete and continuous random variables. In this dissertation, we use copula densities to model class conditional distributions. Such types of densities are useful when the marginal densities of a pattern vector are not normally distributed. This type of models are also useful for a mixed discrete and continuous feature types. Finite mixture density models are very flexible in building classifier and clustering, and for uncovering hidden structures in the data. We use finite mixture Gaussian copula and copula of the Archimedean family based mixture densities to build classifier. The complexities of the estimation are presented. Under such mixture models, maximum likelihood estimation methods are not suitable and regular expectation maximization algorithm may not converge, and if it does, not efficiently. We propose a new estimation method to evaluate such densities and build the classifier based on finite mixture of copula densities. We develop simulations scenarios to compare the performance of the copula based classifier with classical normal distribution based models, the logistic regression based model and the Independent model. We also apply the techniques to real data, and present the misclassification errors

Mixed Cumulative Distribution Networks

Directed acyclic graphs (DAGs) are a popular framework to express multivariate probability distributions. Acyclic directed mixed graphs (ADMGs) are generalizations of DAGs that can succinctly capture much richer sets of conditional independencies, and are especially useful in modeling the effects of latent variables implicitly. Unfortunately there are currently no good parameterizations of general ADMGs. In this paper, we apply recent work on cumulative distribution networks and copulas to propose one one general construction for ADMG models. We consider a simple parameter estimation approach, and report some encouraging experimental results.Comment: 11 pages, 4 figure