Constructive Approximation and Learning by Greedy Algorithms

This thesis develops several kernel-based greedy algorithms for different machine learning problems and analyzes their theoretical and empirical properties. Greedy approaches have been extensively used in the past for tackling problems in combinatorial optimization where finding even a feasible solution can be a computationally hard problem (i.e., not solvable in polynomial time). A key feature of greedy algorithms is that a solution is constructed recursively from the smallest constituent parts. In each step of the constructive process a component is added to the partial solution from the previous step and, thus, the size of the optimization problem is reduced. The selected components are given by optimization problems that are simpler and easier to solve than the original problem. As such schemes are typically fast at constructing a solution they can be very effective on complex optimization problems where finding an optimal/good solution has a high computational cost. Moreover, greedy solutions are rather intuitive and the schemes themselves are simple to design and easy to implement. There is a large class of problems for which greedy schemes generate an optimal solution or a good approximation of the optimum. In the first part of the thesis, we develop two deterministic greedy algorithms for optimization problems in which a solution is given by a set of functions mapping an instance space to the space of reals. The first of the two approaches facilitates data understanding through interactive visualization by providing means for experts to incorporate their domain knowledge into otherwise static kernel principal component analysis. This is achieved by greedily constructing embedding directions that maximize the variance at data points (unexplained by the previously constructed embedding directions) while adhering to specified domain knowledge constraints. The second deterministic greedy approach is a supervised feature construction method capable of addressing the problem of kernel choice. The goal of the approach is to construct a feature representation for which a set of linear hypotheses is of sufficient capacity — large enough to contain a satisfactory solution to the considered problem and small enough to allow good generalization from a small number of training examples. The approach mimics functional gradient descent and constructs features by fitting squared error residuals. We show that the constructive process is consistent and provide conditions under which it converges to the optimal solution. In the second part of the thesis, we investigate two problems for which deterministic greedy schemes can fail to find an optimal solution or a good approximation of the optimum. This happens as a result of making a sequence of choices which take into account only the immediate reward without considering the consequences onto future decisions. To address this shortcoming of deterministic greedy schemes, we propose two efficient randomized greedy algorithms which are guaranteed to find effective solutions to the corresponding problems. In the first of the two approaches, we provide a mean to scale kernel methods to problems with millions of instances. An approach, frequently used in practice, for this type of problems is the Nyström method for low-rank approximation of kernel matrices. A crucial step in this method is the choice of landmarks which determine the quality of the approximation. We tackle this problem with a randomized greedy algorithm based on the K-means++ cluster seeding scheme and provide a theoretical and empirical study of its effectiveness. In the second problem for which a deterministic strategy can fail to find a good solution, the goal is to find a set of objects from a structured space that are likely to exhibit an unknown target property. This discrete optimization problem is of significant interest to cyclic discovery processes such as de novo drug design. We propose to address it with an adaptive Metropolis–Hastings approach that samples candidates from the posterior distribution of structures conditioned on them having the target property. The proposed constructive scheme defines a consistent random process and our empirical evaluation demonstrates its effectiveness across several different application domains

Topics in social network analysis and network science

This chapter introduces statistical methods used in the analysis of social networks and in the rapidly evolving parallel-field of network science. Although several instances of social network analysis in health services research have appeared recently, the majority involve only the most basic methods and thus scratch the surface of what might be accomplished. Cutting-edge methods using relevant examples and illustrations in health services research are provided

Kiel Declarative Programming Days 2013

This report contains the papers presented at the Kiel Declarative Programming Days 2013, held in Kiel (Germany) during September 11-13, 2013. The Kiel Declarative Programming Days 2013 unified the following events: * 20th International Conference on Applications of Declarative Programming and Knowledge Management (INAP 2013) * 22nd International Workshop on Functional and (Constraint) Logic Programming (WFLP 2013) * 27th Workshop on Logic Programming (WLP 2013) All these events are centered around declarative programming, an advanced paradigm for the modeling and solving of complex problems. These specification and implementation methods attracted increasing attention over the last decades, e.g., in the domains of databases and natural language processing, for modeling and processing combinatorial problems, and for high-level programming of complex, in particular, knowledge-based systems

Planar plane-wave matrix theory at the four loop order

We study SU(N) plane-wave matrix theory up to fourth perturbative order in its large N planar limit. The effective Hamiltonian in the closed su(2) subsector of the model is explicitly computed through a specially tailored computer program to perform large scale distributed symbolic algebra and generation of planar graphs. The number of graphs here was in the deep billions. The outcome of our computation establishes the four-loop integrability of the planar plane-wave matrix model. To elucidate the integrable structure we apply the recent technology of the perturbative asymptotic Bethe Ansatz to our model. The resulting S-matrix turns out to be structurally similar but nevertheless distinct to the so far considered long-range spin-chain S-matrices of Inozemtsev, Beisert-Dippel-Staudacher and Arutyunov-Frolov-Staudacher in the AdS/CFT context. In particular our result displays a breakdown of BMN scaling at the four-loop order. That is, while there exists an appropriate identification of the matrix theory mass parameter with the coupling constant of the N=4 superconformal Yang-Mills theory which yields an eigth order lattice derivative for well seperated impurities (naively implying BMN scaling) the detailed impurity contact interactions ruin this scaling property at the four-loop order. Moreover we study the issue of ``wrapping'' interactions, which show up for the first time at this loop-order through a Konishi descendant length four operator

Null models and complexity science: disentangling signal from noise in complex interacting systems

The constantly increasing availability of fine-grained data has led to a very detailed description of many socio-economic systems (such as financial markets, interbank loans or supply chains), whose representation, however, quickly becomes too complex to allow for any meaningful intuition or insight about their functioning mechanisms. This, in turn, leads to the challenge of disentangling statistically meaningful information from noise without assuming any a priori knowledge on the particular system under study. The aim of this thesis is to develop and test on real world data unsupervised techniques to extract relevant information from large complex interacting systems. The question I try to answer is the following: is it possible to disentangle statistically relevant information from noise without assuming any prior knowledge about the system under study? In particular, I tackle this challenge from the viewpoint of hypothesis testing by developing techniques based on so-called null models, i.e., partially randomised representations of the system under study. Given that complex systems can be analysed both from the perspective of their time evolution and of their time-aggregated properties, I have tested and developed one technique for each of these two purposes. The first technique I have developed is aimed at extracting “backbones” of relevant relationships in complex interacting systems represented as static weighted networks of pairwise interactions and it is inspired by the well-known Pólya urn combinatorial process. The second technique I have developed is instead aimed at identifying statistically relevant events and temporal patterns in single or multiple time series by means of maximum entropy null models based on Ensemble Theory. Both of these methodologies try to exploit the heterogeneity of complex systems data in order to design null models that are tailored to the systems under study, and therefore capable of identifying signals that are genuinely distinctive of the systems themselves

Contributions of Graph Theory and Algorithms to Animal Behaviour and Neuroscience

Η θεωρία γραφημάτων και οι αλγόριθμοι προσφέρουν πολύτιμες εργαλειοθήκες για τη μοντε- λοποίηση καθώς και την ανάλυση πολυάριθμων φαινομένων στις φυσικές επιστήμες. Εδώ παρουσιάζεται μια ανασκόπηση της σύγχρονης βιβλιογραφίας, χωρισμένη σε τέσσερα κύρια κεφάλαια, δίνοντας κάποιες ενδείξεις για το πώς οι έννοιες αυτών των δύο κλάδων μπορούν να χρησιμοποιηθούν για τη μελέτη της συμπεριφοράς των ζώων και της νευροεπιστήμης. Κατ ’εξαίρεση, το πρώτο μέρος του πρώτου κεφαλαίου παρέχει μια σύντομη συζήτηση σχετικά με τις εφαρμογές της θεωρίας γραφημάτων στη μοριακή βιολογία. Η επιλογή αυτή έγινε προκειμένου να καταστεί η εργασία αυτή πληρέστερη και να δοθεί στους αναγνώστες με διαφορετικό υπόβαθρο, όσο το δυνατόν περισσότερο, συνολική άποψη για τη δυνητική χρησι- μότητα τέτοιων διεπιστημονικών προσεγγίσεων. Τα υπόλοιπα δύο τμήματα του πρώτου κεφα- λαίου εστιάζουν σε δίκτυα του εγκεφάλου και σε κεντρικές έννοιες της θεωρίας γραφημάτων, όπως η κεντρικότητα, στη μελέτη τους. Το δεύτερο κεφάλαιο εισάγει μερικές έννοιες της κοινωνικότητας των ζώων και αναφέρεται σε μελέτες της συνεργασίας στο ζωικό βασίλειο, εστιάζοντας στην εξελικτική θεωρία γραφημάτων και παιγνίων. Επιπλέον, στη τελευταία ενότητα αυτού του κεφαλαίου συζητείται η συλλογική κίνηση ομάδων ζώων, παρέχοντας εκτός των άλλων, εισαγωγή βασικών όρων για το επόμενο τρίτο κεφάλαιο. Η διεπιστημονική έρευνα, με στόχο την ενοποίηση μεθόδων από διαφορετικούς τομείς, λαμβάνει χώρα ευρέως για να απαντήσει βιολογικά ερωτήματα. Εντούτοις, όπως παρουσιάζεται παρακάτω, η έρευνα στους αλγορίθμους και στη βιολογία μπορούν να συμβάλλουν στην ανάπτυξη η μια της άλλης. Ως εκ τούτου, το τρίτο κεφάλαιο παρέχει πληροφορίες σχετικά με αλγόριθμους των οποίων ο σχεδιασμός έχει εμπνευστεί από τη (συλλογική) συμπεριφορά των ζώων στο φυσικό περιβάλλον. Τέλος, το τέταρτο κεφάλαιο αποκλίνει εκ νέου από το επίκεντρο των προηγούμε- νων κεφαλαίων και κάνει μια σύντομη εισαγωγή στο σημαντικό, αλλά και αμφιλεγόμενο, υπολογιστικό χαρακτήρα της νόησης και κατ’ επέκταση της συμπεριφοράς. Συνολικά, μπορεί κανείς να παρατηρήσει ότι η συνεργασία των προαναφερθέντων πεδίων είναι εκτεταμένη ενώ η πραγματοποιημένη έρευνα ανοίγει νέα ερωτήματα που μπορούν να μελετηθούν μόνο υπό το φως τέτοιων διεπιστημονικών συνεργασιών.Graph theory and algorithms offer precious toolboxes for the modelling as well as the analysis of numerous phenomena in natural sciences. Here a review of the modern bibliography is pre- sented, divided in four main chapters, giving some indications on how the concepts of these two disciplines can be used for the study of animal behaviour and neuroscience. As an exception the premier part of the first chapter provides a short discussion on the applications of graph theory on molecular biology. This choice made in order to make this work more complete and give to the readers from various backgrounds an, as much as possible, overall view of the future potential of such interdisciplinary approaches. The rest two sections of the first chapter deals with brain networks and central terms of graph theory, such as centrality, in their study. The second chapter introduces some concepts of animal sociality and refers to studies of animal cooperation, focusing on evolutionary graph and game theory. Moreover, in the last section of this chapter the collective motion of animal groups is discussed providing, into the bargain, an introduction of basic terms for the subsequent third chapter. Interdisciplinary research, aiming to unite methods from different fields, is vastly used in order to answer biological questions. Although, as it is presented below, both the fields of algorithms and biology can contribute to the elaboration of each other. Hence, the third chapter provides information about algorithms whose design has been inspired by the (collective) behaviour of animals in the nature. Finally, the fourth chapter deviates anew from the central focus of the previous chapters and makes a short introduction in the substantial controversial computational nature of cognition and by extension behaviour. Overall, one can observe that the cooperation of the above mentioned fields is extensive while the accomplished research opens new questions which can be studied only in the light of such collaborations