Kernel methods in machine learning

We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on nonvectorial data. We cover a wide range of methods, ranging from binary classifiers to sophisticated methods for estimation with structured data.Comment: Published in at http://dx.doi.org/10.1214/009053607000000677 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Probabilistic Programming Concepts

A multitude of different probabilistic programming languages exists today, all extending a traditional programming language with primitives to support modeling of complex, structured probability distributions. Each of these languages employs its own probabilistic primitives, and comes with a particular syntax, semantics and inference procedure. This makes it hard to understand the underlying programming concepts and appreciate the differences between the different languages. To obtain a better understanding of probabilistic programming, we identify a number of core programming concepts underlying the primitives used by various probabilistic languages, discuss the execution mechanisms that they require and use these to position state-of-the-art probabilistic languages and their implementation. While doing so, we focus on probabilistic extensions of logic programming languages such as Prolog, which have been developed since more than 20 years

A survey of statistical network models

Networks are ubiquitous in science and have become a focal point for discussion in everyday life. Formal statistical models for the analysis of network data have emerged as a major topic of interest in diverse areas of study, and most of these involve a form of graphical representation. Probability models on graphs date back to 1959. Along with empirical studies in social psychology and sociology from the 1960s, these early works generated an active network community and a substantial literature in the 1970s. This effort moved into the statistical literature in the late 1970s and 1980s, and the past decade has seen a burgeoning network literature in statistical physics and computer science. The growth of the World Wide Web and the emergence of online networking communities such as Facebook, MySpace, and LinkedIn, and a host of more specialized professional network communities has intensified interest in the study of networks and network data. Our goal in this review is to provide the reader with an entry point to this burgeoning literature. We begin with an overview of the historical development of statistical network modeling and then we introduce a number of examples that have been studied in the network literature. Our subsequent discussion focuses on a number of prominent static and dynamic network models and their interconnections. We emphasize formal model descriptions, and pay special attention to the interpretation of parameters and their estimation. We end with a description of some open problems and challenges for machine learning and statistics.Comment: 96 pages, 14 figures, 333 reference

Probabilistic Constraint Logic Programming

This paper addresses two central problems for probabilistic processing models: parameter estimation from incomplete data and efficient retrieval of most probable analyses. These questions have been answered satisfactorily only for probabilistic regular and context-free models. We address these problems for a more expressive probabilistic constraint logic programming model. We present a log-linear probability model for probabilistic constraint logic programming. On top of this model we define an algorithm to estimate the parameters and to select the properties of log-linear models from incomplete data. This algorithm is an extension of the improved iterative scaling algorithm of Della-Pietra, Della-Pietra, and Lafferty (1995). Our algorithm applies to log-linear models in general and is accompanied with suitable approximation methods when applied to large data spaces. Furthermore, we present an approach for searching for most probable analyses of the probabilistic constraint logic programming model. This method can be applied to the ambiguity resolution problem in natural language processing applications.Comment: 35 pages, uses sfbart.cl

Chromosome classification and speech recognition using inferred Markov networks with empirical landmarks.

by Law Hon Man.Thesis (M.Phil.)--Chinese University of Hong Kong, 1993.Includes bibliographical references (leaves 67-70).Chapter 1 --- Introduction --- p.1Chapter 2 --- Automated Chromosome Classification --- p.4Chapter 2.1 --- Procedures in Chromosome Classification --- p.6Chapter 2.2 --- Sample Preparation --- p.7Chapter 2.3 --- Low Level Processing and Measurement --- p.9Chapter 2.4 --- Feature Extraction --- p.11Chapter 2.5 --- Classification --- p.15Chapter 3 --- Inference of Markov Networks by Dynamic Programming --- p.17Chapter 3.1 --- Markov Networks --- p.18Chapter 3.2 --- String-to-String Correction --- p.19Chapter 3.3 --- String-to-Network Alignment --- p.21Chapter 3.4 --- Forced Landmarks in String-to-Network Alignment --- p.31Chapter 4 --- Landmark Finding in Markov Networks --- p.34Chapter 4.1 --- Landmark Finding without a priori Knowledge --- p.34Chapter 4.2 --- Chromosome Profile Processing --- p.37Chapter 4.3 --- Analysis of Chromosome Networks --- p.39Chapter 4.4 --- Classification Results --- p.45Chapter 5 --- Speech Recognition using Inferred Markov Networks --- p.48Chapter 5.1 --- Linear Predictive Analysis --- p.48Chapter 5.2 --- TIMIT Speech Database --- p.50Chapter 5.3 --- Feature Extraction --- p.51Chapter 5.4 --- Empirical Landmarks in Speech Networks --- p.52Chapter 5.5 --- Classification Results --- p.55Chapter 6 --- Conclusion --- p.57Chapter 6.1 --- Suggested Improvements --- p.57Chapter 6.2 --- Concluding remarks --- p.61Appendix A --- p.63Reference --- p.6