17 research outputs found
Natively probabilistic computation
Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2009.Includes bibliographical references (leaves 129-135).I introduce a new set of natively probabilistic computing abstractions, including probabilistic generalizations of Boolean circuits, backtracking search and pure Lisp. I show how these tools let one compactly specify probabilistic generative models, generalize and parallelize widely used sampling algorithms like rejection sampling and Markov chain Monte Carlo, and solve difficult Bayesian inference problems. I first introduce Church, a probabilistic programming language for describing probabilistic generative processes that induce distributions, which generalizes Lisp, a language for describing deterministic procedures that induce functions. I highlight the ways randomness meshes with the reflectiveness of Lisp to support the representation of structured, uncertain knowledge, including nonparametric Bayesian models from the current literature, programs for decision making under uncertainty, and programs that learn very simple programs from data. I then introduce systematic stochastic search, a recursive algorithm for exact and approximate sampling that generalizes a popular form of backtracking search to the broader setting of stochastic simulation and recovers widely used particle filters as a special case. I use it to solve probabilistic reasoning problems from statistical physics, causal reasoning and stereo vision. Finally, I introduce stochastic digital circuits that model the probability algebra just as traditional Boolean circuits model the Boolean algebra.(cont.) I show how these circuits can be used to build massively parallel, fault-tolerant machines for sampling and allow one to efficiently run Markov chain Monte Carlo methods on models with hundreds of thousands of variables in real time. I emphasize the ways in which these ideas fit together into a coherent software and hardware stack for natively probabilistic computing, organized around distributions and samplers rather than deterministic functions. I argue that by building uncertainty and randomness into the foundations of our programming languages and computing machines, we may arrive at ones that are more powerful, flexible and efficient than deterministic designs, and are in better alignment with the needs of computational science, statistics and artificial intelligence.by Vikash Kumar Mansinghka.Ph.D
Visual scene recognition with biologically relevant generative models
This research focuses on developing visual object categorization methodologies that are based on machine learning techniques and biologically inspired generative models of visual scene recognition. Modelling the statistical variability in visual patterns, in the space of features extracted from them by an appropriate low level signal processing technique, is an important matter of investigation for both humans and machines. To study this problem, we have examined in detail two recent probabilistic models of vision: a simple multivariate Gaussian model as suggested by (Karklin & Lewicki, 2009) and a restricted Boltzmann machine (RBM) proposed by (Hinton, 2002). Both the models have been widely used for visual object classification and scene analysis tasks before. This research highlights that these models on their own are not plausible enough to perform the classification task, and suggests Fisher kernel as a means of inducing discrimination into these models for classification power. Our empirical results on standard benchmark data sets reveal that the classification performance of these generative models could be significantly boosted near to the state of the art performance, by drawing a Fisher kernel from compact generative models that computes the data labels in a fraction of total computation time. We compare the proposed technique with other distance based and kernel based classifiers to show how computationally efficient the Fisher kernels are. To the best of our knowledge, Fisher kernel has not been drawn from the RBM before, so the work presented in the thesis is novel in terms of its idea and application to vision problem
Probabilistic models for human behavior learning
The problem of human behavior learning is a popular interdisciplinary research topic that
has been explored from multiple perspectives, with a principal branch of study in the
context of computer vision systems and activity recognition. However, the statistical methods
used in these frameworks typically assume short time scales, usually of minutes or even
seconds. The emergence of mobile electronic devices, such as smartphones and wearables,
has changed this paradigm as long as we are now able to massively collect digital records
from users. This collection of smartphone-generated data, whose attributes are obtained in
an unobtrusive manner from the devices via multiple sensors and apps, shape the behavioral
footprint that is unique for everyone of us. At an individual level, the data projection also
di ers from person to person, as not all sensors are equal, neither the apps installed, or the
devices used in the real life. This point actually reflects that learning the human behavior
from the digital signature of users is an arduous task, that requires to fuse irregular data.
For instance, collections of samples that are corrupted, heterogeneous, outliers or have shortterm
correlations. The statistical modelling of this sort of objects is one of the principal
contributions of this thesis, that we study from the perspective of Gaussian processes (gp).
In the particular case of humans, as well as many other life species in our world, we are
inherently conditioned to the diurnal and nocturnal cycles that everyday shape our behavior,
and hence, our data. We can study these cycles in our behavioral representation to see that
there exists a perpetual circadian rhytm in everyone of us. This tempo is the 24h periodic
component that shapes the baseline temporal structure of our behavior, not the particular
patterns that change for every person. Looking to the trajectories and variabilities that our
behavior may take in the data, we can appreciate that there is not a single repetitive behavior.
Instead, there are typically several patterns or routines, sampled from our own dictionary,
that we choose for every special situation. At the same time, these routines are arbitrary
combinations of di erents timescales, correlations, levels of mobility, social interaction, sleep
quality or will for working during the same hours on weekdays. Together, the properties of
human behavior already indicate to us how we shall proceed to model its structure, not as
unique functions, but as a dictionary of latent behavioral profiles. To discover them, we have
considered latent variable models.
The main application of the statistical methods developed for human behavior learning
appears as we look to medicine. Having a personalized model that is accurately fitted to
the behavioral patterns of some patient of interest, sudden changes in them could be early
indicators of future relapses. From a technical point of view, the traditional question use to
be if newer observations conform or not to the expected behavior indicated by the already
fitted model. The problem can be analyzed from two perspectives that are interrelated, one
more oriented to the characterization of that single object as outlier, typically named as
anomaly detection, and another focused in refreshing the learning model if no longer fits to
the new sequential data. This last problem, widely known as change-point detection (cpd)
is another pillar of this thesis. These methods are oriented to mental health applications,
and particularly to the passive detection of crisis events. The final goal is to provide an
early detection methodology based on probabilistic modeling for early intervention, e.g. prevent
suicide attempts, on psychiatric outpatients with severe a ective disorders of higher
prevalence, such as depression or bipolar diseases.El problema de aprendizaje del comportamiento humano es un tema de investigación interdisciplinar
que ha sido explorado desde múltiples perspectivas, con una lÃnea de estudio
principal en torno a los sistemas de visión por ordenador y el reconocimiento de actividades.
Sin embargo, los métodos estadÃsticos usados en estos casos suelen asumir escalas de tiempo
cortas, generalmente de minutos o incluso segundos. La aparición de tecnologÃas móviles,
tales como teléfonos o relojes inteligentes, ha cambiado este paradigma, dado que ahora es
posible recolectar ingentes colecciones de datos a partir de los usuarios. Este conjunto de
datos generados a partir de nuestro teléfono, cuyos atributos se obtienen de manera no invasiva
desde múltiples sensores y apps, conforman la huella de comportamiento que es única
para cada uno de nosotros. A nivel individual, la proyección sobre los datos difiere de persona
a persona, dado que no todos los sensores son iguales, ni las apps instaladas asà como
los dispositivos utilizados en la vida real. Esto precisamente refleja que el aprendizaje del
comportamiento humano a partir de la huella digital de los usuarios es una ardua tarea,
que requiere principalmente fusionar datos irregulares. Por ejemplo, colecciones de muestras
corruptas, heterogéneas, con outliers o poseedoras de correlaciones cortas. El modelado estadÃstico de este tipo de objetos es una de las contribuciones principales de esta tesis, que
estudiamos desde la perspectiva de los procesos Gaussianos (gp).
En el caso particular de los humanos, asà como para muchas otras especies en nuestro
planeta, estamos inherentemente condicionados a los ciclos diurnos y nocturnos que cada
dÃa dan forma a nuestro comportamiento, y por tanto, a nuestros datos. Podemos estudiar
estos ciclos en la representación del comportamiento que obtenemos y ver que realmente
existe un ritmo circadiano perpetuo en cada uno de nosotros. Este tempo es en realidad
la componente periódica de 24 horas que construye la base sobre la que se asienta nuestro
comportamiento, no únicamente los patrones que cambian para cada persona. Mirando a las
trayectorias y variabilidades que nuestro comportamiento puede plasmar en los datos, podemos
apreciar que no existe un comportamiento único y repetitivo. En su lugar, hay varios
patrones o rutinas, obtenidas de nuestro propio diccionario, que elegimos para cada situación
especial. Al mismo tiempo, estas rutinas son combinaciones arbitrarias de diferentes escalas
de tiempo, correlaciones, niveles de movilidad, interacción social, calidad del sueño o iniciativa
para trabajar durante las mismas horas cada dÃa laborable. Juntas, estas propiedades
del comportamiento humano nos indican como debemos proceder a modelar su estructura,
no como funciones únicas, sino como un diccionario de perfiles ocultos de comportamiento,
Para descubrirlos, hemos considerado modelos de variables latentes.
La aplicación principal de los modelos estadÃsticos desarrollados para el aprendizaje de
comportamiento humano aparece en cuanto miramos a la medicina. Teniendo un modelo
personalizado que está ajustado de una manera precisa a los patrones de comportamiento
de un paciente, los cambios espontáneos en ellos pueden ser indicadores de futuras recaÃdas.
Desde un punto de vista técnico, la pregunta clásica suele ser si nuevas observaciones encajan
o no con lo indicado por el modelo. Este problema se puede enfocar desde dos perspectivas
que están interrelacionadas, una más orientada a la caracterización de aquellos objetos como
outliers, que usualmente se conoce como detección de anomalÃas, y otro enfocado en refrescar
el modelo de aprendizaje si este deja de ajustarse debidamente a los nuevos datos secuenciales.
Este último problema, ampliamente conocido como detección de puntos de cambio (cpd) es otro de los pilares de esta tesis. Estos métodos se han orientado a aplicaciones de salud
mental, y particularmente, a la detección pasiva de eventos crÃticos. El objetivo final es
proveer de una metodologÃa de detección temprana basada en el modelado probabilÃstico
para intervenciones rápidas. Por ejemplo, de cara a prever intentos de suicidio en pacientes
fuera de hospitales con trastornos afectivos severos de gran prevalencia, como depresión o
sÃndrome bipolar.Programa de Doctorado en Multimedia y Comunicaciones por la Universidad Carlos III de Madrid y la Universidad Rey Juan CarlosPresidente: Pablo MartÃnez Olmos.- Secretario: Daniel Hernández Lobato.- Vocal: Javier González Hernánde