Mutual dependency-based modeling of relevance in co-occurrence data

In the analysis of large data sets it is increasingly important to distinguish the relevant information from the irrelevant. This thesis outlines how to find what is relevant in so-called co-occurrence data, where there are two or more representations for each data sample. The modeling task sets the limits to what we are interested in, and in its part defines the relevance. In this work, the problem of finding what is relevant in data is formalized via dependence, that is, the variation that is found in both (or all) co-occurring data sets was deemed to be more relevant than variation that is present in only one (or some) of the data sets. In other words, relevance is defined through dependencies between the data sets. The method development contributions of this thesis are related to latent topic models and methods of dependency exploration. The dependency-seeking models were extended to nonparametric models, and computational algorithms were developed for the models. The methods are applicable to mutual dependency modeling and co-occurrence data in general, without restriction to the applications presented in the publications of this work. The application areas of the publications included modeling of user interest, relevance prediction of text based on eye movements, analysis of brain imaging with fMRI and modeling of gene regulation in bioinformatics. Additionally, frameworks for different application areas were suggested. Until recently it has been a prevalent convention to assume the data to be normally distributed when modeling dependencies between different data sets. Here, a distribution-free nonparametric extension of Canonical Correlation Analysis (CCA) was suggested, together with a computationally more efficient semi-parametric variant. Furthermore, an alternative view to CCA was derived which allows a new kind of interpretation of the results and using CCA in feature selection that regards dependency as the criterion of relevance. Traditionally, latent topic models are one-way clustering models, that is, one of the variables is clustered by the latent variable. We proposed a latent topic model that generalizes in two ways and showed that when only a small amount of data has been gathered, two-way generalization becomes necessary. In the field of brain imaging, natural stimuli in fMRI studies imitate real-life situations and challenge the analysis methods used. A novel two-step framework was proposed for analyzing brain imaging measurements from fMRI. This framework seems promising for the analysis of brain signal data measured under natural stimulation, once such measurements are more widely available

Multi-Modal Learning For Adaptive Scene Understanding

Modern robotics systems typically possess sensors of different modalities. Segmenting scenes observed by the robot into a discrete set of classes is a central requirement for autonomy. Equally, when a robot navigates through an unknown environment, it is often necessary to adjust the parameters of the scene segmentation model to maintain the same level of accuracy in changing situations. This thesis explores efficient means of adaptive semantic scene segmentation in an online setting with the use of multiple sensor modalities. First, we devise a novel conditional random field(CRF) inference method for scene segmentation that incorporates global constraints, enforcing particular sets of nodes to be assigned the same class label. To do this efficiently, the CRF is formulated as a relaxed quadratic program whose maximum a posteriori(MAP) solution is found using a gradient-based optimization approach. These global constraints are useful, since they can encode "a priori" information about the final labeling. This new formulation also reduces the dimensionality of the original image-labeling problem. The proposed model is employed in an urban street scene understanding task. Camera data is used for the CRF based semantic segmentation while global constraints are derived from 3D laser point clouds. Second, an approach to learn CRF parameters without the need for manually labeled training data is proposed. The model parameters are estimated by optimizing a novel loss function using self supervised reference labels, obtained based on the information from camera and laser with minimum amount of human supervision. Third, an approach that can conduct the parameter optimization while increasing the model robustness to non-stationary data distributions in the long trajectories is proposed. We adopted stochastic gradient descent to achieve this goal by using a learning rate that can appropriately grow or diminish to gain adaptability to changes in the data distribution

Quantifying scale relationships in snow distributions

2007 Summer.Includes bibliographic references.Spatial distributions of snow in mountain environments represent the time integration of accumulation and ablation processes, and are strongly and dynamically linked to mountain hydrologic, ecologic, and climatic systems. Accurate measurement and modeling of the spatial distribution and variability of the seasonal mountain snowpack at different scales are imperative for water supply and hydropower decision-making, for investigations of land-atmosphere interaction or biogeochemical cycling, and for accurate simulation of earth system processes and feedbacks. Assessment and prediction of snow distributions in complex terrain are heavily dependent on scale effects, as the pattern and magnitude of variability in snow distributions depends on the scale of observation. Measurement and model scales are usually different from process scales, and thereby introduce a scale bias to the estimate or prediction. To quantify this bias, or to properly design measurement schemes and model applications, the process scale must be known or estimated. Airborne Light Detection And Ranging (lidar) products provide high-resolution, broad-extent altimetry data for terrain and snowpack mapping, and allow an application of variogram fractal analysis techniques to characterize snow depth scaling properties over lag distances from 1 to 1000 meters. Snow depth patterns as measured by lidar at three Colorado mountain sites exhibit fractal (power law) scaling patterns over two distinct scale ranges, separated by a distinct break at the 15-40 m lag distance, depending on the site. Each fractal range represents a range of separation distances over which snow depth processes remain consistent. The scale break between fractal regions is a characteristic scale at which snow depth process relationships change fundamentally. Similar scale break distances in vegetation topography datasets suggest that the snow depth scale break represents a change in wind redistribution processes from wind/vegetation interactions at small lags to wind/terrain interactions at larger lags. These snow depth scale characteristics are interannually consistent, directly describe the scales of action of snow accumulation, redistribution, and ablation processes, and inform scale considerations for measurement and modeling. Snow process models are designed to represent processes acting over specific scale ranges. However, since the incorporated processes vary with scale, the model performance cannot be scale-independent. Thus, distributed snow models must represent the appropriate process interactions at each scale in order to produce reasonable simulations of snow depth or snow water equivalent (SWE) variability. By comparing fractal dimensions and scale break lengths of modeled snow depth patterns to those derived from lidar observations, the model process representations can be evaluated and subsequently refined. Snow depth simulations from the SnowModel seasonal snow process model exhibit fractal patterns, and a scale break can be produced by including a sub-model that simulates fine-scale wind drifting patterns. The fractal dimensions provide important spatial scaling information that can inform refinement of process representations. This collection of work provides a new application of methods developed in other geophysical fields for quantifying scale and variability relationships

Object detection for big data

"May 2014."Dissertation supervisor: Dr. Tony X. Han.Includes vita.We have observed significant advances in object detection over the past few decades and gladly seen the related research has began to contribute to the world: Vehicles could automatically stop before hitting any pedestrian; Face detectors have been integrated into smart phones and tablets; Video surveillance systems could locate the suspects and stop crimes. All these applications demonstrate the substantial research progress on object detection. However learning a robust object detector is still quite challenging due to the fact that object detection is a very unbalanced big data problem. In this dissertation, we aim at improving the object detector's performance from different aspects. For object detection, the state-of-the-art performance is achieved through supervised learning. The performances of object detectors of this kind are mainly determined by two factors: features and underlying classification algorithms. We have done thorough research on both of these factors. Our contribution involves model adaption, local learning, contextual boosting, template learning and feature development. Since the object detection is an unbalanced problem, in which positive examples are hard to be collected, we propose to adapt a general object detector for a specific scenario with a few positive examples; To handle the large intra-class variation problem lying in object detection task, we propose a local adaptation method to learn a set of efficient and effective detectors for a single object category; To extract the effective context from the huge amount of negative data in object detection, we introduce a novel contextual descriptor to iteratively improve the detector; To detect object with a depth sensor, we design an effective depth descriptor; To distinguish the object categories with the similar appearance, we propose a local feature embedding and template selection algorithm, which has been successfully incorporated into a real-world fine-grained object recognition application. All the proposed algorithms and featuIncludes bibliographical references (pages 117-130)

Cell Memory in the Mitogen-Activated Protein Kinase Signaling Pathway

Cells process information from their environment, such as the stimuli to grow, divide, or die, via cell signaling. Deregulated processing of extracellular stimuli can lead to aberrant cell responses and cause cancer. Given that the in vivo cell environment constantly changes, it is important to understand how cells incorporate the context of their environment into their decision making processes. The idea of responding to relative, not absolute, changes in stimuli was first proposed in studies of human perception and became known as Weber's Law. Although, evidence of Weber's Law at the molecular level has been previously presented in studies of several organisms, to the best of our knowledge, it has never been explored in the case of relative sensing of extracellular stimuli in mammalian signaling cascades. The Mitogen-Activated Protein Kinase (MAPK) signaling pathway has been implicated in multiple human diseases, including cancers, and therefore cell signaling through this pathway is an important subject of research. Here we present a theoretical framework and an experimental validation of the mechanism of Weber's Law in the ability of cells to sense relative changes in the levels of extracellular stimuli in the MAPK signaling pathway. In particular, in this work we consider relative sensing in levels of Epidermal Growth Factor (EGF) in the MAPK pathway. We derive an analytical model of steady state behavior of the MAPK signaling pathway stimulated with constant doses of EGF. We demonstrate a mechanism that produces phosphorylation responses proportional to relative changes in ligand concentrations. The mechanism of Weber's Law presented here entails the retention of memory of the dose of the past chronic stimulation with EGF. The molecular mechanisms responsible for Weber's Law in MAPK signaling are likely to contribute to many other receptors signaling systems. Therefore, the mechanism of relative sensing of extracellular ligand concentrations derived here can be generalized beyond the EGF-activated MAPK signaling pathway to many other cell signaling systems. This thesis also presents a probabilistic framework to explore the parameter space of a detailed mechanistic ODE model of EGFR signaling cascades. The application of the model simulation allows us to generate probabilistic predictions of EGFR system behavior and to explore structure-to-function relationships between the model's parameter space and EGFR system responses. Overall, this work suggests an alternative view on the role of cellular endocytosis in the MAPK signaling in vivo. Specifically, traditionally viewed as a mechanism to downregulate and terminate cell signaling, endocytosis may enable cells to dynamically adjust their sensitivity to extracellular stimuli, and hence allow cells to integrate information about the past stimulations into the cell responses to the consequent stimulations and thus, cell fate decisions

Modeling, Predicting and Capturing Human Mobility

Realistic models of human mobility are critical for modern day applications, specifically for recommendation systems, resource planning and process optimization domains. Given the rapid proliferation of mobile devices equipped with Internet connectivity and GPS functionality today, aggregating large sums of individual geolocation data is feasible. The thesis focuses on methodologies to facilitate data-driven mobility modeling by drawing parallels between the inherent nature of mobility trajectories, statistical physics and information theory. On the applied side, the thesis contributions lie in leveraging the formulated mobility models to construct prediction workflows by adopting a privacy-by-design perspective. This enables end users to derive utility from location-based services while preserving their location privacy. Finally, the thesis presents several approaches to generate large-scale synthetic mobility datasets by applying machine learning approaches to facilitate experimental reproducibility

Evolutionary games on graphs

Game theory is one of the key paradigms behind many scientific disciplines from biology to behavioral sciences to economics. In its evolutionary form and especially when the interacting agents are linked in a specific social network the underlying solution concepts and methods are very similar to those applied in non-equilibrium statistical physics. This review gives a tutorial-type overview of the field for physicists. The first three sections introduce the necessary background in classical and evolutionary game theory from the basic definitions to the most important results. The fourth section surveys the topological complications implied by non-mean-field-type social network structures in general. The last three sections discuss in detail the dynamic behavior of three prominent classes of models: the Prisoner's Dilemma, the Rock-Scissors-Paper game, and Competing Associations. The major theme of the review is in what sense and how the graph structure of interactions can modify and enrich the picture of long term behavioral patterns emerging in evolutionary games.Comment: Review, final version, 133 pages, 65 figure

Modeling Pedestrian Behavior in Video

The purpose of this dissertation is to address the problem of predicting pedestrian movement and behavior in and among crowds. Specifically, we will focus on an agent based approach where pedestrians are treated individually and parameters for an energy model are trained by real world video data. These learned pedestrian models are useful in applications such as tracking, simulation, and artificial intelligence. The applications of this method are explored and experimental results show that our trained pedestrian motion model is beneficial for predicting unseen or lost tracks as well as guiding appearance based tracking algorithms. The method we have developed for training such a pedestrian model operates by optimizing a set of weights governing an aggregate energy function in order to minimize a loss function computed between a model\u27s prediction and annotated ground-truth pedestrian tracks. The formulation of the underlying energy function is such that using tight convex upper bounds, we are able to efficiently approximate the derivative of the loss function with respect to the parameters of the model. Once this is accomplished, the model parameters are updated using straightforward gradient descent techniques in order to achieve an optimal solution. This formulation also lends itself towards the development of a multiple behavior model. The multiple pedestrian behavior styles, informally referred to as stereotypes , are common in real data. In our model we show that it is possible, due to the unique ability to compute the derivative of the loss function, to build a new model which utilizes a soft-minimization of single behavior models. This allows unsupervised training of multiple different behavior models in parallel. This novel extension makes our method unique among other methods in the attempt to accurately describe human pedestrian behavior for the myriad of applications that exist. The ability to describe multiple behaviors shows significant improvements in the task of pedestrian motion prediction

Vadose Zone Transport Field Study: Summary Report

From FY 2000 through FY 2003, a series of vadose zone transport field experiments were conducted as part of the U.S. Department of Energy’s Groundwater/Vadose Zone Integration Project Science and Technology Project, now known as the Remediation and Closure Science Project, and managed by the Pacific Northwest National Laboratory (PNNL). The series of experiments included two major field campaigns, one at a 299-E24-11 injection test site near PUREX and a second at a clastic dike site off Army Loop Road. The goals of these experiments were to improve our understanding of vadose zone transport processes; to develop data sets to validate and calibrate vadose zone flow and transport models; and to identify advanced monitoring techniques useful for evaluating flow-and-transport mechanisms and delineating contaminant plumes in the vadose zone at the Hanford Site. This report summarizes the key findings from the field studies and demonstrates how data collected from these studies are being used to improve conceptual models and develop numerical models of flow and transport in Hanford’s vadose zone. Results of these tests have led to a better understanding of the vadose zone. Fine-scale geologic heterogeneities, including grain fabric and lamination, were observed to have a strong effect on the large-scale behavior of contaminant plumes, primarily through increased lateral spreading resulting from anisotropy. Conceptual models have been updated to include lateral spreading and numerical models of unsaturated flow and transport have revised accordingly. A new robust model based on the concept of a connectivity tensor was developed to describe saturation-dependent anisotropy in strongly heterogeneous soils and has been incorporated into PNNL’s Subsurface Transport Over Multiple Phases (STOMP) simulator. Application to field-scale transport problems have led to a better understanding plume behavior at a number of sites where lateral spreading may have dominated waste migration (e.g. BC Cribs and Trenches). The improved models have been also coupled with inverse models and newly-developed parameter scaling techniques to allow estimation of field-scale and effective transport parameters for the vadose zone. The development and utility of pedotransfer functions for describing fine-scale hydrogeochemical heterogeneity and for incorporating this heterogeneity into reactive transport models was explored. An approach based on grain-size statistics appears feasible and has been used to describe heterogeneity in hydraulic properties and sorption properties, such as the cation exchange capacity and the specific surface area of Hanford sediments. This work has also led to the development of inverse modeling capabilities for time-dependent, subsurface, reactive transport with transient flow fields using an automated optimization algorithm. In addition, a number of geophysical techniques investigated for their potential to provide detailed information on the subtle changes in lithology and bedding surfaces; plume delineation, leak detection. High-resolution resistivity is now being used for detecting saline plumes at several waste sites at Hanford, including tank farms. Results from the field studies and associated analysis have appeared in more than 46 publications generated over the past 4 years. These publications include test plans and status reports, in addition to numerous technical notes and peer reviewed papers