Generating Labels for Regression of Subjective Constructs using Triplet Embeddings

Human annotations serve an important role in computational models where the target constructs under study are hidden, such as dimensions of affect. This is especially relevant in machine learning, where subjective labels derived from related observable signals (e.g., audio, video, text) are needed to support model training and testing. Current research trends focus on correcting artifacts and biases introduced by annotators during the annotation process while fusing them into a single annotation. In this work, we propose a novel annotation approach using triplet embeddings. By lifting the absolute annotation process to relative annotations where the annotator compares individual target constructs in triplets, we leverage the accuracy of comparisons over absolute ratings by human annotators. We then build a 1-dimensional embedding in Euclidean space that is indexed in time and serves as a label for regression. In this setting, the annotation fusion occurs naturally as a union of sets of sampled triplet comparisons among different annotators. We show that by using our proposed sampling method to find an embedding, we are able to accurately represent synthetic hidden constructs in time under noisy sampling conditions. We further validate this approach using human annotations collected from Mechanical Turk and show that we can recover the underlying structure of the hidden construct up to bias and scaling factors.Comment: 9 pages, 5 figures, accepted journal pape

Decision-Making with Heterogeneous Sensors - A Copula Based Approach

Statistical decision making has wide ranging applications, from communications and signal processing to econometrics and finance. In contrast to the classical one source-one receiver paradigm, several applications have been identified in the recent past that require acquiring data from multiple sources or sensors. Information from the multiple sensors are transmitted to a remotely located receiver known as the fusion center which makes a global decision. Past work has largely focused on fusion of information from homogeneous sensors. This dissertation extends the formulation to the case when the local sensors may possess disparate sensing modalities. Both the theoretical and practical aspects of multimodal signal processing are considered. The first and foremost challenge is to \u27adequately\u27 model the joint statistics of such heterogeneous sensors. We propose the use of copula theory for this purpose. Copula models are general descriptors of dependence. They provide a way to characterize the nonlinear functional relationships between the multiple modalities, which are otherwise difficult to formalize. The important problem of selecting the `best\u27 copula function from a given set of valid copula densities is addressed, especially in the context of binary hypothesis testing problems. Both, the training-testing paradigm, where a training set is assumed to be available for learning the copula models prior to system deployment, as well as generalized likelihood ratio test (GLRT) based fusion rule for the online selection and estimation of copula parameters are considered. The developed theory is corroborated with extensive computer simulations as well as results on real-world data. Sensor observations (or features extracted thereof) are most often quantized before their transmission to the fusion center for bandwidth and power conservation. A detection scheme is proposed for this problem assuming unifom scalar quantizers at each sensor. The designed rule is applicable for both binary and multibit local sensor decisions. An alternative suboptimal but computationally efficient fusion rule is also designed which involves injecting a deliberate disturbance to the local sensor decisions before fusion. The rule is based on Widrow\u27s statistical theory of quantization. Addition of controlled noise helps to \u27linearize\u27 the higly nonlinear quantization process thus resulting in computational savings. It is shown that although the introduction of external noise does cause a reduction in the received signal to noise ratio, the proposed approach can be highly accurate when the input signals have bandlimited characteristic functions, and the number of quantization levels is large. The problem of quantifying neural synchrony using copula functions is also investigated. It has been widely accepted that multiple simultaneously recorded electroencephalographic signals exhibit nonlinear and non-Gaussian statistics. While the existing and popular measures such as correlation coefficient, corr-entropy coefficient, coh-entropy and mutual information are limited to being bivariate and hence applicable only to pairs of channels, measures such as Granger causality, even though multivariate, fail to account for any nonlinear inter-channel dependence. The application of copula theory helps alleviate both these limitations. The problem of distinguishing patients with mild cognitive impairment from the age-matched control subjects is also considered. Results show that the copula derived synchrony measures when used in conjunction with other synchrony measures improve the detection of Alzheimer\u27s disease onset

Underwater target detection using multiple disparate sonar platforms

2010 Fall.Includes bibliographical references.The detection of underwater objects from sonar imagery presents a difficult problem due to various factors such as variations in the operating and environmental conditions, presence of spatially varying clutter, and variations in target shapes, compositions, and orientation. Additionally, collecting data from multiple platforms can present more challenging questions such as "how should I collaboratively perform detection to achieve optimal performance?", "how many platforms should be employed?", "when do we reach a point of diminishing return when adding platforms?", or more importantly "when does adding an additional platform not help at all?". To perform multi-platform detection and answer these questions we use the coherent information among all disparate sources of information and perform detection on the premise that the amount of coherent information will be greater in situations where a target is present in a region of interest within an image versus a situation where our observation strictly consists of background clutter. To exploit the coherent information among the different sources, we recast the standard Neyman-Pearson, Gauss-Gauss detector into the Multi-Channel Coherence Analysis (MCA) framework. The MCA framework allows one to optimally decompose the multi-channel data into a new appropriate coordinate system in order to analyze their linear dependence or coherence in a more meaningful fashion. To do this, new expressions for the log-likelihood ratio and J-divergence are formulated in this multichannel coordinate system. Using the MCA framework, the data of each channel is first whitened individually, hence removing the second-order information from each channel. Then, a set of linear mapping matrices are obtained which maximizes the sum of the cross-correlations among the channels in the mapped domain. To perform detection in the coordinate system provided by MCA, we first of all construct a model suited to this multiple sensor platform problem and subsequently represent observations in their MCA coordinates associated with the H1 hypothesis. Performing detection in the MCA framework results in a log-likelihood ratio that is written in terms of the MCA correlations and mapping vectors as well as a local signal-to-noise ratio matrix. In this coordinate system, the J-divergence, which is a measure of the difference in means of the likelihood ratio, can effectively be represented in terms of the multi-channel correlations and mapping vectors. Using this J-divergence expression, one can get a more clear picture of the amount of discriminatory information available for detection by analyzing the amount of coherent information present among the channels. New analytical and experimental results are also presented to provide better insight on the effects of adding a new piece of data to the multi-channel Gauss-Gauss detector represented in the MCA framework. To answer questions like those posed in the first paragraph, one must carefully analyze the amount of discriminatory information that is brought to the detection process when adding observations from an additional channel. Rather than attempting to observe the increase (or lack thereof) from the full detection problem it is advantageous to look at the change incrementally. To accomplish this goal, new updating equations for the likelihood ratio are derived that involve linearly estimating the new data from the old (already existing) and updating the likelihood ratio accordingly. In this case, the change in J-divergence can be written in terms of error covariance matrices under each hypothesis. We then derive a change in coordinate system that can be used to perform dimensionality reduction. This especially becomes useful when the data we wish to add exists in high-dimensional space. To demonstrate the usefulness of log-likelihood updating, we conduct two simulation studies. The first simulation corresponds to detecting the presence of dynamical structure in data we have observed and corresponds to a temporal updating scheme. The second is concerned with detecting the presence of a single narrow-band source using multiple linear sensor arrays in which case we consider a platform (or channel) updating scheme. A comprehensive study is carried out on the MCA-based detector on three data sets acquired from the Naval Surface Warfare Center (NSWC) in Panama City, FL. The first data set consists of one high frequency (HF) and three broadband (BB) side-looking sonar imagery coregistered over the same region on the sea floor captured from an Autonomous Underwater Vehicle (AUV) platform. For this data set we consider three different detection schemes using different combinations of these sonar channels. The next data set consists of one HF and only one BB beamformed sonar imagery again coregistered over the same region on the sea floor. This data set consists of not only target objects but also lobster traps giving us experimental intuition as how the multi-channel correlations change for different object compositions. The use of multiple disparate sonar images, e.g., a high frequency, high resolution sonar with good target definition and a multitude of lower resolution broadband sonar with good clutter suppression ability significantly improves the detection and false alarm rates comparing to situations where only single sonar is utilized. Finally, a data set consisting of synthetically generated images of targets with differing degrees of disparity such as signal-to-noise ratio (SNR), aspect angle, resolution, etc., is used to conduct a thorough sensitivity analysis in order to study the effects of different SNR, target types, and disparateness in aspect angle

Deep Multimodal Image-Repurposing Detection

Nefarious actors on social media and other platforms often spread rumors and falsehoods through images whose metadata (e.g., captions) have been modified to provide visual substantiation of the rumor/falsehood. This type of modification is referred to as image repurposing, in which often an unmanipulated image is published along with incorrect or manipulated metadata to serve the actor's ulterior motives. We present the Multimodal Entity Image Repurposing (MEIR) dataset, a substantially challenging dataset over that which has been previously available to support research into image repurposing detection. The new dataset includes location, person, and organization manipulations on real-world data sourced from Flickr. We also present a novel, end-to-end, deep multimodal learning model for assessing the integrity of an image by combining information extracted from the image with related information from a knowledge base. The proposed method is compared against state-of-the-art techniques on existing datasets as well as MEIR, where it outperforms existing methods across the board, with AUC improvement up to 0.23.Comment: To be published at ACM Multimeda 2018 (orals

Distributed Regression in Sensor Networks: Training Distributively with Alternating Projections

Wireless sensor networks (WSNs) have attracted considerable attention in recent years and motivate a host of new challenges for distributed signal processing. The problem of distributed or decentralized estimation has often been considered in the context of parametric models. However, the success of parametric methods is limited by the appropriateness of the strong statistical assumptions made by the models. In this paper, a more flexible nonparametric model for distributed regression is considered that is applicable in a variety of WSN applications including field estimation. Here, starting with the standard regularized kernel least-squares estimator, a message-passing algorithm for distributed estimation in WSNs is derived. The algorithm can be viewed as an instantiation of the successive orthogonal projection (SOP) algorithm. Various practical aspects of the algorithm are discussed and several numerical simulations validate the potential of the approach.Comment: To appear in the Proceedings of the SPIE Conference on Advanced Signal Processing Algorithms, Architectures and Implementations XV, San Diego, CA, July 31 - August 4, 200

Of McKay Correspondence, Non-linear Sigma-model and Conformal Field Theory

The ubiquitous ADE classification has induced many proposals of often mysterious correspondences both in mathematics and physics. The mathematics side includes quiver theory and the McKay Correspondence which relates finite group representation theory to Lie algebras as well as crepant resolutions of Gorenstein singularities. On the physics side, we have the graph-theoretic classification of the modular invariants of WZW models, as well as the relation between the string theory nonlinear $\sigma$-models and Landau-Ginzburg orbifolds. We here propose a unification scheme which naturally incorporates all these correspondences of the ADE type in two complex dimensions. An intricate web of inter-relations is constructed, providing a possible guideline to establish new directions of research or alternate pathways to the standing problems in higher dimensions.Comment: 35 pages, 4 figures; minor corrections, comments on toric geometry and references adde