Incentivizing Exploration with Heterogeneous Value of Money

Recently, Frazier et al. proposed a natural model for crowdsourced exploration of different a priori unknown options: a principal is interested in the long-term welfare of a population of agents who arrive one by one in a multi-armed bandit setting. However, each agent is myopic, so in order to incentivize him to explore options with better long-term prospects, the principal must offer the agent money. Frazier et al. showed that a simple class of policies called time-expanded are optimal in the worst case, and characterized their budget-reward tradeoff. The previous work assumed that all agents are equally and uniformly susceptible to financial incentives. In reality, agents may have different utility for money. We therefore extend the model of Frazier et al. to allow agents that have heterogeneous and non-linear utilities for money. The principal is informed of the agent's tradeoff via a signal that could be more or less informative. Our main result is to show that a convex program can be used to derive a signal-dependent time-expanded policy which achieves the best possible Lagrangian reward in the worst case. The worst-case guarantee is matched by so-called "Diamonds in the Rough" instances; the proof that the guarantees match is based on showing that two different convex programs have the same optimal solution for these specific instances. These results also extend to the budgeted case as in Frazier et al. We also show that the optimal policy is monotone with respect to information, i.e., the approximation ratio of the optimal policy improves as the signals become more informative.Comment: WINE 201

A Factor Graph Approach to Automated Design of Bayesian Signal Processing Algorithms

The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming paradigm, namely message passing in Forney-style factor graphs (FFGs), in the context of automated design of efficient Bayesian signal processing algorithms. To this end, we developed "ForneyLab" (https://github.com/biaslab/ForneyLab.jl) as a Julia toolbox for message passing-based inference in FFGs. We show by example how ForneyLab enables automatic derivation of Bayesian signal processing algorithms, including algorithms for parameter estimation and model comparison. Crucially, due to the modular makeup of the FFG framework, both the model specification and inference methods are readily extensible in ForneyLab. In order to test this framework, we compared variational message passing as implemented by ForneyLab with automatic differentiation variational inference (ADVI) and Monte Carlo methods as implemented by state-of-the-art tools "Edward" and "Stan". In terms of performance, extensibility and stability issues, ForneyLab appears to enjoy an edge relative to its competitors for automated inference in state-space models.Comment: Accepted for publication in the International Journal of Approximate Reasonin

Security, trust and cooperation in wireless sensor networks

Wireless sensor networks are a promising technology for many real-world applications such as critical infrastructure monitoring, scientific data gathering, smart buildings, etc.. However, given the typically unattended and potentially unsecured operation environment, there has been an increased number of security threats to sensor networks. In addition, sensor networks have very constrained resources, such as limited energy, memory, computational power, and communication bandwidth. These unique challenges call for new security mechanisms and algorithms. In this dissertation, we propose novel algorithms and models to address some important and challenging security problems in wireless sensor networks. The first part of the dissertation focuses on data trust in sensor networks. Since sensor networks are mainly deployed to monitor events and report data, the quality of received data must be ensured in order to make meaningful inferences from sensor data. We first study a false data injection attack in the distributed state estimation problem and propose a distributed Bayesian detection algorithm, which could maintain correct estimation results when less than one half of the sensors are compromised. To deal with the situation where more than one half of the sensors may be compromised, we introduce a special class of sensor nodes called \textit{trusted cores}. We then design a secure distributed trust aggregation algorithm that can utilize the trusted cores to improve network robustness. We show that as long as there exist some paths that can connect each regular node to one of these trusted cores, the network can not be subverted by attackers. The second part of the dissertation focuses on sensor network monitoring and anomaly detection. A sensor network may suffer from system failures due to loss of links and nodes, or malicious intrusions. Therefore, it is critical to continuously monitor the overall state of the network and locate performance anomalies. The network monitoring and probe selection problem is formulated as a budgeted coverage problem and a Markov decision process. Efficient probing strategies are designed to achieve a flexible tradeoff between inference accuracy and probing overhead. Based on the probing results on traffic measurements, anomaly detection can be conducted. To capture the highly dynamic network traffic, we develop a detection scheme based on multi-scale analysis of the traffic using wavelet transforms and hidden Markov models. The performance of the probing strategy and of the detection scheme are extensively evaluated in malicious scenarios using the NS-2 network simulator. Lastly, to better understand the role of trust in sensor networks, a game theoretic model is formulated to mathematically analyze the relation between trust and cooperation. Given the trust relations, the interactions among nodes are modeled as a network game on a trust-weighted graph. We then propose an efficient heuristic method that explores network heterogeneity to improve Nash equilibrium efficiency