83,704 research outputs found
The Statistical Performance of Collaborative Inference
The statistical analysis of massive and complex data sets will require the
development of algorithms that depend on distributed computing and
collaborative inference. Inspired by this, we propose a collaborative framework
that aims to estimate the unknown mean of a random variable . In
the model we present, a certain number of calculation units, distributed across
a communication network represented by a graph, participate in the estimation
of by sequentially receiving independent data from while
exchanging messages via a stochastic matrix defined over the graph. We give
precise conditions on the matrix under which the statistical precision of
the individual units is comparable to that of a (gold standard) virtual
centralized estimate, even though each unit does not have access to all of the
data. We show in particular the fundamental role played by both the non-trivial
eigenvalues of and the Ramanujan class of expander graphs, which provide
remarkable performance for moderate algorithmic cost
Gaussian-Gamma collaborative filtering: a hierarchical Bayesian model for recommender systems
The traditional collaborative filtering (CF) suffers from two key challenges, namely, the normal assumption that it is not robust, and it is difficult to set in advance the penalty terms of the latent features. We therefore propose a hierarchical Bayesian model-based CF and the related inference algorithm. Specifically, we impose a Gaussian-Gamma prior on the ratings, and the latent features. We show the model is more robust, and the penalty terms can be adapted automatically in the inference. We use Gibbs sampler for the inference and provide a statistical explanation. We verify the performance using both synthetic and real dataset
Predictive intelligence to the edge through approximate collaborative context reasoning
We focus on Internet of Things (IoT) environments where a network of sensing and computing devices are responsible to locally process contextual data, reason and collaboratively infer the appearance of a specific phenomenon (event). Pushing processing and knowledge inference to the edge of the IoT network allows the complexity of the event reasoning process to be distributed into many manageable pieces and to be physically located at the source of the contextual information. This enables a huge amount of rich data streams to be processed in real time that would be prohibitively complex and costly to deliver on a traditional centralized Cloud system. We propose a lightweight, energy-efficient, distributed, adaptive, multiple-context perspective event reasoning model under uncertainty on each IoT device (sensor/actuator). Each device senses and processes context data and infers events based on different local context perspectives: (i) expert knowledge on event representation, (ii) outliers inference, and (iii) deviation from locally predicted context. Such novel approximate reasoning paradigm is achieved through a contextualized, collaborative belief-driven clustering process, where clusters of devices are formed according to their belief on the presence of events. Our distributed and federated intelligence model efficiently identifies any localized abnormality on the contextual data in light of event reasoning through aggregating local degrees of belief, updates, and adjusts its knowledge to contextual data outliers and novelty detection. We provide comprehensive experimental and comparison assessment of our model over real contextual data with other localized and centralized event detection models and show the benefits stemmed from its adoption by achieving up to three orders of magnitude less energy consumption and high quality of inference
Collaborative Training in Sensor Networks: A graphical model approach
Graphical models have been widely applied in solving distributed inference
problems in sensor networks. In this paper, the problem of coordinating a
network of sensors to train a unique ensemble estimator under communication
constraints is discussed. The information structure of graphical models with
specific potential functions is employed, and this thus converts the
collaborative training task into a problem of local training plus global
inference. Two important classes of algorithms of graphical model inference,
message-passing algorithm and sampling algorithm, are employed to tackle
low-dimensional, parametrized and high-dimensional, non-parametrized problems
respectively. The efficacy of this approach is demonstrated by concrete
examples
Collaborative Uploading in Heterogeneous Networks: Optimal and Adaptive Strategies
Collaborative uploading describes a type of crowdsourcing scenario in
networked environments where a device utilizes multiple paths over neighboring
devices to upload content to a centralized processing entity such as a cloud
service. Intermediate devices may aggregate and preprocess this data stream.
Such scenarios arise in the composition and aggregation of information, e.g.,
from smartphones or sensors. We use a queuing theoretic description of the
collaborative uploading scenario, capturing the ability to split data into
chunks that are then transmitted over multiple paths, and finally merged at the
destination. We analyze replication and allocation strategies that control the
mapping of data to paths and provide closed-form expressions that pinpoint the
optimal strategy given a description of the paths' service distributions.
Finally, we provide an online path-aware adaptation of the allocation strategy
that uses statistical inference to sequentially minimize the expected waiting
time for the uploaded data. Numerical results show the effectiveness of the
adaptive approach compared to the proportional allocation and a variant of the
join-the-shortest-queue allocation, especially for bursty path conditions.Comment: 15 pages, 11 figures, extended version of a conference paper accepted
for publication in the Proceedings of the IEEE International Conference on
Computer Communications (INFOCOM), 201
- …