1,979 research outputs found
ElfStore: A Resilient Data Storage Service for Federated Edge and Fog Resources
Edge and fog computing have grown popular as IoT deployments become
wide-spread. While application composition and scheduling on such resources are
being explored, there exists a gap in a distributed data storage service on the
edge and fog layer, instead depending solely on the cloud for data persistence.
Such a service should reliably store and manage data on fog and edge devices,
even in the presence of failures, and offer transparent discovery and access to
data for use by edge computing applications. Here, we present Elfstore, a
first-of-its-kind edge-local federated store for streams of data blocks. It
uses reliable fog devices as a super-peer overlay to monitor the edge
resources, offers federated metadata indexing using Bloom filters, locates data
within 2-hops, and maintains approximate global statistics about the
reliability and storage capacity of edges. Edges host the actual data blocks,
and we use a unique differential replication scheme to select edges on which to
replicate blocks, to guarantee a minimum reliability and to balance storage
utilization. Our experiments on two IoT virtual deployments with 20 and 272
devices show that ElfStore has low overheads, is bound only by the network
bandwidth, has scalable performance, and offers tunable resilience.Comment: 24 pages, 14 figures, To appear in IEEE International Conference on
Web Services (ICWS), Milan, Italy, 201
Providing Transaction Class-Based QoS in In-Memory Data Grids via Machine Learning
Elastic architectures and the ”pay-as-you-go” resource pricing model offered by many cloud infrastructure providers may seem the right choice for companies dealing with data centric applications characterized by high variable workload. In such a context, in-memory transactional data grids have demonstrated to be particularly suited for exploiting advantages provided by elastic computing platforms, mainly thanks to their ability to be dynamically (re-)sized and tuned. Anyway, when specific QoS requirements have to be met, this kind of architectures have revealed to be complex to be managed by humans. Particularly, their management is a very complex task without the stand of mechanisms supporting run-time automatic sizing/tuning of the data platform and the underlying (virtual) hardware resources provided by the cloud. In this paper, we present a neural network-based architecture where the system is constantly and automatically re-configured, particularly in terms of computing resources
Speculative Approximations for Terascale Analytics
Model calibration is a major challenge faced by the plethora of statistical
analytics packages that are increasingly used in Big Data applications.
Identifying the optimal model parameters is a time-consuming process that has
to be executed from scratch for every dataset/model combination even by
experienced data scientists. We argue that the incapacity to evaluate multiple
parameter configurations simultaneously and the lack of support to quickly
identify sub-optimal configurations are the principal causes. In this paper, we
develop two database-inspired techniques for efficient model calibration.
Speculative parameter testing applies advanced parallel multi-query processing
methods to evaluate several configurations concurrently. The number of
configurations is determined adaptively at runtime, while the configurations
themselves are extracted from a distribution that is continuously learned
following a Bayesian process. Online aggregation is applied to identify
sub-optimal configurations early in the processing by incrementally sampling
the training dataset and estimating the objective function corresponding to
each configuration. We design concurrent online aggregation estimators and
define halting conditions to accurately and timely stop the execution. We apply
the proposed techniques to distributed gradient descent optimization -- batch
and incremental -- for support vector machines and logistic regression models.
We implement the resulting solutions in GLADE PF-OLA -- a state-of-the-art Big
Data analytics system -- and evaluate their performance over terascale-size
synthetic and real datasets. The results confirm that as many as 32
configurations can be evaluated concurrently almost as fast as one, while
sub-optimal configurations are detected accurately in as little as a
fraction of the time
- …