3,079 research outputs found
Optimal Detection of Faulty Traffic Sensors Used in Route Planning
In a smart city, real-time traffic sensors may be deployed for various
applications, such as route planning. Unfortunately, sensors are prone to
failures, which result in erroneous traffic data. Erroneous data can adversely
affect applications such as route planning, and can cause increased travel
time. To minimize the impact of sensor failures, we must detect them promptly
and accurately. However, typical detection algorithms may lead to a large
number of false positives (i.e., false alarms) and false negatives (i.e.,
missed detections), which can result in suboptimal route planning. In this
paper, we devise an effective detector for identifying faulty traffic sensors
using a prediction model based on Gaussian Processes. Further, we present an
approach for computing the optimal parameters of the detector which minimize
losses due to false-positive and false-negative errors. We also characterize
critical sensors, whose failure can have high impact on the route planning
application. Finally, we implement our method and evaluate it numerically using
a real-world dataset and the route planning platform OpenTripPlanner.Comment: Proceedings of The 2nd Workshop on Science of Smart City Operations
and Platforms Engineering (SCOPE 2017), Pittsburgh, PA USA, April 2017, 6
page
Scalable and fault-tolerant data stream processing on multi-core architectures
With increasing data volumes and velocity, many applications are shifting from the classical “process-after-store” paradigm to a stream processing model: data is produced and consumed as continuous streams. Stream processing captures latency-sensitive applications as diverse as credit card fraud detection and high-frequency trading. These applications are expressed as queries of algebraic operations (e.g., aggregation) over the most recent data using windows, i.e., finite evolving views over the input streams. To guarantee correct results, streaming applications require precise window semantics (e.g., temporal ordering) for operations that maintain state.
While high processing throughput and low latency are performance desiderata for stateful streaming applications, achieving both poses challenges. Computing the state of overlapping windows causes redundant aggregation operations: incremental execution (i.e., reusing previous results) reduces latency but prevents parallelization; at the same time, parallelizing window execution for stateful operations with precise semantics demands ordering guarantees and state access coordination. Finally, streams and state must be recovered to produce consistent and repeatable results in the event of failures.
Given the rise of shared-memory multi-core CPU architectures and high-speed networking, we argue that it is possible to address these challenges in a single node without compromising window semantics, performance, or fault-tolerance. In this thesis, we analyze, design, and implement stream processing engines (SPEs) that achieve high performance on multi-core architectures. To this end, we introduce new approaches for in-memory processing that address the previous challenges: (i) for overlapping windows, we provide a family of window aggregation techniques that enable computation sharing based on the algebraic properties of aggregation functions; (ii) for parallel window execution, we balance parallelism and incremental execution by developing abstractions for both and combining them to a novel design; and (iii) for reliable single-node execution, we enable strong fault-tolerance guarantees without sacrificing performance by reducing the required disk I/O bandwidth using a novel persistence model. We combine the above to implement an SPE that processes hundreds of millions of tuples per second with sub-second latencies. These results reveal the opportunity to reduce resource and maintenance footprint by replacing cluster-based SPEs with single-node deployments.Open Acces
When Things Matter: A Data-Centric View of the Internet of Things
With the recent advances in radio-frequency identification (RFID), low-cost
wireless sensor devices, and Web technologies, the Internet of Things (IoT)
approach has gained momentum in connecting everyday objects to the Internet and
facilitating machine-to-human and machine-to-machine communication with the
physical world. While IoT offers the capability to connect and integrate both
digital and physical entities, enabling a whole new class of applications and
services, several significant challenges need to be addressed before these
applications and services can be fully realized. A fundamental challenge
centers around managing IoT data, typically produced in dynamic and volatile
environments, which is not only extremely large in scale and volume, but also
noisy, and continuous. This article surveys the main techniques and
state-of-the-art research efforts in IoT from data-centric perspectives,
including data stream processing, data storage models, complex event
processing, and searching in IoT. Open research issues for IoT data management
are also discussed
Saber: window-based hybrid stream processing for heterogeneous architectures
Modern servers have become heterogeneous, often combining multicore CPUs with many-core GPGPUs. Such heterogeneous architectures have the potential to improve the performance of data-intensive stream processing applications, but they are not supported by current relational stream processing engines. For an engine to exploit a heterogeneous architecture, it must execute streaming SQL queries with sufficient data-parallelism to fully utilise all available heterogeneous processors, and decide how to use each in the most effective way. It must do this while respecting the semantics of streaming SQL queries, in particular with regard to window handling. We describe SABER, a hybrid high-performance relational stream processing engine for CPUs and GPGPUs. SABER executes windowbased streaming SQL queries in a data-parallel fashion using all available CPU and GPGPU cores. Instead of statically assigning query operators to heterogeneous processors, SABER employs a new adaptive heterogeneous lookahead scheduling strategy, which increases the share of queries executing on the processor that yields the highest performance. To hide data movement costs, SABER pipelines the transfer of stream data between different memory types and the CPU/GPGPU. Our experimental comparison against state-ofthe-art engines shows that SABER increases processing throughput while maintaining low latency for a wide range of streaming SQL queries with small and large windows sizes
A fault fuzzy-ontology for large scale fault-tolerant wireless sensor networks
International audienceFault tolerance is a key research area for many of applications such as those based on sensor network technologies. In a large scale wireless sensor network (WSN), it becomes important to find new methods for fault-tolerance that can meet new application requirements like Internet of things, urbane intelligence and observation systems. The challenge is beyond the limit of a single wireless sensor network and concerns multiple widely interconnected sub networks. The domain of fault grows considerably because of this new configuration. In this context, the paper proposes a fault fuzzy-ontology (FFO) for large scale WSNs to be used within a Web service architecture for diagnosis and testing
- …