1,000 research outputs found
Time Series Management Systems:A Survey
The collection of time series data increases as more monitoring and
automation are being deployed. These deployments range in scale from an
Internet of things (IoT) device located in a household to enormous distributed
Cyber-Physical Systems (CPSs) producing large volumes of data at high velocity.
To store and analyze these vast amounts of data, specialized Time Series
Management Systems (TSMSs) have been developed to overcome the limitations of
general purpose Database Management Systems (DBMSs) for times series
management. In this paper, we present a thorough analysis and classification of
TSMSs developed through academic or industrial research and documented through
publications. Our classification is organized into categories based on the
architectures observed during our analysis. In addition, we provide an overview
of each system with a focus on the motivational use case that drove the
development of the system, the functionality for storage and querying of time
series a system implements, the components the system is composed of, and the
capabilities of each system with regard to Stream Processing and Approximate
Query Processing (AQP). Last, we provide a summary of research directions
proposed by other researchers in the field and present our vision for a next
generation TSMS.Comment: 20 Pages, 15 Figures, 2 Tables, Accepted for publication in IEEE TKD
Remote sensing big data computing: challenges and opportunities
As we have entered an era of high resolution earth observation, the RS data are undergoing an explosive
growth. The proliferation of data also give rise to the increasing complexity of RS data, like the diversity
and higher dimensionality characteristic of the data. RS data are regarded as RS ‘‘Big Data’’. Fortunately, we are witness the coming technological leapfrogging. In this paper, we give a brief overview on the Big Data and data-intensive problems, including the analysis of RS Big Data, Big Data challenges, current techniques and works for processing RS Big Data
Storage Solutions for Big Data Systems: A Qualitative Study and Comparison
Big data systems development is full of challenges in view of the variety of
application areas and domains that this technology promises to serve.
Typically, fundamental design decisions involved in big data systems design
include choosing appropriate storage and computing infrastructures. In this age
of heterogeneous systems that integrate different technologies for optimized
solution to a specific real world problem, big data system are not an exception
to any such rule. As far as the storage aspect of any big data system is
concerned, the primary facet in this regard is a storage infrastructure and
NoSQL seems to be the right technology that fulfills its requirements. However,
every big data application has variable data characteristics and thus, the
corresponding data fits into a different data model. This paper presents
feature and use case analysis and comparison of the four main data models
namely document oriented, key value, graph and wide column. Moreover, a feature
analysis of 80 NoSQL solutions has been provided, elaborating on the criteria
and points that a developer must consider while making a possible choice.
Typically, big data storage needs to communicate with the execution engine and
other processing and visualization technologies to create a comprehensive
solution. This brings forth second facet of big data storage, big data file
formats, into picture. The second half of the research paper compares the
advantages, shortcomings and possible use cases of available big data file
formats for Hadoop, which is the foundation for most big data computing
technologies. Decentralized storage and blockchain are seen as the next
generation of big data storage and its challenges and future prospects have
also been discussed
Advancing Urban Flood Resilience With Smart Water Infrastructure
Advances in wireless communications and low-power electronics are enabling a new generation of smart water systems that will employ real-time sensing and control to solve our most pressing water challenges. In a future characterized by these systems, networks of sensors will detect and communicate flood events at the neighborhood scale to improve disaster response. Meanwhile, wirelessly-controlled valves and pumps will coordinate reservoir releases to halt combined sewer overflows and restore water quality in urban streams. While these technologies promise to transform the field of water resources engineering, considerable knowledge gaps remain with regards to how smart water systems should be designed and operated. This dissertation presents foundational work towards building the smart water systems of the future, with a particular focus on applications to urban flooding. First, I introduce a first-of-its-kind embedded platform for real-time sensing and control of stormwater systems that will enable emergency managers to detect and respond to urban flood events in real-time. Next, I introduce new methods for hydrologic data assimilation that will enable real-time geolocation of floods and water quality hazards. Finally, I present theoretical contributions to the problem of controller placement in hydraulic networks that will help guide the design of future decentralized flood control systems. Taken together, these contributions pave the way for adaptive stormwater infrastructure that will mitigate the impacts of urban flooding through real-time response.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/163144/1/mdbartos_1.pd
Flexi-WVSNP-DASH: A Wireless Video Sensor Network Platform for the Internet of Things
abstract: Video capture, storage, and distribution in wireless video sensor networks
(WVSNs) critically depends on the resources of the nodes forming the sensor
networks. In the era of big data, Internet of Things (IoT), and distributed
demand and solutions, there is a need for multi-dimensional data to be part of
the Sensor Network data that is easily accessible and consumable by humanity as
well as machinery. Images and video are expected to become as ubiquitous as is
the scalar data in traditional sensor networks. The inception of video-streaming
over the Internet, heralded a relentless research for effective ways of
distributing video in a scalable and cost effective way. There has been novel
implementation attempts across several network layers. Due to the inherent
complications of backward compatibility and need for standardization across
network layers, there has been a refocused attention to address most of the
video distribution over the application layer. As a result, a few video
streaming solutions over the Hypertext Transfer Protocol (HTTP) have been
proposed. Most notable are Apple’s HTTP Live Streaming (HLS) and the Motion
Picture Experts Groups Dynamic Adaptive Streaming over HTTP (MPEG-DASH). These
frameworks, do not address the typical and future WVSN use cases. A highly
flexible Wireless Video Sensor Network Platform and compatible DASH (WVSNP-DASH)
are introduced. The platform's goal is to usher video as a data element that
can be integrated into traditional and non-Internet networks. A low cost,
scalable node is built from the ground up to be fully compatible with the
Internet of Things Machine to Machine (M2M) concept, as well as the ability to
be easily re-targeted to new applications in a short time. Flexi-WVSNP design
includes a multi-radio node, a middle-ware for sensor operation and
communication, a cross platform client facing data retriever/player framework,
scalable security as well as a cohesive but decoupled hardware and software
design.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201
A Map-algebra-inspired Approach for Interacting With Wireless Sensor Networks, Cyber-physical Systems or Internet of Things
The typical approach for consuming data from wireless sensor networks (WSN) and Internet of Things (IoT) has been to send data back to central servers for processing and analysis. This thesis develops an alternative strategy for processing and acting on data directly in the environment referred to as Active embedded Map Algebra (AeMA). Active refers to the near real time production of data, and embedded refers to the architecture of distributed embedded sensor nodes. Network macroprogramming, a style of programming adopted for wireless sensor networks and IoT, addresses the challenges of coordinating the behavior of multiple connected devices through a high-level programming model. Several macroprogramming models have been proposed, but none to date has adopted a comprehensive spatial model. This thesis takes the unique approach of adapting the well-known Map Algebra model from Geographic Information Science to extend the functionality of WSN/IoT and the opportunities for user interaction with WSN/IoT. As an inherently spatial model, the Map Algebra-inspired metaphor supports the types of computation desired from a network of geographically dispersed WSN nodes. The AeMA data model aligns with the conceptual model of GIS layers and specific layer operations from Map Algebra. A declarative query and network tasking language, based on Map Algebra operations, provides the basis for operations and interactions. The model adds functionality to calculate and store time series and specific temporal summary-type composite objects as an extension to traditional Map Algebra. The AeMA encodes Map Algebra-inspired operations into an extensible Virtual Machine Runtime system, called MARS (Map Algebra Runtime System) that supports Map Algebra in an efficient and extensible way. Map algebra-like operations are performed in a distributed manner. Data do not leave the network but are analyzed and consumed in place. As a consequence, collected information is available in-situ to drive local actions. The conceptual model and tasking language are designed to direct nodes as active entities, able to perform some actions on their environment. This Map Algebra inspired network macroprogramming model has many potential applications for spatially deployed WSN/IoT networks. In particular the thesis notes its utility for precision agriculture applications
Vertical Optimizations of Convolutional Neural Networks for Embedded Systems
L'abstract è presente nell'allegato / the abstract is in the attachmen
Préserver la vie privée des individus grâce aux Systèmes Personnels de Gestion des Données
Riding the wave of smart disclosure initiatives and new privacy-protection regulations, the Personal Cloud paradigm is emerging through a myriad of solutions offered to users to let them gather and manage their whole digital life. On the bright side, this opens the way to novel value-added services when crossing multiple sources of data of a given person or crossing the data of multiple people. Yet this paradigm shift towards user empowerment raises fundamental questions with regards to the appropriateness of the functionalities and the data management and protection techniques which are offered by existing solutions to laymen users. Our work addresses these questions on three levels. First, we review, compare and analyze personal cloud alternatives in terms of the functionalities they provide and the threat models they target. From this analysis, we derive a general set of functionality and security requirements that any Personal Data Management System (PDMS) should consider. We then identify the challenges of implementing such a PDMS and propose a preliminary design for an extensive and secure PDMS reference architecture satisfying the considered requirements. Second, we focus on personal computations for a specific hardware PDMS instance (i.e., secure token with mass storage of NAND Flash). In this context, we propose a scalable embedded full-text search engine to index large document collections and manage tag-based access control policies. Third, we address the problem of collective computations in a fully-distributed architecture of PDMSs. We discuss the system and security requirements and propose protocols to enable distributed query processing with strong security guarantees against an attacker mastering many colluding corrupted nodes.Surfant sur la vague des initiatives de divulgation restreinte de données et des nouvelles réglementations en matière de protection de la vie privée, le paradigme du Cloud Personnel émerge à travers une myriade de solutions proposées aux utilisateurs leur permettant de rassembler et de gérer l'ensemble de leur vie numérique. Du côté positif, cela ouvre la voie à de nouveaux services à valeur ajoutée lors du croisement de plusieurs sources de données d'un individu ou du croisement des données de plusieurs personnes. Cependant, ce changement de paradigme vers la responsabilisation de l'utilisateur soulève des questions fondamentales quant à l'adéquation des fonctionnalités et des techniques de gestion et de protection des données proposées par les solutions existantes aux utilisateurs lambda. Notre travail aborde ces questions à trois niveaux. Tout d'abord, nous passons en revue, comparons et analysons les alternatives de cloud personnel au niveau des fonctionnalités fournies et des modèles de menaces ciblés. De cette analyse, nous déduisons un ensemble général d'exigences en matière de fonctionnalité et de sécurité que tout système personnel de gestion des données (PDMS) devrait prendre en compte. Nous identifions ensuite les défis liés à la mise en œuvre d'un tel PDMS et proposons une conception préliminaire pour une architecture PDMS étendue et sécurisée de référence répondant aux exigences considérées. Ensuite, nous nous concentrons sur les calculs personnels pour une instance matérielle spécifique du PDMS (à savoir, un dispositif personnel sécurisé avec un stockage de masse de type NAND Flash). Dans ce contexte, nous proposons un moteur de recherche plein texte embarqué et évolutif pour indexer de grandes collections de documents et gérer des politiques de contrôle d'accès basées sur des étiquettes. Troisièmement, nous abordons le problème des calculs collectifs dans une architecture entièrement distribuée de PDMS. Nous discutons des exigences d'architectures système et de sécurité et proposons des protocoles pour permettre le traitement distribué des requêtes avec de fortes garanties de sécurité contre un attaquant maîtrisant de nombreux nœuds corrompus
- …