2,066 research outputs found
Analytics-as-a-Service in a Multi-Cloud Environment through Semantically-enabled Hierarchical Data Processing
yesA large number of cloud middleware platforms and tools are deployed to support a variety of Internet
of Things (IoT) data analytics tasks. It is a common practice that such cloud platforms are only used
by its owners to achieve their primary and predefined objectives, where raw and processed data are only
consumed by them. However, allowing third parties to access processed data to achieve their own objectives
significantly increases intergation, cooperation, and can also lead to innovative use of the data. Multicloud,
privacy-aware environments facilitate such data access, allowing different parties to share processed
data to reduce computation resource consumption collectively. However, there are interoperability issues in
such environments that involve heterogeneous data and analytics-as-a-service providers. There is a lack of
both - architectural blueprints that can support such diverse, multi-cloud environments, and corresponding
empirical studies that show feasibility of such architectures. In this paper, we have outlined an innovative
hierarchical data processing architecture that utilises semantics at all the levels of IoT stack in multicloud
environments. We demonstrate the feasibility of such architecture by building a system based on this
architecture using OpenIoT as a middleware, and Google Cloud and Microsoft Azure as cloud environments.
The evaluation shows that the system is scalable and has no significant limitations or overheads
Constructing distributed time-critical applications using cognitive enabled services
Time-critical analytics applications are increasingly making use of distributed service interfaces (e.g., micro-services) that support the rapid construction of new applications by dynamically linking the services into different workflow configurations. Traditional service-based applications, in fixed networks, are typically constructed and managed centrally and assume stable service endpoints and adequate network connectivity. Constructing and maintaining such applications in dynamic heterogeneous wireless networked environments, where limited bandwidth and transient connectivity are commonplace, presents significant challenges and makes centralized application construction and management impossible. In this paper we present an architecture which is capable of providing an adaptable and resilient method for on-demand decentralized construction and management of complex time-critical applications in such environments. The approach uses a Vector Symbolic Architecture (VSA) to compactly represent an application as a single semantic vector that encodes the service interfaces, workflow, and the time-critical constraints required. By extending existing services interfaces, with a simple cognitive layer that can interpret and exchange the vectors, we show how the required services can be dynamically discovered and interconnected in a completely decentralized manner. We demonstrate the viability of this approach by using a VSA to encode various time-critical data analytics workflows. We show that these vectors can be used to dynamically construct and run applications using services that are distributed across an emulated Mobile Ad-Hoc Wireless Network (MANET). Scalability is demonstrated via an empirical evaluation
Recent advances in industrial wireless sensor networks towards efficient management in IoT
With the accelerated development of Internet-of- Things (IoT), wireless sensor networks (WSN) are gaining importance in the continued advancement of information and communication technologies, and have been connected and integrated with Internet in vast industrial applications. However, given the fact that most wireless sensor devices are resource constrained and operate on batteries, the communication overhead and power consumption are therefore important issues for wireless sensor networks design. In order to efficiently manage these wireless sensor devices in a unified manner, the industrial authorities should be able to provide a network infrastructure supporting various WSN applications and services that facilitate the management of sensor-equipped real-world entities. This paper presents an overview of industrial ecosystem, technical architecture, industrial device management standards and our latest research activity in developing a WSN management system. The key approach to enable efficient and reliable management of WSN within such an infrastructure is a cross layer design of lightweight and cloud-based RESTful web service
Storage Solutions for Big Data Systems: A Qualitative Study and Comparison
Big data systems development is full of challenges in view of the variety of
application areas and domains that this technology promises to serve.
Typically, fundamental design decisions involved in big data systems design
include choosing appropriate storage and computing infrastructures. In this age
of heterogeneous systems that integrate different technologies for optimized
solution to a specific real world problem, big data system are not an exception
to any such rule. As far as the storage aspect of any big data system is
concerned, the primary facet in this regard is a storage infrastructure and
NoSQL seems to be the right technology that fulfills its requirements. However,
every big data application has variable data characteristics and thus, the
corresponding data fits into a different data model. This paper presents
feature and use case analysis and comparison of the four main data models
namely document oriented, key value, graph and wide column. Moreover, a feature
analysis of 80 NoSQL solutions has been provided, elaborating on the criteria
and points that a developer must consider while making a possible choice.
Typically, big data storage needs to communicate with the execution engine and
other processing and visualization technologies to create a comprehensive
solution. This brings forth second facet of big data storage, big data file
formats, into picture. The second half of the research paper compares the
advantages, shortcomings and possible use cases of available big data file
formats for Hadoop, which is the foundation for most big data computing
technologies. Decentralized storage and blockchain are seen as the next
generation of big data storage and its challenges and future prospects have
also been discussed
Knowledge-infused and Consistent Complex Event Processing over Real-time and Persistent Streams
Emerging applications in Internet of Things (IoT) and Cyber-Physical Systems
(CPS) present novel challenges to Big Data platforms for performing online
analytics. Ubiquitous sensors from IoT deployments are able to generate data
streams at high velocity, that include information from a variety of domains,
and accumulate to large volumes on disk. Complex Event Processing (CEP) is
recognized as an important real-time computing paradigm for analyzing
continuous data streams. However, existing work on CEP is largely limited to
relational query processing, exposing two distinctive gaps for query
specification and execution: (1) infusing the relational query model with
higher level knowledge semantics, and (2) seamless query evaluation across
temporal spaces that span past, present and future events. These allow
accessible analytics over data streams having properties from different
disciplines, and help span the velocity (real-time) and volume (persistent)
dimensions. In this article, we introduce a Knowledge-infused CEP (X-CEP)
framework that provides domain-aware knowledge query constructs along with
temporal operators that allow end-to-end queries to span across real-time and
persistent streams. We translate this query model to efficient query execution
over online and offline data streams, proposing several optimizations to
mitigate the overheads introduced by evaluating semantic predicates and in
accessing high-volume historic data streams. The proposed X-CEP query model and
execution approaches are implemented in our prototype semantic CEP engine,
SCEPter. We validate our query model using domain-aware CEP queries from a
real-world Smart Power Grid application, and experimentally analyze the
benefits of our optimizations for executing these queries, using event streams
from a campus-microgrid IoT deployment.Comment: 34 pages, 16 figures, accepted in Future Generation Computer Systems,
October 27, 201
System Abstractions for Scalable Application Development at the Edge
Recent years have witnessed an explosive growth of Internet of Things (IoT) devices, which collect or generate huge amounts of data. Given diverse device capabilities and application requirements, data processing takes place across a range of settings, from on-device to a nearby edge server/cloud and remote cloud. Consequently, edge-cloud coordination has been studied extensively from the perspectives of job placement, scheduling and joint optimization. Typical approaches focus on performance optimization for individual applications. This often requires domain knowledge of the applications, but also leads to application-specific solutions. Application development and deployment over diverse scenarios thus incur repetitive manual efforts. There are two overarching challenges to provide system-level support for application development at the edge. First, there is inherent heterogeneity at the device hardware level. The execution settings may range from a small cluster as an edge cloud to on-device inference on embedded devices, differing in hardware capability and programming environments. Further, application performance requirements vary significantly, making it even more difficult to map different applications to already heterogeneous hardware. Second, there are trends towards incorporating edge and cloud and multi-modal data. Together, these add further dimensions to the design space and increase the complexity significantly. In this thesis, we propose a novel framework to simplify application development and deployment over a continuum of edge to cloud. Our framework provides key connections between different dimensions of design considerations, corresponding to the application abstraction, data abstraction and resource management abstraction respectively. First, our framework masks hardware heterogeneity with abstract resource types through containerization, and abstracts away the application processing pipelines into generic flow graphs. Further, our framework further supports a notion of degradable computing for application scenarios at the edge that are driven by multimodal sensory input. Next, as video analytics is the killer app of edge computing, we include a generic data management service between video query systems and a video store to organize video data at the edge. We propose a video data unit abstraction based on a notion of distance between objects in the video, quantifying the semantic similarity among video data. Last, considering concurrent application execution, our framework supports multi-application offloading with device-centric control, with a userspace scheduler service that wraps over the operating system scheduler
City Data Fusion: Sensor Data Fusion in the Internet of Things
Internet of Things (IoT) has gained substantial attention recently and play a
significant role in smart city application deployments. A number of such smart
city applications depend on sensor fusion capabilities in the cloud from
diverse data sources. We introduce the concept of IoT and present in detail ten
different parameters that govern our sensor data fusion evaluation framework.
We then evaluate the current state-of-the art in sensor data fusion against our
sensor data fusion framework. Our main goal is to examine and survey different
sensor data fusion research efforts based on our evaluation framework. The
major open research issues related to sensor data fusion are also presented.Comment: Accepted to be published in International Journal of Distributed
Systems and Technologies (IJDST), 201
- …