An efficient time optimized scheme for progressive analytics in big data

Big data analytics is the key research subject for future data driven decision making applications. Due to the large amount of data, progressive analytics could provide an efficient way for querying big data clusters. Each cluster contains only a piece of the examined data. Continuous queries over these data sources require intelligent mechanisms to result the final outcome (query response) in the minimum time with the maximum performance. A Query Controller (QC) is responsible to manage continuous/sequential queries and return the final outcome to users or applications. In this paper, we propose a mechanism that can be adopted by the QC. The proposed mechanism is capable of managing partial results retrieved by a number of processors each one responsible for each cluster. Each processor executes a query over a specific cluster of data. Our mechanism adopts two sequential decision making models for handling the incoming partial results. The first model is based on a finite horizon time-optimized model and the second one is based on an infinite horizon optimally scheduled model. We provide mathematical formulations for solving the discussed problem and present simulation results. Through a large number of experiments, we reveal the advantages of the proposed models and give numerical results comparing them with a deterministic model. These results indicate that the proposed models can efficiently reduce the required time for returning the final outcome to the user/application while keeping the quality of the aggregated result at high levels

Distributed localized contextual event reasoning under uncertainty

We focus on Internet of Things (IoT) environments where sensing and computing devices (nodes) are responsible to observe, reason, report and react to a specific phenomenon. Each node captures context from data streams and reasons on the presence of an event. We propose a distributed predictive analytics scheme for localized context reasoning under uncertainty. Such reasoning is achieved through a contextualized, knowledge-driven clustering process, where the clusters of nodes are formed according to their belief on the presence of the phenomenon. Each cluster enhances its localized opinion about the presence of an event through consensus realized under the principles of Fuzzy Logic (FL). The proposed FLdriven consensus process is further enhanced with semantics adopting Type-2 Fuzzy Sets to handle the uncertainty related to the identification of an event. We provide a comprehensive experimental evaluation and comparison assessment with other schemes over real data and report on the benefits stemmed from its adoption in IoT environments

A Stock Options Metaphor for Content Delivery Networks

The concept of Stock Options is used to address the scarcity of resources, not adequately addressed by the previous tools of our Prediction Mechanism. Using a Predictive Reservation Scheme, network and disk resources are being monitored through well-established techniques (Kernel Regression Estimators) in a given time frame. Next, an Secondary Market mechanism significantly improves the efficiency and robustness of our Predictive Reservation Scheme by allowing the fast exchange of unused (remaining) resources between the Origin Servers (CDN Clients). This exchange can happen, either by implementing socially optimal practices or by allowing automatic electronic auctions at the end of the day or at shorter time intervals. Finally, we further enhance our Prediction Mechanism; Stock Options are obtained and exercised, depending on the lack of resources at the end of day. As a result, Origin Servers may acquire resources (if required) at a normal price. The effectiveness of our mechanism further improves.Comment: 35 pages, 13 figure

Data Storage in Internet of Things: A Proposed Distributed Model

In Internet of Things (IoT), numerous devices are able to collect and report data while they can execute simple processing tasks to produce knowledge. IoT Nodes exhibit limited computational re-sources, thus, they can only perform a limited number of tasks and store a short version of the collected data. In this paper, we propose a scheme that focuses on a distributed model for data storage in a group of IoT nodes. Nodes cooperate each other exchanging statistical information for their data. Our work aims to provide a model for the selection of the node where the incoming data should be stored irrelevantly of the node in which they are initially reported. The selection process involves a decision making process that adopts a statistical similarity model of the incoming data with the datasets present in the group, the estimation of the load of each node and the in-network communication cost. All these parameters are fed into a multi-class classification scheme for the final decision. Our aim is to have a view on the statistics of the available datasets beforehand, thus, facilitating the post-processing and the production of knowledge. We report on the evaluation of our scheme and present experimental results towards the presentation of pros and cons of our model

Assessing dynamic models for high priority waste collection in smart cities

Waste Management (WM) represents an important part of Smart Cities (SCs) with significant impact on modern societies. WM involves a set of processes ranging from waste collection to the recycling of the collected materials. The proliferation of sensors and actuators enable the new era of Internet of Things (IoT) that can be adopted in SCs and help in WM. Novel approaches that involve dynamic routing models combined with the IoT capabilities could provide solutions that outperform existing models. In this paper, we focus on a SC where a number of collection bins are located in different areas with sensors attached to them. We study a dynamic waste collection architecture, which is based on data retrieved by sensors. We pay special attention to the possibility of immediate WM service in high priority areas, e.g., schools or hospitals where, possibly, the presence of dangerous waste or the negative effects on human quality of living impose the need for immediate collection. This is very crucial when we focus on sensitive groups of citizens like pupils, elderly or people living close to areas where dangerous waste is rejected. We propose novel algorithms aiming at providing efficient and scalable solutions to the dynamic waste collection problem through the management of the trade-off between the immediate collection and its cost. We describe how the proposed system effectively responds to the demand as realized by sensor observations and alerts originated in high priority areas. Our aim is to minimize the time required for serving high priority areas while keeping the average expected performance at high level. Comprehensive simulations on top of the data retrieved by a SC validate the proposed algorithms on both quantitative and qualitative criteria which are adopted to analyze their strengths and weaknesses. We claim that, local authorities could choose the model that best matches their needs and resources of each city

Data fusion and type-2 fuzzy inference in contextual data stream monitoring

Data stream monitoring provides the basis for building intelligent context-aware applications over contextual data streams. A number of wireless sensors could be spread in a specific area and monitor contextual parameters for identifying phenomena e.g., fire or flood. A back-end system receives measurements and derives decisions for possible abnormalities related to negative effects. We propose a mechanism, which based on multivariate sensors data streams, provides real-time identification of phenomena. The proposed framework performs contextual information fusion over consensus theory for the efficient measurements aggregation while time-series prediction is adopted to result future insights on the aggregated values. The unanimous fused and predicted pieces of context are fed into a Type-2 fuzzy inference system to derive highly accurate identification of events. The Type-2 inference process offers reasoning capabilities under the uncertainty of the phenomena identification. We provide comprehensive experimental evaluation over real contextual data and report on the advantages and disadvantages of the proposed mechanism. Our mechanism is further compared with Type-1 fuzzy inference and other mechanisms to demonstrate its false alarms minimization capability

Stepwise correlation of multivariate IoT event data based on first-order Markov chains

Correlating events in complex and dynamic IoT environments is a challenging task not only because of the amount of available data that needs to be processed but also due to the call for time efficient data processing. In this paper, we discuss the major steps that should be performed in real- or near real-time event management focusing on event detection and event correlation. We investigate the adoption of a univariate change detection algorithm for real-time event detection and we propose a stepwise event correlation scheme based on a first-order Markov model. The proposed theory is applied on the maritime domain and is validated through extensive experimentation with real sensor streams originating from large-scale sensor networks deployed in a maritime fleet of ships.Comment: arXiv admin note: substantial text overlap with arXiv:1803.0563

Knowledge-centric Analytics Queries Allocation in Edge Computing Environments

The Internet of Things involves a huge number of devices that collect data and deliver them to the Cloud. The processing of data at the Cloud is characterized by increased latency in providing responses to analytics queries defined by analysts or applications. Hence, Edge Computing (EC) comes into the scene to provide data processing close to the source. The collected data can be stored in edge devices and queries can be executed there to reduce latency. In this paper, we envision a case where entities located in the Cloud undertake the responsibility of receiving analytics queries and decide on the most appropriate edge nodes for queries execution. The decision is based on statistical signatures of the datasets of nodes and the statistical matching between statistics and analytics queries. Edge nodes regularly update their statistical signatures to support such decision process. Our performance evaluation shows the advantages and the shortcomings of our proposed schema in edge computing environments

Uncertainty-driven Ensemble Forecasting of QoS in Software Defined Networks

Software Defined Networking (SDN) is the key technology for combining networking and Cloud solutions to provide novel applications. SDN offers a number of advantages as the existing resources can be virtualized and orchestrated to provide new services to the end users. Such a technology should be accompanied by powerful mechanisms that ensure the end-to-end quality of service at high levels, thus, enabling support for complex applications that satisfy end users needs. In this paper, we propose an intelligent mechanism that agglomerates the benefits of SDNs with real-time “Big Data” forecasting analytics. The proposed mechanism, as part of the SDN controller, supports predictive intelligence by monitoring a set of network performance parameters, forecasting their future values, and deriving indications on potential service quality violations. By treating the performance measurements as time-series, our mechanism employs a novel ensemble forecasting methodology to estimate their future values. Such predictions are fed to a Type-2 Fuzzy Logic system to deliver, in real-time, decisions related to service quality violations. Such decisions proactively assist the SDN controller for providing the best possible orchestration of the virtualized resources. We evaluate the proposed mechanism w.r.t. precision and recall metrics over synthetic data