Run Time Approximation of Non-blocking Service Rates for Streaming Systems

Stream processing is a compute paradigm that promises safe and efficient parallelism. Modern big-data problems are often well suited for stream processing's throughput-oriented nature. Realization of efficient stream processing requires monitoring and optimization of multiple communications links. Most techniques to optimize these links use queueing network models or network flow models, which require some idea of the actual execution rate of each independent compute kernel within the system. What we want to know is how fast can each kernel process data independent of other communicating kernels. This is known as the "service rate" of the kernel within the queueing literature. Current approaches to divining service rates are static. Modern workloads, however, are often dynamic. Shared cloud systems also present applications with highly dynamic execution environments (multiple users, hardware migration, etc.). It is therefore desirable to continuously re-tune an application during run time (online) in response to changing conditions. Our approach enables online service rate monitoring under most conditions, obviating the need for reliance on steady state predictions for what are probably non-steady state phenomena. First, some of the difficulties associated with online service rate determination are examined. Second, the algorithm to approximate the online non-blocking service rate is described. Lastly, the algorithm is implemented within the open source RaftLib framework for validation using a simple microbenchmark as well as two full streaming applications.Comment: technical repor

Predictive intelligence to the edge through approximate collaborative context reasoning

We focus on Internet of Things (IoT) environments where a network of sensing and computing devices are responsible to locally process contextual data, reason and collaboratively infer the appearance of a specific phenomenon (event). Pushing processing and knowledge inference to the edge of the IoT network allows the complexity of the event reasoning process to be distributed into many manageable pieces and to be physically located at the source of the contextual information. This enables a huge amount of rich data streams to be processed in real time that would be prohibitively complex and costly to deliver on a traditional centralized Cloud system. We propose a lightweight, energy-efficient, distributed, adaptive, multiple-context perspective event reasoning model under uncertainty on each IoT device (sensor/actuator). Each device senses and processes context data and infers events based on different local context perspectives: (i) expert knowledge on event representation, (ii) outliers inference, and (iii) deviation from locally predicted context. Such novel approximate reasoning paradigm is achieved through a contextualized, collaborative belief-driven clustering process, where clusters of devices are formed according to their belief on the presence of events. Our distributed and federated intelligence model efficiently identifies any localized abnormality on the contextual data in light of event reasoning through aggregating local degrees of belief, updates, and adjusts its knowledge to contextual data outliers and novelty detection. We provide comprehensive experimental and comparison assessment of our model over real contextual data with other localized and centralized event detection models and show the benefits stemmed from its adoption by achieving up to three orders of magnitude less energy consumption and high quality of inference

Keeping Context In Mind: Automating Mobile App Access Control with User Interface Inspection

Recent studies observe that app foreground is the most striking component that influences the access control decisions in mobile platform, as users tend to deny permission requests lacking visible evidence. However, none of the existing permission models provides a systematic approach that can automatically answer the question: Is the resource access indicated by app foreground? In this work, we present the design, implementation, and evaluation of COSMOS, a context-aware mediation system that bridges the semantic gap between foreground interaction and background access, in order to protect system integrity and user privacy. Specifically, COSMOS learns from a large set of apps with similar functionalities and user interfaces to construct generic models that detect the outliers at runtime. It can be further customized to satisfy specific user privacy preference by continuously evolving with user decisions. Experiments show that COSMOS achieves both high precision and high recall in detecting malicious requests. We also demonstrate the effectiveness of COSMOS in capturing specific user preferences using the decisions collected from 24 users and illustrate that COSMOS can be easily deployed on smartphones as a real-time guard with a very low performance overhead.Comment: Accepted for publication in IEEE INFOCOM'201

Building a context rich interface to low level sensor data

Sensor networks play an important role in our modern information society. These networks are used for a variety of activities in different domains, including traffic monitoring, environmental analysis, transport and personal health. In general, systems generate data in their own format with little or no associated semantics. As a result, data must be managed individually and significant human effort is required to analyze data and develop ad-hoc applications for different end-user requirements. The research presented here proposes a holistic and comprehensive approach to significantly reduce the human effort in analyzing networks of sensors. The goal is to facilitate any form of sensor network, enabling users to combine related semantics with sensor data, and facilitate the end-user transformation of data necessary to provide more complex query expressions, and thus meet the analytical requirements

Wireless Sensor Networking in Challenging Environments

Recent years have witnessed growing interest in deploying wireless sensing applications in real-world environments. For example, home automation systems provide fine-grained metering and control of home appliances in residential settings. Similarly, assisted living applications employ wireless sensors to provide continuous health and wellness monitoring in homes. However, real deployments of Wireless Sensor Networks (WSNs) pose significant challenges due to their low-power radios and uncontrolled ambient environments. Our empirical study in over 15 real-world apartments shows that low-power WSNs based on the IEEE 802.15.4 standard are highly susceptible to external interference beyond user control, such as Wi-Fi access points, Bluetooth peripherals, cordless phones, and numerous other devices prevalent in residential environments that share the unlicensed 2.4 GHz ISM band with IEEE 802.15.4 radios. To address these real-world challenges, we developed two practical wireless network protocols including the Adaptive and Robust Channel Hopping (ARCH) protocol and the Adaptive Energy Detection Protocol (AEDP). ARCH enhances network reliability through opportunistically changing radio\u27s frequency to avoid interference and environmental noise and AEDP reduces false wakeups in noisy wireless environments by dynamically adjusting the wakeup threshold of low-power radios. Another major trend in WSNs is the convergence with smart phones. To deal with the dynamic wireless conditions and varying application requirements of mobile users, we developed the Self-Adapting MAC Layer (SAML) to support adaptive communication between smart phones and wireless sensors. SAML dynamically selects and switches Medium Access Control protocols to accommodate changes in ambient conditions and application requirements. Compared with the residential and personal wireless systems, industrial applications pose unique challenges due to their critical demands on reliability and real-time performance. We developed an experimental testbed by realizing key network mechanisms of industrial Wireless Sensor and Actuator Networks (WSANs) and conducted an empirical study that revealed the limitations and potential enhancements of those mechanisms. Our study shows that graph routing is more resilient to interference and its backup routes may be heavily used in noisy environments, which demonstrate the necessity of path diversity for reliable WSANs. Our study also suggests that combining channel diversity with retransmission may effectively reduce the burstiness of transmission failures and judicious allocation of multiple transmissions in a shared slot can effectively improve network capacity without significantly impacting reliability

In-Network Volumetric DDoS Victim Identification Using Programmable Commodity Switches

Volumetric distributed Denial-of-Service (DDoS) attacks have become one of the most significant threats to modern telecommunication networks. However, most existing defense systems require that detection software operates from a centralized monitoring collector, leading to increased traffic load and delayed response. The recent advent of Data Plane Programmability (DPP) enables an alternative solution: threshold-based volumetric DDoS detection can be performed directly in programmable switches to skim only potentially hazardous traffic, to be analyzed in depth at the controller. In this paper, we first introduce the BACON data structure based on sketches, to estimate per-destination flow cardinality, and theoretically analyze it. Then we employ it in a simple in-network DDoS victim identification strategy, INDDoS, to detect the destination IPs for which the number of incoming connections exceeds a pre-defined threshold. We describe its hardware implementation on a Tofino-based programmable switch using the domain-specific P4 language, proving that some limitations imposed by real hardware to safeguard processing speed can be overcome to implement relatively complex packet manipulations. Finally, we present some experimental performance measurements, showing that our programmable switch is able to keep processing packets at line-rate while performing volumetric DDoS detection, and also achieves a high F1 score on DDoS victim identification.Comment: Accepted by IEEE Transactions on Network and Service Management Special issue on Latest Developments for Security Management of Networks and Service