1,080 research outputs found

    Hardware-Aware Algorithm Designs for Efficient Parallel and Distributed Processing

    Get PDF
    The introduction and widespread adoption of the Internet of Things, together with emerging new industrial applications, bring new requirements in data processing. Specifically, the need for timely processing of data that arrives at high rates creates a challenge for the traditional cloud computing paradigm, where data collected at various sources is sent to the cloud for processing. As an approach to this challenge, processing algorithms and infrastructure are distributed from the cloud to multiple tiers of computing, closer to the sources of data. This creates a wide range of devices for algorithms to be deployed on and software designs to adapt to.In this thesis, we investigate how hardware-aware algorithm designs on a variety of platforms lead to algorithm implementations that efficiently utilize the underlying resources. We design, implement and evaluate new techniques for representative applications that involve the whole spectrum of devices, from resource-constrained sensors in the field, to highly parallel servers. At each tier of processing capability, we identify key architectural features that are relevant for applications and propose designs that make use of these features to achieve high-rate, timely and energy-efficient processing.In the first part of the thesis, we focus on high-end servers and utilize two main approaches to achieve high throughput processing: vectorization and thread parallelism. We employ vectorization for the case of pattern matching algorithms used in security applications. We show that re-thinking the design of algorithms to better utilize the resources available in the platforms they are deployed on, such as vector processing units, can bring significant speedups in processing throughout. We then show how thread-aware data distribution and proper inter-thread synchronization allow scalability, especially for the problem of high-rate network traffic monitoring. We design a parallelization scheme for sketch-based algorithms that summarize traffic information, which allows them to handle incoming data at high rates and be able to answer queries on that data efficiently, without overheads.In the second part of the thesis, we target the intermediate tier of computing devices and focus on the typical examples of hardware that is found there. We show how single-board computers with embedded accelerators can be used to handle the computationally heavy part of applications and showcase it specifically for pattern matching for security-related processing. We further identify key hardware features that affect the performance of pattern matching algorithms on such devices, present a co-evaluation framework to compare algorithms, and design a new algorithm that efficiently utilizes the hardware features.In the last part of the thesis, we shift the focus to the low-power, resource-constrained tier of processing devices. We target wireless sensor networks and study distributed data processing algorithms where the processing happens on the same devices that generate the data. Specifically, we focus on a continuous monitoring algorithm (geometric monitoring) that aims to minimize communication between nodes. By deploying that algorithm in action, under realistic environments, we demonstrate that the interplay between the network protocol and the application plays an important role in this layer of devices. Based on that observation, we co-design a continuous monitoring application with a modern network stack and augment it further with an in-network aggregation technique. In this way, we show that awareness of the underlying network stack is important to realize the full potential of the continuous monitoring algorithm.The techniques and solutions presented in this thesis contribute to better utilization of hardware characteristics, across a wide spectrum of platforms. We employ these techniques on problems that are representative examples of current and upcoming applications and contribute with an outlook of emerging possibilities that can build on the results of the thesis

    System Abstractions for Scalable Application Development at the Edge

    Get PDF
    Recent years have witnessed an explosive growth of Internet of Things (IoT) devices, which collect or generate huge amounts of data. Given diverse device capabilities and application requirements, data processing takes place across a range of settings, from on-device to a nearby edge server/cloud and remote cloud. Consequently, edge-cloud coordination has been studied extensively from the perspectives of job placement, scheduling and joint optimization. Typical approaches focus on performance optimization for individual applications. This often requires domain knowledge of the applications, but also leads to application-specific solutions. Application development and deployment over diverse scenarios thus incur repetitive manual efforts. There are two overarching challenges to provide system-level support for application development at the edge. First, there is inherent heterogeneity at the device hardware level. The execution settings may range from a small cluster as an edge cloud to on-device inference on embedded devices, differing in hardware capability and programming environments. Further, application performance requirements vary significantly, making it even more difficult to map different applications to already heterogeneous hardware. Second, there are trends towards incorporating edge and cloud and multi-modal data. Together, these add further dimensions to the design space and increase the complexity significantly. In this thesis, we propose a novel framework to simplify application development and deployment over a continuum of edge to cloud. Our framework provides key connections between different dimensions of design considerations, corresponding to the application abstraction, data abstraction and resource management abstraction respectively. First, our framework masks hardware heterogeneity with abstract resource types through containerization, and abstracts away the application processing pipelines into generic flow graphs. Further, our framework further supports a notion of degradable computing for application scenarios at the edge that are driven by multimodal sensory input. Next, as video analytics is the killer app of edge computing, we include a generic data management service between video query systems and a video store to organize video data at the edge. We propose a video data unit abstraction based on a notion of distance between objects in the video, quantifying the semantic similarity among video data. Last, considering concurrent application execution, our framework supports multi-application offloading with device-centric control, with a userspace scheduler service that wraps over the operating system scheduler

    A Comprehensive Survey on Networking over TV White Spaces

    Full text link
    The 2008 Federal Communication Commission (FCC) ruling in the United States opened up new opportunities for unlicensed operation in the TV white space spectrum. Networking protocols over the TV white spaces promise to subdue the shortcomings of existing short-range multi-hop wireless architectures and protocols by offering more availability, wider bandwidth, and longer-range communication. The TV white space protocols are the enabling technologies for sensing and monitoring, Internet-of-Things (IoT), wireless broadband access, real-time, smart and connected community, and smart utility applications. In this paper, we perform a retrospective review of the protocols that have been built over the last decade and also the new challenges and the directions for future work. To the best of our knowledge, this is the first comprehensive survey to present and compare existing networking protocols over the TV white spaces.Comment: 19 page

    Big data analytics for large-scale wireless networks: Challenges and opportunities

    Full text link
    © 2019 Association for Computing Machinery. The wide proliferation of various wireless communication systems and wireless devices has led to the arrival of big data era in large-scale wireless networks. Big data of large-scale wireless networks has the key features of wide variety, high volume, real-time velocity, and huge value leading to the unique research challenges that are different from existing computing systems. In this article, we present a survey of the state-of-art big data analytics (BDA) approaches for large-scale wireless networks. In particular, we categorize the life cycle of BDA into four consecutive stages: Data Acquisition, Data Preprocessing, Data Storage, and Data Analytics. We then present a detailed survey of the technical solutions to the challenges in BDA for large-scale wireless networks according to each stage in the life cycle of BDA. Moreover, we discuss the open research issues and outline the future directions in this promising area

    Channel Access Management in Data Intensive Sensor Networks

    Get PDF
    There are considerable challenges for channel access in Data Intensive Sensor Networks - DISN, supporting Data Intensive Applications like Structural Health Monitoring. As the data load increases, considerable degradation of the key performance parameters of such sensor networks is observed. Successful packet delivery ratio drops due to frequent collisions and retransmissions. The data glut results in increased latency and energy consumption overall. With the considerable limitations on sensor node resources like battery power, this implies that excessive transmissions in response to sensor queries can lead to premature network death. After a certain load threshold the performance characteristics of traditional WSNs become unacceptable. Research work indicates that successful packet delivery ratio in 802.15.4 networks can drop from 95% to 55% as the offered network load increases from 1 packet/sec to 10 packets/sec. This result in conjunction with the fact that it is common for sensors in an SHM system to generate 6-8 packets/sec of vibration data makes it important to design appropriate channel access schemes for such data intensive applications.In this work, we address the problem of significant performance degradation in a special-purpose DISN. Our specific focus is on the medium access control layer since it gives a fine-grained control on managing channel access and reducing energy waste. The goal of this dissertation is to design and evaluate a suite of channel access schemes that ensure graceful performance degradation in special-purpose DISNs as the network traffic load increases.First, we present a case study that investigates two distinct MAC proposals based on random access and scheduling access. The results of the case study provide the motivation to develop hybrid access schemes. Next, we introduce novel hybrid channel access protocols for DISNs ranging from a simple randomized transmission scheme that is robust under channel and topology dynamics to one that utilizes limited topological information about neighboring sensors to minimize collisions and energy waste. The protocols combine randomized transmission with heuristic scheduling to alleviate network performance degradation due to excessive collisions and retransmissions. We then propose a grid-based access scheduling protocol for a mobile DISN that is scalable and decentralized. The grid-based protocol efficiently handles sensor mobility with acceptable data loss and limited overhead. Finally, we extend the randomized transmission protocol from the hybrid approaches to develop an adaptable probability-based data transmission method. This work combines probabilistic transmission with heuristics, i.e., Latin Squares and a grid network, to tune transmission probabilities of sensors, thus meeting specific performance objectives in DISNs. We perform analytical evaluations and run simulation-based examinations to test all of the proposed protocols
    • …
    corecore