Efficient reliable broadcast for commodity clusters

High-speed collective communication is the key to achieve high-performance computing in parallel computing. In the past, collective operations are usually implemented using unicast operations. We proposed a new architecture EQA (Enhanced Queue Architecture) for implementing high-speed collective operations in a cluster. With the incorporation of EQA and the hardware broadcast facility in network switches, an efficient reliable broadcast operation is implemented in a DP-SMP communication subsystem. With EQA, the computation, memory and network resources can be utilized efficiently. We evaluated the performance of the broadcast operation in a commodity cluster with fast Ethernet connection. We found that the hardware-based broadcast from DP-SMP with EQA outperforms the software-based broadcast operation. The use of EQA in broadcast operation could reduce the memory consumption by almost 40%. DP-SMP with EQA has proven to be an efficient communication mechanism for coupling commodity clusters.published_or_final_versio

Broadcasting in cycles with chords

Broadcasting is the process of information dissemination in which one node, the originator, knows a single piece of information and using a series of calls must inform every other node in the network of this information. We assume that at any given time, a node can communicate the message to another node, with which it shares an edge, by acting as either a sender or receiver, but not both. Multiple message broadcasting considers the case when the originator has m messages, where m \u3e 1, to disseminate. Whereas broadcasting limits the communication of a message from one node to another node via a single edge, line broadcasting allows one node to send a message to any other node in the network as long as a simple path exists between the sending node and the receiving node and every edge along the path is not in use.;In this dissertation, we consider the problem of broadcasting in a cycle with chords and we develop broadcast schemes for this type of network.;We begin by investigating the problem of broadcasting in a cycle with one and two chords, respectively. Then, we consider the problem of multiple message broadcasting in cycles with one and two chords. Finally, we consider the problem of line broadcasting in cycles with chords.;Through our investigations, we develop two algorithms for the problem of broadcasting in a cycle with one and two chords, respectively and we analyze the correctness and complexity of these algorithms. Then, we discuss problems associated with multiple message broadcasting in cycles with one and two chords. Finally, we use techniques developed for line broadcasting in cycles to create minimum time broadcast schemes for cycles through the addition of chords.;Using techniques developed in this dissertation, we are able to broadcast in minimum time in cycles with chords. In cycles whose size is a power of 2, we have proved that the number of chords that we add to the cycle is the minimum number of chords required to broadcast in minimum time in such a cycle

Design and Performance analysis of a relational replicated database systems

The hardware organization and software structure of a new database system are presented. This system, the relational replicated database system (RRDS), is based on a set of replicated processors operating on a partitioned database. Performance improvements and capacity growth can be obtained by adding more processors to the configuration. Based on designing goals a set of hardware and software design questions were developed. The system then evolved according to a five-phase process, based on simulation and analysis, which addressed and resolved the design questions. Strategies and algorithms were developed for data access, data placement, and directory management for the hardware organization. A predictive performance analysis was conducted to determine the extent to which original design goals were satisfied. The predictive performance results, along with an analytical comparison with three other relational multi-backend systems, provided information about the strengths and weaknesses of our design as well as a basis for future research

A new connectivity strategy for wireless mesh networks using dynamic spectrum access

The introduction of Dynamic Spectrum Access (DSA) marked an important juncture in the evolution of wireless networks. DSA is a spectrum assignment paradigm where devices are able to make real-time adjustment to their spectrum usage and adapt to changes in their spectral environment to meet performance objectives. DSA allows spectrum to be used more efficiently and may be considered as a viable approach to the ever increasing demand for spectrum in urban areas and the need for coverage extension to unconnected communities. While DSA can be applied to any spectrum band, the initial focus has been in the Ultra-High Frequency (UHF) band traditionally used for television broadcast because the band is lightly occupied and also happens to be ideal spectrum for sparsely populated rural areas. Wireless access in general is said to offer the most hope in extending connectivity to rural and unconnected peri-urban communities. Wireless Mesh Networks (WMN) in particular offer several attractive characteristics such as multi-hopping, ad-hoc networking, capabilities of self-organising and self-healing, hence the focus on WMNs. Motivated by the desire to leverage DSA for mesh networking, this research revisits the aspect of connectivity in WMNs with DSA. The advantages of DSA when combined with mesh networking not only build on the benefits, but also creates additional challenges. The study seeks to address the connectivity challenge across three key dimensions, namely network formation, link metric and multi-link utilisation. To start with, one of the conundrums faced in WMNs with DSA is that the current 802.11s mesh standard provides limited support for DSA, while DSA related standards such as 802.22 provide limited support for mesh networking. This gap in standardisation complicates the integration of DSA in WMNs as several issues are left outside the scope of the applicable standard. This dissertation highlights the inadequacy of the current MAC protocol in ensuring TVWS regulation compliance in multi-hop environments and proposes a logical link MAC sub-layer procedure to fill the gap. A network is considered compliant in this context if each node operates on a channel that it is allowed to use as determined for example, by the spectrum database. Using a combination of prototypical experiments, simulation and numerical analysis, it is shown that the proposed protocol ensures network formation is accomplished in a manner that is compliant with TVWS regulation. Having tackled the compliance problem at the mesh formation level, the next logical step was to explore performance improvement avenues. Considering the importance of routing in WMNs, the study evaluates link characterisation to determine suitable metric for routing purposes. Along this dimension, the research makes two main contributions. Firstly, A-link-metric (Augmented Link Metric) approach for WMN with DSA is proposed. A-link-metric reinforces existing metrics to factor in characteristics of a DSA channel, which is essential to improve the routing protocol's ranking of links for optimal path selection. Secondly, in response to the question of “which one is the suitable metric?”, the Dynamic Path Metric Selection (DPMeS) concept is introduced. The principal idea is to mechanise the routing protocol such that it assesses the network via a distributed probing mechanism and dynamically binds the routing metric. Using DPMeS, a routing metric is selected to match the network type and prevailing conditions, which is vital as each routing metric thrives or recedes in performance depending on the scenario. DPMeS is aimed at unifying the years worth of prior studies on routing metrics in WMNs. Simulation results indicate that A-link-metric achieves up to 83.4 % and 34.6 % performance improvement in terms of throughput and end-to-end delay respectively compared to the corresponding base metric (i.e. non-augmented variant). With DPMeS, the routing protocol is expected to yield better performance consistently compared to the fixed metric approach whose performance fluctuates amid changes in network setup and conditions. By and large, DSA-enabled WMN nodes will require access to some fixed spectrum to fall back on when opportunistic spectrum is unavailable. In the absence of fully functional integrated-chip cognitive radios to enable DSA, the immediate feasible solution for the interim is single hardware platforms fitted with multiple transceivers. This configuration results in multi-band multi-radio node capability that lends itself to a variety of link options in terms of transmit/receive radio functionality. The dissertation reports on the experimental performance evaluation of radios operating in the 5 GHz and UHF-TVWS bands for hybrid back-haul links. It is found that individual radios perform differently depending on the operating parameter settings, namely channel, channel-width and transmission power subject to prevailing environmental (both spectral and topographical) conditions. When aggregated, if the radios' data-rates are approximately equal, there is a throughput and round-trip time performance improvement of 44.5 - 61.8 % and 7.5 - 41.9 % respectively. For hybrid links comprising radios with significantly unequal data-rates, this study proposes an adaptive round-robin (ARR) based algorithm for efficient multilink utilisation. Numerical analysis indicate that ARR provides 75 % throughput improvement. These results indicate that network optimisation overall requires both time and frequency division duplexing. Based on the experimental test results, this dissertation presents a three-layered routing framework for multi-link utilisation. The top layer represents the nodes' logical interface to the WMN while the bottom layer corresponds to the underlying physical wireless network interface cards (WNIC). The middle layer is an abstract and reductive representation of the possible and available transmission, and reception options between node pairs, which depends on the number and type of WNICs. Drawing on the experimental results and insight gained, the study builds criteria towards a mechanism for auto selection of the optimal link option. Overall, this study is anticipated to serve as a springboard to stimulate the adoption and integration of DSA in WMNs, and further development in multi-link utilisation strategies to increase capacity. Ultimately, it is hoped that this contribution will collectively contribute effort towards attaining the global goal of extending connectivity to the unconnected

Movies, TV programs and Youtube channels

학위논문(박사) -- 서울대학교대학원 : 공과대학 산업공학과, 2021.8. 조성준.The content market, including video content market, is a high-risk, high-return industry. Because the cost of copying and distributing the created video content is very low, large profit can be generated upon success. However, as content is an experience good, its quality cannot be judged before purchase. Hence, marketing has an important role in the content market because of the asymmetry of information between suppliers and consumers. Additionally, it has the characteristics of One Source Multi Use; if it is successful, additional profits can be created through various channels. Therefore, it is important for the content industry to correctly distinguish content with a high probability of success from the one without it and to conduct effective marketing activities to familiarize consumers with the product. Herein, we propose a methodology to assist in data-based decision-making using machine learning models and help in identifying problematic issues in video content markets such as movies, TV programs, and over-the-top (OTT) market. In the film market, although marketing is very important, decisions are still made based on the sense of practitioners. We used the market research data collected through online and offline surveys to learn a model that can predict the number of audiences on the opening-week Saturday, and then use the learned model to propose a method for effective marketing activities. In the TV program market, programming is performed to improve the overall viewership by matching TV programs and viewer groups well. We learn a model that predicts the audience rating of a program using the characteristics of the program and the audience-rating information of the programs before, after, and at the same time, and use the resulting data to assist in decision-making to find the optimal programming scenario. The OTT market is facing a new problem of user's perception bias caused by the “recent recommendation” system. In the fields of politics and news particularly, if the user does not have access to different viewpoints because of the recommendation service, it may create and/or deepen a bias toward a specific political view without the user being aware of it. In order to compensate for this, it is important to use the recommended channel while the user is well aware of what kind of channel it is. We built a channel network in the news/political field using the data extracted from the comments left by users on the videos of each channel. In addition, we propose a method to compensate for the bias by classifying networks into conservative and progressive channel clusters and presenting the topography of the political tendencies of YouTube channels.1 Introduction 1 2 Prediction of Movie Audience on First Saturday with Decision Trees 5 2.1 Background 5 2.2 Related work 9 2.3 Predictive model construction 15 2.3.1 Data 15 2.3.2 Target variable 17 2.3.3 Predictor variable 19 2.3.4 Decision Tree and ensemble prediction models 28 2.4 Prediction model evaluation 29 2.5 Summary 37 3 Prediction of TV program ratings with Decision Trees 40 3.1 Background 40 3.2 Related work 42 3.2.1 Research on the ratings themselves 42 3.2.2 Research on broadcasting programming 44 3.3 Predictive model construction 45 3.3.1 Target variable 45 3.3.2 Predictor variable 46 3.3.3 Prediction Model 48 3.4 Prediction model evaluation 50 3.4.1 Data 50 3.4.2 Experimental results 51 3.5 Optimization strategy using the predictive model 54 3.5.1 Broadcasting programming change process 56 3.5.2 Case Study 57 3.6 Summary 60 4 Relation detection of YouTube channels 62 4.1 Background 62 4.2 Related work 65 4.3 Method 67 4.3.1 Channel representation 68 4.3.2 Channel clustering with large k and merging clusters by keywords 71 4.3.3 Relabeling with RWR 73 4.3.4 Isolation score 74 4.4 Result 74 4.4.1 Channel representation 74 4.4.2 Channel clustering with large k and merging clusters by keywords 76 4.4.3 Relabeling with RWR 77 4.4.4 Isolation score 79 4.5 Discussion 80 4.5.1 On the Representativeness of the Channel Preferences of the Users from Their Comments 80 4.5.2 On Relabeling with RWR 82 4.6 Summary 83 5 Conclusion 85 5.1 Contribution 85 5.2 Future Direction 87 Bibliography 91 국문초록 110박

Task Partitioning for Distributed Assembly

This thesis addresses the problem of how to plan a strategy for a team of robots to cooperatively build a structure, henceforth referred to as the distributed assembly problem. The problem of distributed assembly requires a range of capabilities for successful completion of the task. These include accurate sensing and manipulation using a mobile robot, the ability to continuously adhere to precedence constraints on placements, and the ability to guarantee static stability at every stage of construction. The fundamental contribution of this work is to propose methods to address task allocation problems in the presence of constraints on task ordering. Algorithms are presented to partition 2- and 3-D assembly tasks into separate subtasks that satisfy local and global precedence constraints between the assembly components. The objective is to achieve a partitioning that minimizes completion time by minimizing the workload imbalance between the robots, and maximizes assembly parallelization. Towards this objective four approaches are presented. The first is an approach where each robot runs a simultaneous Dijkstra's Algorithm with its own root. The second approach incorporates online workload balancing and error correction by adding a communication scheme and a scanning robot equipped with a visual depth sensor. The third approach addresses the task partitioning using an algorithm inspired by Ant Colony Optimization. Finally, the problem of cooperative manipulation for tasks that require close coordination is addressed. All approaches are tested in both simulation and experiment.Ph.D., Mechanical Engineering -- Drexel University, 201

Probabilistic methods for distributed information dissemination

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 457-484).The ever-increasing growth of modern networks comes with a paradigm shift in network operation. Networks can no longer be abstracted as deterministic, centrally controlled systems with static topologies but need to be understood as highly distributed, dynamic systems with inherent unreliabilities. This makes many communication, coordination and computation tasks challenging and in many scenarios communication becomes a crucial bottleneck. In this thesis, we develop new algorithms and techniques to address these challenges. In particular we concentrate on broadcast and information dissemination tasks and introduce novel ideas on how randomization can lead to powerful, simple and practical communication primitives suitable for these modern networks. In this endeavor we combine and further develop tools from different disciplines trying to simultaneously addresses the distributed, information theoretic and algorithmic aspects of network communication. The two main probabilistic techniques developed to disseminate information in a network are gossip and random linear network coding. Gossip is an alternative to classical flooding approaches: Instead of nodes repeatedly forwarding information to all their neighbors, gossiping nodes forward information only to a small number of (random) neighbors. We show that, when done right, gossip disperses information almost as quickly as flooding, albeit with a drastically reduced communication overhead. Random linear network coding (RLNC) applies when a large amount of information or many messages are to be disseminated. Instead of routing messages through intermediate nodes, that is, following a classical store-and-forward approach, RLNC mixes messages together by forwarding random linear combinations of messages. The simplicity and topology-obliviousness of this approach makes RLNC particularly interesting for the distributed settings considered in this thesis. Unfortunately the performance of RLNC was not well understood even for the simplest such settings. We introduce a simple yet powerful analysis technique that allows us to prove optimal performance guarantees for all settings considered in the literature and many more that were not analyzable so far. Specifically, we give many new results for RLNC gossip algorithms, RLNC algorithms for dynamic networks, and RLNC with correlated data. We also provide a novel highly efficient distributed implementation of RLNC that achieves these performance guarantees while buffering only a minimal amount of information at intermediate nodes. We then apply our techniques to improve communication primitives in multi-hop radio networks. While radio networks inherently support broadcast communications, e.g., from one node to all surrounding nodes, interference of simultaneous transmissions makes multihop broadcast communication an interesting challenge. We show that, again, randomization holds the key for obtaining simple, efficient and distributed information dissemination protocols. In particular, using random back-off strategies to coordinate access to the shared medium leads to optimal gossip-like communications and applying RLNC achieves the first throughput-optimal multi-message communication primitives. Lastly we apply our probabilistic approach for analyzing simple, distributed propagation protocols in a broader context by studying algorithms for the Lovász Local Lemma. These algorithms find solutions to certain local constraint satisfaction problems by randomly fixing and propagating violations locally. Our two main results show that, firstly, there are also efficient deterministic propagation strategies achieving the same and, secondly, using the random fixing strategy has the advantage of producing not just an arbitrary solution but an approximately uniformly random one. Both results lead to simple, constructions for a many locally consistent structures of interest that were not known to be efficiently constructable before.by Bernhard Haeupler.Ph.D

Algorithms for Fundamental Problems in Computer Networks.

Traditional studies of algorithms consider the sequential setting, where the whole input data is fed into a single device that computes the solution. Today, the network, such as the Internet, contains of a vast amount of information. The overhead of aggregating all the information into a single device is too expensive, so a distributed approach to solve the problem is often preferable. In this thesis, we aim to develop efficient algorithms for the following fundamental graph problems that arise in networks, in both sequential and distributed settings. Graph coloring is a basic symmetry breaking problem in distributed computing. Each node is to be assigned a color such that adjacent nodes are assigned different colors. Both the efficiency and the quality of coloring are important measures of an algorithm. One of our main contributions is providing tools for obtaining colorings of good quality whose existence are non-trivial. We also consider other optimization problems in the distributed setting. For example, we investigate efficient methods for identifying the connectivity as well as the bottleneck edges in a distributed network. Our approximation algorithm is almost-tight in the sense that the running time matches the known lower bound up to a poly-logarithmic factor. For another example, we model how the task allocation can be done in ant colonies, when the ants may have different capabilities in doing different tasks. The matching problems are one of the classic combinatorial optimization problems. We study the weighted matching problems in the sequential setting. We give a new scaling algorithm for finding the maximum weight perfect matching in general graphs, which improves the long-standing Gabow-Tarjan's algorithm (1991) and matches the running time of the best weighted bipartite perfect matching algorithm (Gabow and Tarjan, 1989). Furthermore, for the maximum weight matching problem in bipartite graphs, we give a faster scaling algorithm whose running time is faster than Gabow and Tarjan's weighted bipartite {it perfect} matching algorithm.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/113540/1/hsinhao_1.pd

LIPIcs, Volume 251, ITCS 2023, Complete Volume

LIPIcs, Volume 251, ITCS 2023, Complete Volum

LIPIcs, Volume 261, ICALP 2023, Complete Volume

LIPIcs, Volume 261, ICALP 2023, Complete Volum