91 research outputs found
The state of peer-to-peer network simulators
Networking research often relies on simulation in order to test and evaluate new ideas. An important requirement of this process is that results must be reproducible so that other researchers can replicate, validate and extend existing work. We look at the landscape of simulators for research in peer-to-peer (P2P) networks by conducting a survey of a combined total of over 280 papers from before and after 2007 (the year of the last survey in this area), and comment on the large quantity of research using bespoke, closed-source simulators. We propose a set of criteria that P2P simulators should meet, and poll the P2P research community for their agreement. We aim to drive the community towards performing their experiments on simulators that allow for others to validate their results
Data Storage and Dissemination in Pervasive Edge Computing Environments
Nowadays, smart mobile devices generate huge amounts of data in all sorts of gatherings.
Much of that data has localized and ephemeral interest, but can be of great use if shared
among co-located devices. However, mobile devices often experience poor connectivity,
leading to availability issues if application storage and logic are fully delegated to a
remote cloud infrastructure. In turn, the edge computing paradigm pushes computations
and storage beyond the data center, closer to end-user devices where data is generated
and consumed. Hence, enabling the execution of certain components of edge-enabled
systems directly and cooperatively on edge devices.
This thesis focuses on the design and evaluation of resilient and efficient data storage
and dissemination solutions for pervasive edge computing environments, operating with
or without access to the network infrastructure. In line with this dichotomy, our goal can
be divided into two specific scenarios. The first one is related to the absence of network
infrastructure and the provision of a transient data storage and dissemination system
for networks of co-located mobile devices. The second one relates with the existence of
network infrastructure access and the corresponding edge computing capabilities.
First, the thesis presents time-aware reactive storage (TARS), a reactive data storage
and dissemination model with intrinsic time-awareness, that exploits synergies between
the storage substrate and the publish/subscribe paradigm, and allows queries within a
specific time scope. Next, it describes in more detail: i) Thyme, a data storage and dis-
semination system for wireless edge environments, implementing TARS; ii) Parsley, a
flexible and resilient group-based distributed hash table with preemptive peer relocation
and a dynamic data sharding mechanism; and iii) Thyme GardenBed, a framework
for data storage and dissemination across multi-region edge networks, that makes use of
both device-to-device and edge interactions.
The developed solutions present low overheads, while providing adequate response
times for interactive usage and low energy consumption, proving to be practical in a
variety of situations. They also display good load balancing and fault tolerance properties.Resumo
Hoje em dia, os dispositivos mĂłveis inteligentes geram grandes quantidades de dados
em todos os tipos de aglomeraçÔes de pessoas. Muitos desses dados tĂȘm interesse loca-
lizado e efĂȘmero, mas podem ser de grande utilidade se partilhados entre dispositivos
co-localizados. No entanto, os dispositivos mĂłveis muitas vezes experienciam fraca co-
nectividade, levando a problemas de disponibilidade se o armazenamento e a lĂłgica das
aplicaçÔes forem totalmente delegados numa infraestrutura remota na nuvem. Por sua
vez, o paradigma de computação na periferia da rede leva as computaçÔes e o armazena-
mento para além dos centros de dados, para mais perto dos dispositivos dos utilizadores
finais onde os dados são gerados e consumidos. Assim, permitindo a execução de certos
componentes de sistemas direta e cooperativamente em dispositivos na periferia da rede.
Esta tese foca-se no desenho e avaliação de soluçÔes resilientes e eficientes para arma-
zenamento e disseminação de dados em ambientes pervasivos de computação na periferia
da rede, operando com ou sem acesso Ă infraestrutura de rede. Em linha com esta dico-
tomia, o nosso objetivo pode ser dividido em dois cenĂĄrios especĂficos. O primeiro estĂĄ
relacionado com a ausĂȘncia de infraestrutura de rede e o fornecimento de um sistema
efĂȘmero de armazenamento e disseminação de dados para redes de dispositivos mĂłveis
co-localizados. O segundo diz respeito Ă existĂȘncia de acesso Ă infraestrutura de rede e
aos recursos de computação na periferia da rede correspondentes.
Primeiramente, a tese apresenta armazenamento reativo ciente do tempo (ARCT), um
modelo reativo de armazenamento e disseminação de dados com percepção intrĂnseca
do tempo, que explora sinergias entre o substrato de armazenamento e o paradigma pu-
blicação/subscrição, e permite consultas num escopo de tempo especĂfico. De seguida,
descreve em mais detalhe: i) Thyme, um sistema de armazenamento e disseminação de
dados para ambientes sem fios na periferia da rede, que implementa ARCT; ii) Pars-
ley, uma tabela de dispersĂŁo distribuĂda flexĂvel e resiliente baseada em grupos, com
realocação preventiva de nós e um mecanismo de particionamento dinùmico de dados; e
iii) Thyme GardenBed, um sistema para armazenamento e disseminação de dados em
redes multi-regionais na periferia da rede, que faz uso de interaçÔes entre dispositivos e
com a periferia da rede.
As soluçÔes desenvolvidas apresentam baixos custos, proporcionando tempos de res-
posta adequados para uso interativo e baixo consumo de energia, demonstrando serem
pråticas nas mais diversas situaçÔes. Estas soluçÔes também exibem boas propriedades de balanceamento de carga e tolerùncia a faltas
Towards a Framework for DHT Distributed Computing
Distributed Hash Tables (DHTs) are protocols and frameworks used by peer-to-peer (P2P) systems. They are used as the organizational backbone for many P2P file-sharing systems due to their scalability, fault-tolerance, and load-balancing properties. These same properties are highly desirable in a distributed computing environment, especially one that wants to use heterogeneous components. We show that DHTs can be used not only as the framework to build a P2P file-sharing service, but as a P2P distributed computing platform. We propose creating a P2P distributed computing framework using distributed hash tables, based on our prototype system ChordReduce. This framework would make it simple and efficient for developers to create their own distributed computing applications. Unlike Hadoop and similar MapReduce frameworks, our framework can be used both in both the context of a datacenter or as part of a P2P computing platform. This opens up new possibilities for building platforms to distributed computing problems. One advantage our system will have is an autonomous load-balancing mechanism. Nodes will be able to independently acquire work from other nodes in the network, rather than sitting idle. More powerful nodes in the network will be able use the mechanism to acquire more work, exploiting the heterogeneity of the network. By utilizing the load-balancing algorithm, a datacenter could easily leverage additional P2P resources at runtime on an as needed basis. Our framework will allow MapReduce-like or distributed machine learning platforms to be easily deployed in a greater variety of contexts
Efficient service discovery in wide area networks
Living in an increasingly networked world, with an abundant number
of services available to consumers, the consumer electronics market
is enjoying a boom. The average consumer in the developed world may
own several networked devices such as games consoles, mobile phones,
PDAs, laptops and desktops, wireless picture frames and printers to
name but a few. With this growing number of networked devices comes
a growing demand for services, defined here as functions requested
by a client and provided by a networked node. For example, a client
may wish to download and share music or pictures, find and use
printer services, or lookup information (e.g. train times, cinema
bookings).
It is notable that a significant proportion of networked devices are
now mobile. Mobile devices introduce a new dynamic to the service
discovery problem, such as lower battery and processing power and
more expensive bandwidth. Device owners expect to access services
not only in their immediate proximity, but further afield (e.g. in
their homes and offices). Solving these problems is the focus of
this research.
This Thesis offers two alternative approaches to service discovery
in Wide Area Networks (WANs). Firstly, a unique combination of the
Session Initiation Protocol (SIP) and the OSGi middleware technology
is presented to provide both mobility and service discovery
capability in WANs. Through experimentation, this technique is shown
to be successful where the number of operating domains is small, but
it does not scale well.
To address the issue of scalability, this Thesis proposes the use of
Peer-to-Peer (P2P) service overlays as a medium for service
discovery in WANs. To confirm that P2P overlays can in fact support
service discovery, a technique to utilise the Distributed Hash Table
(DHT) functionality of distributed systems is used to store and
retrieve service advertisements. Through simulation, this is shown
to be both a scalable and a flexible service discovery technique.
However, the problems associated with P2P networks with respect to
efficiency are well documented.
In a novel approach to reduce messaging costs in P2P networks,
multi-destination multicast is used. Two well known P2P overlays are
extended using the Explicit Multi-Unicast (XCAST) protocol. The
resulting analysis of this extension provides a strong argument for
multiple P2P maintenance algorithms co-existing in a single P2P
overlay to provide adaptable performance. A novel multi-tier P2P
overlay system is presented, which is tailored for service rich
mobile devices and which provides an efficient platform for service
discovery
Dynamic data placement and discovery in wide-area networks
The workloads of online services and applications such as social networks, sensor data platforms and web search engines have become increasingly global and dynamic, setting new challenges to providing users with low latency access to data. To achieve this, these services typically leverage a multi-site wide-area networked infrastructure. Data access latency in such an infrastructure depends on the network paths between users and data, which is determined by the data placement and discovery strategies. Current strategies are static, which offer low latencies upon deployment but worse performance under a dynamic workload.
We propose dynamic data placement and discovery strategies for wide-area networked infrastructures, which adapt to the data access workload. We achieve this with data activity correlation (DAC), an application-agnostic approach for determining the correlations between data items based on access pattern similarities. By dynamically clustering data according to DAC, network traffic in clusters is kept local. We utilise DAC as a key component in reducing access latencies for two application scenarios, emphasising different aspects of the problem:
The first scenario assumes the fixed placement of data at sites, and thus focusses on data discovery. This is the case for a global sensor discovery platform, which aims to provide low latency discovery of sensor metadata. We present a self-organising hierarchical infrastructure consisting of multiple DAC clusters, maintained with an online and distributed split-and-merge algorithm. This reduces the number of sites visited, and thus latency, during discovery for a variety of workloads.
The second scenario focusses on data placement. This is the case for global online services that leverage a multi-data centre deployment to provide users with low latency access to data. We present a geo-dynamic partitioning middleware, which maintains DAC clusters with an online elastic partition algorithm. It supports the geo-aware placement of partitions across data centres according to the workload. This provides globally distributed users with low latency access to data for static and dynamic workloads.Open Acces
Resource discovery for distributed computing systems: A comprehensive survey
Large-scale distributed computing environments provide a vast amount of heterogeneous computing resources from different sources for resource sharing and distributed computing. Discovering appropriate resources in such environments is a challenge which involves several different subjects. In this paper, we provide an investigation on the current state of resource discovery protocols, mechanisms, and platforms for large-scale distributed environments, focusing on the design aspects. We classify all related aspects, general steps, and requirements to construct a novel resource discovery solution in three categories consisting of structures, methods, and issues. Accordingly, we review the literature, analyzing various aspects for each category
A framework for the dynamic management of Peer-to-Peer overlays
Peer-to-Peer (P2P) applications have been associated with inefficient operation, interference with other network services and large operational costs for network providers. This thesis presents a framework which can help ISPs address these issues by means of intelligent management of peer behaviour. The proposed approach involves limited control of P2P overlays without interfering with the fundamental characteristics of peer autonomy and decentralised operation.
At the core of the management framework lays the Active Virtual Peer (AVP). Essentially intelligent peers operated by the network providers, the AVPs interact with the overlay from within, minimising redundant or inefficient traffic, enhancing overlay stability and facilitating the efficient and balanced use of available peer and network resources. They offer an âinsiderâsâ view of the overlay and permit the management of P2P functions in a compatible and non-intrusive manner. AVPs can support multiple P2P protocols and coordinate to perform functions collectively.
To account for the multi-faceted nature of P2P applications and allow the incorporation of modern techniques and protocols as they appear, the framework is based on a modular architecture. Core modules for overlay control and transit traffic minimisation are presented. Towards the latter, a number of suitable P2P content caching strategies are proposed.
Using a purpose-built P2P network simulator and small-scale experiments, it is demonstrated that the introduction of AVPs inside the network can significantly reduce inter-AS traffic, minimise costly multi-hop flows, increase overlay stability and load-balancing and offer improved peer transfer performance
Using Redis supported by NVRAM in HPC applications
Nowadays, the efficiency of a storage systems is a bottleneck in many moern HPC clusters. High performance in traditional approach â processing using files â is often difficult to obtain because of model complexity and its read/write patterns. Alternative approach is applying a key-value database, which usually has low latency and scales well. On the other hand, many key-value stores suffer from limitation of memory capacity and vulnerability to serious faiures, which is caused by processing in RAM. Moreover, some research suggests, that scientific data models are not applicable to storage structures of key-value databases. In this paper, the author proposes resolving mentioned issues by replacing RAM with NVRAM. Practical example is based on Redis NoSQL. The article contains also a three domain specific APIs, that show the idea bhind transformation from HPC data model to Redis structures, as well as two micro-benchmarks results
- âŠ