113 research outputs found
DEPAS: A Decentralized Probabilistic Algorithm for Auto-Scaling
The dynamic provisioning of virtualized resources offered by cloud computing
infrastructures allows applications deployed in a cloud environment to
automatically increase and decrease the amount of used resources. This
capability is called auto-scaling and its main purpose is to automatically
adjust the scale of the system that is running the application to satisfy the
varying workload with minimum resource utilization. The need for auto-scaling
is particularly important during workload peaks, in which applications may need
to scale up to extremely large-scale systems.
Both the research community and the main cloud providers have already
developed auto-scaling solutions. However, most research solutions are
centralized and not suitable for managing large-scale systems, moreover cloud
providers' solutions are bound to the limitations of a specific provider in
terms of resource prices, availability, reliability, and connectivity.
In this paper we propose DEPAS, a decentralized probabilistic auto-scaling
algorithm integrated into a P2P architecture that is cloud provider
independent, thus allowing the auto-scaling of services over multiple cloud
infrastructures at the same time. Our simulations, which are based on real
service traces, show that our approach is capable of: (i) keeping the overall
utilization of all the instantiated cloud resources in a target range, (ii)
maintaining service response times close to the ones obtained using optimal
centralized auto-scaling approaches.Comment: Submitted to Springer Computin
PiCasso: enabling information-centric multi-tenancy at the edge of community mesh networks
© 2019 Elsevier. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/Edge computing is radically shaping the way Internet services are run by enabling computations to be available close to the users - thus mitigating the latency and performance challenges faced in today’s Internet infrastructure. Emerging markets, rural and remote communities are further away from the cloud and edge computing has indeed become an essential panacea. Many solutions have been recently proposed to facilitate efficient service delivery in edge data centers. However, we argue that those solutions cannot fully support the operations in Community Mesh Networks (CMNs) since the network connection may be less reliable and exhibit variable performance. In this paper, we propose to leverage lightweight virtualisation, Information-Centric Networking (ICN), and service deployment algorithms to overcome these limitations. The proposal is implemented in the PiCasso system, which utilises in-network caching and name based routing of ICN, combined with our HANET (HArdware and NETwork Resources) service deployment heuristic, to optimise the forwarding path of service delivery in a network zone. We analyse the data collected from the Guifi.net Sants network zone, to develop a smart heuristic for the service deployment in that zone. Through a real deployment in Guifi.net, we show that HANET improves the response time up to 53% and 28.7% for stateless and stateful services respectively. PiCasso achieves 43% traffic reduction on service delivery in our real deployment, compared to the traditional host-centric communication. The overall effect of our ICN platform is that most content and service delivery requests can be satisfied very close to the client device, many times just one hop away, decoupling QoS from intra-network traffic and origin server load.Peer ReviewedPostprint (author's final draft
Recommended from our members
HARD: Hybrid Adaptive Resource Discovery for Jungle Computing
In recent years, Jungle Computing has emerged as a distributed computing paradigm based on simultaneous combination of various hierarchical and distributed computing environments which are composed by large number of heterogeneous resources. In such a computing environment, the resources and the underlying computation and communication infrastructures are highly-hierarchical and heterogeneous. This creates a lot of difficulty and complexity for finding the proper resources in a precise way in order to run a particular job on the system efficiently. This paper proposes Hybrid Adaptive Resource Discovery (HARD), a novel efficient and highly scalable resource-discovery approach which is built upon a virtual hierarchical overlay based on self-organization and self-adaptation of processing resources in the system, where the computing resources are organized into distributed hierarchies according to a proposed hierarchical multi-layered resource description model. The proposed approach supports distributed query processing within and across hierarchical layers by deploying various distributed resource discovery services and functionalities in the system which are implemented using different adapted algorithms and mechanisms in each level of hierarchy. The proposed approach addresses the requirements for resource discovery in Jungle Computing environments such as high-hierarchy, high-heterogeneity, high-scalability and dynamicity. Simulation results show significant scalability and efficiency of the proposed approach over highly heterogeneous, hierarchical and dynamic computing environments
Resource discovery for distributed computing systems: A comprehensive survey
Large-scale distributed computing environments provide a vast amount of heterogeneous computing resources from different sources for resource sharing and distributed computing. Discovering appropriate resources in such environments is a challenge which involves several different subjects. In this paper, we provide an investigation on the current state of resource discovery protocols, mechanisms, and platforms for large-scale distributed environments, focusing on the design aspects. We classify all related aspects, general steps, and requirements to construct a novel resource discovery solution in three categories consisting of structures, methods, and issues. Accordingly, we review the literature, analyzing various aspects for each category
Software-Defined Cloud Computing: Architectural Elements and Open Challenges
The variety of existing cloud services creates a challenge for service
providers to enforce reasonable Software Level Agreements (SLA) stating the
Quality of Service (QoS) and penalties in case QoS is not achieved. To avoid
such penalties at the same time that the infrastructure operates with minimum
energy and resource wastage, constant monitoring and adaptation of the
infrastructure is needed. We refer to Software-Defined Cloud Computing, or
simply Software-Defined Clouds (SDC), as an approach for automating the process
of optimal cloud configuration by extending virtualization concept to all
resources in a data center. An SDC enables easy reconfiguration and adaptation
of physical resources in a cloud infrastructure, to better accommodate the
demand on QoS through a software that can describe and manage various aspects
comprising the cloud environment. In this paper, we present an architecture for
SDCs on data centers with emphasis on mobile cloud applications. We present an
evaluation, showcasing the potential of SDC in two use cases-QoS-aware
bandwidth allocation and bandwidth-aware, energy-efficient VM placement-and
discuss the research challenges and opportunities in this emerging area.Comment: Keynote Paper, 3rd International Conference on Advances in Computing,
Communications and Informatics (ICACCI 2014), September 24-27, 2014, Delhi,
Indi
Descoberta de recursos para sistemas de escala arbitrarias
Doutoramento em InformáticaTecnologias de Computação Distribuída em larga escala tais como Cloud,
Grid, Cluster e Supercomputadores HPC estão a evoluir juntamente com a
emergência revolucionária de modelos de múltiplos núcleos (por exemplo:
GPU, CPUs num único die, Supercomputadores em single die, Supercomputadores
em chip, etc) e avanços significativos em redes e soluções de
interligação. No futuro, nós de computação com milhares de núcleos podem
ser ligados entre si para formar uma única unidade de computação
transparente que esconde das aplicações a complexidade e a natureza distribuída desses sistemas com múltiplos núcleos. A fim de beneficiar de forma
eficiente de todos os potenciais recursos nesses ambientes de computação
em grande escala com múltiplos núcleos ativos, a descoberta de recursos é um elemento crucial para explorar ao máximo as capacidade de todos
os recursos heterogéneos distribuídos, através do reconhecimento preciso e
localização desses recursos no sistema. A descoberta eficiente e escalável
de recursos ´e um desafio para tais sistemas futuros, onde os recursos e as
infira-estruturas de computação e comunicação subjacentes são altamente
dinâmicas, hierarquizadas e heterogéneas. Nesta tese, investigamos o problema
da descoberta de recursos no que diz respeito aos requisitos gerais da
escalabilidade arbitrária de ambientes de computação futuros com múltiplos
núcleos ativos. A principal contribuição desta tese ´e a proposta de uma
entidade de descoberta de recursos adaptativa híbrida (Hybrid Adaptive
Resource Discovery - HARD), uma abordagem de descoberta de recursos eficiente
e altamente escalável, construída sobre uma sobreposição hierárquica
virtual baseada na auto-organizaçãoo e auto-adaptação de recursos de processamento
no sistema, onde os recursos computacionais são organizados
em hierarquias distribuídas de acordo com uma proposta de modelo de
descriçãoo de recursos multi-camadas hierárquicas. Operacionalmente, em
cada camada, que consiste numa arquitetura ponto-a-ponto de módulos que,
interagindo uns com os outros, fornecem uma visão global da disponibilidade
de recursos num ambiente distribuído grande, dinâmico e heterogéneo. O
modelo de descoberta de recursos proposto fornece a adaptabilidade e flexibilidade
para executar consultas complexas através do apoio a um conjunto
de características significativas (tais como multi-dimensional, variedade e
consulta agregada) apoiadas por uma correspondência exata e parcial, tanto
para o conteúdo de objetos estéticos e dinâmicos. Simulações mostram
que o HARD pode ser aplicado a escalas arbitrárias de dinamismo, tanto
em termos de complexidade como de escala, posicionando esta proposta
como uma arquitetura adequada para sistemas futuros de múltiplos núcleos.
Também contribuímos com a proposta de um regime de gestão eficiente
dos recursos para sistemas futuros que podem utilizar recursos distribuíos
de forma eficiente e de uma forma totalmente descentralizada. Além disso,
aproveitando componentes de descoberta (RR-RPs) permite que a nossa
plataforma de gestão de recursos encontre e aloque dinamicamente recursos
disponíeis que garantam os parâmetros de QoS pedidos.Large scale distributed computing technologies such as Cloud, Grid, Cluster
and HPC supercomputers are progressing along with the revolutionary emergence
of many-core designs (e.g. GPU, CPUs on single die, supercomputers
on chip, etc.) and significant advances in networking and interconnect solutions.
In future, computing nodes with thousands of cores may be connected
together to form a single transparent computing unit which hides from applications
the complexity and distributed nature of these many core systems. In
order to efficiently benefit from all the potential resources in such large scale
many-core-enabled computing environments, resource discovery is the vital
building block to maximally exploit the capabilities of all distributed heterogeneous
resources through precisely recognizing and locating those resources
in the system. The efficient and scalable resource discovery is challenging for
such future systems where the resources and the underlying computation and
communication infrastructures are highly-dynamic, highly-hierarchical and
highly-heterogeneous. In this thesis, we investigate the problem of resource
discovery with respect to the general requirements of arbitrary scale future
many-core-enabled computing environments. The main contribution of this
thesis is to propose Hybrid Adaptive Resource Discovery (HARD), a novel
efficient and highly scalable resource-discovery approach which is built upon
a virtual hierarchical overlay based on self-organization and self-adaptation
of processing resources in the system, where the computing resources are
organized into distributed hierarchies according to a proposed hierarchical
multi-layered resource description model. Operationally, at each layer, it
consists of a peer-to-peer architecture of modules that, by interacting with
each other, provide a global view of the resource availability in a large,
dynamic and heterogeneous distributed environment. The proposed resource
discovery model provides the adaptability and flexibility to perform complex
querying by supporting a set of significant querying features (such as
multi-dimensional, range and aggregate querying) while supporting exact
and partial matching, both for static and dynamic object contents. The
simulation shows that HARD can be applied to arbitrary scales of dynamicity,
both in terms of complexity and of scale, positioning this proposal as a
proper architecture for future many-core systems. We also contributed to
propose a novel resource management scheme for future systems which
efficiently can utilize distributed resources in a fully decentralized fashion.
Moreover, leveraging discovery components (RR-RPs) enables our resource
management platform to dynamically find and allocate available resources
that guarantee the QoS parameters on demand
Probabilistic best-fit multi-dimensional range query in Self-Organizing Cloud
With virtual machine (VM) technology being increasingly mature, computing resources in modern Cloud systems can be partitioned in fine granularity and allocated on demand with 'pay-as-you-go' model. In this work, we study the resource query and allocation problems in a Self- Organizing Cloud (SOC), where host machines are connected by a peer-to-peer (P2P) overlay network on the Internet. To run a user task in SOC, the requester needs to perform a multi-dimensional range search over the P2P network for locating host machines that satisfy its minimal demand on each type of resources. The multi-dimensional range search problem is known to be challenging as contentions along multiple dimensions could happen in the presence of the uncoordinated analogous queries. Moreover, low resource matching rate may happen while restricting query delay and network traffic. We design a novel resource discovery protocol, namely Proactive Index Diffusion CAN (PID-CAN), which can proactively diffuse resource indexes over the nodes and randomly route query messages among them. Such a protocol is especially suitable for the range query that needs to maximize its best-fit resource shares under possible competition along multiple resource dimensions. Via simulation, we show that PID-CAN could keep stable and optimized searching performance with low query delay and traffic overhead, for various test cases under different distributions of query ranges and competition degrees. It also performs satisfactorily in dynamic node-churning situation. © 2011 IEEE.published_or_final_versionThe 40th International Conference on Parallel Processing (ICPP-2011), Taipei City, Taiwan, 13-16 September 2011. In Proceedings of the 40th ICPP, 2011, p. 763-77
Data Storage and Dissemination in Pervasive Edge Computing Environments
Nowadays, smart mobile devices generate huge amounts of data in all sorts of gatherings.
Much of that data has localized and ephemeral interest, but can be of great use if shared
among co-located devices. However, mobile devices often experience poor connectivity,
leading to availability issues if application storage and logic are fully delegated to a
remote cloud infrastructure. In turn, the edge computing paradigm pushes computations
and storage beyond the data center, closer to end-user devices where data is generated
and consumed. Hence, enabling the execution of certain components of edge-enabled
systems directly and cooperatively on edge devices.
This thesis focuses on the design and evaluation of resilient and efficient data storage
and dissemination solutions for pervasive edge computing environments, operating with
or without access to the network infrastructure. In line with this dichotomy, our goal can
be divided into two specific scenarios. The first one is related to the absence of network
infrastructure and the provision of a transient data storage and dissemination system
for networks of co-located mobile devices. The second one relates with the existence of
network infrastructure access and the corresponding edge computing capabilities.
First, the thesis presents time-aware reactive storage (TARS), a reactive data storage
and dissemination model with intrinsic time-awareness, that exploits synergies between
the storage substrate and the publish/subscribe paradigm, and allows queries within a
specific time scope. Next, it describes in more detail: i) Thyme, a data storage and dis-
semination system for wireless edge environments, implementing TARS; ii) Parsley, a
flexible and resilient group-based distributed hash table with preemptive peer relocation
and a dynamic data sharding mechanism; and iii) Thyme GardenBed, a framework
for data storage and dissemination across multi-region edge networks, that makes use of
both device-to-device and edge interactions.
The developed solutions present low overheads, while providing adequate response
times for interactive usage and low energy consumption, proving to be practical in a
variety of situations. They also display good load balancing and fault tolerance properties.Resumo
Hoje em dia, os dispositivos móveis inteligentes geram grandes quantidades de dados
em todos os tipos de aglomerações de pessoas. Muitos desses dados têm interesse loca-
lizado e efêmero, mas podem ser de grande utilidade se partilhados entre dispositivos
co-localizados. No entanto, os dispositivos móveis muitas vezes experienciam fraca co-
nectividade, levando a problemas de disponibilidade se o armazenamento e a lógica das
aplicações forem totalmente delegados numa infraestrutura remota na nuvem. Por sua
vez, o paradigma de computação na periferia da rede leva as computações e o armazena-
mento para além dos centros de dados, para mais perto dos dispositivos dos utilizadores
finais onde os dados são gerados e consumidos. Assim, permitindo a execução de certos
componentes de sistemas direta e cooperativamente em dispositivos na periferia da rede.
Esta tese foca-se no desenho e avaliação de soluções resilientes e eficientes para arma-
zenamento e disseminação de dados em ambientes pervasivos de computação na periferia
da rede, operando com ou sem acesso à infraestrutura de rede. Em linha com esta dico-
tomia, o nosso objetivo pode ser dividido em dois cenários específicos. O primeiro está
relacionado com a ausência de infraestrutura de rede e o fornecimento de um sistema
efêmero de armazenamento e disseminação de dados para redes de dispositivos móveis
co-localizados. O segundo diz respeito à existência de acesso à infraestrutura de rede e
aos recursos de computação na periferia da rede correspondentes.
Primeiramente, a tese apresenta armazenamento reativo ciente do tempo (ARCT), um
modelo reativo de armazenamento e disseminação de dados com percepção intrínseca
do tempo, que explora sinergias entre o substrato de armazenamento e o paradigma pu-
blicação/subscrição, e permite consultas num escopo de tempo específico. De seguida,
descreve em mais detalhe: i) Thyme, um sistema de armazenamento e disseminação de
dados para ambientes sem fios na periferia da rede, que implementa ARCT; ii) Pars-
ley, uma tabela de dispersão distribuída flexível e resiliente baseada em grupos, com
realocação preventiva de nós e um mecanismo de particionamento dinâmico de dados; e
iii) Thyme GardenBed, um sistema para armazenamento e disseminação de dados em
redes multi-regionais na periferia da rede, que faz uso de interações entre dispositivos e
com a periferia da rede.
As soluções desenvolvidas apresentam baixos custos, proporcionando tempos de res-
posta adequados para uso interativo e baixo consumo de energia, demonstrando serem
práticas nas mais diversas situações. Estas soluções também exibem boas propriedades de balanceamento de carga e tolerância a faltas
- …