A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and mapping also helps to provide an easy way for new practitioners to understand this complex area of research.Comment: 46 pages, 16 figures, Technical Repor

Predictive CDN Selection for Video Delivery Based on LSTM Network Performance Forecasts and Cost-Effective Trade-Offs

Owing to increasing consumption of video streams and demand for higher quality content and more advanced displays, future telecommunication networks are expected to outperform current networks in terms of key performance indicators (KPIs). Currently, content delivery networks (CDNs) are used to enhance media availability and delivery performance across the Internet in a cost-effective manner. The proliferation of CDN vendors and business models allows the content provider (CP) to use multiple CDN providers simultaneously. However, extreme concurrency dynamics can affect CDN capacity, causing performance degradation and outages, while overestimated demand affects costs. 5G standardization communities envision advanced network functions executing video analytics to enhance or boost media services. Network accelerators are required to enforce CDN resilience and efficient utilization of CDN assets. In this regard, this study investigates a cost-effective service to dynamically select the CDN for each session and video segment at the Media Server, without any modification to the video streaming pipeline being required. This service performs time series forecasts by employing a Long Short-Term Memory (LSTM) network to process real time measurements coming from connected video players. This service also ensures reliable and cost-effective content delivery through proactive selection of the CDN that fits with performance and business constraints. To this end, the proposed service predicts the number of players that can be served by each CDN at each time; then, it switches the required players between CDNs to keep the (Quality of Service) QoS rates or to reduce the CP's operational expenditure (OPEX). The proposed solution is evaluated by a real server, CDNs, and players and delivering dynamic adaptive streaming over HTTP (MPEG-DASH), where clients are notified to switch to another CDN through a standard MPEG-DASH media presentation description (MPD) update mechanismThis work was supported in part by the EC projects Fed4Fire+, under Grant 732638 (H2020-ICT-13-2016, Research and Innovation Action), and in part by Open-VERSO project (Red Cervera Program, Spanish Government's Centre for the Development of Industrial Technology

Service-centric networking for distributed heterogeneous clouds

Optimal placement and selection of service instances in a distributed heterogeneous cloud is a complex trade-off between application requirements and resource capabilities that requires detailed information on the service, infrastructure constraints, and the underlying IP network. In this article we first posit that from an analysis of a snapshot of today's centralized and regional data center infrastructure, there is a sufficient number of candidate sites for deploying many services while meeting latency and bandwidth constraints. We then provide quantitative arguments why both network and hardware performance needs to be taken into account when selecting candidate sites to deploy a given service. Finally, we propose a novel architectural solution for service-centric networking. The resulting system exploits the availability of fine-grained execution nodes across the Internet and uses knowledge of available computational and network resources for deploying, replicating and selecting instances to optimize quality of experience for a wide range of services

The role of topology and contracts in internet content delivery

The Internet depends on economic relationships between ASes (Autonomous Systems), which come in different shapes and sizes - transit, content, and access networks. CDNs (Content delivery networks) are also a pivotal part of the Internet ecosystem and construct their overlays for faster content delivery. With the evolving Internet topology and traffic growth, there is a need to study the cache deployments of CDNs to optimize cost while meeting performance requirements. The bilateral contracts enforce the routing of traffic between neighbouring ASes and are applied recursively: traffic that an AS sends to its neighbour is then controlled by the contracts of that neighbour. The lack of routing flexibility, little control over the quality of the end-to-end path are some of the limitations with the existing bilateral model, and they need to be overcome for achieving end-to-end performance guarantees. Furthermore, due to general reluctance of ASes to disclose their interconnection agreements, inference of inter-AS economic relationships depend on routing and forwarding data from measurements. Since the inferences are imperfect, this necessitates building robust algorithmic strategies to characterize ASes with a significantly higher accuracy. In this thesis, we first study the problem of optimizing multi-AS deployments of CDN caches in the Internet core. Our work is of significant practical relevance since it formalizes the planning process that all CDN operators must follow to reduce the operational cost of their overlay networks, while meeting the performance requirements of their end users. Next, we focus on developing a temporal cone (TC) algorithm that detects PFS (Provider-free ASes). By delivering a significant portion of Internet traffic, PFS is highly relevant to the overall resilience of the Internet. We detect PFS from public datasets of inter-AS economic relationships, utilizing topological statistics (customer cones of ASes) and temporal diversity. Finally, we focus on a multilateral contractual arrangement and develop algorithms for optimizing the cost of transit and access ASes. In particular, we implement Bertsekas auction algorithm for the optimal cost assignment of access ASes to transit ASes. Furthermore, we implement an epsilon-greedy bandit algorithm for optimizing the price of transit ASes and show its learning potential.This work has been supported by IMDEA Networks Institute.Programa Oficial de Doctorado en Ingeniería TelemáticaPresidente: Jordi Domingo Pascual.- Secretario: Francisco Valera Pintor.- Vocal: Pedro Andrés Aranda Gutiérre

Divided Diasporas: Southern Africans in Canada

The protracted economic and political crisis in Zimbabwe led directly to a major increase in mixed migration flows to South Africa. Migrants were drawn from every sector of society, all education and skill levels, equal numbers of both sexes, and all ages (including unaccompanied child migration). Many migrants claimed asylum in South Africa which gave them the right to work while they waited for a refugee hearing. Many others were arrested and deported back to Zimbabwe. Migrants who were unable to find employment in the formal economy turned to employment and self-employment in the informal economy. These migrant entrepreneurs used personal savings to establish small and micro enterprises in many urban areas. The businesses focused on retail trading, manufacturing and services and contributed to the South African economy in various ways, including providing employment for South Africans. Nationwide xenophobic violence in 2008 targeted all migrants, irrespective of origin and legal status. From 2008 onwards, violent attacks on migrant-owned informal businesses began to escalate. This culminated in a second round of nationwide xenophobic violence in early 2015 when migrant-owned businesses were targeted by mobs. Migrants send essential remittances to family in Zimbabwe and return migration is not a viable or long-term response until Zimbabwe’s economic crisis is resolved. As a result, informal migrant entrepreneurs have adapted to hostile business conditions by adopting a range of strategies to avoid and protect themselves and their businesses from xenophobia. Against this backdrop, this report first discusses the nature of the crisis in Zimbabwe and its connections with large-scale out-migration, particularly to South Africa. The South African response to crisis-driven migration is reviewed showing how the government shifted from a predominantly coercive and control-oriented policy towards a more realistic assessment of the need to accommodate migrants through an immigration amnesty and the right to work in the formal and informal sector. One of the major challenges facing migrants and all stakeholders in South Africa is xenophobic violence. Nationwide attacks on migrants and refugees in 2008 and 2015 have been interspersed with ongoing lower-level episodes of violence. These attacks have increasingly targeted migrants and refugees, including many Zimbabweans, seeking to make a living in the country’s urban informal economy. The research for this report focused on the business activities and responses to xenophobic violence of Zimbabweans in the informal economy. Amongst the key findings were the following: Between 20-30% of Zimbabwean migrants in South African cities are involved in the informal economy and the importance of informal sector employment to Zimbabweans has increased over time. Zimbabweans operating enterprises in the informal economy are predominantly young (50- 75% under the age of 35) and male (60-70%). Nearly two-thirds of the migrant entrepreneurs arrived in South Africa in the peak years of the Zimbabwean crisis between 2000 and 2010 (42%). Another 32% migrated after 2010. Less than 2% migrated to South Africa before the end of apartheid. Economic hardship, unemployment and political persecution are the main push drivers of migration to South Africa. Pull drivers include the assistance of relatives already in South Africa and the prospect of employment. The majority of the Zimbabwean migrant enterprises are in the retail, trade and wholesale sector, followed by services and manufacturing. Around three-quarters of the migrants relied on their personal savings to start their businesses and many worked in the formal economy first. Business expansion has occurred despite the prime obligation of the entrepreneurs to support family still in Zimbabwe. Instead of reinvesting all of the business profits into further expansion, a portion is therefore diverted into remittance channels. Over one-third remit funds at least once per month and only 12% never send remittances. A significant number of the entrepreneurs had been victims of or knew other who had been victims of crime such as looting and robbery, xenophobic abuse and police misconduct abuse. The report then presents the results of in-depth interviews with Zimbabwean business-owners who had experienced xenophobic violence in 2008 and 2015 or at other times. The narratives of the migrants provide insights into the unpredictable nature of the violence, their vulnerability to attack, the loss of business goods and property during mob violence and the need to restart from scratch, and the various strategies that they adopt to reduce risk. These strategies include operating in safer areas (not feasible for all), avoiding areas where corrupt police tend to operate, paying for protection and flight when xenophobic violence erupts. Return to Zimbabwe is not considered a viable option because of the economic conditions there. The interviews also provide insights into the migrants’ perceptions of government and stakeholder responses to the xenophobic violence. Almost without exception, the migrants felt that neither government (the Zimbabwean or South African) had done anything to protect or assist them during and after the violence. This perception of inaction also extended to international and non-governmental organisations. The migrants were particularly harsh in their comments about the police who were widely seen as either conniving in the violence or uninterested in protecting migrants. The perceptions of the migrants that nothing is done may simply be a function of who was interviewed and does not necessarily reflect the actual reality. The report therefore evaluates the response of the South African government to the ongoing crisis of xenophobia and concludes that some actions – such as sending in the army – are taken during episodes of nationwide violence but that ongoing daily and weekly attacks are generally ignored. There is a strong official line that these attacks are not motivated by xenophobia and. Indeed, that xenophobia does not even exist. This is clearly contradicted by the migrants who view the attacks as motivated by xenophobia. A second element of the official response is that the migrants are partially to blame for what happens to them as their business success builds resentment amongst South Africans. Government has yet to acknowledge that migrant-owned informal enterprises make a valuable contribution to the economy of the country, including through job creation for South Africans. The primary response to the violence of 2015 was the launching of a military-style Operation Fiela which was justified as a crime-fighting initiative but appears to have targeted migrant enterprises. The final sections of the report examine the responses and programmes of various non-governmental and international organisations to the crisis of xenophobia. During large-scale xenophobic violence there is considerable mobilisation of anti-xenophobia civil society organisations to offer protection and protest. Their effectiveness and impact tends to dissipate when the violence is more scattered and random. The South African Human Rights Commission (SAHRC) has played a major role in the past in holding government to account and articulating extensive recommendations for remedial action, most of which have not been taken up and many of which are still highly relevant. International organisations have tended to target integration and education programming at the community level but there has only been one systematic evaluation (of the UNHCR’s response) which was highly critical of the organisation. These organisations and other governments are considerably hamstrung by xenophobia denialism at the highest level because it means that government will avoid the kinds of partnership that are urgently needed to address this endemic crisis