One Trillion Edges: Graph Processing at Facebook-Scale

ABSTRACT Analyzing large graphs provides valuable insights for social networking and web companies in content ranking and recommendations. While numerous graph processing systems have been developed and evaluated on available benchmark graphs of up to 6.6B edges, they often face significant difficulties in scaling to much larger graphs. Industry graphs can be two orders of magnitude larger -hundreds of billions or up to one trillion edges. In addition to scalability challenges, real world applications often require much more complex graph processing workflows than previously evaluated. In this paper, we describe the usability, performance, and scalability improvements we made to Apache Giraph, an open-source graph processing system, in order to use it on Facebook-scale graphs of up to one trillion edges. We also describe several key extensions to the original Pregel model that make it possible to develop a broader range of production graph applications and workflows as well as improve code reuse. Finally, we report on real-world operations as well as performance characteristics of several large-scale production applications

Proceedings of the First Karlsruhe Service Summit Workshop - Advances in Service Research, Karlsruhe, Germany, February 2015 (KIT Scientific Reports ; 7692)

Since April 2008 KSRI fosters interdisciplinary research in order to support and advance the progress in the service domain. KSRI brings together academia and industry while serving as a European research hub with respect to service science. For KSS2015 Research Workshop, we invited submissions of theoretical and empirical research dealing with the relevant topics in the context of services including energy, mobility, health care, social collaboration, and web technologies

Towards intelligent transport systems: geospatial ontological framework and agent simulation

In an Intelligent Transport System (ITS) environment, the communication component is of high significance as it supports interactions between vehicles and the roadside infrastructure. Existing studies focus on the physical capability and capacity of the communication technologies, but the equally important development of suitable and efficient semantic content for transmission has received notably less attention. Using an ontology is one promising approach for context modelling in ubiquitous computing environments. In the transport domain, an ontology can be used both for context modelling and semantic contents for vehicular communications. This research explores the development of an ontological framework implementing a geosemantic messaging model to support vehicle-to-vehicle communications. To develop an ontology model, two scenarios (an ambulance situation and a breakdown on the motorway) are constructed to describe specific situations using short-range communication in an ITS environment. In the scenarios, spatiotemporal relations and semantic relations among vehicles and road facilities are extracted and defined as classes, objects, and properties/relations in the ontology model. For the ontology model, some functions and query templates are also developed to update vehicles’ movements and to provide some logical procedures that vehicles need to follow in emergency situations. To measure the effects of the vehicular communication based on the ontology model, an agent-based approach is adopted to dynamically simulate the moving vehicles and their communications following the scenarios. The simulation results demonstrate that the ontology model can support vehicular communications to update each vehicle’s context model and assist its decision-making process to resolve the emergency situations. The results also show the effect of vehicular communications on the efficiency trends of traffic in emergency situations, where some vehicles have a communication device, and others do not. The efficiency trends, based on the percentage of vehicles having a communication device, can be useful to set a transition period plan for implanting communication devices onto vehicles and the infrastructure. The geospatial ontological framework and agent simulation may contribute to increase the intelligence of ITS by supporting data-level and application-level implementation of autonomous vehicle agents to share knowledge in local contexts. This work can be easily extended to support more complex interactions amongst vehicles and the infrastructure

Managing Device and Platform Heterogeneity through the Web of Things

The chaotic growth of the IoT determined a fragmented landscape with a huge number of devices, technologies, and platforms available on the market, and consequential issues of interoperability on many system deployments. The Web of Things (WoT) architecture recently proposed by the W3C consortium constitutes a novel solution to enable interoperability across IoT Platforms and application domains. At the same time, in order to see an effective improvement, a wide adoption of the W3C WoT solutions from the academic and industrial communities is required; this translates into the need of accurate and complete support tools to ease the deployment of W3C WoT applications, as well as reference guidelines about how to enable the WoT on top of existing IoT scenarios and how to deploy WoT scenarios from scratch. In this thesis, we bring three main contributions for filling such gap: (1) we introduce the WoT Store, a novel platform for managing and easing the deployment of Things and applications on the W3C WoT, and additional strategies for bringing old legacy IoT systems into the WoT. The WoT Store allows the dynamic discovery of the resources available in the environment, i.e. the Things, and to interact with each of them through a dashboard by visualizing their properties, executing commands, or observing the notifications produced. (2) We map three different IoT scenarios to WoT scenarios: a generic heterogeneous environmental monitoring scenario, a structural health monitoring scenario and an Industry4.0 scenario. (3) We make proposals to improve both the W3C standard and the node-wot software stack design: in the first case, new vocabularies are needed in order to handle particular protocols employed in industrial scenarios, while in the second case we present some contributions required for the dynamic instantiation and the migration of Web Things and WoT services in a cloud-to-edge continuum environment