Object level physics data replication in the Grid

To support distributed physics analysis on a scale as foreseen by the LHC experiments, 'Grid' systems are needed that manage and streamline data distribution, replication, and synchronization. We report on the development of a tool that allows large physics datasets to be managed and replicated at the granularity level of single objects. Efficient and convenient support for data extraction and replication at the level of individual objects and events will enable for types of interactive data analysis that would be too inconvenient or costly to perform with tools that work on a file level only. Our tool development effort is intended as both a demonstrator project for various types of existing Grid technology, and as a research effort to develop Grid technology further. The basic use case supported by our tool is one in which a physicist repeatedly selects some physics objects located at a central repository, and replicates them to a local site. The selection can be done using 'tag' or 'ntuple' analysis at the local site. The tool replicates the selected objects, and merges all replicated objects into a single single coherent 'virtual' dataset. This allows all objects to be used together seamlessly, even if they were replicated at different times or from different locations. The version of the tool that is reported on in this paper replicates ORCA based physics data created by CMS in its ongoing high level trigger design studies. The basic capabilities and limitations of the tool are discussed, together with some performance results. Some tool internals are also presented. Finally we will report on experiences so far and on future plans

Managed ecosystems of networked objects

Small embedded devices such as sensors and actuators will become the cornerstone of the Future Internet. To this end, generic, open and secure communication and service platforms are needed in order to be able to exploit the new business opportunities these devices bring. In this paper, we evaluate the current efforts to integrate sensors and actuators into the Internet and identify the limitations at the level of cooperation of these Internet-connected objects and the possible intelligence at the end points. As a solution, we propose the concept of Managed Ecosystem of Networked Objects, which aims to create a smart network architecture for groups of Internet-connected objects by combining network virtualization and clean-slate end-to-end protocol design. The concept maps to many real-life scenarios and should empower application developers to use sensor data in an easy and natural way. At the same time, the concept introduces many new challenging research problems, but their realization could offer a meaningful contribution to the realization of the Internet of Things

Towards a Scalable Dynamic Spatial Database System

With the rise of GPS-enabled smartphones and other similar mobile devices, massive amounts of location data are available. However, no scalable solutions for soft real-time spatial queries on large sets of moving objects have yet emerged. In this paper we explore and measure the limits of actual algorithms and implementations regarding different application scenarios. And finally we propose a novel distributed architecture to solve the scalability issues.Comment: (2012

Analysis domain model for shared virtual environments

The field of shared virtual environments, which also encompasses online games and social 3D environments, has a system landscape consisting of multiple solutions that share great functional overlap. However, there is little system interoperability between the different solutions. A shared virtual environment has an associated problem domain that is highly complex raising difficult challenges to the development process, starting with the architectural design of the underlying system. This paper has two main contributions. The first contribution is a broad domain analysis of shared virtual environments, which enables developers to have a better understanding of the whole rather than the part(s). The second contribution is a reference domain model for discussing and describing solutions - the Analysis Domain Model

VINEA: a policy-based virtual network embedding architecture

Network virtualization has enabled new business models by allowing infrastructure providers to lease or share their physical network. To concurrently run multiple customized virtual network services, such infrastructure providers need to run a virtual network embedding protocol. The virtual network embedding is the (NP-hard) problem of matching constrained virtual networks onto the physical network. We present the design and implementation of a policy-based architecture for the virtual network embedding problem. By policy, we mean a variant aspect of any of the (invariant) embedding mechanisms: resource discovery, virtual network mapping, and allocation on the physical infrastructure. Our architecture adapts to different scenarios by instantiating appropriate policies, and has bounds on embedding efficiency and on convergence embedding time, over a single provider, or across multiple federated providers. The performance of representative novel policy configurations are compared over a prototype implementation. We also present an object model as a foundation for a protocol specification, and we release a testbed to enable users to test their own embedding policies, and to run applications within their virtual networks. The testbed uses a Linux system architecture to reserve virtual node and link capacities.National Science Foundation (CNS-0963974

A distributed multi-agent framework for shared resources scheduling

Nowadays, manufacturers have to share some of their resources with partners due to the competitive economic environment. The management of the availability periods of shared resources causes a problem because it is achieved by the scheduling systems which assume a local environment where all resources are on the same site. Therefore, distributed scheduling with shared resources is an important research topic in recent years. In this communication, we introduce the architecture and behavior of DSCEP framework (distributed, supervisor, customer, environment, and producer) under shared resources situation with disturbances. We are using a simple example of manufacturing system to illustrate the ability of DSCEP framework to solve the shared resources scheduling problem in complex systems

PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development

This paper describes PlinyCompute, a system for development of high-performance, data-intensive, distributed computing tools and libraries. In the large, PlinyCompute presents the programmer with a very high-level, declarative interface, relying on automatic, relational-database style optimization to figure out how to stage distributed computations. However, in the small, PlinyCompute presents the capable systems programmer with a persistent object data model and API (the "PC object model") and associated memory management system that has been designed from the ground-up for high performance, distributed, data-intensive computing. This contrasts with most other Big Data systems, which are constructed on top of the Java Virtual Machine (JVM), and hence must at least partially cede performance-critical concerns such as memory management (including layout and de/allocation) and virtual method/function dispatch to the JVM. This hybrid approach---declarative in the large, trusting the programmer's ability to utilize PC object model efficiently in the small---results in a system that is ideal for the development of reusable, data-intensive tools and libraries. Through extensive benchmarking, we show that implementing complex objects manipulation and non-trivial, library-style computations on top of PlinyCompute can result in a speedup of 2x to more than 50x or more compared to equivalent implementations on Spark.Comment: 48 pages, including references and Appendi