Data Provenance and Management in Radio Astronomy: A Stream Computing Approach

New approaches for data provenance and data management (DPDM) are required for mega science projects like the Square Kilometer Array, characterized by extremely large data volume and intense data rates, therefore demanding innovative and highly efficient computational paradigms. In this context, we explore a stream-computing approach with the emphasis on the use of accelerators. In particular, we make use of a new generation of high performance stream-based parallelization middleware known as InfoSphere Streams. Its viability for managing and ensuring interoperability and integrity of signal processing data pipelines is demonstrated in radio astronomy. IBM InfoSphere Streams embraces the stream-computing paradigm. It is a shift from conventional data mining techniques (involving analysis of existing data from databases) towards real-time analytic processing. We discuss using InfoSphere Streams for effective DPDM in radio astronomy and propose a way in which InfoSphere Streams can be utilized for large antennae arrays. We present a case-study: the InfoSphere Streams implementation of an autocorrelating spectrometer, and using this example we discuss the advantages of the stream-computing approach and the utilization of hardware accelerators

An Open Framework for Integrating Widely Distributed Hypermedia Resources

The success of the WWW has served as an illustration of how hypermedia functionality can enhance access to large amounts of distributed information. However, the WWW and many other distributed hypermedia systems offer very simple forms of hypermedia functionality which are not easily applied to existing applications and data formats, and cannot easily incorporate alternative functions which would aid hypermedia navigation to and from existing documents that have not been developed with hypermedia access in mind. This paper describes the extension to a distributed environment of the open hypermedia functionality of the Microcosm system, which is designed to support the provision of hypermedia access to a wide range of source material and application, and to offer straightforward extension of the system to incorporate new forms of information access

Static Analysis for Extracting Permission Checks of a Large Scale Framework: The Challenges And Solutions for Analyzing Android

A common security architecture is based on the protection of certain resources by permission checks (used e.g., in Android and Blackberry). It has some limitations, for instance, when applications are granted more permissions than they actually need, which facilitates all kinds of malicious usage (e.g., through code injection). The analysis of permission-based framework requires a precise mapping between API methods of the framework and the permissions they require. In this paper, we show that naive static analysis fails miserably when applied with off-the-shelf components on the Android framework. We then present an advanced class-hierarchy and field-sensitive set of analyses to extract this mapping. Those static analyses are capable of analyzing the Android framework. They use novel domain specific optimizations dedicated to Android.Comment: IEEE Transactions on Software Engineering (2014). arXiv admin note: substantial text overlap with arXiv:1206.582

Split and Migrate: Resource-Driven Placement and Discovery of Microservices at the Edge

Microservices architectures combine the use of fine-grained and independently-scalable services with lightweight communication protocols, such as REST calls over HTTP. Microservices bring flexibility to the development and deployment of application back-ends in the cloud. Applications such as collaborative editing tools require frequent interactions between the front-end running on users\u27 machines and a back-end formed of multiple microservices. User-perceived latencies depend on their connection to microservices, but also on the interaction patterns between these services and their databases. Placing services at the edge of the network, closer to the users, is necessary to reduce user-perceived latencies. It is however difficult to decide on the placement of complete stateful microservices at one specific core or edge location without trading between a latency reduction for some users and a latency increase for the others. We present how to dynamically deploy microservices on a combination of core and edge resources to systematically reduce user-perceived latencies. Our approach enables the split of stateful microservices, and the placement of the resulting splits on appropriate core and edge sites. Koala, a decentralized and resource-driven service discovery middleware, enables REST calls to reach and use the appropriate split, with only minimal changes to a legacy microservices application. Locality awareness using network coordinates further enables to automatically migrate services split and follow the location of the users. We confirm the effectiveness of our approach with a full prototype and an application to ShareLatex, a microservices-based collaborative editing application

CERN Storage Systems for Large-Scale Wireless

The project aims at evaluating the use of CERN computing infrastructure for next generation sensor networks data analysis. The proposed system allows the simulation of a large-scale sensor array for traffic analysis, streaming data to CERN storage systems in an efficient way. The data are made available for offline and quasi-online analysis, enabling both long term planning and fast reaction on the environment

Assessing Risks in Two Projects: A Strategic Opportunity and a Necessary Evil

McFarlan\u27s IT Project Risk Assessment Framework (Applegate, et al., 1996), identifies three main areas of IT project risk: project size, project structure, and technology familiarity. According to this framework, if two IT projects are of similar size, a project which is designed primarily around emerging technologies will entail significantly higher risks than a project which is designed primarily around traditional technologies. This paper analyzes two comparably-sized IT projects. One - a telemedicine initiative at Fletcher-Allen Health Care in Vermont -- is designed primarily around emerging technologies. The other - the year 2000 compliance program at the New York Metropolitan Transportation Authority (MTA) - is focused primarily on fixing and testing existing systems on traditional platforms. Our assessment identified two additional salient criteria which, when applied to the two projects revealed higher risks at the MTA. These criteria are time constraints (i.e., the immovable deadline of the year 2000) and system interdependence (i.e., the need for applications to share data with other applications, both within the MTA and with numerous external parties). When these two factors are taken into account, it becomes evident that Year 2000 initiatives represent far higher project risks than the emerging technology projects that are considered to be on the bleeding edge

Future Open Mobile Services

The major barriers for the success of mobile data services are the lack of comprehensible mobile service architectures, their confusing business models and the complexity combined with the inconsistency of the technology enablers. This paper attempts to present a more structured and comprehensive analysis of the current mobile service architectures and their technology enablers. The paper starts with a thorough study of the evolution of mobile services and their business models, and a collection of expectations of the different actors, including the end-user. Next, starting from the original mobile services architecture and environment, an attempt to place the different technology enablers in relation to each other and in relation to their position in the mobile system, will be carried out. Each technology enabler together with their contribution in the enhancement of mobile services are then summarised in a complete and comprehensive way. The paper concludes with a recapitulation of the achievement of the state-of-the-art technology enablers and an identification of future improvements

Sensor Observation Streams Within Cloud-based IoT Platforms: Challenges and Directions

Observation streams can be considered as a special case of data streams produced by sensors. With the growth of the Internet of Things (IoT), more and more connected sensors will produce unbounded observation streams. In order to bridge the gap between sensors and observation consumers, we have witnessed the design and the development of Cloud-based IoT platforms. Such systems raise new research challenges, in particular regarding observation collection, processing and consumption. These new research challenges are related to observation streams and should be addressed from the implementation phase by developers to build platforms able to meet other non-functional requirements later. Unlike existing surveys, this paper is intended for developers that would like to design and implement a Cloud-based IoT platform capable of handling sensor observation streams. It provides a comprehensive way to understand main observation-related challenges, as well as non-functional requirements of IoT platforms such as platform adaptation, scalability and availability. Last but not the least, it gives recommendations and compares some relevant open-source software that can speed up the development process

Entropy/IP: Uncovering Structure in IPv6 Addresses

In this paper, we introduce Entropy/IP: a system that discovers Internet address structure based on analyses of a subset of IPv6 addresses known to be active, i.e., training data, gleaned by readily available passive and active means. The system is completely automated and employs a combination of information-theoretic and machine learning techniques to probabilistically model IPv6 addresses. We present results showing that our system is effective in exposing structural characteristics of portions of the IPv6 Internet address space populated by active client, service, and router addresses. In addition to visualizing the address structure for exploration, the system uses its models to generate candidate target addresses for scanning. For each of 15 evaluated datasets, we train on 1K addresses and generate 1M candidates for scanning. We achieve some success in 14 datasets, finding up to 40% of the generated addresses to be active. In 11 of these datasets, we find active network identifiers (e.g., /64 prefixes or `subnets') not seen in training. Thus, we provide the first evidence that it is practical to discover subnets and hosts by scanning probabilistically selected areas of the IPv6 address space not known to contain active hosts a priori.Comment: Paper presented at the ACM IMC 2016 in Santa Monica, USA (https://dl.acm.org/citation.cfm?id=2987445). Live Demo site available at http://www.entropy-ip.com