Storage Solutions for Big Data Systems: A Qualitative Study and Comparison

Big data systems development is full of challenges in view of the variety of application areas and domains that this technology promises to serve. Typically, fundamental design decisions involved in big data systems design include choosing appropriate storage and computing infrastructures. In this age of heterogeneous systems that integrate different technologies for optimized solution to a specific real world problem, big data system are not an exception to any such rule. As far as the storage aspect of any big data system is concerned, the primary facet in this regard is a storage infrastructure and NoSQL seems to be the right technology that fulfills its requirements. However, every big data application has variable data characteristics and thus, the corresponding data fits into a different data model. This paper presents feature and use case analysis and comparison of the four main data models namely document oriented, key value, graph and wide column. Moreover, a feature analysis of 80 NoSQL solutions has been provided, elaborating on the criteria and points that a developer must consider while making a possible choice. Typically, big data storage needs to communicate with the execution engine and other processing and visualization technologies to create a comprehensive solution. This brings forth second facet of big data storage, big data file formats, into picture. The second half of the research paper compares the advantages, shortcomings and possible use cases of available big data file formats for Hadoop, which is the foundation for most big data computing technologies. Decentralized storage and blockchain are seen as the next generation of big data storage and its challenges and future prospects have also been discussed

Towards a flexible and transparent database evolution

Applications refactorings that imply the schema evolution are common activities in programming practices. Although modern object-oriented databases provide transparent schema evolution mechanisms, those refactorings continue to be time consuming tasks for programmers. In this paper we address this problem with a novel approach based on aspect-oriented programming and orthogonal persistence paradigms, as well as our meta-model. An overview of our framework is presented. This framework, a prototype based on that approach, provides applications with aspects of persistence and database evolution. It also provides a new pointcut/advice language that enables the modularization of the instance adaptation crosscutting concern of classes, which were subject to a schema evolution. We also present an application that relies on our framework. This application was developed without any concern regarding persistence and database evolution. However, its data is recovered in each execution, as well as objects, in previous schema versions, remain available, transparently, by means of our framework

On the standardisation of Web service management operations

Given the current interest in TCP/IP network management research towards Web services, it is important to recognise how standardisation can be achieved. This paper mainly focuses on the standardisation of operations and not management information. We state that standardisation should be done by standardising the abstract parts of a WSDL document, i.e. the interfaces and the messages. Operations can vary in granularity and parameter transparency, creating four extreme operation signatures, all of which have advantages and disadvantages

Modularizing application and database evolution - an aspect-oriented framework for orthogonal persistence

In the maintenance of software applications, database evolution is one common difficulty. In object‐oriented databases, this process comprises schema evolution and instance adaptation. Both tasks usually require significant effort from programmers and database administrators. In this paper, we propose orthogonal persistence and aspect‐oriented programming to support semi‐transparent database evolution. A default mechanism for instance evolution is defined, but the user may provide modularized solutions using the aspect‐oriented paradigm. We present our framework AOF4OOP to test the feasibility of our proposed approach. This prototype allows programmes to transparently access data in other versions of the database schema. We evaluate our framework, comparing it to related approaches using two real applications and measuring the improvement of the productivity of the programmer.info:eu-repo/semantics/publishedVersio

Web based system architecture for long pulse remote experimentation

Remote experimentation (RE) methods will be essential in next generation fusion devices. Requirements for long pulse RE will be: on-line data visualization, on-line data acquisition processes monitoring and on-line data acquisition systems interactions (start, stop or set-up modifications). Note that these methods are not oriented to real-time control of fusion plant devices. INDRA Sistemas S.A., CIEMAT (Centro de Investigaciones Energéticas Medioambientales y Tecnológicas) and UPM (Universidad Politécnica de Madrid) have designed a specific software architecture for these purposes. The architecture can be supported on the BeansNet platform, whose integration with an application server provides an adequate solution to the requirements. BeansNet is a JINI based framework developed by INDRA, which makes easy the implementation of a remote experimentation model based on a Service Oriented Architecture. The new software architecture has been designed on the basis of the experience acquired in the development of an upgrade of the TJ-II remote experimentation system

TEMPOS: A Platform for Developing Temporal Applications on Top of Object DBMS

This paper presents TEMPOS: a set of models and languages supporting the manipulation of temporal data on top of object DBMS. The proposed models exploit object-oriented technology to meet some important, yet traditionally neglected design criteria related to legacy code migration and representation independence. Two complementary ways for accessing temporal data are offered: a query language and a visual browser. The query language, namely TempOQL, is an extension of OQL supporting the manipulation of histories regardless of their representations, through fully composable functional operators. The visual browser offers operators that facilitate several time-related interactive navigation tasks, such as studying a snapshot of a collection of objects at a given instant, or detecting and examining changes within temporal attributes and relationships. TEMPOS models and languages have been formalized both at the syntactical and the semantical level and have been implemented on top of an object DBMS. The suitability of the proposals with regard to applications' requirements has been validated through concrete case studies

A framework for multi-tier type evolution and data migration

This paper describes a framework that supports the simultaneous evolution of objectoriented data models and relational schemas with respect to a tool- supported object-relational mapping. Thereby the proposed framework accounts for non-trivial data migration induced by type evolution from the outset. The support for data migration is offered on the level of transparent data access. The framework consists of the following integrated parts: an automatic model change detection mechanism, a generator for schema evolution code and a generator for data migration APIs. The framework has been concepted in the IMIS project. IMIS is an information system for environmental radioactivity measurements. Though the indicated domain especially demands a solution like the one discussed in the paper, the achievements are of general purpose for multi-tier system architectures with object-relational mapping

Web and Semantic Web Query Languages

A number of techniques have been developed to facilitate powerful data retrieval on the Web and Semantic Web. Three categories of Web query languages can be distinguished, according to the format of the data they can retrieve: XML, RDF and Topic Maps. This article introduces the spectrum of languages falling into these categories and summarises their salient aspects. The languages are introduced using common sample data and query types. Key aspects of the query languages considered are stressed in a conclusion

Database evolution on an orthogonal persistent programming system: A semi-transparent approach

In this paper the problem of the evolution of an object-oriented database in the context of orthogonal persistent programming systems is addressed. We have observed two characteristics in that type of systems that offer particular conditions to implement the evolution in a semi-transparent fashion. That transparency can further be enhanced with the obliviousness provided by the Aspect-Oriented Programming techniques. Was conceived a meta-model and developed a prototype to test the feasibility of our approach. The system allows programs, written to a schema, access semi-transparently to data in other versions of the schema