531 research outputs found
Storage Solutions for Big Data Systems: A Qualitative Study and Comparison
Big data systems development is full of challenges in view of the variety of
application areas and domains that this technology promises to serve.
Typically, fundamental design decisions involved in big data systems design
include choosing appropriate storage and computing infrastructures. In this age
of heterogeneous systems that integrate different technologies for optimized
solution to a specific real world problem, big data system are not an exception
to any such rule. As far as the storage aspect of any big data system is
concerned, the primary facet in this regard is a storage infrastructure and
NoSQL seems to be the right technology that fulfills its requirements. However,
every big data application has variable data characteristics and thus, the
corresponding data fits into a different data model. This paper presents
feature and use case analysis and comparison of the four main data models
namely document oriented, key value, graph and wide column. Moreover, a feature
analysis of 80 NoSQL solutions has been provided, elaborating on the criteria
and points that a developer must consider while making a possible choice.
Typically, big data storage needs to communicate with the execution engine and
other processing and visualization technologies to create a comprehensive
solution. This brings forth second facet of big data storage, big data file
formats, into picture. The second half of the research paper compares the
advantages, shortcomings and possible use cases of available big data file
formats for Hadoop, which is the foundation for most big data computing
technologies. Decentralized storage and blockchain are seen as the next
generation of big data storage and its challenges and future prospects have
also been discussed
SchemaAnalyst: Search-Based Test Data Generation for Relational Database Schemas
Data stored in relational databases plays a vital role
in many aspects of society. When this data is incorrect, the
services that depend on it may be compromised. The database
schema is the artefact responsible for maintaining the integrity
of stored data. Because of its critical function, the proper testing
of the database schema is a task of great importance. Employing
a search-based approach to generate high-quality test data for
database schemas, SchemaAnalyst is a tool that supports testing
this key software component. This presented tool is extensible
and includes both an evaluation framework for assessing the
quality of the generated tests and full-featured documentation.
In addition to describing the design and implementation of
SchemaAnalyst and overviewing its efficiency and effectiveness,
this paper coincides with the tool’s public release, thereby enhancing
practitioners’ ability to test relational database schemas
On the performance impact of using JSON, beyond impedance mismatch
NOSQL database management systems adopt semi-structured data models, such as JSON, to easily accommodate schema evolution and overcome the overhead generated from transforming internal structures to tabular data (i.e., impedance mismatch). There exist multiple, and equivalent, ways to physically represent semi-structured data, but there is a lack of evidence about the potential impact on space and query performance. In this paper, we embark on the task of quantifying that, precisely for document stores. We empirically compare multiple ways of representing semi-structured data, which allows us to derive a set of guidelines for efficient physical database design considering both JSON and relational options in the same palette.Partly funded by the European Commission through the programme “EM IT4BI-DC”.Peer ReviewedPostprint (author's final draft
- …