An Evaluation of Physical Disk I/Os for Complex Object Processing

In order to obtain the performance required for nonstandard database environments, a hierarchical complex object model with object references is used as a storage structure for complex objects. Several storage models for these complex objects, as well as a benchmark to evaluate their performance, are described. A cost model for analytical performance evaluation is developed, and the analytical results are validated by means of measurements on the DASDBS, complex object storage system. The results show which storage structures for complex objects are the most efficient under which circumstance

PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development

This paper describes PlinyCompute, a system for development of high-performance, data-intensive, distributed computing tools and libraries. In the large, PlinyCompute presents the programmer with a very high-level, declarative interface, relying on automatic, relational-database style optimization to figure out how to stage distributed computations. However, in the small, PlinyCompute presents the capable systems programmer with a persistent object data model and API (the "PC object model") and associated memory management system that has been designed from the ground-up for high performance, distributed, data-intensive computing. This contrasts with most other Big Data systems, which are constructed on top of the Java Virtual Machine (JVM), and hence must at least partially cede performance-critical concerns such as memory management (including layout and de/allocation) and virtual method/function dispatch to the JVM. This hybrid approach---declarative in the large, trusting the programmer's ability to utilize PC object model efficiently in the small---results in a system that is ideal for the development of reusable, data-intensive tools and libraries. Through extensive benchmarking, we show that implementing complex objects manipulation and non-trivial, library-style computations on top of PlinyCompute can result in a speedup of 2x to more than 50x or more compared to equivalent implementations on Spark.Comment: 48 pages, including references and Appendi

Servicing the federation : the case for metadata harvesting

The paper presents a comparative analysis of data harvesting and distributed computing as complementary models of service delivery within large-scale federated digital libraries. Informed by requirements of flexibility and scalability of federated services, the analysis focuses on the identification and assessment of model invariants. In particular, it abstracts over application domains, services, and protocol implementations. The analytical evidence produced shows that the harvesting model offers stronger guarantees of satisfying the identified requirements. In addition, it suggests a first characterisation of services based on their suitability to either model and thus indicates how they could be integrated in the context of a single federated digital library

A model for Business Intelligence Systems’ Development

Often, Business Intelligence Systems (BIS) require historical data or data collected from var-ious sources. The solution is found in data warehouses, which are the main technology used to extract, transform, load and store data in the organizational Business Intelligence projects. The development cycle of a data warehouse involves lots of resources, time, high costs and above all, it is built only for some specific tasks. In this paper, we’ll present some of the aspects of the BI systems’ development such as: architecture, lifecycle, modeling techniques and finally, some evaluation criteria for the system’s performance.BIS (Business Intelligence Systems), Data Warehouses, OLAP (On-Line Analytical Processing), Object-Oriented Modeling

Data Warehouse Design and Management: Theory and Practice

The need to store data and information permanently, for their reuse in later stages, is a very relevant problem in the modern world and now affects a large number of people and economic agents. The storage and subsequent use of data can indeed be a valuable source for decision making or to increase commercial activity. The next step to data storage is the efficient and effective use of information, particularly through the Business Intelligence, at whose base is just the implementation of a Data Warehouse. In the present paper we will analyze Data Warehouses with their theoretical models, and illustrate a practical implementation in a specific case study on a pharmaceutical distribution companyData warehouse, database, data model.

Storage Solutions for Big Data Systems: A Qualitative Study and Comparison

Big data systems development is full of challenges in view of the variety of application areas and domains that this technology promises to serve. Typically, fundamental design decisions involved in big data systems design include choosing appropriate storage and computing infrastructures. In this age of heterogeneous systems that integrate different technologies for optimized solution to a specific real world problem, big data system are not an exception to any such rule. As far as the storage aspect of any big data system is concerned, the primary facet in this regard is a storage infrastructure and NoSQL seems to be the right technology that fulfills its requirements. However, every big data application has variable data characteristics and thus, the corresponding data fits into a different data model. This paper presents feature and use case analysis and comparison of the four main data models namely document oriented, key value, graph and wide column. Moreover, a feature analysis of 80 NoSQL solutions has been provided, elaborating on the criteria and points that a developer must consider while making a possible choice. Typically, big data storage needs to communicate with the execution engine and other processing and visualization technologies to create a comprehensive solution. This brings forth second facet of big data storage, big data file formats, into picture. The second half of the research paper compares the advantages, shortcomings and possible use cases of available big data file formats for Hadoop, which is the foundation for most big data computing technologies. Decentralized storage and blockchain are seen as the next generation of big data storage and its challenges and future prospects have also been discussed

Communications software performance prediction

Software development can be costly and it is important that confidence in a software system be established as early as possible in the design process. Where the software supports communication services, it is essential that the resultant system will operate within certain performance constraints (e.g. response time). This paper gives an overview of work in progress on a collaborative project sponsored by BT which aims to offer performance predictions at an early stage in the software design process. The Permabase architecture enables object-oriented software designs to be combined with descriptions of the network configuration and workload as a basis for the input to a simulation model which can predict aspects of the performance of the system. The prototype implementation of the architecture uses a combination of linked design and simulation tools

Wide, long, or nested data? Reconciling the machine and human viewpoints

Data expressed in tables may be re-arranged in various forms, while conveying the same information. This can create a tension when one form is easier to comprehend by a human reader, but another form is more convenient for processing by machine. This problem has received considerable attention for data scientists writing code, but rather less for end user analysts using spreadsheets. We propose a new data model, the “lish”, which supports a spreadsheet-like flexibility of layout, while capturing sufficient structure to facilitate processing. Using a typical example in a prototype editor, we demonstrate how it might help users resolve the tension between the two forms. A user study is in preparation