Storage Solutions for Big Data Systems: A Qualitative Study and Comparison

Big data systems development is full of challenges in view of the variety of application areas and domains that this technology promises to serve. Typically, fundamental design decisions involved in big data systems design include choosing appropriate storage and computing infrastructures. In this age of heterogeneous systems that integrate different technologies for optimized solution to a specific real world problem, big data system are not an exception to any such rule. As far as the storage aspect of any big data system is concerned, the primary facet in this regard is a storage infrastructure and NoSQL seems to be the right technology that fulfills its requirements. However, every big data application has variable data characteristics and thus, the corresponding data fits into a different data model. This paper presents feature and use case analysis and comparison of the four main data models namely document oriented, key value, graph and wide column. Moreover, a feature analysis of 80 NoSQL solutions has been provided, elaborating on the criteria and points that a developer must consider while making a possible choice. Typically, big data storage needs to communicate with the execution engine and other processing and visualization technologies to create a comprehensive solution. This brings forth second facet of big data storage, big data file formats, into picture. The second half of the research paper compares the advantages, shortcomings and possible use cases of available big data file formats for Hadoop, which is the foundation for most big data computing technologies. Decentralized storage and blockchain are seen as the next generation of big data storage and its challenges and future prospects have also been discussed

The Horcrux Protocol: A Method for Decentralized Biometric-based Self-sovereign Identity

Most user authentication methods and identity proving systems rely on a centralized database. Such information storage presents a single point of compromise from a security perspective. If this system is compromised it poses a direct threat to users' digital identities. This paper proposes a decentralized authentication method, called the Horcrux protocol, in which there is no such single point of compromise. The protocol relies on decentralized identifiers (DIDs) under development by the W3C Verifiable Claims Community Group and the concept of self-sovereign identity. To accomplish this, we propose specification and implementation of a decentralized biometric credential storage option via blockchains using DIDs and DID documents within the IEEE 2410-2017 Biometric Open Protocol Standard (BOPS)

Performance Analysis of Blockchain Platforms

Blockchain technologies have drawn massive attention to the world these past few years mostly because of the burst of cryptocurrencies like Bitcoin, Etherium, Ripple and many others. A Blockchain, also known as distributed ledger technology, has demonstrated huge potential in saving time and costs. This open-source technology which generates a decentralized public ledger of transactions is widely appreciated for ensuring a high level of privacy through encryption and thus sharing the transaction details only amongst the participants involved in the transactions. The Blockchain is used not only for cryptocurrency but also by various companies to meet their business ends, such as efficient management of supply chains and logistics. The rise and fall of numerous crypto-currencies based on blockchain technology have generated debate among tech-giants and regulatory bodies. There are various groups which are working on standardizing the blockchain technology. At the same time, numerous groups are actively working, developing and fine-tuning their own blockchain platforms. Platforms such as etherium, hyperledger, parity, etc. have their own pros and cons. This research is focused on the performance analysis of blockchain platforms which gives a comparative understanding of these platforms

Implementation and testing of a Neighborhood Office Center (NOC) and integration of the NOC with an administrative correspondence management information system

The application of telecommunications and telecomputing was investigated as a means of reducing NASA's consumption of natural resources and the proliferation of paper copies of correspondence. The feasibility, operational advantages, and limitations of decentralized (remote) neighborhood offices (NOC) linked through an electronic network are demonstrated. These offices are joined to a management information system for correspondence tracking, and to an administrative office center service based on the use of magnetic medium word processing typewriters which handle the daily typing load. In connection with an augmented teleconference network, a uniform means is provided for creating, storing, and retrieving administrative documents, records, and data, while simultaneously permitting users of the system to track their status. Information will be transferred without using paper - merely through digital electronic communication and display, as a step toward the establishment of an agency-wide electronic mail system

Evaluation of Link Traversal Query Execution over Decentralized Environments with Structural Assumptions

To counter societal and economic problems caused by data silos on the Web, efforts such as Solid strive to reclaim private data by storing it in permissioned documents over a large number of personal vaults across the Web. Building applications on top of such a decentralized Knowledge Graph involves significant technical challenges: centralized aggregation prior to query processing is excluded for legal reasons, and current federated querying techniques cannot handle this large scale of distribution at the expected performance. We propose an extension to Link Traversal Query Processing (LTQP) that incorporates structural properties within decentralized environments to tackle their unprecedented scale. In this article, we analyze the structural properties of the Solid decentralization ecosystem that are relevant for query execution, and provide the SolidBench benchmark to simulate Solid environments representatively. We introduce novel LTQP algorithms leveraging these structural properties, and evaluate their effectiveness. Our experiments indicate that these new algorithms obtain accurate results in the order of seconds for non-complex queries, which existing algorithms cannot achieve. Furthermore, we discuss limitations with respect to more complex queries. This work reveals that a traversal-based querying method using structural assumptions can be effective for large-scale decentralization, but that advances are needed in the area of query planning for LTQP to handle more complex queries. These insights open the door to query-driven decentralized applications, in which declarative queries shield developers from the inherent complexity of a decentralized landscape.Comment: Not peer-reviewe