Storage Solutions for Big Data Systems: A Qualitative Study and Comparison

Big data systems development is full of challenges in view of the variety of application areas and domains that this technology promises to serve. Typically, fundamental design decisions involved in big data systems design include choosing appropriate storage and computing infrastructures. In this age of heterogeneous systems that integrate different technologies for optimized solution to a specific real world problem, big data system are not an exception to any such rule. As far as the storage aspect of any big data system is concerned, the primary facet in this regard is a storage infrastructure and NoSQL seems to be the right technology that fulfills its requirements. However, every big data application has variable data characteristics and thus, the corresponding data fits into a different data model. This paper presents feature and use case analysis and comparison of the four main data models namely document oriented, key value, graph and wide column. Moreover, a feature analysis of 80 NoSQL solutions has been provided, elaborating on the criteria and points that a developer must consider while making a possible choice. Typically, big data storage needs to communicate with the execution engine and other processing and visualization technologies to create a comprehensive solution. This brings forth second facet of big data storage, big data file formats, into picture. The second half of the research paper compares the advantages, shortcomings and possible use cases of available big data file formats for Hadoop, which is the foundation for most big data computing technologies. Decentralized storage and blockchain are seen as the next generation of big data storage and its challenges and future prospects have also been discussed

LRCoin: Leakage-resilient Cryptocurrency Based on Bitcoin for Data Trading in IoT

Currently, the number of Internet of Thing (IoT) devices making up the IoT is more than 11 billion and this number has been continuously increasing. The prevalence of these devices leads to an emerging IoT business model called Device-as-a-service(DaaS), which enables sensor devices to collect data disseminated to all interested devices. The devices sharing data with other devices could receive some financial reward such as Bitcoin. However, side-channel attacks, which aim to exploit some information leaked from the IoT devices during data trade execution, are possible since most of the IoT devices are vulnerable to be hacked or compromised. Thus, it is challenging to securely realize data trading in IoT environment due to the information leakage such as leaking the private key for signing a Bitcoin transaction in Bitcoin system. In this paper, we propose LRCoin, a kind of leakage-resilient cryptocurrency based on bitcoin in which the signature algorithm used for authenticating bitcoin transactions is leakage-resilient. LRCoin is suitable for the scenarios where information leakage is inevitable such as IoT applications. Our core contribution is proposing an efficient bilinear-based continual-leakage-resilient ECDSA signature. We prove the proposed signature algorithm is unforgeable against adaptively chosen messages attack in the generic bilinear group model under the continual leakage setting. Both the theoretical analysis and the implementation demonstrate the practicability of the proposed scheme.Comment: 9 pages, 3 figures, 1 tabl

Blockchain access control Ecosystem for Big Data security

In recent years, the advancement in modern technologies has experienced an explosion of huge data sets being captured and recorded in different fields, but also given rise to concerns the security and protection of data storage, transmission, processing, and access to data. The blockchain is a distributed ledger that records transactions in a secure, flexible, verifiable and permanent way. Transactions in a blockchain can be an exchange of an asset, the execution of the terms of a smart contract, or an update to a record. In this paper, we have developed a blockchain access control ecosystem that gives asset owners the sovereign right to effectively manage access control of large data sets and protect against data breaches. The Linux Foundation's Hyperledger Fabric blockchain is used to run the business network while the Hyperledger composer tool is used to implement the smart contracts or transaction processing functions that run on the blockchain network

Blockchains Meet Distributed Hash Tables: Decoupling Validation from State Storage

The first obstacle that regular users encounter when setting up a node for a public blockchain is the time taken for downloading all the data needed for the node to start operating correctly. In fact, this may last from hours to weeks for the major networks. Our contribution is twofold. Firstly, we show a design that enables mining and validation of new blocks keeping only a very small state. Secondly, we show that it is possible to store the state of the blockchain in a distributed hash table obtaining a wide spectrum of trade-offs between storage committed by the nodes and replication factor. Our proposal is independent from the consensus algorithm adopted, and copes well with transactions that involve smart contracts.Comment: 13 pages, 3 figures, conference (DLT19

NEWSTRADCOIN: A Blockchain Based Privacy Preserving Secure NEWS Trading Network

In order to stay up to date with world issues and cutting-edge technol-ogies, the newspaper plays a crucial role. However, collecting news is not a very easy task. Currently, news publishers are collecting news from their correspond-ents through social networks, email, phone call, fax etc. and sometimes they buy news from the agencies. However, the existing news sharing networks may not provide security for data integrity and any third party may obstruct the regular flow of news sharing. Moreover, the existing news schemes are very vulnerable in case of disclosing the identity. Therefore, a universal platform is needed in the era of globalization where anyone can share and trade news from anywhere in the world securely, without the interference of third-party, and without disclosing the identity of an individual. Recently, blockchain has gained popularity because of its security mechanism over data, identity, etc. Blockchain enables a distrib-uted way of managing transactions where each participant of the network holds the same copy of the transactions. Therefore, with the help of pseudonymity, fault-tolerance, immutability and the distributed structure of blockchain, a scheme (termed as NEWSTRADCOIN) is presented in this paper in which not only news can be shared securely but also anyone can earn money by selling news. The proposed NEWSTRADCOIN can provide a universal platform where publishers can directly obtain news from news-gatherers in a secure way by main-taining data integrity, without experiencing the interference of a third-party, and without disclosing the identity of the news gatherer and publishers.Comment: 12 pages, 3 figure

Biometric Template Storage with Blockchain: A First Look into Cost and Performance Tradeoffs

We explore practical tradeoffs in blockchain-based biometric template storage. We first discuss opportunities and challenges in the integration of blockchain and biometrics, with emphasis in biometric template storage and protection, a key problem in biometrics still largely unsolved. Blockchain technologies provide excellent architectures and practical tools for securing and managing the sensitive and private data stored in biometric templates, but at a cost. We explore experimentally the key tradeoffs involved in that integration, namely: latency, processing time, economic cost, and biometric performance. We experimentally study those factors by implementing a smart contract on Ethereum for biometric template storage, whose cost-performance is evaluated by varying the complexity of state-of-the-art schemes for face and handwritten signature biometrics. We report our experiments using popular benchmarks in biometrics research, including deep learning approaches and databases captured in the wild. As a result, we experimentally show that straightforward schemes for data storage in blockchain (i.e., direct and hash-based) may be prohibitive for biometric template storage using state-of-the-art biometric methods. A good cost-performance tradeoff is shown by using a blockchain approach based on Merkle trees

Exploiting The Laws of Order in Smart Contracts

We investigate a family of bugs in blockchain-based smart contracts, which we call event-ordering (or EO) bugs. These bugs are intimately related to the dynamic ordering of contract events, i.e., calls of its functions on the blockchain, and enable potential exploits of millions of USD worth of Ether. Known examples of such bugs and prior techniques to detect them have been restricted to a small number of event orderings, typicall 1 or 2. Our work provides a new formulation of this general class of EO bugs as finding concurrency properties arising in long permutations of such events. The technical challenge in detecting our formulation of EO bugs is the inherent combinatorial blowup in path and state space analysis, even for simple contracts. We propose the first use of partial-order reduction techniques, using happen-before relations extracted automatically for contracts, along with several other optimizations built on a dynamic symbolic execution technique. We build an automatic tool called ETHRACER that requires no hints from users and runs directly on Ethereum bytecode. It flags 7-11% of over ten thousand contracts analyzed in roughly 18.5 minutes per contract, providing compact event traces that human analysts can run as witnesses. These witnesses are so compact that confirmations require only a few minutes of human effort. Half of the flagged contracts have subtle EO bugs, including in ERC-20 contracts that carry hundreds of millions of dollars worth of Ether. Thus, ETHRACER is effective at detecting a subtle yet dangerous class of bugs which existing tools miss.Comment: 18 pages, 12 figure

Decentralized Smart Surveillance through Microservices Platform

Connected societies require reliable measures to assure the safety, privacy, and security of members. Public safety technology has made fundamental improvements since the first generation of surveillance cameras were introduced, which aims to reduce the role of observer agents so that no abnormality goes unnoticed. While the edge computing paradigm promises solutions to address the shortcomings of cloud computing, e.g., the extra communication delay and network security issues, it also introduces new challenges. One of the main concerns is the limited computing power at the edge to meet the on-site dynamic data processing. In this paper, a Lightweight IoT (Internet of Things) based Smart Public Safety (LISPS) framework is proposed on top of microservices architecture. As a computing hierarchy at the edge, the LISPS system possesses high flexibility in the design process, loose coupling to add new services or update existing functions without interrupting the normal operations, and efficient power balancing. A real-world public safety monitoring scenario is selected to verify the effectiveness of LISPS, which detects, tracks human objects and identify suspicious activities. The experimental results demonstrate the feasibility of the approach.Comment: 2019 SPIE Defense + Commercial Sensin

Differential Privacy in Blockchain Technology: A Futuristic Approach

Blockchain has received a widespread attention because of its decentralized, tamper-proof, and transparent nature. Blockchain works over the principle of distributed, secured, and shared ledger, which is used to record, and track data within a decentralized network. This technology has successfully replaced certain systems of economic transactions in organizations and has the potential to overtake various industrial business models in future. Blockchain works over peer-to-peer (P2P) phenomenon for its operation and does not require any trusted-third party authorization for data tracking and storage. The information stored in blockchain is distributed throughout the decentralized network and is usually protected using cryptographic hash functions. Since the beginning of blockchain technology, its use in different applications is increasing exponentially, but this increased use has also raised some questions regarding privacy and security of data being stored in it. Protecting privacy of blockchain data using data perturbation strategy such as differential privacy could be a novel approach to overcome privacy issues in blockchain. In this article, we cover the topic of integration of differential privacy in each layer of blockchain and in certain blockchain based scenarios. Moreover, we highlight some future challenges and application scenarios in which integration of differential privacy in blockchain can produce fruitful results.Comment: Submitted to Journa

A Case Study for Grain Quality Assurance Tracking based on a Blockchain Business Network

One of the key processes in Agriculture is quality measurement throughout the transportation of grains along its complex supply chain. This procedure is suitable for failures, such as delays to final destinations, poor monitoring, and frauds. To address the grain quality measurement challenge through the transportation chain, novel technologies, such as Distributed Ledger and Blockchain, can bring more efficiency and resilience to the process. Particularly, Blockchain is a new type of distributed database in which transactions are securely appended using cryptography and hashed pointers. Those transactions can be generated and ruled by special network-embedded software -- known as smart contracts -- that may be public to all nodes of the network or may be private to a specific set of peer nodes. This paper analyses the implementation of Blockchain technology targeting grain quality assurance tracking in a real scenario. Preliminary results support a potential demand for a Blockchain-based certification that would lead to an added valuation of around 15% for GM-free soy in the scope of a Grain Exporter Business Network in Brazil