Storage Solutions for Big Data Systems: A Qualitative Study and Comparison

Big data systems development is full of challenges in view of the variety of application areas and domains that this technology promises to serve. Typically, fundamental design decisions involved in big data systems design include choosing appropriate storage and computing infrastructures. In this age of heterogeneous systems that integrate different technologies for optimized solution to a specific real world problem, big data system are not an exception to any such rule. As far as the storage aspect of any big data system is concerned, the primary facet in this regard is a storage infrastructure and NoSQL seems to be the right technology that fulfills its requirements. However, every big data application has variable data characteristics and thus, the corresponding data fits into a different data model. This paper presents feature and use case analysis and comparison of the four main data models namely document oriented, key value, graph and wide column. Moreover, a feature analysis of 80 NoSQL solutions has been provided, elaborating on the criteria and points that a developer must consider while making a possible choice. Typically, big data storage needs to communicate with the execution engine and other processing and visualization technologies to create a comprehensive solution. This brings forth second facet of big data storage, big data file formats, into picture. The second half of the research paper compares the advantages, shortcomings and possible use cases of available big data file formats for Hadoop, which is the foundation for most big data computing technologies. Decentralized storage and blockchain are seen as the next generation of big data storage and its challenges and future prospects have also been discussed

Challenges of Internet of Things and Big Data Integration

The Internet of Things anticipates the conjunction of physical gadgets to the In-ternet and their access to wireless sensor data which makes it expedient to restrain the physical world. Big Data convergence has put multifarious new opportunities ahead of business ventures to get into a new market or enhance their operations in the current market. considering the existing techniques and technologies, it is probably safe to say that the best solution is to use big data tools to provide an analytical solution to the Internet of Things. Based on the current technology deployment and adoption trends, it is envisioned that the Internet of Things is the technology of the future, while to-day's real-world devices can provide real and valuable analytics, and people in the real world use many IoT devices. Despite all the advertisements that companies offer in connection with the Internet of Things, you as a liable consumer, have the right to be suspicious about IoT advertise-ments. The primary question is: What is the promise of the Internet of things con-cerning reality and what are the prospects for the future.Comment: Proceedings of the International Conference on International Conference on Emerging Technologies in Computing 2018 (iCETiC '18), 23rd -24th August, 2018, at London Metropolitan University, London, UK, Published by Springer-Verla

What can AI do for you?

Simply put, most organizations do not know how to approach the incorporation of AI into their businesses, and few are knowledgeable enough to understand which concepts are applicable to their business models. Doing nothing and waiting is not an option: Mahidar and Davenport (2018) argue that companies that try to play catch-up will ultimately lose to those who invested and began learning early. But how do we bridge the gap between skepticism and adoption? We propose a toolkit, inclusive of people, processes, and technologies, to help companies with discovery and readiness to start their AI journey. Our toolkit will deliver specific and actionable answers to the operative question: What can AI do for you

Big Data Management in Education Sector: an Overview

The advancement in technological innovation has given rise to a new trend known as Big Data today. Given the soaring popularity of big data technology, organisations are profoundly attracted to and interested in it to transform their organisation by improving their businesses. Big data is enabling organisations to outpace their competitors and save cost. Similarly, the application of Big Data management in Universities is an essential aspect to institutions that have Big Data to manage; as the use of Big Data in the higher education sector is increasing day by day. Many studies have been carried out on big data and analytics with little interest in its management. Big Data management is a reality that represents a set of challenges involving Big Data modeling, storage, and retrieval, analysis, and visualization for several areas in organizations. This paper introduces and contributes to the conceptual and theoretical understanding of Big Data management within higher education as it outlines its relevance to higher education institutions. It describes the opportunities this growing research area brings to higher education as well as major challenges associated with it

Big Data Management in Education Sector: an Overview

The advancement in technological innovation has given rise to a new trend known as Big Data today. Given the soaring popularity of big data technology, organisations are profoundly attracted to and interested in it to transform their organisation by improving their businesses. Big data is enabling organisations to outpace their competitors and save cost. Similarly, the application of Big Data management in Universities is an essential aspect to institutions that have Big Data to manage; as the use of Big Data in the higher education sector is increasing day by day. Many studies have been carried out on big data and analytics with little interest in its management. Big Data management is a reality that represents a set of challenges involving Big Data modeling, storage, and retrieval, analysis, and visualization for several areas in organizations. This paper introduces and contributes to the conceptual and theoretical understanding of Big Data management within higher education as it outlines its relevance to higher education institutions. It describes the opportunities this growing research area brings to higher education as well as major challenges associated with it

Development of a real-time business intelligence (BI) framework based on hex-elementization of data points for accurate business decision-making

The desire to use business intelligence (BI) to enhance efficiency and effectiveness of business decisions is neither new nor revolutionary. The promise of BI is to provide the ability to capture interrelationship from data and information to guide action towards a business goal. Although BI has been around since the 1960s, businesses still cannot get competitive information in the form they want, when they want and how they want. Business decisions are already full of challenges. The challenges in business decision-making include the use of a vast amount of data, adopting new technologies, and making decisions on a real-time basis. To address these challenges, businesses spend valuable time and resources on data, technologies and business processes. Integration of data in decision-making is crucial for modern businesses. This research aims to propose and validate a framework for organic integration of data into business decision-making. This proposed framework enables efficient business decisions in real-time. The core of this research is to understand and modularise the pre-established set of data points into intelligent and granular “hex-elements” (stated simply, hex-element is a data point with six properties). These intelligent hex-elements build semi-automatic relationships using their six properties between the large volume and high-velocity data points in a dynamic, automated and integrated manner. The proposed business intelligence framework is called “Hex-Elementization” (or “Hex-E” for short). Evolution of technology presents ongoing challenges to BI. These challenges emanate from the challenging nature of the underlying new-age data characterised by large volume, high velocity and wide variety. Efficient and effective analysis of such data depends on the business context and the corresponding technical capabilities of the organisation. Technologies like Big Data, Internet of Things (IoT), Artificial Intelligence (AI) and Machine Learning (ML), play a key role in capitalising on the variety, volume and veracity of data. Extricating the “value” from data in its various forms, depth and scale require synchronizing technologies with analytics and business processes. Transforming data into useful and actionable intelligence is the discipline of data scientists. Data scientists and data analysts use sophisticated tools to crunch data into information which, in turn, are converted into intelligence. The transformation of data into information and its final consumption as actionable business intelligence is an end-to-end journey. This end-to-end transformation of data to intelligence is complex, time-consuming and resource-intensive. This research explores approaches to ease the challenges the of end-to-end transformation of data into intelligence. This research presents Hex-E as a simplified and semi-automated framework to integrate, unify, correlate and coalesce data (from diverse sources and disparate formats) into intelligence. Furthermore, this framework aims to unify data from diverse sources and disparate formats to help businesses make accurate and timely decisions

Optimizing Data Warehousing Startegies

Database technologies have evolved over the last two decades into different constructs to support the ever-growing information needs for organizations spanning the spectrum of operational and analytical processing. The paper examines the characteristics of transactional databases, operational data stores, data warehouses and virtual data warehouses. A framework is developed for an optimal data warehousing strategy based on organizational needs classified by the types of business processes defined by the requirements of supporting functional areas and the levels of decision structures. Enterprise architecture is described to provide an integrated and complimentary view of various data warehousing constructs