A Survey on Big Data, Hadoop and it’s Ecosystem

Now days, The 21st century is emphasized by a rapid and enormous change in the field of information technology. It is a non-separable part of our daily life and of multiple other industries like education, genetics, entertainment, science & technology, business etc. In this information age, a vast amount of data generation takes place. This vast amount of data is referred as Big Data. There is a number of challenges present in the Big Data such as capturing data, data analysis, searching of data, sharing of data, filtering of data etc. Today Big Data is applied in various fields like shopping websites such as Amazon, Flipkart, Social networking sites such as Twitter, Facebook, and so on. It is reviewed from some literature that, the Big data tends to use different analysis methods, like predictive analysis, user analysis etc. This paper represents the fact that, Big Data required an open source technology for operating and storing huge amount of data. This paper greatly emphasizes on Apache Hadoop, which has become dominant due to its applicability for processing of big data.Hadoop supports thousands of terabytes of data. Hadoop framework facilitates the analysis of big data and its processing methodologies as well as the structure of an ecosystem

Implementing Graph Pattern Mining for Big Data in the Cloud

With the increasing popularity of various social networking sites, there is an explosive growth in data associated with these, so mining big data has become an important problem in the graph pattern mining research area. Graph mining helps to explore the patterns from networks or databases. Till now various graph mining techniques exist for mining frequent patterns for a graph database which contains relatively small sized graphs. But with the rapid arrival of the era of big data, traditional graph mining approaches have been unable to meet large data analysis needs. In this context, this paper proposes an adaptation to the big graph data mining approach especially in the field of social networks. The proposed approach is based on Hadoop plateform, and improves the efficiency by processing big data in distributed fashion. Again the proposed approach can be adapted to cloud environment which has the merits – load balancing, scalability and efficiency. Experiments have been conducted with real Facebook data set. The approach can be also adapted to dataset larger than experimented data. DOI: 10.17762/ijritcc2321-8169.150514

Small fish in a big pond: an architectural approach to users privacy, rights and security in the age of big data

We focus on the challenges and issues associated with Big Data, and propose a novel architecture that uses the principles of Separation of Concerns and distributed computing to overcome many of the challenges associated with storage, analysis and integrity. We address the issue of asymmetrical distribution of power between the originators of data and the organizations and institutions that make use of that data by taking a systemic perspective to include both sides in our architectural design, shifting from a customer-provider relationship to a more symbiotic one in which control over access to customer data resides with the customer. We illustrate the affordances of the proposed architecture by describing its application in the domain of Social Networking Sites, where we furnish a mechanism to address problems of privacy and identity, and create the potential to open up online social networking to a richer set of possible applications

Towards the Development of a Framework for Socially Responsible Software by Analyzing Social Media Big Data on Cloud Through Ontological Engineering

AbstractA socially responsible internet is the need of the hour considering its huge potential and role in educating and transforming the society. Social computing is emerging as an important area as far as development of next generation web is concerned. With the proliferation of social networking applications, vast amount of data is available on cloud, which may be analyzed to gain useful insight into behavioral and linguistic patterns of different cultural and socio-economic groups further classified on the basis of gender and age etc. The idea is to come up with an appropriate framework for socially responsible software artifacts. These artifacts will monitor online social network data and analyze it from the perspective of socially responsible behavior based on ontological engineering concepts. Identification of socially responsible agents is such an example, though based on a different approach. More examples may be taken from literature dealing with microblog analytics, social semantic web, upper ontology for social web, and social-network-sourced big data analytics. In the present work, it is proposed to focus on analysis/monitoring of socially responsible behavior of social media big data and develop an upper level ontology as the framework/tool for such an analytics

Preserving Social Media: the Problem of Access

As the applications and services made possible through Web 2.0 continue to proliferate and influence the way individuals exchange information, the landscape of social science research, as well as research in the humanities and the arts, has the potential to change dramatically and to be enriched by a wealth of new, user-generated data. In response to this phenomenon, the UK Data Service have commissioned the Digital Preservation Coalition to undertake a 12-month study into the preservation of social media as part of the ‘Big Data Network’ programme funded by the Economic and Social Research Council (ESRC). The larger study focuses on the potential uses and accompanying challenges of data generated by social networking applications. This paper, ‘Preserving Social Media: the Problem of Access’, comprises an excerpt of that longer study, allowing the authors a space to explore in closer detail the issue of making social media archives accessible to researchers and students now and in the future. To do this, the paper addresses use cases that demonstrate the potential value of social media to academic social science. Furthermore, it examines how researchers and collecting institutions acquire and preserve social media data within a context of curatorial and legislative restrictions that may prove an even greater obstacle to access than any technical restrictions. Based on analysis of these obstacles, it will examine existing methods of curating and preserving social media archives, and second, make some recommendations for how collecting institutions might approach the long-term preservation of social media in a way that protects the individuals represented in the data and complies with the conditions of third party platforms. With the understanding that web-based communication technologies will continue to evolve, this paper will focus on the overarching properties of social media, analysing and comparing current methods of curation and preservation that provide sustainable solutions

Evolution of the Field of Social Media Research through Science Maps (2008-2017)

The objectives of this work were to discover the main points of interest in the field of research on Social Media, within the scientific area of Communication, and to analyse how it has evolved. A methodology based on the analysis of co-words and visualisation techniques was applied. The data was obtained from scientific publications indexed in the Web of Science (WoS) database, during the periods 2008-2012 and 2013-2017. The resulting maps showed that, during the period 2008-2012, the main areas of interest were web 2.0 and the internet in terms of social networking sites. However, during the period 2013-2017, there was a strong upward trend in the impact of social networks and platforms, especially Twitter and Facebook, in many areas (such as social movements, public relations and publicity, distribution of content, crisis communication, participatory journalism, political communication, or the configuration of public identities through social platforms, with special emphasis on youth). Finally, new scientific challenges were found in automatic analysis of content and management of big data. In conclusion, it was possible to transform a complex, underlying, dynamic and multidimensional reality into visible representations that could help experts in the field to better understand the evolution of research on Social Media

A Novel Approach for Clustering Big Data based on MapReduce

Clustering is one of the most important applications of data mining. It has attracted attention of researchers in statistics and machine learning. It is used in many applications like information retrieval, image processing and social network analytics etc. It helps the user to understand the similarity and dissimilarity between objects. Cluster analysis makes the users understand complex and large data sets more clearly. There are different types of clustering algorithms analyzed by various researchers. Kmeans is the most popular partitioning based algorithm as it provides good results because of accurate calculation on numerical data. But Kmeans give good results for numerical data only. Big data is combination of numerical and categorical data. Kprototype algorithm is used to deal with numerical as well as categorical data. Kprototype combines the distance calculated from numeric and categorical data. With the growth of data due to social networking websites, business transactions, scientific calculation etc., there is vast collection of structured, semi-structured and unstructured data. So, there is need of optimization of Kprototype so that these varieties of data can be analyzed efficiently.In this work, Kprototype algorithm is implemented on MapReduce in this paper. Experiments have proved that Kprototype implemented on Mapreduce gives better performance gain on multiple nodes as compared to single node. CPU execution time and speedup are used as evaluation metrics for comparison.Intellegent splitter is proposed in this paper which splits mixed big data into numerical and categorical data. Comparison with traditional algorithms proves that proposed algorithm works better for large scale of data

How Big Data Analytics Impacts the Retail Management on the European and American Markets?

Many studies on big data analytics focused on specialized use cases in business environments. Studies have been performed on the use of big data analytics in order to learn about consumer associations and expertise, among others. Nevertheless, there is an absence of investigation within the retail industry contemplating the big data management, looking at the adverse effect on organizational performance and customer satisfaction. Merchants investigate analytics to obtain a unified picture of their operations and customers throughout online channels or stores and make strategic choices towards the management of retail. Thereof, this analysis was carried out by focusing heavily on the European and American retail sector to demonstrate the impact of big data analytics. A quantitative study technique was used to analyze 450 individuals in the European and American retail sector. The outcomes on the analysis mentioned that among the various big data analytics used inside the European and American retail sector, the individuals majorly emphasized social networking analytics. Future scientists can do research on the forthcoming retail fashion on the European and American markets, and the way the consequences of big data evaluation evolved within the previous couple of years and contend with the unpredicted long-term recessions within the European and American retail sector