Testing Big Data Applications

Today big data has become the basis of discussion for the organizations. The big task associated with big data stream is coping with its various challenges and performing the appropriate testing for the optimal analysis of the data which may benefit the processing of various activities, especially from a business perspective. Big data term follows the massive volume of data, (might be in units of petabytes or exabytes) exceeding the processing and analytical capacity of the conventional systems and thereby raising the need for analyzing and testing the big data before applications can be put into use. Testing such huge data coming from the various number of sources like the internet, smartphones, audios, videos, media, etc. is a challenge itself. The most favourable solution to test big data follows the automated/programmed approach. This paper outlines the big data characteristics, and various challenges associated with it followed by the approach, strategy, and proposed framework for testing big data applications

D-SPACE4Cloud: A Design Tool for Big Data Applications

The last years have seen a steep rise in data generation worldwide, with the development and widespread adoption of several software projects targeting the Big Data paradigm. Many companies currently engage in Big Data analytics as part of their core business activities, nonetheless there are no tools and techniques to support the design of the underlying hardware configuration backing such systems. In particular, the focus in this report is set on Cloud deployed clusters, which represent a cost-effective alternative to on premises installations. We propose a novel tool implementing a battery of optimization and prediction techniques integrated so as to efficiently assess several alternative resource configurations, in order to determine the minimum cost cluster deployment satisfying QoS constraints. Further, the experimental campaign conducted on real systems shows the validity and relevance of the proposed method

The Implications of Diverse Applications and Scalable Data Sets in Benchmarking Big Data Systems

Now we live in an era of big data, and big data applications are becoming more and more pervasive. How to benchmark data center computer systems running big data applications (in short big data systems) is a hot topic. In this paper, we focus on measuring the performance impacts of diverse applications and scalable volumes of data sets on big data systems. For four typical data analysis applications---an important class of big data applications, we find two major results through experiments: first, the data scale has a significant impact on the performance of big data systems, so we must provide scalable volumes of data sets in big data benchmarks. Second, for the four applications, even all of them use the simple algorithms, the performance trends are different with increasing data scales, and hence we must consider not only variety of data sets but also variety of applications in benchmarking big data systems.Comment: 16 pages, 3 figure

Distributed optimization for big data applications

ope

Big Data Applications in Digital Marketing

Every year, a set of new trends arise that change the course of the digital marketing process and make it easier for marketers to do their work and save time continuously. One of the most critical new trends that have greatly influenced digital marketing and are expected to sustain its impact in the future is Big Data. This article aimed to outline the role of big data in digital marketing by discussing its various applications in digital marketing operations. This article was based on the systematic review methodology by reviewing the previous literature in the study area. The results obtained from the literature showed various applications of big data analytics in digital marketing, including (improving customer experience, measuring and analyzing competitors, innovation and product development....etc.). The article also discovered that companies regularly employ big data to improve the accuracy of different marketing decisions, such as enhancing customer knowledge, providing highly customized promotional content, increasing sales, and measuring the effectiveness of digital marketing campaigns. This article will provide a theoretical base for future researchers to conduct a field study on Turkish companies to examine to what extent they are using big data analytics in digital marketing

Statistical learning and big data applications

The amount of data generated in the field of laboratory medicine has grown to an extent that conventional laboratory information systems (LISs) are struggling to manage and analyze this complex, entangled information (“Big Data”). Statistical learning, a generalized framework from machine learning (ML) and artificial intelligence (AI) is predestined for processing “Big Data” and holds the potential to revolutionize the field of laboratory medicine. Personalized medicine may in particular benefit from AI-based systems, especially when coupled with readily available wearables and smartphones which can collect health data from individual patients and offer new, cost-effective access routes to healthcare for patients worldwide. The amount of personal data collected, however, also raises concerns about patient-privacy and calls for clear ethical guidelines for “Big Data” research, including rigorous quality checks of data and algorithms to eliminate underlying bias and enable transparency. Likewise, novel federated privacy-preserving data processing approaches may reduce the need for centralized data storage. Generative AI-systems including large language models such as ChatGPT currently enter the stage to reshape clinical research, clinical decision-support systems, and healthcare delivery. In our opinion, AI-based systems have a tremendous potential to transform laboratory medicine, however, their opportunities should be weighed against the risks carefully. Despite all enthusiasm, we advocate for stringent added-value assessments, just as for any new drug or treatment. Human experts should carefully validate AI-based systems, including patient-privacy protection, to ensure quality, transparency, and public acceptance. In this opinion paper, data prerequisites, recent developments, chances, and limitations of statistical learning approaches are highlighted

CloudJet4BigData: Streamlining Big Data via an Accelerated Socket Interface

Big data needs to feed users with fresh processing results and cloud platforms can be used to speed up big data applications. This paper describes a new data communication protocol (CloudJet) for long distance and large volume big data accessing operations to alleviate the large latencies encountered in sharing big data resources in the clouds. It encapsulates a dynamic multi-stream/multi-path engine at the socket level, which conforms to Portable Operating System Interface (POSIX) and thereby can accelerate any POSIX-compatible applications across IP based networks. It was demonstrated that CloudJet accelerates typical big data applications such as very large database (VLDB), data mining, media streaming and office applications by up to tenfold in real-world tests