An End-to-End Big Data Analytics Platform for IoT-enabled Smart Factories: A Case Study of Battery Module Assembly System for Electric Vehicles

Within the concept of factories of the future, big data analytics systems play a critical role in supporting decision-making at various stages across enterprise processes. However, the design and deployment of industry-ready, lightweight, modular, flexible, and low-cost big data analytics solutions remains one of the main challenges towards the Industry 4.0 enabled digital transformation. This paper presents an end-to-end IoT-based big data analytics platform that consists of five interconnected layers and several components for data acquisition, integration, storage, analytics and visualisation purposes. The platform architecture benefits from state-of-the-art technologies and integrates them in a systematic and interoperable way with clear information flows. The developed platform has been deployed in an Electric Vehicle (EV) battery module smart assembly automation system designed by the Automation Systems Group (ASG) at the University of Warwick, UK. The developed proof-of-concept solution demonstrates how a wide variety of tools and methods can be orchestrated to work together aiming to support decision-making and to improve both process and product qualities in smart manufacturing environments

Simplifying Big Data Analytics System with A Reference Architecture

The internet and pervasive technology like the Internet of Things (i.e. sensors and smart devices) have exponentially increased the scale of data collection and availability. This big data not only challenges the structure of existing enterprise analytics systems but also offer new opportunities to create new knowledge and competitive advantage. Businesses have been exploiting these opportunities by implementing and operating big data analytics capabilities. Social network companies such as Facebook, LinkedIn, Twitter and Video streaming company like Netflix have implemented big data analytics and subsequently published related literatures. However, these use cases did not provide a simplified and coherent big data analytics reference architecture as well as currently, there still remains limited reference architecture of big data analytics. This paper aims to simplify big data analytics by providing a reference architecture based on existing four use cases and subsequently verified the reference architecture with Amazon and Google analytics services

Speculative Approximations for Terascale Analytics

Model calibration is a major challenge faced by the plethora of statistical analytics packages that are increasingly used in Big Data applications. Identifying the optimal model parameters is a time-consuming process that has to be executed from scratch for every dataset/model combination even by experienced data scientists. We argue that the incapacity to evaluate multiple parameter configurations simultaneously and the lack of support to quickly identify sub-optimal configurations are the principal causes. In this paper, we develop two database-inspired techniques for efficient model calibration. Speculative parameter testing applies advanced parallel multi-query processing methods to evaluate several configurations concurrently. The number of configurations is determined adaptively at runtime, while the configurations themselves are extracted from a distribution that is continuously learned following a Bayesian process. Online aggregation is applied to identify sub-optimal configurations early in the processing by incrementally sampling the training dataset and estimating the objective function corresponding to each configuration. We design concurrent online aggregation estimators and define halting conditions to accurately and timely stop the execution. We apply the proposed techniques to distributed gradient descent optimization -- batch and incremental -- for support vector machines and logistic regression models. We implement the resulting solutions in GLADE PF-OLA -- a state-of-the-art Big Data analytics system -- and evaluate their performance over terascale-size synthetic and real datasets. The results confirm that as many as 32 configurations can be evaluated concurrently almost as fast as one, while sub-optimal configurations are detected accurately in as little as a $1/20^{\text{th}}$ fraction of the time

The UAE Employees’ Perceptions towards Factors for Sustaining Big Data Implementation and Continuous Impact on Their Organization’s Performance

The UAE has officially launched the Big Data initiative in the year 2022; however, the interest in and adoption of Big Data technologies and strategies had started much earlier in the private and public sectors. This research aims to explore the perceptions of the UAE employees on factors needed to implement sustainable Big Data and the continuous impact on their organizational performance. A total of 257 employees were randomly selected for an online survey, and data were collected using a Likert-style five-point scale that was tested for validity and reliability. The findings indicate that employees believe that Big Data Sustainable Implementation leads to Business Performance. Additionally, employees consider factors such as Big Data Architecture Quality, Human Cognitive Factors, and Organizational Readiness to significantly impact on Sustainable Implementation. Further, a moderating impact of Human Cognitive Factors was found on the relationship between Big Data Architecture Quality and Sustainable Implementation. The study provides managerial insights and recommendations for policymaking

Speedup Your Analytics : Automatic Parameter Tuning for Databases and Big Data Systems

Database and big data analytics systems such as Hadoop and Spark have a large number of configuration parameters that control memory distribution, I/O optimization, parallelism, and compression. Improper parameter settings can cause significant performance degradation and stability issues. However, regular users and even expert administrators struggle to understand and tune them to achieve good performance. In this tutorial, we review existing approaches on automatic parameter tuning for databases, Hadoop, and Spark, which we classify into six categories: rule-based, cost modeling, simulation-based, experiment-driven, machine learning, and adaptive tuning. We describe the foundations of different automatic parameter tuning algorithms and present pros and cons of each approach. We also highlight real-world applications and systems, and identify research challenges for handling cloud services, resource heterogeneity, and real-time analytics.Peer reviewe

Classification of customers based on temporal load profile patterns

[EN] The deployment of Advanced Metering Infrastructure (AMI) is providing to utilities large amounts of energy consumption data from their customers, in form of daily load profiles with energy consumed per hour or a smaller period. These data can yield valuable results when analyzed, in order to extract useful knowledge about the typical patterns of consumption of energy from the customers. The proper mechanisms and tools have to be developed and implemented for this objective. Big Data and Big Data Analytics systems will contribute to analyze this information and help to extract knowledge from the data, summarized in form of patterns or other mining knowledge, that will aid experts in decision support. In the present work a classification of customers based on their temporal load profiles is proposed. This classification procedure could be implemented in the current Big Data Analytics software systems, providing an added value to their statistical analysis options. Previous works in the literature present algorithms that allow to classify load profiles from customers by processing batch datasets and obtaining static patterns of load profiles. The proposed technique allows to analyze patterns not only in shape but also in their evolution or trend of energy consumption at each hour of the day through time. Specific quantitative indicators that characterize the patterns (and the consumers associated to them) are described and tested for this purpose.Benítez Sánchez, IJ.; Quijano Lopez, A.; Delgado Espinos, I.; Diez Ruano, JL. (2017). Classification of customers based on temporal load profile patterns. Cigre Science & engineering. (7):143-148. http://hdl.handle.net/10251/104883S143148

An Exploratory-Descriptive Review of Main Big Data Analytics Reference Architectures – an IT Service Management Approach

Big Data Analytics (BDA) aims to create decision-making business value by applying multiple analytical procedures from the Statistics, Operations Research and Artificial Intelligence disciplines to huge internal and external business datasets. However, BDA requires high investments in IT resources – computing, storage, network, software, data, and environment -, and consequently the selection of the right-sized implementation is a hard business managerial decision. Parallelly, IT Service Management (ITSM) frameworks have provided best processes-practices to deliver value to end-users through the concept of IT services, and the provision of BDA as Service (BDAaaS) has now emerged. Consequently, from a dual BDA-ITSM perspective, delivering BDAaaS demands the design and implementation of a concrete BDAaaS architecture. Practitioner and academic literature on BDAaaS architectures is abundant but fragmented, disperse and uses a non-standard terminology. ITSM managers and academics involved on the problematic to deliver BDAaaS, thus, face the lack of mature practical guidelines and theoretical frameworks on BDAaaS architectures. In this research, consequently, with an exploratory-descriptive purpose, we contributed with an updated review of three main non-proprietary BDAaaS reference architectures to ITSM managers, and with a hybrid functional-deployment architectural view to the BDAaaS literature. However, given its exploratory status, further conceptual and empirical research is encouraged