AUTO-CDD: automatic cleaning dirty data using machine learning techniques

Cleaning the dirty data has become very critical significance for many years, especially in medical sectors. This is the reason behind widening research in this sector. To initiate the research, a comparison between currently used functions of handling missing values and Auto-CDD is presented. The developed system will guarantee to overcome processing unwanted outcomes in data Analytical process; second, it will improve overall data processing. Our motivation is to create an intelligent tool that will automatically predict the missing data. Starting with feature selection using Random Forest Gini Index values. Then by using three Machine Learning Paradigm trained model was developed and evaluated by two datasets from UCI (i.e. Diabetics and Student Performance). Evaluated outcomes of accuracy proved Random Forest Classifier and Logistic Regression gives constant accuracy at around 90%. Finally, it concludes that this process will help to get clean data for further analytical process

AUTO-CDD: automatic cleaning dirty data using machine learning techniques

Cleaning the dirty data has become very critical significance for many years, especially in medical sectors. This is the reason behind widening research in this sector. To initiate the research, a comparison between currently used functions of handling missing values and Auto-CDD is presented. The developed system will guarantee to overcome processing unwanted outcomes in data Analytical process; second, it will improve overall data processing. Our motivation is to create an intelligent tool that will automatically predict the missing data. Starting with feature selection using Random Forest Gini Index values. Then by using three Machine Learning Paradigm trained model was developed and evaluated by two datasets from UCI (i.e. Diabetics and Student Performance). Evaluated outcomes of accuracy proved Random Forest Classifier and Logistic Regression gives constant accuracy at around 90%. Finally, it concludes that this process will help to get clean data for further analytical process

A Study on the Aspects of Quality of Big Data on Online Business and Recent Tools and Trends Towards Cleaning Dirty Data

The reliability, efficiency, and accuracy of e-business depend on the quality of data that is associated with a buyer, seller, brokers, e-business portals, admins, managers, decision-makers and so on. However, maintaining the quality of data in e-business is very challenging. It is because e-business data typically comes from different communication channels and sources. Integrating and managing the data quality of different sources is generally much troublesome than dealing with traditional business data. Even though there are several data cleaning methods and tools exist those methods and tools have some constraints. None of them directly working, particularly on e-business data that motivates to do research to highlight the aspects of big data quality related to e-business. Therefore, this research demonstrates the problems related to data quality related to online business, discusses the existing literature of data quality, the current tools and techniques that are being used for data quality and provides a research finding highlighting the weaknesses of current tools to address the problem of online business

Towards machine learning-based self-tuning of Hadoop-Spark system

Apache Spark is an open source distributed platform which uses the concept of distributed memory for processing big data. Spark has more than 180 predominant configuration parameter. Configuration settings directly control the efficiency of Apache spark while processing big data, to get the best outcome yet a challenging task as it has many configuration parameters. Currently, these predominant parameters are tuned manually by trial and error. To overcome this manual tuning problem in this paper proposed and developed a self-tuning approach using machine learning. This approach can tune the parameter value when it’s required. The approach was implemented on Dell server and experiment was done on five different sizes of the dataset and parameter. A comparison is provided to highlight the experimented result of the proposed approach with default Spark configuration system. The results demonstrate that the execution is speeded-up by about 33% (on an average) compared to the default configuration

Bangladeshi crops leaf disease detection using YOLOv8

The agricultural sector in Bangladesh is a cornerstone of the nation’s economy, with key crops such as rice, corn, wheat, potato, and tomato playing vital roles. However, these crops are highly vulnerable to various leaf diseases, which pose significant threats to crop yields and food security if not promptly addressed. Consequently, there is an urgent need for an automated system that can accurately identify and categorize leaf diseases, enabling early intervention and management. This study explores the efficacy of the latest state-of-the-art object detection model, YOLOv8 (You Only Look Once), in surpassing previous models for the automated detection and categorization of leaf diseases in these five major crops. By leveraging modern computer vision techniques, the goal is to enhance the efficiency of disease detection and management. A dataset comprising 19 classes, each with 150 images, totaling 2850 images, was meticulously curated and annotated for training and evaluation. The YOLOv8 framework, known for its capability to detect multiple objects simultaneously, was employed to train a deep neural network. The system’s performance was evaluated using standard metrics such as mean Average Precision (mAP) and F1 score. The findings demonstrate that the YOLOv8 framework successfully identifies leaf diseases, achieving a high mAP of 98% and an F1 score of 97%. These results underscore the significant potential of this approach to enhance crop disease management, thereby improving food security and promoting agricultural sustainability in Bangladesh