MapReduce Integrated Multi-algorithm for HPC Running State Analysis

High-performance computer clusters are major seismic processing platforms in the oil industry and have a frequent occurrence of failures. In this study, K-means and the Naive Bayes algorithm were programmed into MapReduce and run on Hadoop. The accumulated high-performance computer cluster running status data were first clustered by K-means, and then the results were used for Naive Bayes training. Finally, the test data were discriminated for the knowledge base and equipment failure. Experiments indicate that K-means returned good results, the Naive Bayes algorithm had a high rate of discrimination, and the multi-algorithm used in MapReduce achieved an intelligent prediction mechanism

Learning Continuous Time Bayesian Network Classifiers Using MapReduce

Parameter and structural learning on continuous time Bayesian network classifiers are challenging tasks when you are dealing with big data. This paper describes an efficient scalable parallel algorithm for parameter and structural learning in the case of complete data using the MapReduce framework. Two popular instances of classifiers are analyzed, namely the continuous time naive Bayes and the continuous time tree augmented naive Bayes. Details of the proposed algorithm are presented using Hadoop, an open-source implementation of a distributed file system and the MapReduce framework for distributed data processing. Performance evaluation of the designed algorithm shows a robust parallel scaling