B Malone

I Tsamardinos

JR Quinlan

N Friedman

Yuan Tan

Z Liu

Zhifa Liu

BMC Bioinformatics

English

Crossref

MEETING ABSTRACT Open AccessFeature selection and prediction with a Markovblanket structure learning algorithmYuan Tan1, Zhifa Liu2*From 12th Annual UT-ORNL-KBRIN Bioinformatics Summit 2013Buchanan, TN, USA. 22-24 March 2013BackgroundClassification and prediction are common tasks inmachine learning. For example, many studies haveattempted to predict gene expression given information,such as DNA sequence, expression of other genes or epi-genetic modifications. Many existing methods, such asneural networks and support vector machines, have beenused to make these predictions. Unfortunately, these blackbox techniques offer little insight into the reasoningbehind the predictions. In many cases, relatively few attri-butes contribute to the classification accuracy. Bayesiannetworks explicitly encode the relationships among attri-butes to make predictions. In a Bayesian network, theMarkov blanket (MB) of the class variable gives all of the* Correspondence: Zhifa.Liu@stjude.org2Department of Biostatistics, St. Jude Children’s Research Hospital, Memphis,TN 38105, USAFull list of author information is available at the end of the articleFigure 1 The implementation of Markov Blanket Feature Selection Algorithm.Tan and Liu BMC Bioinformatics 2013, 14(Suppl 17):A3http://www.biomedcentral.com/1471-2105/14/S17/A3© 2013 Tan and Liu; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative CommonsAttribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction inany medium, provided the original work is properly cited.information necessary to predict its value. In this work,we propose an algorithm to learn only the MB of theclass variable; other attributes are removed. Therefore,our algorithm combines classification and feature selec-tion. Results on benchmark machine learning datasetsindicate that our feature selection technique usuallyreduces the size of the dataset more than 80% on somedatasets. Accuracy results suggest that the classificationability of our algorithm is competitive with existing stateof the art techniques.Materials and methodsIn a classification problem, we are given a dataset con-sisting of a set of attributes A and a class variable C.Furthermore, the dataset is split into a training set Dtrand a testing set Dte. The goal is to learn a classifierfrom Dtr that correctly predicts C in Dte. In this study,we compared the performance of our Markov Blanketstructure with other classical classifiers such as C4.5 [1], optimal Bayesian network [2], and Tree AugmentedNaïve Bayes Network [3] and Markov Blank Hill Climbing[4]. Here is a general introduction for those classifiers.Markov blanket feature selection algorithmThe intuition of this algorithm is that an attribute iseither a parent, child or spouse of C, or the attribute isnot in C’s MB. Hence we only add each attribute to theMB according to an ordering and score for the newnetwork. And we do not add attributes that make thescore worse. Meanwhile, we keep the MB with the bestscore among all orderings. To this end, our algorithm per-forms feature selection and finds MB structure which hasthe maximum classification ability. The return structure isa Bayesian classifier for classification variable C. The gen-eral idea of the algorithm is shown in the Figure 1.ExperimentsWe compared our feature selection algorithm to severalstate of the art classification methods on several bench-mark datasets. All of the classification methods weselected learn a “human readable” model. In order torepresent a wide variety of data domains, we downloaded14 datasets from UCI machine learning repository [5]. Thedata processing and the classification steps in Figure 2 wasfollowed a similar data procedure in [6].ResultsAs shown in Figure 3 for compression ratio of thesebenchmark datasets, our feature selection often achievedquite high compression ratios by ignoring attributeswhich do not help predict C. From this, we infer thatonly a few attributes are necessary to predict C.The accuracy results in Figure 4 demonstrate that,despite compressing the data over 80% in some cases,MB feature selection is still competitive in terms ofaccuracy with state of the art methods.Figure 2 The procedure of data processing and classification.Figure 3 The Compression Ratio for Benchmark Datasets by Markov Blanket Feature Selection Algorithm.Tan and Liu BMC Bioinformatics 2013, 14(Suppl 17):A3http://www.biomedcentral.com/1471-2105/14/S17/A3Page 2 of 3Discussion and conclusionsThe compression ratio decreases as the number of vari-ables in the dataset increases. This suggests that, even asdataset sizes increase, only a few attributes are helpful inpredicting the class variable. The compression ratio isunaffected by the number of records in the dataset. Thissuggests that even when given many records, our algo-rithm does not pick many attributes in an attempt tooverfit the dataset. Ignoring unimportant attributes doesnot significantly affect the classification accuracy.Despite compressing the data on average more than70%, the classification accuracy is rarely more that 5%below the best classifier. Identifying MB variables couldsignificantly reduce the cost of diagnostic lab tests byfocusing interest on only the most relevant attributes.Authors’ details1Department of Statistics, Mississippi State University, Starkville, MS 39759,USA. 2Department of Biostatistics, St. Jude Children’s Research Hospital,Memphis, TN 38105, USA.Published: 22 October 2013References1. Quinlan JR: C4.5: programs for machine learning. Machine Learning 1994,16(3):235-240.2. Malone B, Yuan C, Hansen E, Bridges S: Improving the scalability ofoptimal Bayesian network learning with external-memory frontierbreadth-first branch and bound search. In Proceedings of the Twenty-Seventh Annual Conference on Uncertainty in Artificial Intelligence. Volume2011. Barcelona: AUAI Press;Fabio G. Cozman and Avi Pfeffer 2011:479-488.3. Friedman N, Geiger D, Goldszmidt M: Bayesian network classifiers.Machine Learning 1997, 29:131-163.4. Tsamardinos I, Brown LE, Aliferis CF: The max-min hill-climbing Bayesiannetwork structure learning algorithm. Machine Learning 2006, 65(1):31-78.5. Bache K, Lichman M: UCI Machine Learning Repository.[http://archive.ics.uci.edu/ml].6. Liu Z, Malone B, Yuan C: Empirical evaluation of scoring functions forBayesian network model selection. BMC Bioinformatics 2012, 13(Suppl 15):S14.doi:10.1186/1471-2105-14-S17-A3Cite this article as: Tan and Liu: Feature selection and prediction with aMarkov blanket structure learning algorithm. BMC Bioinformatics 2013 14(Suppl 17):A3.Submit your next manuscript to BioMed Centraland take full advantage of: • Convenient online submission• Thorough peer review• No space constraints or color figure charges• Immediate publication on acceptance• Inclusion in PubMed, CAS, Scopus and Google Scholar• Research which is freely available for redistributionSubmit your manuscript at www.biomedcentral.com/submitFigure 4 The classification results for Benchmark Datasets by Markov Blanket Feature Selection Algorithm.Tan and Liu BMC Bioinformatics 2013, 14(Suppl 17):A3http://www.biomedcentral.com/1471-2105/14/S17/A3Page 3 of 3

Feature selection and prediction with a Markov blanket structure learning algorithm

https://bmcbioinformatics.biomedcentral.com/track/pdf/10.1186/1471-2105-14-S17-A3?site=bmcbioinformatics.biomedcentral.com

Feature selection and prediction with a Markov blanket structure learning algorithm

Abstract

Similar works

Full text

Available Versions

Crossref