Optimal trees selection for classification via out-of-bag assessment and sub-bagging

The effect of training data size on machine learning methods has been well investigated over the past two decades. The predictive performance of tree based machine learning methods, in general, improves with a decreasing rate as the size of training data increases. We investigate this in optimal trees ensemble (OTE) where the method fails to learn from some of the training observations due to internal validation. Modified tree selection methods are thus proposed for OTE to cater for the loss of training observations in internal validation. In the first method, corresponding out-of-bag (OOB) observations are used in both individual and collective performance assessment for each tree. Trees are ranked based on their individual performance on the OOB observations. A certain number of top ranked trees is selected and starting from the most accurate tree, subsequent trees are added one by one and their impact is recorded by using the OOB observations left out from the bootstrap sample taken for the tree being added. A tree is selected if it improves predictive accuracy of the ensemble. In the second approach, trees are grown on random subsets, taken without replacement-known as sub-bagging, of the training data instead of bootstrap samples (taken with replacement). The remaining observations from each sample are used in both individual and collective assessments for each corresponding tree similar to the first method. Analysis on 21 benchmark datasets and simulations studies show improved performance of the modified methods in comparison to OTE and other state-of-the-art methods

Random Projection Optimal Trees Ensemble

Ensemble classifiers, formed by the combination of multiple weak learners, have been shown to outperform ordinary classification methods in that the former decrease bias, variance and/or improve predictions. These classifiers, however, can still result in low prediction performance when used with the wrong choice of their hyper-parameters values and/or when there are noisy features in the data. Thus, feature selection and fine tuning hyper-parameter could improve predictive accuracy of ensemble classifiers. This thesis first investigates the effect of feature selection on three methods: Random Forest (RF), Optimal Trees Ensemble (OTE) and Random Projection Ensembles (RP) in high dimensional settings. To this end, LASSO has been considered for selecting the most important features based on training data for dimension reduction. Additionally, the influence of various hyper-parameters regulating the three methods has also been assessed. Secondly, this thesis proposes a novel idea to use random projection method in conjunction with optimal tree selection to get an improved trees ensemble. This is achieved by randomly projecting the training data into lower dimension and classification trees are grown on bootstrap samples taken from the newly projected datasets. The best performing trees are selected based on out-of-bag error rate and combined to get the final Optimal Random Projection Trees Ensemble (ORPTE). The results of ORPTE are compared with those of Tree, RF, OTE, RP, k -NN, XGBoost and SVM. Analysis on several benchmark datasets is given to illustrate the effect of feature selection and hyper-parameter tuning on the methods and the efficiency of the proposed method. The results reveal that feature selection improves the predictive performance of the RP method in addition to reducing the computational burden on benchmark and example datasets. The performance of OTE and RF is less influenced by feature selection. Moreover, ORPTE has outperformed in terms of prediction accuracy in majority of the cases

Functional roles and novel tools for improving‐oxidative stability of polyunsaturated fatty acids: A comprehensive review

Abstract Polyunsaturated fatty acids may be derived from a variety of sources and could be incorporated into a balanced diet. They protect against a wide range of illnesses, including cancer osteoarthritis and autoimmune problems. The PUFAs, ω‐6, and ω‐3 fatty acids, which are found in both the marine and terrestrial environments, are given special attention. The primary goal is to evaluate the significant research papers in relation to the human health risks and benefits of ω‐6 and ω‐3 fatty acid dietary resources. This review article highlights the types of fatty acids, factors affecting the stability of polyunsaturated fatty acids, methods used for the mitigation of oxidative stability, health benefits of polyunsaturated fatty acids, and future perspectives in detail