Melanoma classification on dermoscopy skin images is a demanding task because of the low contrast of the lesion images, the intra-structural variation of melanomas, the high degree of visual similarity between melanoma and non-melanoma lesions, and the existence of hair and ruler marker artifacts. In this study, the malignant melanoma skin cancer classification system is proposed with the aid of correctly classify melanoma skin cancer. The system involves three main steps: segmentation, feature extraction and classification. Ahead of the segmentation step, the preprocessing skin lesion images is processed for getting rid of the covered hair artifacts. In the segmentation step, the input preprocessed lesion image is segmented by using the proposed texture filter-based segmentation method. Then, the features according to the underlying ABCD (Asymmetry, Border, Color, Differential Texture) dermatology rules using shape, edge, color and texture features are extracted from the segmented region. Finally, the extracted features are classified to identify whether the skin image is malignant melanoma or non-melanoma with the use of bag tree ensemble classifier. The system performance is evaluated with the use of the benchmarking datasets: PH2 dataset, ISBI2016 dataset and ISIC2017 dataset. According to the experimental results, the proposed design allows for both reliable classification of real world dermoscopy images and feasible operation time with today’s standard PC computing platforms. To address the class imbalance in the dataset and to yield the improved classification performance, the experiments are also analyzed not only on original imbalanced dataset but also on balancing datasets: undersampled and oversampled datasets. The system works well and provides both high sensitivity and specificity according to the experimental results on the oversampled dataset with bag tree ensemble classifier to leading to statistically better performance compared to original imbalanced dataset