One method of image recognition that can be used is a convolutional neural network (CNN). However, the training model of CNN is not an easy thing; it takes tuning parameters that take a long time in the training process. This research will do Batik pattern recognition by using CNN. From the experiment that we conducted, the result shows that the feature extraction, selection, and reduction give the accuracy more significant than raw image dataset. The feature selection and reduction also can improve the execution time. Parameters value that gave best accuracy are: epoch = 200, batch_size = 20, optimizer = adam, learning_rate = 0.01, network weight initialization = lecun_uniform, neuron activation function = linear

Ayumi, Vina

Noprisson, Handrie

Nurhaida, Ida

English

Center for Scientific Publication

INTERNATIONAL JOURNAL OF COMPUTING SCIENCE AND APPLIED MATHEMATICS, VOL. 8, NO. 1, FEBRUARY 2022 5Implementation of Convolutional Neural Networksfor Batik Image DatasetVina Ayumi, Ida Nurhaida and Handrie NoprissonAbstract—One method of image recognition that can be usedis a convolutional neural network (CNN). However, the trainingmodel of CNN is not an easy thing; it takes tuning parametersthat take a long time in the training process. This research willdo Batik pattern recognition by using CNN. From the experimentthat we conducted, the result shows that the feature extraction,selection, and reduction give the accuracy more significant thanraw image dataset. The feature selection and reduction alsocan improve the execution time. Parameters value that gavebest accuracy are: epoch = 200, batch size = 20, optimizer= adam, learning rate = 0.01, network weight initialization =lecun uniform, neuron activation function = linear.Index Terms—Image recognition, convolutional neural net-works, optimization of tuning parameters.I. INTRODUCTIONINDONESIA has a cultural heritage in the form of diversityBatik patterns [1]. In general, batik has a pattern of batikmotifs geometry and non-geometry [2]. The existence ofpatterns on this batik motif raises the topic of research in thefield of pattern recognition, especially image processing [3],[4], [5].Image processing technology lately can be used to help theprocess of recognition of batik motifs. Methods that exist inimage processing can help machine learning and recognizingthe motif on batik.Previous research that has been done in the field of imageprocessing technology with the object of batik image hasbeen started in 2009 to 2012 using various methods such asGray-Level Co-Occurrence Metrics (GLCM), Scale InvariantFeature Transform, Canny Edge Detection, Gabor Filter, K-mean, Log Gabor Filter and Color Histogram [3], [6], [7], [8],[9].One method of image recognition that can be used is a con-volutional neural network (CNN). This research is conductedfor Batik pattern recognition by using CNN.II. LITERATURE REVIEWIntroduced by Yann & Yoshua (1998), Convolutional NeuralNetworks (CNN) is inspired by the workings of nerves in thevisual cortex that have complex structures [10], [11].V. Ayumi and I. Nurhaida are with the Faculty of Computer Science, Univer-sitas Mercu Buana, Jakarta, Indonesia e-mail: vina.ayumi@mercubuana.ac.id,ida.nurhaida@mercubuana.ac.id.H. Noprisson is with the Faculty of Computer Science, Universi-tas Mercu Buana, Jakarta, Indonesia and the e-Government and e-Business Research Lab, Universitas Indonesia, Depok, Indonesia e-mail:handrie.noprisson@mercubuana.ac.id.Manuscript received March 31, 2019; accepted January 12, 2022.The visual cortex is one part of the brain that has a rolein processing information that enters through the five senses.In the visual cortex has cells that are very sensitive to thereceptive field (receptive field) is a small part of the area inthe input space. CNN works on input space locally and is goodat searching for strong local spatial correlations in an image[10].CNN belongs to one type of Multilayer Neural Network.Like other neural network theories, CNN is trained to useback-propagation algorithms. CNN is designed to recognizethe visual pattern of image pixels directly by minimizing thepre-process [10].The advantages of CNN are able to recognize patterns thathave different variations, as well as robust against simplegeometric distortion and transformation. CNN uses three archi-tectural ideas to know some degree of shift, scale and distor-tion invariance. These three architectural ideas are respectivelylocal receptive, shared weights, and spatial or temporal sub-sampling.An example of one of the CNN architectures used forcharacter recognition is the LeNet-5 architecture, shown inFig. 1.The CNN network consists of a set of layers where eachlayer consists of one or more sub-layers. Each sub-layer unitreceives input from the surrounding environment that resideson the previous layer. Using the local receptive field, CNN canextract basic visual features such as the orientation of edges,endpoints, and corners. In the next layer, these features arecombined to detect features at a higher level.Figure 2 shows a standard neural network with two hiddenlayers before Dropout (a) is applied, and after a dropout (b)is applied. Implementing a dropout on a neural network isequivalent to reducing the number of networks within a neuralnetwork to a thinned network. This thinner network consistsof all network units that survive the dropout process. Neuralnetwork with n units can be seen as a collection of possiblethinner 2n networks. The whole network shares the weightsso that the number of parameters remains O(n2) or less thanO(n2) [12].III. METHODOLOGYIn this research, we will recognize Batik pattern by usingCNN optimized by regularisation technique namely Dropout.The complete research phases can be seen in Fig. 3.This research work is completed through five phases:1) First, we prepared Batik Image as dataset based on thedefined criteria.INTERNATIONAL JOURNAL OF COMPUTING SCIENCE AND APPLIED MATHEMATICS, VOL. 8, NO. 1, FEBRUARY 2022 6Fig. 1. Architecture of LeNet5 [10].Fig. 2. Dropout Neural Net Model [12].Fig. 3. Phases of research.2) Second, we pre-processed Batik image in order to beready for data augmentation. This phase is completedby increasing the contrast of the image and convertingthe image into a suitable format.3) Third, we conducted data augmentation to generate newdata. The purpose of this phase is to enrich data fortraining. Data augmentation process is elaborated below:a) Image rotation randomly in (0-180) degree.b) Image translation randomly in vertical or horizon-tal.c) Shearing transformation.d) Zoom range to zoom out image randomly.e) Flip horizontal randomly.f) Fill mode to fill out new pixel after image rotationor translation.4) Fourth, we trained a model using CNN with dropoutregularisation.5) Fifth, we tested a model to know the performance of theproposed model.6) Sixth, we evaluated result by computing error rate inpredicting data testing.IV. EXPERIMENTAL RESULTSA. Experimental SetupThe experiment to classify batik pattern was conducted inUbuntu 14.04 LTS 64-bit on a PC with Processor Intel®Core™ i7-6500U CPU @ 2.50GHz × 4, Memory DDR2 RAM8 GB and Hard Disk 160 GB. In this study, we implementedexperiments in Python using Keras deep learning library. Weimplement Deep Learning method, i.e., Convolutional NeuralNetwork with Dropout regularisation. We performed cross-validation and divided the dataset into training and testing data.The testing data took 30% of the dataset while 70% for datatraining. Then the data training samples are trained using deeplearning methods to generate the model.B. Feature ExtractionIn this study we extract four texture features; they areGabor filters, Log Gabor filters, GLCM, and LBP. The featureINTERNATIONAL JOURNAL OF COMPUTING SCIENCE AND APPLIED MATHEMATICS, VOL. 8, NO. 1, FEBRUARY 2022 7TABLE ICLASSIFICATION REPORTClass Precision Recall F1-score Support1 0.76 0.90 0.83 4482 0.50 0.17 0.25 723 0.70 0.59 0.64 974 0.74 0.66 0.70 1865 0.77 0.77 0.77 147avg / total 0.73 0.75 0.73 950extraction and fusion process successfully generated a featurevector with a length of 217 for each image. After the featureextraction and the feature fusion, we make the feature selectionand reduction process using PCA.C. Feature SelectionWe experiment using PCA features selection and reductionto select the best component features. In this experiment weused the same batch size and epoch value; we used batch size= 32 and epoch = 50. We experimented PCA using 25, 50, 100,150, 200, 210, and all component to select the best componentof the feature vector. The PCA features selection and reductionresult shown in Table VI.From Table VI, the best accuracy in this experiment is0.7463 achieved when the number of components (N ) is 50.From this feature selection and reduction process, we canreduce the vector feature to new feature vector with length50, so that we can decrease the computational time, and givesmore accuracy than using all features. Table I presented theclassification report in each class for the N = 50.As presented in Table I, the class “1” has the best score inrecall and f1-score, while the class “5” has the best score inprecision.D. Tuning HyperparametersIn this scenario, we conducted the parallel processing totuning the hyperparameters. As a result in an experimentusing PCA features selection, in this experiment, we used50 features component that achieved the best result in PCAfeatures selection. In this experiment, we used Grid Searchalgorithm that provided in the GridSearchCV class in Scikit-learn. We imported KerasClassifier and GridSearchCV in ourcode; this will allow us to use sklearn’s Grid Search to tuninghyperparameters in parallel processing. The hyperparametersthat we tuned they are: epoch, batch size, training optimisationalgorithm, learning rate, network weight initialisation, andneuron activation function.1) Epoch and batch size: The result in tuning epoch andbatch size gives the best accuracy 75.56% and std score 0.006using some epoch 200, and batch size 20. The computationaltime in this experiment took 264.059723 seconds.2) Optimiser algorithm: The result in tuning optimiseralgorithm gives the best accuracy 74.29% and std score 0.014using Adam optimiser algorithm. The computational time inthis experiment took time 323.357711 second.TABLE IIEPOCH AND BATCH SIZEEpoch/Epoch=50 Epoch=100 Epoch=200batch sizeBs=200.693315 0.740289 0.755646(0.010280) (0.003556) (0.006093)Bs=320.688799 0.719964 0.740289(0.014939) (0.008004) (0.016122)Bs=400.666667 0.746612 0.741192(0.009581) (0.007978) (0.009581)TABLE IIIOPTIMIZER ALGORITHMOptimizer AccuracyAdadelta0.447606(0.010039)Adam0.742999(0.014009)Adamax0.729901(0.013106)TABLE IVLEARNING RATELearning rate Accuracy0.0010.738482(0.012320)0.010.447606(0.010039)0.10.447606(0.010039)0.20.447606(0.010039)0.30.347787(0.135001)TABLE VNETWORK WEIGHT INITIALIZATIONInit mode Accuracyuniform0.735321(0.009011)lecun uniform0.755194(0.005569)normal0.747516(0.005110)3) Learning rate: Table IV presented the result in tuninglearning rate; the result shows that learning rate 0.001 givesthe best accuracy and the processing time in this experimenttook 288.3194 seconds.4) Network weight initialisation: Table V shows thatthe best network weight initialisation on the dataset is le-cun uniform, the accuracy achieved is 75.51% and the pro-cessing time took 270.827701 seconds.INTERNATIONAL JOURNAL OF COMPUTING SCIENCE AND APPLIED MATHEMATICS, VOL. 8, NO. 1, FEBRUARY 2022 8TABLE VIPCA FEATURES SELECTION AND REDUCTION RESULTSParameter all component = 217 N = 25 N = 50 N = 100 N = 150 N = 200 N = 210Accuracy 0.70631 0.71263 0.74631 0.72526 0.73157 0.72210 0.71578F1-Score 0.67940 0.68884 0.72897 0.71226 0.70985 0.70005 0.69216Precision 0.70015 0.71908 0.73148 0.70936 0.73157 0.71836 0.70187Recall 0.70631 0.71263 0.74631 0.72526 0.71571 0.72210 0.71578Kappa 0.55153 0.57890 0.62013 0.59991 0.59605 0.58703 0.56700Time (second) 197.766 33.487 64.212 90.866 135.505 176.733 185.864TABLE VIINEURON ACTIVATION FUNCTIONActivation function Accuracysigmoid0.447606(0.010039)hard sigmoid0.447606(0.010039)linear0.759711(0.003556)5) Neuron activation function: Experiment in tuning theneuron activation function gives the result accuracy 75.97%using the linear function. The computational time in thisscenario took 295.212576 s.V. CONCLUSIONFrom the experiment that we conducted, the result showsthat the feature extraction, selection, and reduction give theaccuracy more significant than raw image dataset. The featureselection and reduction also can improve the execution time.The parameters tuning and using the dropout in layers canimprove the accuracy more. Parameters value that gave bestaccuracy are: epoch = 200, batch size = 20, optimizer =Adam, learning rate = 0.01, network weight initialization =lecun uniform, neuron activation function = linear.ACKNOWLEDGMENTThis research was fully supported Centre of Research,Universitas Mercu Buana through an internal research grant(named penelitian internal).REFERENCES[1] A. Wulandari, Batik Nusantara : Makna Filosofis, Cara Pembuatan &Industri Batik. Yogyakarta, Indonesia: Penerbit ANDI, 2011.[2] S. Samsi, Techniques, Motifs, Patterns Batik Yogya and Solo. TitianFoundation, 2011.[3] I. Nurhaida, R. Manurung, and A. Arymurthy, “Performance comparisonanalysis features extraction methods for batik recognition,” in Inter-national Conference on Advanced Computer Science and InformationSystems (ICACSIS), 2012, pp. 207–212.[4] H. Noprisson, E. Hidayat, and N. Zulkarnaim, “A preliminary study ofmodelling interconnected systems initiatives for preserving indigenousknowledge in indonesia,” in International Conference on InformationTechnology Systems and Innovation (ICITSI), 2015, pp. 1–6.[5] M. Sadikin and I. Wasito, “Toward object interaction mining by startingwith object extraction based on pattern learning method,” in Proceedingsof the Pattern Learning Method Asia-Pacific Materials Science andInformation Technology Conference (APMSIT’14), 2014.[6] I. Nurhaida, A. Noviyanto, R. Manurung, and A. Arymurthy, “Automaticindonesian’s batik pattern recognition using sift approach,” ProcediaComputer Science, vol. 59, pp. 567–576, 2015.[7] R. Akta, “Batik motif classification using scale invariant feature trans-form method,” Ph.D. dissertation, Universitas Indonesia, 2012.[8] K.-S. Loke and M. Cheong, “Efficient textile recognition via decompo-sition of co-occurrence matrices,” in IEEE International Conference onSignal and Image Processing Applications, 2009, pp. 257–261.[9] L. Rahadianti, R. Manurung, and A. Murni, “Clustering batik imagesbased on log-gabor and colour histogram features,” University of In-donesia, 2010.[10] Y. LeCun, Y. Bengio et al., “Convolutional networks for images, speech,and time series,” The handbook of brain theory and neural networks,vol. 3361, no. 10, p. 1995, 1995.[11] D. Hubel and T. Wiesel, “Receptive fields and functional architecture ofmonkey striate cortex,” The Journal of physiology, vol. 195, no. 1, pp.215–243, 1968.[12] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhut-dinov, “Dropout: a simple way to prevent neural networks from over-fitting,” The journal of machine learning research, vol. 15, no. 1, pp.1929–1958, 2014.

Implementation of Convolutional Neural Networks for Batik Image Dataset

https://iptek.its.ac.id/index.php/ijcsam/article/download/5053/6529

Implementation of Convolutional Neural Networks for Batik Image Dataset

Abstract

Similar works

Full text

Available Versions

Center for Scientific Publication