Garcia-Gasulla, Dario

Labarta Mancho, Jesús José

Parés Sabatés, Ferran

UPCommons. Portal del coneixement obert de la UPC

Training CNNs using high-resolution images ofvariable shapeFerran Pare´s∗, Dario Garcia-Gasulla∗, Jesu´s Labarta∗†∗Barcelona Supercomputing Center, Barcelona, Spain†Universitat Polite`cnica de Catalunya, Barcelona, SpainE-mail: {ferran.pares, dario.garcia, jesus.labarta}@bsc.esKeywords—Neural Network, CNN, Medium dataset, High-memory.I. EXTENDED ABSTRACTNeural Networks, concretely Convolutional Neural Net-works, have proven empirically to be capable of successfullysolve image recognition problems. Usually, image recognitiondatasets are composed by a set of images that have been pre-processed to have exactly the same width and height, sinceconsistency between input shapes is a mandatory conditionfor typical Neural Network architectures and a really helpfulcondition for training them successfully. Handling images ofvarying size and varying aspect ratio during a CNN trainingprocess have not been tackled in the literature and presentsseveral challenges like adapting its architecture to avoid theequal image shape limitation, how to group images in batches,how the CNN is going to learn patterns at this wide range ofsizes, how to generalize this new unprocessed problem, amongothers.Additionally, image recognition images are usually down-sampled to a size of 256x256 or similar, which is a reallylow resolution that allows to fit in memory all the activationsproduced in the typical CNN architectures. In fact, usingimages of greater size arises memory consumption issues andnew challenges from the point of view of learning.So, the main objective of this work is to successfully traina CNN using high resolution images of varying size and aspectratio.A. DatasetIn this project we used the Medium dataset, a subset ofthe recently available Met dataset [1]. This dataset consist ofseveral museum pieces like sculptures, paintings, photographsand its main material they are composed by. As you mayexpect, the problem is to predict the main material of the mu-seum piece based on its image. This dataset contains images ofvarying size and aspect ratios, so it is suitable for this project.Smaller images have 0.25MP with shape 500x500 (width xheight) and larger images have 16MP with shape 4000x4000(Figure 1a), while aspect ratios vary from landscape imageslike 2000x1173 (Figure 1b) to portrait images with 572x1026dimensions (Figure 1c).B. CNN architectureAs said before, traditional CNN architectures are unable towork with images of varying shape. That’s mainly due to fullyconnected layers, since these layers require a fixed input shape.In contrast, convolutional layers can work with different inputshapes. So, there exists in the literature a specific type of CNNarchitecture called fully-convolutional that allows to workusing variable input shapes. The main characteristic of thisarchitecture is that it aggregates all the spatial information toa single value before feeding fully-connected layers, effectivelyfixing its shape allowing fully-connected usage.The fully-convolutional layers may solve the ability tofeed images of different shapes, but it will be really hard forthe CNN to learn patterns at this wide range of scales. Forthis purpose we propose to use the Spatial Pyramid Pooling(SPP) [2] to handle several pattern scales before discriminatingclasses.C. Input pipelineThe architecture proposed in the previous subsection allowsto use input images of different shapes on each step, but stillneeds to join images together in batches. It is possible to joinseveral images of different shapes by padding small images tomatch biggest ones in the same batch but, it is interesting to(a) Vase made of terracotta(b) Spoon made of silver (c) Vase made of porcelainFig. 1: Medium dataset: Subset of images6th BSC Severo Ochoa Doctoral Symposium60Fig. 2: Memory profile during training process using imagesfrom a single bucket. In this case, the training process itera-tively doubles the batch size every 3 steps. Initial batch sizeis 8 and it crashes when trying to use batch size of 64.group similar images in the same batch to keep in memory asless non-informative padding as possible. Another reason tominimize the padding is the difficulties it generates in termsof learning. It is unknown in the literature how much paddingaffects training when it gets increased too much.In order to group images by its shape similarity, we havealready designed an input pipeline that we call Buckets toBuckets (B2B). B2B pipeline consists of 2 steps. The first onegroups images in superbuckets based on the number of pixelsand then, the second one groups images from superbucketsinto smaller buckets based on their aspect ratio. However,the performance of this method resides in the right choice ofthe most appropriate size and aspect ratio boundaries used toseparate superbuckets and buckets.Size and aspect ratio boundaries are chosen based on anheuristic technique. This technique tries to find the set ofboundaries that groups images producing as less padding aspossible. So, the heuristic orders all the images based on theirsize or their aspect ratio, places arbitrary boundaries to splitthem in groups and, finally, iteratively move these boundariesback and forth to reduce the padding of all groups.D. Dynamic batch sizeSince we have grouped images in superbuckets based on itsimage size, there will be groups with small images and groupswith larger ones. So, when using large images the pipeline willbatch as much as possible while fitting in memory, but whenusing smaller ones the pipeline will be capable of batchingmore images and still fit in memory. So, on each step, thebatch size used will vary depending on the bucket images arecoming from.However, the dynamic batch size also brings some draw-backs. From the learning point of view the system has to adaptits learning rate depending on the batch size in order to gettraining stability [3], and from the machine point of view, thedynamic batch size brings memory usage variability.E. Memory profilingMemory profiling has been a great tool in this project todetect memory instability, memory reduction when applyingdifferent B2B boundaries, batch size limits, etc. Memoryprofiles are build from tracking the process resident memoryduring training and then visualized in plots like the one inFigure 2.F. Current limitations to overcomeCurrently this project uses Tensorflow and Marenostrum4nodes. The main reason to use Marenostrum4 nodes its theamount of memory that nodes offer (96GB on regular nodesand even more on the high-memory nodes). Regardless ofhaving 96GB available on the MN4 node, we are only capableof using around 40GB of this memory due to a Tensorflowlimitation on its implementation. Tensorflow indexes some ofits tensors with an int32 variable, so the library is unable touse tensors bigger than 2.147.483.648 positions.G. ConclusionsThis project offers a lot of challenges from the point ofview of Artificial Intelligence and High-Performance Comput-ing. There is no standard procedure to train CNN using imagesof varying size and aspect ratio in the AI literature, while theuse of high-resolution images introduces a challenge in termsof memory consumption.Some of the problems are already tackled and explained inthis extended abstract and some of them remain to be solved.The main goal of this project is to successfully train a CNNunder these dataset conditions and, train it as fast as possibleusing greater memory consumption.REFERENCES[1] M. Museum, “Open access initiative,”https://github.com/metmuseum/openaccesse, 2017.[2] K. He, X. Zhang, S. Ren, and J. Sun, “Spatial pyramid pooling in deepconvolutional networks for visual recognition,” IEEE transactions onpattern analysis and machine intelligence, vol. 37, no. 9, pp. 1904–1916,2015.[3] P. Goyal, P. Dolla´r, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola,A. Tulloch, Y. Jia, and K. He, “Accurate, large minibatch sgd: Trainingimagenet in 1 hour,” arXiv preprint arXiv:1706.02677, 2017.Ferran Pare´s received his BSc degree in Electronicsin the Universitat Polite`cnica de Catalunya (UPC),Spain in 2014. Afterward, he completed his MScdegree in Artificial Intelligence in 2016 from a MScprogram with three participating universities: Uni-versitat Polite`cnica de Catalunya (UPC), Universitatde Barcelona (UB) and Universitat Rovira i Virgili(URV). The same year, he joined Barcelona Super-computing Center (BSC) in the High-PerformanceArtificial Intelligence group and started his PhD inArtificial Intelligence from Universitat Polite`cnica deCatalunya (UPC) in the following year (2017), Spain.6th BSC Severo Ochoa Doctoral Symposium61

Training CNNs using high-resolution images of variable shape

UPCommons

https://upcommons.upc.edu/bitstream/2117/167734/1/25_Training_CNNs_using.pdf

Training CNNs using high-resolution images of variable shape

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC

UPCommons