Multi-task deep learning with incomplete training samples for the image-based prediction of variables describing silk fabrics

Abstract

This paper presents a method for the classification of images of silk fabrics with the aim to predict properties such as the placeand time of origin and the production technique. The proposed method was developed in the context of the EU project SILKNOW(http://silknow.eu/). In the context of classification, we address the problem of limited as well as not fully labelled data andinvestigate the connection between the distinct variables. A pre-trained Convolutional Neural Network (CNN) is used for thefeature extraction and a classification network realizing Multi-task learning (MTL) is trained based on these features. The trainingprocedure is adapted to enable the consideration of images that do not have a label for all tasks. Additionally, MTL with fullylabeled training data is investigated for the classification of silk fabrics. The impact of both MTL approaches is compared to singletask learning based on two different class structures. We achieve overall accuracies of 92-95% and average F1-scores of 88-90% inour best experiments. © 2019 Authors