Open Source Dataset and Machine Learning Techniques for Automatic Recognition of Historical Graffiti

Machine learning techniques are presented for automatic recognition of the historical letters (XI-XVIII centuries) carved on the stoned walls of St.Sophia cathedral in Kyiv (Ukraine). A new image dataset of these carved Glagolitic and Cyrillic letters (CGCL) was assembled and pre-processed for recognition and prediction by machine learning methods. The dataset consists of more than 4000 images for 34 types of letters. The explanatory data analysis of CGCL and notMNIST datasets shown that the carved letters can hardly be differentiated by dimensionality reduction methods, for example, by t-distributed stochastic neighbor embedding (tSNE) due to the worse letter representation by stone carving in comparison to hand writing. The multinomial logistic regression (MLR) and a 2D convolutional neural network (CNN) models were applied. The MLR model demonstrated the area under curve (AUC) values for receiver operating characteristic (ROC) are not lower than 0.92 and 0.60 for notMNIST and CGCL, respectively. The CNN model gave AUC values close to 0.99 for both notMNIST and CGCL (despite the much smaller size and quality of CGCL in comparison to notMNIST) under condition of the high lossy data augmentation. CGCL dataset was published to be available for the data science community as an open source resource.Comment: 11 pages, 9 figures, accepted for 25th International Conference on Neural Information Processing (ICONIP 2018), 14-16 December, 2018 (Siem Reap, Cambodia

Метод розпізнавання історичних графіті

Дисертація присвячена розробці та дослідженню методу розпізнавання історичних графіті. Розроблений метод розпізнавання історичних графіті демонструє високу точність класифікації кириличних літер , викарбуваних на стінах собору Софії Київської, продемонстровано результативність запропонованого критерію оптимізації глибинних нейронних мереж при навчанні на малих наборах даних на прикладі застосування до дослідження розпізнання історичних графіті. Запропонована нова структура датасету історичних графіті, що сформована за стандартом ISO 15924, та опублікована у відкритому доступі.The dissertation is devoted to the development and research of the method for historical graffiti image recognition. The developed method of historical graffiti image recognition providing the high performance model for classification task of historical graffiti carved on the walls of st. Sofia cathedral (Kyiv, Ukraine), was suggested method of optimization deep neural networks for small datasets and result was demonstrated on exploration of historical graffiti recognition task. Was suggested a new structure of historical graffiti dataset in accordance with ISO 15924 standard, which available as open source