Troika Generative Adversarial Network (T-GAN): A Synthetic Image Generator That Improves Neural Network Training for Handwriting Classification

Training an artificial neural network for handwriting classification requires a sufficiently sized annotated dataset in order to avoid overfitting. In the absence of sufficient instances, data augmentation techniques are normally considered. In this paper, we propose the troika generative adversarial network (T-GAN) for data augmentation to address the scarcity of publicly labeled handwriting datasets. T-GAN has three generator subnetworks architectured to have some weight-sharing in order to learn the joint distribution from three specific domains. We used T-GAN to augment the data from a subset of the IAM Handwriting Database. We then compared this with other data augmentation techniques by measuring the improvements brought by each technique to the handwriting classification accuracies in three types of artificial neural networks (ANNs): deep ANN, convolutional neural network (CNN), and deep CNN. The data augmentation technique involving the T-GAN yielded the highest accuracy improvements in each of the three ANN classifier types – outperforming the standard techniques of image rotation, affine transformation, and combination of these two – as well as the technique that uses another GAN-based model, the coupled GAN (CoGAN). Furthermore, a paired t-test between the 10-fold cross-validation results of the T-GAN and CoGAN, the second-best augmentation technique in this study, on a deep CNN-made classifier confirmed the superiority of the data augmentation technique that uses the T-GAN. Finally, when the generated synthetic data instances from the T-GAN were further enhanced using the pepper noise removal and median filter, the classification accuracy of the trained CNN and deep CNN classifiers were further improved to 93.54% and 95.45%, respectively. Each of these is a big improvement from the original accuracies of 67.43% and 68.32%, respectively of the 2 classifiers trained on the original unaugmented dataset. Thus, data augmentation using T-GAN – coupled with the mentioned two image noise removal techniques – can be a preferred pre-training technique for augmenting handwriting datasets with insufficient data samples

Multi-Content GAN for Few-Shot Font Style Transfer

In this work, we focus on the challenge of taking partial observations of highly-stylized text and generalizing the observations to generate unobserved glyphs in the ornamented typeface. To generate a set of multi-content images following a consistent style from very few examples, we propose an end-to-end stacked conditional GAN model considering content along channels and style along network layers. Our proposed network transfers the style of given glyphs to the contents of unseen ones, capturing highly stylized fonts found in the real-world such as those on movie posters or infographics. We seek to transfer both the typographic stylization (ex. serifs and ears) as well as the textual stylization (ex. color gradients and effects.) We base our experiments on our collected data set including 10,000 fonts with different styles and demonstrate effective generalization from a very small number of observed glyphs