Leveraging Expert Models for Training Deep Neural Networks in Scarce Data Domains: Application to Offline Handwritten Signature Verification

This paper introduces a novel approach to leverage the knowledge of existing expert models for training new Convolutional Neural Networks, on domains where task-specific data are limited or unavailable. The presented scheme is applied in offline handwritten signature verification (OffSV) which, akin to other biometric applications, suffers from inherent data limitations due to regulatory restrictions. The proposed Student-Teacher (S-T) configuration utilizes feature-based knowledge distillation (FKD), combining graph-based similarity for local activations with global similarity measures to supervise student's training, using only handwritten text data. Remarkably, the models trained using this technique exhibit comparable, if not superior, performance to the teacher model across three popular signature datasets. More importantly, these results are attained without employing any signatures during the feature extraction training process. This study demonstrates the efficacy of leveraging existing expert models to overcome data scarcity challenges in OffSV and potentially other related domains

Siamese-Network Based Signature Verification using Self Supervised Learning

The use of signatures is often encountered in various public documents ranging from academic documents to business documents that are a sign that the existence of signatures is crucial in various administrative processes. The frequent use of signatures does not mean a procedure without loopholes, but we must remain vigilant against signature falsification carried out with various motives behind it. Therefore, in this study, a signature verification system was developed that could prevent the falsification of signatures in public documents by using digital imagery of existing signatures. This study used neural networks with siamese network-based architectures that also empower self-supervised learning techniques to improve accuracy in the realm of limited data. The final evaluation of the machine learning method used gets a maximum accuracy of 83% and this result is better than the machine learning model that does not involve self-supervised learning methods

Deep Adaptive Learning for Writer Identification based on Single Handwritten Word Images

There are two types of information in each handwritten word image: explicit information which can be easily read or derived directly, such as lexical content or word length, and implicit attributes such as the author's identity. Whether features learned by a neural network for one task can be used for another task remains an open question. In this paper, we present a deep adaptive learning method for writer identification based on single-word images using multi-task learning. An auxiliary task is added to the training process to enforce the emergence of reusable features. Our proposed method transfers the benefits of the learned features of a convolutional neural network from an auxiliary task such as explicit content recognition to the main task of writer identification in a single procedure. Specifically, we propose a new adaptive convolutional layer to exploit the learned deep features. A multi-task neural network with one or several adaptive convolutional layers is trained end-to-end, to exploit robust generic features for a specific main task, i.e., writer identification. Three auxiliary tasks, corresponding to three explicit attributes of handwritten word images (lexical content, word length and character attributes), are evaluated. Experimental results on two benchmark datasets show that the proposed deep adaptive learning method can improve the performance of writer identification based on single-word images, compared to non-adaptive and simple linear-adaptive approaches.Comment: Under view of Pattern Recognitio

A White-Box False Positive Adversarial Attack Method on Contrastive Loss-Based Offline Handwritten Signature Verification Models

In this paper, we tackle the challenge of white-box false positive adversarial attacks on contrastive loss-based offline handwritten signature verification models. We propose a novel attack method that treats the attack as a style transfer between closely related but distinct writing styles. To guide the generation of deceptive images, we introduce two new loss functions that enhance the attack success rate by perturbing the Euclidean distance between the embedding vectors of the original and synthesized samples, while ensuring minimal perturbations by reducing the difference between the generated image and the original image. Our method demonstrates state-of-the-art performance in white-box attacks on contrastive loss-based offline handwritten signature verification models, as evidenced by our experiments. The key contributions of this paper include a novel false positive attack method, two new loss functions, effective style transfer in handwriting styles, and superior performance in white-box false positive attacks compared to other white-box attack methods.Comment: 8 pages, 3 figure

DeepSign: Deep On-Line Signature Verification

Deep learning has become a breathtaking technology in the last years, overcoming traditional handcrafted approaches and even humans for many different tasks. However, in some tasks, such as the verification of handwritten signatures, the amount of publicly available data is scarce, what makes difficult to test the real limits of deep learning. In addition to the lack of public data, it is not easy to evaluate the improvements of novel proposed approaches as different databases and experimental protocols are usually considered. The main contributions of this study are: i) we provide an in-depth analysis of state-of-the-art deep learning approaches for on-line signature verification, ii) we present and describe the new DeepSignDB on-line handwritten signature biometric public database, iii) we propose a standard experimental protocol and benchmark to be used for the research community in order to perform a fair comparison of novel approaches with the state of the art, and iv) we adapt and evaluate our recent deep learning approach named Time-Aligned Recurrent Neural Networks (TA-RNNs) for the task of on-line handwritten signature verification. This approach combines the potential of Dynamic Time Warping and Recurrent Neural Networks to train more robust systems against forgeries. Our proposed TA-RNN system outperforms the state of the art, achieving results even below 2.0% EER when considering skilled forgery impostors and just one training signature per user

Automatic intrapersonal variability modeling for offline signature augmentation

Orientador: Luiz Eduardo Soares de OliveiraCoorientadores: Robert Sabourin e Alceu de Souza Britto Jr..Tese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 19/07/2021Inclui referências: p. 93-102Área de concentração: Ciência da ComputaçãoResumo: Normalmente, em um cenario do mundo real, poucas assinaturas estao disponiveis para treinar um sistema de verificacao automatica de assinaturas (SVAA). Para resolver esse problema, diversas abordagens para a duplicacao de assinaturas estaticas foram propostas ao longo dos anos. Essas abordagens geram novas amostras de assinaturas sinteticas aplicando algumas transformacoes na imagem original da assinatura. Algumas delas geram amostras realistas, especialmente o duplicator. Este metodo utiliza um conjunto de parametros para modelar o comportamento do escritor (variabilidade do escritor) ao assinar. No entanto, esses parametros so empiricamente definidos. Este tipo de abordagem pode ser demorado e pode selecionar parametros que nao descrevem a real variabilidade do escritor. A principal hipotese desse trabalho e que a variabilidade do escritor observada no dominio da imagem tambem pode ser transferido para o dominio de caracteristicas. Portanto, este trabalho propoe um novo metodo para modelar automaticamente a variabilidade do escritor para a posterior duplicacao de assinaturas no dominio de imagem (duplicator) e dominio de caracteristicas (filtro Gaussiano e variacao do metodo de Knop). Este trabalho tambem propoe um novo metodo de duplicacao de assinaturas estaticas, que gera as amostras sinteticas diretamente no dominio de caracteristicas usando um filtro Gaussiano. Alem disso, uma nova abordagem para avaliar a qualidade de amostras sinteticas no dominio de caracteristicas e apresentada. As limitacoes e vantagens de ambas as abordagens de duplicacao de assinaturas tambem sao exploradas. Alem de usar a nova abordagem para avaliar a qualidade das amostras, o desempenho de um SVAA e avaliado usando as amostras e tres bases de assinaturas estaticas bem conhecidas: a GPDS-300, a MCYT-75 e a CEDAR. Para a mais utilizada, GPDS-300, quando o classificador SVM foi treinando com somente uma assinatura genuina por escritor, ele obteve um Equal Error Rate (EER) de 5,71%. Quando o classificador tambem utilizou as amostras sinteticas geradas no dominio de imagem, o EER caiu para 1,08%. Quando o classificador foi treinado com as amostras geradas pelo filtro Gaussiano, o EER caiu para 1,04%.Abstract: Normally, in a real-world scenario, there are few signatures available to train an automatic signature verification system (ASVS). To address this issue, several offline signature duplication approaches have been proposed along the years. These approaches generate a new synthetic signature sample applying some transformations in the original signature image. Some of them generate realistic samples, specially the duplicator. This method uses a set of parameters to model the writer's behavior (writer variability) during the signing act. However, these parameters are empirically defined. This kind of approach can be time consuming and can select parameters that do not describe the real writer variability. The main hypothesis of this work is that the writer variability observed in the image space can be transferred to the feature space as well. Therefore, this work proposes a new method to automatically model the writer variability for further signature duplication in the image (duplicator) and the feature space (Gaussian filter and a variation of Knop's method). This work also proposes a new offline signature duplication method, which directly generates the synthetic samples in the feature space using a Gaussian filter. Furthermore, a new approach to assess the quality of the synthetic samples in the feature space is introduced. The limitations and advantages of both signature augmentation approaches are also explored. Despite using the new approach to assess the quality of the samples, the performance of an ASVS was assessed using them and three well-known offline signature datasets: GPDS-300, MCYT-75, and CEDAR. For the most used one, GPDS-300, when the SVM classifier was trained with only one genuine signature per writer, it achieved an Equal Error Rate (EER) of 5.71%. When the classifier also was trained with the synthetic samples generated in the image space, the EER dropped to 1.08%. When the classifier was trained using the synthetic samples generated by the Gaussian filter, the EER dropped to 1.04%

Offline signature verification using writer-dependent ensembles and static classifier selection with handcraft features

Orientador: Eduardo TodtDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 17/02/2022Inclui referências: p. 85-94Área de concentração: Ciência da ComputaçãoResumo: Reconhecimento e identificação de assinaturas em documentos e manuscritos são tarefas desafiadoras que ao longo do tempo vêm sendo estudadas, em especial na questão de discernir assinaturas genuínas de falsificações. Com o recente avanço das tecnologias, principalmente no campo da computação, pesquisas nesta área têm se tornado cada vez mais frequentes, possibilitando o uso de novos métodos de análise das assinaturas, aumentando a precisão e a confiança na verificação delas. Ainda há muito o que se explorar em pesquisas desta área dentro da computação. Verificações de assinaturas consistem, de forma geral, em obter características acerca de um a assinatura e utilizá-las para discerni-la das demais. Estudos propondo variados tipos de métodos foram realizados nos últimos anos a fim de aprimorar os resultados obtidos por sistemas de verificação e identificação de assinaturas. Diferentes formas de extrair características têm sido exploradas, com o o uso de redes neurais artificiais voltadas especificam ente para verificação de assinaturas, como a ResNet e a SigNet, representando o estado-da-arte nesta área de pesquisa. Apesar disso, métodos mais simples de extração de características ainda são muito utilizados, como o histograma de gradientes orientados (HOG), o Local Binary Patterns (LBP) e Local Phase Quantization (LPQ) por exemplo, apresentando, em muitos casos, resultados similares ao estado-da-arte. Não apenas isso, mas diferentes formas de combinar informações de extratores de características e resultados de classificadores têm sido propostos, como é o caso dos seletores de características, métodos de comitê de máquinas e algoritmos de análise da qualidade das características. D esta form a, o trabalho realizado consiste em explorar diferentes métodos de extração de características com binados em um conjunto de classificadores, de maneira que cada conjunto seja construído de forma dependente do autor e seja especificam ente adaptado para reconhecer as melhores características para cada autor, aprendendo quais com binações de classificadores com determinado grupo de características melhor se adaptam para reconhecer suas assinaturas. O desempenho e a funcionalidade do sistema foram comparados com os principais trabalhos da área desenvolvidos nos últimos anos, tendo sido realizados testes com as databases CEDAR, M CYT e UTSig. A pesar de não superar o estado-da-arte, o sistema apresentou bom desempenho, podendo ser com parado com alguns outros trabalhos importantes na área. Além disso, o sistema mostrou a eficiência dos classificadores Support Vector M achine(SVM ) e votadores para a realização da meta-classificação, bem como o potencial de alguns extratores de características para a área de verificação de assinaturas, com o foi o caso do Compound Local Binary Pattern(CLBP).Abstract: Signature recognition and identification in documents and manuscripts are challenging tasks that have been studied over time, especially in the matter of discerning genuine signatures from forgeries. With the recent advancement of technologies, especially in the field of computing, research in this area has become increasingly frequent, enabling the use of new methods of analysis of signatures, increasing accuracy and confidence in their verification. There is still much to be explored in research in this area within computing. Signature verification generally consists in obtaining features about a signature and using them to distinguish it from others. Studies proposing different types o f methods have been carried out in recent years in order to improve the results obtained by signature verification and identification systems. Different ways of extracting features have been explored, such as the use of artificial neural networks specifically aimed at verifying signatures, like ResNet and SigNet, representing the state-of-the-art in this research area. Despite this, simpler methods of feature extraction are still widely used, such as the Histogram of Oriented Gradients (HOG), the Local Binary Patterns (LBP) and the Local Phase Quantization (LPQ) for example, presenting, in many cases, similar results to the state-of-the-art. Not only that, but different ways of combining information from feature extractors and results from classifiers have been proposed, such as feature selectors, machine committee methods and feature quality analysis algorithms. In this way, the developed work consists in exploring different methods of features extractors combined in an ensemble, so that each ensemble is built in a writer-dependent way and is specifically adapted to recognize the best features for each author, learning which combinations of classifiers with a certain group of characteristics is better adapted to recognize their signatures. The performance and functionality of the system were compared w ith the m ain works in the area developed in recent years, w ith tests having been carried out with the CEDAR, M CYT and UTSig databases. Despite not overcoming the state-of-the-art, the system presented good performance, being able to be compared with some other important works in the area. In addition, the system showed the efficiency of Support Vector Machine(SVM ) classifiers and voters to perform the meta-classification, as well as the potential of some feature extractors for the signature verification area, such as the Compound Local Binary Pattern(CLBP)