Offline signature verification using writer-dependent ensembles and static classifier selection with handcraft features

Orientador: Eduardo TodtDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 17/02/2022Inclui referências: p. 85-94Área de concentração: Ciência da ComputaçãoResumo: Reconhecimento e identificação de assinaturas em documentos e manuscritos são tarefas desafiadoras que ao longo do tempo vêm sendo estudadas, em especial na questão de discernir assinaturas genuínas de falsificações. Com o recente avanço das tecnologias, principalmente no campo da computação, pesquisas nesta área têm se tornado cada vez mais frequentes, possibilitando o uso de novos métodos de análise das assinaturas, aumentando a precisão e a confiança na verificação delas. Ainda há muito o que se explorar em pesquisas desta área dentro da computação. Verificações de assinaturas consistem, de forma geral, em obter características acerca de um a assinatura e utilizá-las para discerni-la das demais. Estudos propondo variados tipos de métodos foram realizados nos últimos anos a fim de aprimorar os resultados obtidos por sistemas de verificação e identificação de assinaturas. Diferentes formas de extrair características têm sido exploradas, com o o uso de redes neurais artificiais voltadas especificam ente para verificação de assinaturas, como a ResNet e a SigNet, representando o estado-da-arte nesta área de pesquisa. Apesar disso, métodos mais simples de extração de características ainda são muito utilizados, como o histograma de gradientes orientados (HOG), o Local Binary Patterns (LBP) e Local Phase Quantization (LPQ) por exemplo, apresentando, em muitos casos, resultados similares ao estado-da-arte. Não apenas isso, mas diferentes formas de combinar informações de extratores de características e resultados de classificadores têm sido propostos, como é o caso dos seletores de características, métodos de comitê de máquinas e algoritmos de análise da qualidade das características. D esta form a, o trabalho realizado consiste em explorar diferentes métodos de extração de características com binados em um conjunto de classificadores, de maneira que cada conjunto seja construído de forma dependente do autor e seja especificam ente adaptado para reconhecer as melhores características para cada autor, aprendendo quais com binações de classificadores com determinado grupo de características melhor se adaptam para reconhecer suas assinaturas. O desempenho e a funcionalidade do sistema foram comparados com os principais trabalhos da área desenvolvidos nos últimos anos, tendo sido realizados testes com as databases CEDAR, M CYT e UTSig. A pesar de não superar o estado-da-arte, o sistema apresentou bom desempenho, podendo ser com parado com alguns outros trabalhos importantes na área. Além disso, o sistema mostrou a eficiência dos classificadores Support Vector M achine(SVM ) e votadores para a realização da meta-classificação, bem como o potencial de alguns extratores de características para a área de verificação de assinaturas, com o foi o caso do Compound Local Binary Pattern(CLBP).Abstract: Signature recognition and identification in documents and manuscripts are challenging tasks that have been studied over time, especially in the matter of discerning genuine signatures from forgeries. With the recent advancement of technologies, especially in the field of computing, research in this area has become increasingly frequent, enabling the use of new methods of analysis of signatures, increasing accuracy and confidence in their verification. There is still much to be explored in research in this area within computing. Signature verification generally consists in obtaining features about a signature and using them to distinguish it from others. Studies proposing different types o f methods have been carried out in recent years in order to improve the results obtained by signature verification and identification systems. Different ways of extracting features have been explored, such as the use of artificial neural networks specifically aimed at verifying signatures, like ResNet and SigNet, representing the state-of-the-art in this research area. Despite this, simpler methods of feature extraction are still widely used, such as the Histogram of Oriented Gradients (HOG), the Local Binary Patterns (LBP) and the Local Phase Quantization (LPQ) for example, presenting, in many cases, similar results to the state-of-the-art. Not only that, but different ways of combining information from feature extractors and results from classifiers have been proposed, such as feature selectors, machine committee methods and feature quality analysis algorithms. In this way, the developed work consists in exploring different methods of features extractors combined in an ensemble, so that each ensemble is built in a writer-dependent way and is specifically adapted to recognize the best features for each author, learning which combinations of classifiers with a certain group of characteristics is better adapted to recognize their signatures. The performance and functionality of the system were compared w ith the m ain works in the area developed in recent years, w ith tests having been carried out with the CEDAR, M CYT and UTSig databases. Despite not overcoming the state-of-the-art, the system presented good performance, being able to be compared with some other important works in the area. In addition, the system showed the efficiency of Support Vector Machine(SVM ) classifiers and voters to perform the meta-classification, as well as the potential of some feature extractors for the signature verification area, such as the Compound Local Binary Pattern(CLBP)

Automatic intrapersonal variability modeling for offline signature augmentation

Orientador: Luiz Eduardo Soares de OliveiraCoorientadores: Robert Sabourin e Alceu de Souza Britto Jr..Tese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 19/07/2021Inclui referências: p. 93-102Área de concentração: Ciência da ComputaçãoResumo: Normalmente, em um cenario do mundo real, poucas assinaturas estao disponiveis para treinar um sistema de verificacao automatica de assinaturas (SVAA). Para resolver esse problema, diversas abordagens para a duplicacao de assinaturas estaticas foram propostas ao longo dos anos. Essas abordagens geram novas amostras de assinaturas sinteticas aplicando algumas transformacoes na imagem original da assinatura. Algumas delas geram amostras realistas, especialmente o duplicator. Este metodo utiliza um conjunto de parametros para modelar o comportamento do escritor (variabilidade do escritor) ao assinar. No entanto, esses parametros so empiricamente definidos. Este tipo de abordagem pode ser demorado e pode selecionar parametros que nao descrevem a real variabilidade do escritor. A principal hipotese desse trabalho e que a variabilidade do escritor observada no dominio da imagem tambem pode ser transferido para o dominio de caracteristicas. Portanto, este trabalho propoe um novo metodo para modelar automaticamente a variabilidade do escritor para a posterior duplicacao de assinaturas no dominio de imagem (duplicator) e dominio de caracteristicas (filtro Gaussiano e variacao do metodo de Knop). Este trabalho tambem propoe um novo metodo de duplicacao de assinaturas estaticas, que gera as amostras sinteticas diretamente no dominio de caracteristicas usando um filtro Gaussiano. Alem disso, uma nova abordagem para avaliar a qualidade de amostras sinteticas no dominio de caracteristicas e apresentada. As limitacoes e vantagens de ambas as abordagens de duplicacao de assinaturas tambem sao exploradas. Alem de usar a nova abordagem para avaliar a qualidade das amostras, o desempenho de um SVAA e avaliado usando as amostras e tres bases de assinaturas estaticas bem conhecidas: a GPDS-300, a MCYT-75 e a CEDAR. Para a mais utilizada, GPDS-300, quando o classificador SVM foi treinando com somente uma assinatura genuina por escritor, ele obteve um Equal Error Rate (EER) de 5,71%. Quando o classificador tambem utilizou as amostras sinteticas geradas no dominio de imagem, o EER caiu para 1,08%. Quando o classificador foi treinado com as amostras geradas pelo filtro Gaussiano, o EER caiu para 1,04%.Abstract: Normally, in a real-world scenario, there are few signatures available to train an automatic signature verification system (ASVS). To address this issue, several offline signature duplication approaches have been proposed along the years. These approaches generate a new synthetic signature sample applying some transformations in the original signature image. Some of them generate realistic samples, specially the duplicator. This method uses a set of parameters to model the writer's behavior (writer variability) during the signing act. However, these parameters are empirically defined. This kind of approach can be time consuming and can select parameters that do not describe the real writer variability. The main hypothesis of this work is that the writer variability observed in the image space can be transferred to the feature space as well. Therefore, this work proposes a new method to automatically model the writer variability for further signature duplication in the image (duplicator) and the feature space (Gaussian filter and a variation of Knop's method). This work also proposes a new offline signature duplication method, which directly generates the synthetic samples in the feature space using a Gaussian filter. Furthermore, a new approach to assess the quality of the synthetic samples in the feature space is introduced. The limitations and advantages of both signature augmentation approaches are also explored. Despite using the new approach to assess the quality of the samples, the performance of an ASVS was assessed using them and three well-known offline signature datasets: GPDS-300, MCYT-75, and CEDAR. For the most used one, GPDS-300, when the SVM classifier was trained with only one genuine signature per writer, it achieved an Equal Error Rate (EER) of 5.71%. When the classifier also was trained with the synthetic samples generated in the image space, the EER dropped to 1.08%. When the classifier was trained using the synthetic samples generated by the Gaussian filter, the EER dropped to 1.04%

Large scale visual search

With the ever-growing amount of image data on the web, much attention has been devoted to large scale image search. It is one of the most challenging problems in computer vision for several reasons. First, it must address various appearance transformations such as changes in perspective, rotation and scale existing in the huge amount of image data. Second, it needs to minimize memory requirements and computational cost when generating image representations. Finally, it needs to construct an efficient index space and a suitable similarity measure to reduce the response time to the users. This thesis aims to provide robust image representations that are less sensitive to above mentioned appearance transformations and are suitable for large scale image retrieval. Although this thesis makes a substantial number of contributions to large scale image retrieval, we also presented additional challenges and future research based on the contributions in this thesis.China Scholarship Council (CSC)Computer Systems, Imagery and Medi

Local features for view matching across independently moving cameras.

PhD ThesisMoving platforms, such as wearable and robotic cameras, need to recognise the same place observed from different viewpoints in order to collaboratively reconstruct a 3D scene and to support augmented reality or autonomous navigation. However, matching views is challenging for independently moving cameras that directly interact with each other due to severe geometric and photometric differences, such as viewpoint, scale, and illumination changes, can considerably decrease the matching performance. This thesis proposes novel, compact, local features that can cope with with scale and viewpoint variations. We extract and describe an image patch at different scales of an image pyramid by comparing intensity values between learnt pixel pairs (binary test), and employ a cross-scale distance when matching these features. We capture, at multiple scales, the temporal changes of a 3D point, as observed in the image sequence of a camera, by tracking local binary descriptors. After validating the feature-point trajectories through 3D reconstruction, we reduce, for each scale, the sequence of binary features to a compact, fixed-length descriptor that identifies the most frequent and the most stable binary tests over time. We then propose XC-PR, a cross-camera place recognition approach that stores locally, for each uncalibrated camera, spatio-temporal descriptors, extracted at a single scale, in a tree that is selectively updated, as the camera moves. Cameras exchange descriptors selected from previous frames within an adaptive temporal window and with the highest number of local features corresponding to the descriptors. The other camera locally searches and matches the received descriptors to identify and geometrically validate a previously seen place. Experiments on different scenarios show the improved matching accuracy of the joint multi-scale extraction and temporal reduction through comparisons of different temporal reduction strategies, as well as the cross-camera matching strategy based on Bag of Binary Words, and the application to several binary descriptors. We also show that XC-PR achieves similar accuracy but faster, on average, than a baseline consisting of an incremental list of spatio-temporal descriptors. Moreover, XC-PR achieves similar accuracy of a frame-based Bag of Binary Words approach adapted to our approach, while avoiding to match features that cannot be informative, e.g. for 3D reconstruction

Document image analysis and recognition: a survey

This paper analyzes the problems of document image recognition and the existing solutions. Document recognition algorithms have been studied for quite a long time, but despite this, currently, the topic is relevant and research continues, as evidenced by a large number of associated publications and reviews. However, most of these works and reviews are devoted to individual recognition tasks. In this review, the entire set of methods, approaches, and algorithms necessary for document recognition is considered. A preliminary systematization allowed us to distinguish groups of methods for extracting information from documents of different types: single-page and multi-page, with text and handwritten contents, with a fixed template and flexible structure, and digitalized via different ways: scanning, photographing, video recording. Here, we consider methods of document recognition and analysis applied to a wide range of tasks: identification and verification of identity, due diligence, machine learning algorithms, questionnaires, and audits. The groups of methods necessary for the recognition of a single page image are examined: the classical computer vision algorithms, i.e., keypoints, local feature descriptors, Fast Hough Transforms, image binarization, and modern neural network models for document boundary detection, document classification, document structure analysis, i.e., text blocks and tables localization, extraction and recognition of the details, post-processing of recognition results. The review provides a description of publicly available experimental data packages for training and testing recognition algorithms. Methods for optimizing the performance of document image analysis and recognition methods are described.The reported study was funded by RFBR, project number 20-17-50177. The authors thank Sc. D. Vladimir L. Arlazarov (FRC CSC RAS), Pavel Bezmaternykh (FRC CSC RAS), Elena Limonova (FRC CSC RAS), Ph. D. Dmitry Polevoy (FRC CSC RAS), Daniil Tropin (LLC “Smart Engines Service”), Yuliya Chernysheva (LLC “Smart Engines Service”), Yuliya Shemyakina (LLC “Smart Engines Service”) for valuable comments and suggestions

Topological place recognition for life-long visual localization

Premio Extraordinario de Doctorado de la UAH en el año académico 2016-2017La navegación de vehículos inteligentes o robots móviles en períodos largos de tiempo ha experimentado un gran interés por parte de la comunidad investigadora en los últimos años. Los sistemas basados en cámaras se han extendido ampliamente en el pasado reciente gracias a las mejoras en sus características, precio y reducción de tamaño, añadidos a los progresos en técnicas de visión artificial. Por ello, la localización basada en visión es una aspecto clave para desarrollar una navegación autónoma robusta en situaciones a largo plazo. Teniendo en cuenta esto, la identificación de localizaciones por medio de técnicas de reconocimiento de lugar topológicas puede ser complementaria a otros enfoques como son las soluciones basadas en el Global Positioning System (GPS), o incluso suplementaria cuando la señal GPS no está disponible.El estado del arte en reconocimiento de lugar topológico ha mostrado un funcionamiento satisfactorio en el corto plazo. Sin embargo, la localización visual a largo plazo es problemática debido a los grandes cambios de apariencia que un lugar sufre como consecuencia de elementos dinámicos, la iluminación o la climatología, entre otros. El objetivo de esta tesis es enfrentarse a las dificultades de llevar a cabo una localización topológica eficiente y robusta a lo largo del tiempo. En consecuencia, se van a contribuir dos nuevos enfoques basados en reconocimiento visual de lugar para resolver los diferentes problemas asociados a una localización visual a largo plazo. Por un lado, un método de reconocimiento de lugar visual basado en descriptores binarios es propuesto. La innovación de este enfoque reside en la descripción global de secuencias de imágenes como códigos binarios, que son extraídos mediante un descriptor basado en la técnica denominada Local Difference Binary (LDB). Los descriptores son eficientemente asociados usando la distancia de Hamming y un método de búsqueda conocido como Approximate Nearest Neighbors (ANN). Además, una técnica de iluminación invariante es aplicada para mejorar el funcionamiento en condiciones luminosas cambiantes. El empleo de la descripción binaria previamente introducida proporciona una reducción de los costes computacionales y de memoria.Por otro lado, también se presenta un método de reconocimiento de lugar visual basado en deep learning, en el cual los descriptores aplicados son procesados por una Convolutional Neural Network (CNN). Este es un concepto recientemente popularizado en visión artificial que ha obtenido resultados impresionantes en problemas de clasificación de imagen. La novedad de nuestro enfoque reside en la fusión de la información de imagen de múltiples capas convolucionales a varios niveles y granularidades. Además, los datos redundantes de los descriptores basados en CNNs son comprimidos en un número reducido de bits para una localización más eficiente. El descriptor final es condensado aplicando técnicas de compresión y binarización para realizar una asociación usando de nuevo la distancia de Hamming. En términos generales, los métodos centrados en CNNs mejoran la precisión generando representaciones visuales de las localizaciones más detalladas, pero son más costosos en términos de computación.Ambos enfoques de reconocimiento de lugar visual son extensamente evaluados sobre varios datasets públicos. Estas pruebas arrojan una precisión satisfactoria en situaciones a largo plazo, como es corroborado por los resultados mostrados, que comparan nuestros métodos contra los principales algoritmos del estado del arte, mostrando mejores resultados para todos los casos.Además, también se ha analizado la aplicabilidad de nuestro reconocimiento de lugar topológico en diferentes problemas de localización. Estas aplicaciones incluyen la detección de cierres de lazo basada en los lugares reconocidos o la corrección de la deriva acumulada en odometría visual usando la información proporcionada por los cierres de lazo. Asimismo, también se consideran las aplicaciones de la detección de cambios geométricos a lo largo de las estaciones del año, que son esenciales para las actualizaciones de los mapas en sistemas de conducción autónomos centrados en una operación a largo plazo. Todas estas contribuciones son discutidas al final de la tesis, incluyendo varias conclusiones sobre el trabajo presentado y líneas de investigación futuras