A hypothesize-and-verify framework for Text Recognition using Deep Recurrent Neural Networks

Deep LSTM is an ideal candidate for text recognition. However text recognition involves some initial image processing steps like segmentation of lines and words which can induce error to the recognition system. Without segmentation, learning very long range context is difficult and becomes computationally intractable. Therefore, alternative soft decisions are needed at the pre-processing level. This paper proposes a hybrid text recognizer using a deep recurrent neural network with multiple layers of abstraction and long range context along with a language model to verify the performance of the deep neural network. In this paper we construct a multi-hypotheses tree architecture with candidate segments of line sequences from different segmentation algorithms at its different branches. The deep neural network is trained on perfectly segmented data and tests each of the candidate segments, generating unicode sequences. In the verification step, these unicode sequences are validated using a sub-string match with the language model and best first search is used to find the best possible combination of alternative hypothesis from the tree structure. Thus the verification framework using language models eliminates wrong segmentation outputs and filters recognition errors

Arabic Handwritten Word Recognition based on Bernoulli Mixture HMMs

This thesis presents new approaches in off-line Arabic Handwriting Recognition based on conventional Bernoulli Hidden Markov models. Until now, the off-line handwriting recognition, in particular, the Arabic handwriting recognition is still far away form being perfect. Hidden Markov Models (HMMs) are now widely used for off-line handwriting recognition in many languages and, in particular, in Arabic. As in speech recognition, they are usually built from shared, embedded HMMs at symbol level, in which state-conditional probability density functions are modeled with Gaussian mixtures. In contrast to speech recognition, however, it is unclear which kind of features should be used and, indeed, very different features sets are in use today. Among them, we have recently proposed to simply use columns of raw, binary image pixels, which are directly fed into embedded Bernoulli (mixture) HMMs, that is, embedded HMMs in which the emission probabilities are modeled with Bernoulli mixtures. The idea is to by-pass feature extraction and ensure that no discriminative information is filtered out during feature extraction, which in some sense is integrated into the recognition model. In this thesis, we review this idea along with some extensions that are currently providing state-of-the-art results on Arabic handwritten word recognition.Alkhoury, I. (2010). Arabic Handwritten Word Recognition based on Bernoulli Mixture HMMs. http://hdl.handle.net/10251/11478Archivo delegad

The use of new technologies to access to handwritten historical information in digital form. Galeón Project

Español: La investigación histórica en archivos obliga a realizar un amplio trabajo de revisión de miles de documentos que, en muchos casos, no tienen relación con el tema de estudio, generando un importante gasto en tiempo y recursos. Para dar respuesta a este problema en relación al estudio del patrimonio arqueológico subacuático, desde el CAS-IAPH se ha ideado el Proyecto Galeón, cuyo objetivo es desarrollar soluciones innovadoras para consultar grandes conjuntos digitalizados de documentos históricos manuscritos. Actualmente no es posible la transcripción automatizada de un gran volumen de imágenes de documentos manuscritos, pero el desarrollo tecnológico en el campo del reconocimiento formal de palabras, puede simplificar este proceso. Para ello se ha ideado un modelo teórico de Búsqueda de Palabras Claves (BPC) basado en Grafos de Palabras (GP), que, además de para el patrimonio cultural marítimo, podría utilizarse para otros temas de investigación. Inglés: Historical research in archives forces to realize an extensive work of reviewing thousands of documents that, in many cases, have no connection with the subject matter, generating a significant expenditure of time and resources. To address this problem in relation to the study of underwater archaeological heritage, from the CAS-IAPH has been devised the Galleon Project, which aims to develop innovative solutions to query large sets of historical documents digitized manuscripts. Nowadays It is not possible the automated transcription of a large volume of images from handwritten documents, but the development in the field of formal recognition of words, can simplify this process. For this we have developed a theoretical model to identify Keywords based on Graphs of Words (GP), which, as well as in the maritime cultural heritage, could be used for any research topic

Arabic Printed Word Recognition Using Windowed Bernoulli HMMs

[EN] Hidden Markov Models (HMMs) are now widely used for off-line text recognition in many languages and, in particular, Arabic. In previous work, we proposed to directly use columns of raw, binary image pixels, which are directly fed into embedded Bernoulli (mixture) HMMs, that is, embedded HMMs in which the emission probabilities are modeled with Bernoulli mixtures. The idea was to by-pass feature extraction and to ensure that no discriminative information is filtered out during feature extraction, which in some sense is integrated into the recognition model. More recently, we extended the column bit vectors by means of a sliding window of adequate width to better capture image context at each horizontal position of the word image. However, these models might have limited capability to properly model vertical image distortions. In this paper, we have considered three methods of window repositioning after window extraction to overcome this limitation. Each sliding window is translated (repositioned) to align its center to the center of mass. Using this approach, state-of-art results are reported on the Arabic Printed Text Recognition (APTI) database.The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no 287755. Also supported by the Spanish Government (Plan E, iTrans2 TIN2009-14511 and AECID 2011/2012 grant).Alkhoury, I.; Giménez Pastor, A.; Juan Císcar, A.; Andrés Ferrer, J. (2013). Arabic Printed Word Recognition Using Windowed Bernoulli HMMs. Lecture Notes in Computer Science. 8156:330-339. https://doi.org/10.1007/978-3-642-41181-6_34S3303398156Dehghan, M., et al.: Handwritten Farsi (Arabic) word recognition: a holistic approach using discrete HMM. Pattern Recognition 34(5), 1057–1065 (2001), http://www.sciencedirect.com/science/article/pii/S0031320300000510Giménez, A., Juan, A.: Embedded Bernoulli Mixture HMMs for Handwritten Word Recognition. In: ICDAR 2009, Barcelona, Spain, pp. 896–900 (July 2009)Giménez, A., Khoury, I., Juan, A.: Windowed Bernoulli Mixture HMMs for Arabic Handwritten Word Recognition. In: ICFHR 2010, Kolkata, India, pp. 533–538 (November 2010)Grosicki, E., El Abed, H.: ICDAR 2009 Handwriting Recognition Competition. In: ICDAR 2009, Barcelona, Spain, pp. 1398–1402 (July 2009)Günter, S., et al.: HMM-based handwritten word recognition: on the optimization of the number of states, training iterations and Gaussian components. Pattern Recognition 37, 2069–2079 (2004)Märgner, V., El Abed, H.: ICDAR 2007 - Arabic Handwriting Recognition Competition. In: ICDAR 2007, Curitiba, Brazil, pp. 1274–1278 (September 2007)Märgner, V., El Abed, H.: ICDAR 2009 Arabic Handwriting Recognition Competition. In: ICDAR 2009, Barcelona, Spain, pp. 1383–1387 (July 2009)Pechwitz, M., et al.: IFN/ENIT - database of handwritten Arabic words. In: CIFED 2002, Hammamet, Tunis, pp. 21–23 (October 2002)Rabiner, L., Juang, B.: Fundamentals of speech recognition. Prentice-Hall (1993)Slimane, F., et al.: A new arabic printed text image database and evaluation protocols. In: ICDAR 2009, pp. 946–950 (July 2009)Slimane, F., et al.: ICDAR 2011 - arabic recognition competition: Multi-font multi-size digitally represented text. In: ICDAR 2011 - Arabic Recognition Competition, pp. 1449–1453. IEEE (September 2011)Young, S.: et al.: The HTK Book. Cambridge University Engineering Department (1995

Dynamic signatures: A review of dynamic feature variation and forensic methodology

This article focuses on dynamic signatures and their features. It provides a detailed and critical review of dynamic feature variations and circumstantial parameters affecting dynamic signatures. The state of the art summarizes available knowledge, meant to assist the forensic practitioner in cases presenting extraordinary writing conditions. The studied parameters include hardware-related issues, aging and the influence of time, as well as physical and mental states of the writer. Some parameters, such as drug and alcohol abuse or medication, have very strong effects on handwriting and signature dynamics. Other conditions such as the writer’s posture and fatigue have been found to affect feature variation less severely. The need for further research about the influence of these parameters, as well as handwriting dynamics in general is highlighted. These factors are relevant to the examiner in the assessment of the probative value of the reported features. Additionally, methodology for forensic examination of dynamic signatures is discussed. Available methodology and procedures are reviewed, while pointing out major technical and methodological advances in the field of forensic handwriting examination. The need for sharing the best practice manuals, standard operating procedures and methodologies to favor further progress is accentuated

Combinaison de données hétérogènes pour la reconnaissance d'images de documents

Ce manuscrit est une synthèse de mes travaux de recherche depuis 2008, au sein de l’équipe Intuidoc de l’Irisa. Ces travaux portent sur l’analyse automatique d’images de documents numérisés, et plus particulièrement la reconnaissance de la structure de documents. S’il existe des OCR du commerce de bonne qualité, la reconnaissance d’images de documents reste un problème ouvert pour les documents anciens, abîmés, à structure complexe, ou avec une forte interaction entre écriture manuscrite et texte impriméMes travaux se sont focalisés autour de la méthode DMOS. Il s’agit d’une méthode à base de règles grammaticales, permettant une description physique, syntaxique et sémantique des documents à reconnaître. En particulier, nous avons mis au point le mécanisme de calque perceptif, qui permet une combinaison de données hétérogènes, guidée par une description grammaticale des contenus.Les données hétérogènes peuvent être des primitives basiques extraites de l’image (segments, composantes connexes), des objets construits (lignes de texte, alignements), des résultats de classifieurs, des données issues d’interactions avec l’utilisateur, des résultats de systèmes de reconnaissance d’écriture, et plus récemment des résultats produits par des systèmes à base d’apprentissage profond. Grâce à l’utilisation des calques perceptifs, ces données sont combinées simplement en suivant des règles symboliques décrivant l’organisation physique et logique des documents. Dans ce manuscrit, nous présentons l’intérêt de la combinaison de données hétérogènes sur de nombreuses problématiques concrètes. Nous abordons les tâches de segmentation physique des pages : la localisation de lignes de texte et la segmentation en paragraphes. Nous étudions l’analyse de documents à structure complexe comme la presse ancienne, le corpus hétérogène Maurdor, des diagrammes de type flowchart, ou des documents tabulaires d’histoire de la finance. Nous présentons également des résultats sur des formulaires pré-imprimés. Enfin, nous abordons des perspectives de recherches autour de l’apprentissage avec peu de données, l’utilisation de réseaux de neurones profonds, ainsi que des interactions avec le domaine du traitement automatique de la langue

Multimodal Interactive Transcription of Handwritten Text Images

En esta tesis se presenta un nuevo marco interactivo y multimodal para la transcripción de Documentos manuscritos. Esta aproximación, lejos de proporcionar la transcripción completa pretende asistir al experto en la dura tarea de transcribir. Hasta la fecha, los sistemas de reconocimiento de texto manuscrito disponibles no proporcionan transcripciones aceptables por los usuarios y, generalmente, se requiere la intervención del humano para corregir las transcripciones obtenidas. Estos sistemas han demostrado ser realmente útiles en aplicaciones restringidas y con vocabularios limitados (como es el caso del reconocimiento de direcciones postales o de cantidades numéricas en cheques bancarios), consiguiendo en este tipo de tareas resultados aceptables. Sin embargo, cuando se trabaja con documentos manuscritos sin ningún tipo de restricción (como documentos manuscritos antiguos o texto espontáneo), la tecnología actual solo consigue resultados inaceptables. El escenario interactivo estudiado en esta tesis permite una solución más efectiva. En este escenario, el sistema de reconocimiento y el usuario cooperan para generar la transcripción final de la imagen de texto. El sistema utiliza la imagen de texto y una parte de la transcripción previamente validada (prefijo) para proponer una posible continuación. Despues, el usuario encuentra y corrige el siguente error producido por el sistema, generando así un nuevo prefijo mas largo. Este nuevo prefijo, es utilizado por el sistema para sugerir una nueva hipótesis. La tecnología utilizada se basa en modelos ocultos de Markov y n-gramas. Estos modelos son utilizados aquí de la misma manera que en el reconocimiento automático del habla. Algunas modificaciones en la definición convencional de los n-gramas han sido necesarias para tener en cuenta la retroalimentación del usuario en este sistema.Romero Gómez, V. (2010). Multimodal Interactive Transcription of Handwritten Text Images [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/8541Palanci

Reliable pattern recognition system with novel semi-supervised learning approach

Over the past decade, there has been considerable progress in the design of statistical machine learning strategies, including Semi-Supervised Learning (SSL) approaches. However, researchers still have difficulties in applying most of these learning strategies when two or more classes overlap, and/or when each class has a bimodal/multimodal distribution. In this thesis, an efficient, robust, and reliable recognition system with a novel SSL scheme has been developed to overcome overlapping problems between two classes and bimodal distribution within each class. This system was based on the nature of category learning and recognition to enhance the system's performance in relevant applications. In the training procedure, besides the supervised learning strategy, the unsupervised learning approach was applied to retrieve the "extra information" that could not be obtained from the images themselves. This approach was very helpful for the classification between two confusing classes. In this SSL scheme, both the training data and the test data were utilized in the final classification. In this thesis, the design of a promising supervised learning model with advanced state-of-the-art technologies is firstly presented, and a novel rejection measurement for verification of rejected samples, namely Linear Discriminant Analysis Measurement (LDAM), is defined. Experiments on CENPARMI's Hindu-Arabic Handwritten Numeral Database, CENPARMI's Numerals Database, and NIST's Numerals Database were conducted in order to evaluate the efficiency of LDAM. Moreover, multiple verification modules, including a Writing Style Verification (WSV) module, have been developed according to four newly defined error categories. The error categorization was based on the different costs of misclassification. The WSV module has been developed by the unsupervised learning approach to automatically retrieve the person's writing styles so that the rejected samples can be classified and verified accordingly. As a result, errors on CENPARMI's Hindu-Arabic Handwritten Numeral Database (24,784 training samples, 6,199 testing samples) were reduced drastically from 397 to 59, and the final recognition rate of this HAHNR reached 99.05%, a significantly higher rate compared to other experiments on the same database. When the rejection option was applied on this database, the recognition rate, error rate, and reliability were 97.89%, 0.63%, and 99.28%, respectivel

Drawing, Handwriting Processing Analysis: New Advances and Challenges

International audienceDrawing and handwriting are communicational skills that are fundamental in geopolitical, ideological and technological evolutions of all time. drawingand handwriting are still useful in defining innovative applications in numerous fields. In this regard, researchers have to solve new problems like those related to the manner in which drawing and handwriting become an efficient way to command various connected objects; or to validate graphomotor skills as evident and objective sources of data useful in the study of human beings, their capabilities and their limits from birth to decline