Perceptual Recognition of Arabic Literal Amounts

Since humans are the best readers, one of the most promising trends in automatic handwriting recognition is to get inspiration from psychological reading models. The underlying idea is to derive benefits from studies of human reading, in order to build efficient automatic reading systems. In this context, we propose a human reading inspired system for the recognition of Arabic handwritten literalamounts. Our approach is based on the McClelland and Rumelhart's neural model called IAM, which is one of the most referenced psychological reading models. In this article, we have adapted IAM to suit the Arabic writing characteristics, such as the natural existence of sub-words, and the particularities of the considered literal amounts vocabulary. The core of the proposed system is a neural network classifier with local knowledge representation, structured hierarchically into three levels: perceptual structural features, sub-words and words. In contrast to the classical neural networks, localist approach is more appropriate to our problem. Indeed, it introduces a priori knowledge which leads to a precise structure of the network and avoids the black box aspect as well as the learning phase. Our experimental recognition results are interesting and confirm our expectation that adapting human reading models is a promising issue in automatic handwritten word recognition

An end-to-end, interactive Deep Learning based Annotation system for cursive and print English handwritten text

With the surging inclination towards carrying out tasks on computational devices and digital mediums, any method that converts a task that was previously carried out manually, to a digitized version, is always welcome. Irrespective of the various documentation tasks that can be done online today, there are still many applications and domains where handwritten text is inevitable, which makes the digitization of handwritten documents a very essential task. Over the past decades, there has been extensive research on offline handwritten text recognition. In the recent past, most of these attempts have shifted to Machine learning and Deep learning based approaches. In order to design more complex and deeper networks, and ensure stellar performances, it is essential to have larger quantities of annotated data. Most of the databases present for offline handwritten text recognition today, have either been manually annotated or semi automatically annotated with a lot of manual involvement. These processes are very time consuming and prone to human errors. To tackle this problem, we present an innovative, complete end-to-end pipeline, that annotates offline handwritten manuscripts written in both print and cursive English, using Deep Learning and User Interaction techniques. This novel method, which involves an architectural combination of a detection system built upon a state-of-the-art text detection model, and a custom made Deep Learning model for the recognition system, is combined with an easy-to-use interactive interface, aiming to improve the accuracy of the detection, segmentation, serialization and recognition phases, in order to ensure high quality annotated data with minimal human interaction.Comment: 17 pages, 8 figures, 2 table

Image and interpretation using artificial intelligence to read ancient Roman texts

The ink and stylus tablets discovered at the Roman Fort of Vindolanda are a unique resource for scholars of ancient history. However, the stylus tablets have proved particularly difficult to read. This paper describes a system that assists expert papyrologists in the interpretation of the Vindolanda writing tablets. A model-based approach is taken that relies on models of the written form of characters, and statistical modelling of language, to produce plausible interpretations of the documents. Fusion of the contributions from the language, character, and image feature models is achieved by utilizing the GRAVA agent architecture that uses Minimum Description Length as the basis for information fusion across semantic levels. A system is developed that reads in image data and outputs plausible interpretations of the Vindolanda tablets

Doctor of Philosophy

dissertationAdult second language (L2) learners often experience difficulty with novel L2 phonological contrasts, limiting their ability to establish contrastive lexical representations of L2 words. It has been demonstrated that the availability of orthographic input (OI), and variables interacting with OI, can shape the inferences learners make about L2 words' phonological forms. The present dissertation focuses on grapheme familiarity and congruence, in addition to L2 experience and the effect of instruction, in the case of native English speakers learning L2 Russian(-like) words presented in Cyrillic. Few studies have directly investigated effects of grapheme familiarity and congruence on phono-lexical acquisition simultaneously, systematically investigated the variables' effects on naïve and experienced L2 learners, or investigated how explicit intervention can mediate OI effects. The present dissertation addresses these gaps in our understanding. The two studies in this dissertation employed the artificial L2 lexicon paradigm. Taken together, the results indicate the following: (i) native language orthographic interference effects are robust in L2 word learning, especially when grapheme-phoneme correspondences are incongruent (unfamiliar and congruent stimuli did not cause difficulty); (ii) experience with the Russian language mediates this interference, with advanced learners performing near ceiling on all stimuli types and naïve learners performing least accurately; and (iii) naïve learners do not seem to benefit from textual enhancement and instruction prior to word learning in an experiment. The results of the present dissertation suggest that more research is needed to address the challenges associated with the interference effects of OI in L2 acquisition

A Computational Theory of Contextual Knowledge in Machine Reading

Machine recognition of off–line handwriting can be achieved by either recognising words as individual symbols (word level recognition) or by segmenting a word into parts, usually letters, and classifying those parts (letter level recognition). Whichever method is used, current handwriting recognition systems cannot overcome the inherent ambiguity in writingwithout recourse to contextual information. This thesis presents a set of experiments that use Hidden Markov Models of language to resolve ambiguity in the classification process. It goes on to describe an algorithm designed to recognise a document written by a single–author and to improve recognition by adaptingto the writing style and learning new words. Learning and adaptation is achieved by reading the document over several iterations. The algorithm is designed to incorporate contextual processing, adaptation to modify the shape of known words and learning of new words within a constrained dictionary. Adaptation occurs when a word that has previously been trained in the classifier is recognised at either the word or letter level and the word image is used to modify the classifier. Learning occurs when a new word that has not been in the training set is recognised at the letter level and is subsequently added to the classifier. Words and letters are recognised using a nearest neighbour classifier and used features based on the two–dimensional Fourier transform. By incorporating a measure of confidence based on the distribution of training points around an exemplar, adaptation and learning is constrained to only occur when a word is confidently classified. The algorithm was implemented and tested with a dictionary of 1000 words. Results show that adaptation of the letter classifier improved recognition on average by 3.9% with only 1.6% at the whole word level. Two experiments were carried out to evaluate the learning in the system. It was found that learning accounted for little improvement in the classification results and also that learning new words was prone to misclassifications being propagated

Neuropsychological Studies of Reading and Writing

This thesis investigates the reading and writing of two patients with brain injuries due to cerebro-vascular accidents. Background tests show both patients to be moderately anomic and to have severe impairments in reading and writing nonwords. Investigations of the locus of impairment in AN's nonword reading showed her to have normal orthographic analysis capabilities but impairments in converting single and multiple graphemes into phonemes and in phonemic blending. The central issue studied was the role of lexical but non-semantic processes in reading aloud, writing to dictation and copying. For this purpose a "familiar nonword" paradigm was developed in which the patients learned to read or write a small set of nonwords either with or without any associated semantics. Both AN and AM were able to learn to read nonwords to which no meanings were attached but they could still not read novel nonwords. Both patients were unable to report any meanings for the familiar nonwords when they read them and there was no evidence that learning to read them improved their sub-lexical processing abilities. These results are evidence for a direct lexical route from print to sound that is dedicated to processing whole familiar words. It was also shown with AN that if nonwords are given meanings then learning is faster than if they are not given meanings. Experiments designed to test the hypothesis that nonwords are read by analogy to words found no support for it. Both patients have severe impairments in writing novel nonwords to dictation. As they can repeat spoken nonwords after they have failed to write them, this is not due to a short-term memory impairment. Despite their nonword writing impairments, both patients were able to write to dictation the meaningless nonwords that they had previously learned to read at the first attempt, and AN did so one month after learning to read them. Neither patient however, could write novel nonwords made by reordering the letters of the familiar nonwords. Furthermore, the familiar nonwords used spellings that are of a priori low probability. The familiar nonwords must therefore have been written using lexical knowledge. Tests of semantic association showed that the familiar nonwords evoked no semantic information that the patients could report. Function words dictated to AN evoked little semantic information but she wrote them to dictation significantly better than nonwords made by reordering their letters. These results are evidence for a direct lexical route for writing to dictation. Copying was studied both with and without a five second delay between presentation and response. AN was better at delayed copying of meaningless but familiar nonwords than she was at copying novel nonwords. She was also better at delayed copying of six-letter, bi-syllabic nonwords that she had been trained to copy than she was at copying novel nonwords made by recombining the first and second halves of the familiar nonwords such that these halves retained their positions from the parent nonwords. AN was better at copying function words than nonwords made by reordering their letters. She was also better at copying function words than she was at reading or writing them to dictation. These results are evidence for a direct lexical route for copying. AN and AM were both able to write to dictation nonwords that they had never heard or written before but with which they had been made visually familiar during a visual discrimination task. They must have used lexical knowledge to do so because the spellings used were of a priori very low probability. The creation of lexical orthographic information which can be retrieved from novel auditory input raises difficulties for current models and various possible interpretations are discussed. Finally, some of the possible implications of the re-learning abilities shown by these patients, for rehabilitation procedures are discussed briefly