The notebooks in the Bleek and Lloyd collection contain handwritten stories that metaphorically encode the Bushman culture and are useful to researchers and scholars trying to understand Bushman language and culture. These notebooks, however, only exist as scanned images and therefore the stories they contain cannot be searched, indexed or compared. This research seeks to investigate how accurately the Bushman stories can be automatically converted from images to text, in a process known as transcription, and also to explore the various techniques for doing this. The expected contribution is a measurement of how accurately transcription can be automatically performed as well as a comparison of different techniques for doing this

Suleman, A/Prof Hussein

Williams, Mr Kyle

UCT Computer Science Research Document Archive

Learning to Read BushmanKyle WilliamsDepartment of Computer ScienceUniversity of Cape TownPrivate Bag X3, Rondebosch, 7701South Africakwilliams@cs.uct.ac.zaHussein SulemanDepartment of Computer ScienceUniversity of Cape TownPrivate Bag X3, Rondebosch, 7701South Africahussein@cs.uct.ac.zaABSTRACTThe notebooks in the Bleek and Lloyd collection con-tain handwritten stories that metaphorically encode theBushman culture and are useful to researchers and schol-ars trying to understand Bushman language and culture.These notebooks, however, only exist as scanned imagesand therefore the stories they contain cannot be searched,indexed or compared. This research seeks to investigatehow accurately the Bushman stories can be automaticallyconverted from images to text, in a process known as tran-scription, and also to explore the various techniques fordoing this. The expected contribution is a measurementof how accurately transcription can be automatically per-formed as well as a comparison of different techniques fordoing this.Categories and Subject DescriptorsI.2.6 [Artificial Intelligence]: Learning; I.2.7 [ArtificialIntelligence]: Natural Language Processing; I.7.5 [Doc-ument and Text Processing]: Document CaptureGeneral TermsAlgorithms, Experimentation, PerformanceKeywordsOCR, machine learning, handwritten manuscripts, cul-tural heritage preservation, digital libraries1. INTRODUCTIONThe notebooks in the Bleek and Lloyd Collection [4]are available to researchers and scholars as digital scans.While these digital scans are useful for learning aboutBushman language and culture, they do not allow for theautomatic indexing, searching and comparison of the textthat appears in them. Transcription will allow these andother operations to be performed. However, manual tran-scription is a tedious and impractical task due to the sizeof the collection. This research deals with the automationof this task.Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.SAICSIT ’10, October 11-13, Bela Bela, South AfricaCopyright 2010 ACM 978-1-60558-950-3/10/10 ...$10.00.Figure 1: Diacritics above, below and spanningmultiple characters2. OBJECTIVESHandwriting transcription is a well studied problem [1].However, for the Bushman languages, it is a difficult taskdue to the diacritics that appear above and below char-acters and that span multiple characters (Figure 1). Thelack of a language model also contributes to the difficultyof the task. The diacritics complicate the recognition andsegmentation processes, while the lack of a language modeldoes not allow for the incorporation of knowledge aboutthe language in order to improve results. The objectivesof this research are to determine how accurately hand-written Bushman texts can be automatically transcribedand which of the techniques used in the literature aremost effective for doing this. In this sense, the objectivesinclude determining which combinations of segmentationtechniques, features and machine learning algorithms per-form best, as well as determining the extent to which alanguage model can be built and used to improve results.3. METHODOLOGY3.1 Research QuestionsAn experimental methodology will be used in this re-search in order to answer the following main research ques-tion and sub-research questions.• How accurately can the automatic transcription ofhandwritten Bushman texts be performed?– Which of a selection of Hidden Markov Mod-els, Neural Networks and Support Vector Ma-chines, when used in conjunction with variousfeature sets, performs best when automaticallytranscribing handwritten Bushman texts?– Which segmentation techniques are effective forthe machine learning algorithms used in thisresearch?– To what extent can an n-gram language modelimprove accuracy when automatically transcrib-ing handwritten Bushman texts?The main research question seeks to investigate the gen-eral overall accuracy with which the automatic transcrip-tion of handwritten Bushman texts can be performed,1while the sub-research questions investigate the variousways to go about doing it.3.2 Development and EvaluationTo investigate which machine learning algorithms andfeature sets perform best, Hidden Markov Models, Neu-ral Networks and Support Vector Machines will be builtmaking use of various feature sets. These machine learningalgorithms and feature sets will then be evaluated by di-rectly comparing transcription accuracy, word error ratesand other statistical measures for differences among com-binations.Various segmentation techniques will be employed inorder to to investigate which segmentation techniques arethe most effective for the different machine learning algo-rithms used in this study. These segmentation techniqueswill be evaluated based on the effect that they have ontranscription accuracy, word error rates and other statis-tical measures for differences among combinations.An n-gram language model will be built and incorpo-rated into the system. Doing this will allow for an eval-uation to be performed to determine the extent to whichan n-gram language model can be used to improve accu-racy when automatically transcribing handwritten Bush-man texts. This improvement can be directly measuredby comparing transcription accuracy and word error rateswith and without the n-gram language model.The main research question concerning the accuracywith which handwritten Bushman texts can be automat-ically transcribed can be determined based on the eval-uation above. Furthermore, additional end-to-end evalu-ation will be conducted to gain insight into the tradeoffbetween accuracy and performance and other importantconsiderations in the transcription process.Evaluation will be performed using K -fold cross-validationon a representative set of the collection. The data in therepresentative set will be decided on in discussion withexperts from the Michaelis School of Fine Art.3.3 Pilot StudyA pilot study was conducted to investigate the feasi-bility of the automatic transcription of Bushman charac-ters [5]. In the pilot study, the automatic transcriptionof a limited set of Bushman characters was conducted fora neatly rewritten version of a Bushman story. In thepilot study an accuracy rate of approximately 80% wasachieved, when using a training set created by two au-thors, under-sampled bitmaps [3] as features and a sup-port vector machine [2] for classification and recognition.The pilot study provided useful insight into the problemof automatically transcribing handwritten Bushman texts,however, it over-simplified many of the problems associ-ated with automatically transcribing handwritten Bush-man texts.4. EXPECTED OUTCOMESThe main expected outcome of this research is a mea-surement of the accuracy with which automatic transcrip-tion of Bushman texts can be performed. Another ex-pected outcome is a comparison of a selection of HiddenMarkov Models, Neural Networks and Support Vector Ma-chines used to classify and recognise Bushman languagesand insight into which feature sets are best suited to hand-written characters with diacritics. Furthermore, insightwill be gained into effective segmentation approaches forthe different machine learning algorithms considered.A software system will be produced that is capable oftranscribing handwritten Bushman texts using a varietyof features and the three machine learning algorithms pre-viously mentioned. Additionally, a transcription of someof the Bleek and Lloyd notebooks is expected, with a highlevel of accuracy.Success will largely be judged based on the research pro-viding a strong and convincing comparison of the variousfeature sets and machine learning algorithms considered.5. PRACTICAL IMPLICATIONSIt is expected that this research could have a numberof practical implications, such as being used in text-to-speech applications, reprinting of the stories that appearin the Bleek and Lloyd Collection and allowing for auto-matic translation of the texts using the Bleek and Lloyddictionaries [6]. Furthermore, it is expected it will allowresearchers and scholar to gain insight into Bushman cul-ture and enhance their understanding of some of the ear-liest inhabitants of the Earth. Lastly, it is expected thatthis research will have practical implications for other re-searchers in the field, especially those working with textsthat contain complex diacritics.6. CONCLUSIONSThe aims of this research are to investigate the besttechniques for automatically transcribing handwritten Bush-man texts. The various techniques used in the literaturewill be compared to determine which are most effectivefor Bushman texts and which combinations of techniqueswork best together. It is expected that the findings fromthis research will not only be applicable to the Bushmantexts which appear in the Bleek and Lloyd collection, butalso other texts which contain complex diacritics.7. ACKNOWLEDGMENTSThis research is supported by the University of CapeTown, the Telkom/NSN/Telesciences/THRIP Center ofExcellence and the National Research Foundation.8. REFERENCES[1] H. Bunke. Recognition of cursive roman handwriting:past, present and future. In Document Analysis andRecognition, pages 448 – 459 vol.1, 2003.[2] C. Cortes and V. Vapnik. Support-vector networks.In Machine Learning, pages 273–297, 1995.[3] A. Oliveira, C. Mello, E. Silva Jr, and V. Alves.Optical digit recognition for images of handwrittenhistorical documents. In Neural Networks, 2006.SBRN ’06. Ninth Brazilian Symposium on, pages 166–171, oct. 2006.[4] H. Suleman. Digital libraries without databases: Thebleek and lloyd collection. In 11th EuropeanConference on Research and Advanced Technology forDigital Libraries, pages 392–403, March 2007.[5] K. Williams. Feasibility of automatic transcription ofneatly rewritten bushman texts. Technical report,Department of Computer Science, University of CapeTown, 2010. Technical Report CS10-06-00.[6] K. Williams, S. Manilal, L. Molwantoa, andH. Suleman. A visual dictionary for an extinctlanguage. In G. Chowdhury, C. Koo, and J. Hunter,editors, The Role of Digital Libraries in a Time ofGlobal Change, volume 6102 of Lecture Notes inComputer Science, pages 1–4. Springer, 2010.2

Learning to Read Bushman

http://pubs.cs.uct.ac.za/archive/00000683/01/kwilliams_saicsit2010_pgsymposium.pdf

Learning to Read Bushman

Abstract

Similar works

Full text

Available Versions

UCT Computer Science Research Document Archive