The first step in the text recognition process is represented by the text line segmentation procedures. Only after text lines are correctly identified can the process proceed to the recognition of individual characters. This paper proposes a line segmentation algorithm based on the computation of an information content level, called energy, for each pixel of the image and using it to execute the seam carving procedure. The algorithm proposes the identification of text lines which follow the text more accurately with the expected downside of the computational overhead

Boiangiu, Costin Anton

Ioanitescu, Radu

Tanase, Mihai Cristian

The first step in the text recognition process is represented by the text line segmentationÂ procedures. Only after text lines are correctly identified can the process proceed to theÂ recognition of individual characters. This paper proposes a line segmentation algorithmÂ based on the computation of an information content level, called energy, for each pixel ofÂ the image and using it to execute the seam carving procedure. The algorithm proposes theÂ identification of text lines which follow the text more accurately with the expected downsideÂ of the computational overhead

Agora University Editing House: Journals

INT J COMPUT COMMUN, ISSN 1841-98369(1):8-15, February, 2014.Handwritten Documents Text Line Segmentation based onInformation EnergyC.A. Boiangiu, M.C. Tanase, R. IoanitescuCostin-Anton Boiangiu*, Radu Ioanitescu"Politehnica" University of BucharestRomania, 060042 Bucharest*Corresponding author: icostin.boiangiu@cs.pub.roMihai Cristian TanaseVirtualMetrix DesignRomania, 060104 Bucharestmihaicristian.tanase@gmail.comAbstract: The ﬁrst step in the text recognition process is represented by the textline segmentation procedures. Only after text lines are correctly identiﬁed can theprocess proceed to the recognition of individual characters. This paper proposes aline segmentation algorithm based on the computation of an information content level,called energy, for each pixel of the image and using it to execute the seam carvingprocedure. The algorithm proposes the identiﬁcation of text lines which follow thetext more accurately with the expected downside of the computational overhead.Keywords: text line segmentation, text recognition, information energy, OCR.1 IntroductionThe identiﬁcation of the boundaries of the lines of text represents an essential step in manyalgorithms like the ones for document structure extraction or text recognition. The research inthis ﬁeld has focused mostly on the development of such algorithms for printed documents. Thislimitation of the domain of application reduces the complexity of the problem as the printeddocuments are perfectly formed and the main problem that would need to be solved is the skewangle introduced in the process of printing or scanning, angle which is assumed to be the samefor the entire document,. With such documents, the problem is reduced to the identiﬁcation ofthe skew angle which is assumed to be constant for the entire page because the text lines areparallel with each other. Such methods are presented in [1]- [3].However, when dealing with handwritten documents, the assumptions made by such algo-rithms do not hold anymore. There is no constant skew angle, the lines are not parallel andeven the size and format similarity between the same characters found on diﬀerent areas of thepage cannot be assumed. Even worse, the separation between lines cannot be assumed as forprinted documents because, in handwritten tex, the characters often overlap the line bellow asthey are more compactly spaced on the vertical. Algorithms trying to address these increasedcomplexities have also been attempted in [4] - [13].The pixel of a given document stored as a digital image have diﬀerent levels of information.A white pixel in a big white region or one that is found between two lines of text containsvery little information while a character deﬁning pixel would contain a lot of information. Thispaper proposes the association of an energy level for each pixel of the input image which triesto estimate the importance or the amount of information provided by that pixel. Although theconcept can be applied to general images, printed or handwritten text, we will apply this conceptfor handwritten documents. The information level is then used by the seam carving algorithmfor the segmentation of the text lines.Copyright© 2006-2014 by CCC PublicationsHandwritten Documents Text Line Segmentation based on Information Energy 92 Related WorkThe algorithms that try to locate the text lines in a document are divided mainly by theinformation they take from the input document. As the input documents are the results of adigitization process, they are acquired, generally through scanning, as grayscale images. Thisgrayscale representation is converted into binary or black and white for algorithms which aredesigned to work with this type of documents only. The binary representation conversion is doneusing a previously deﬁned threshold level. All pixels that have gray levels above the thresholdare converted to black, the rest are converted to white. A too high threshold will results in animage containing too little information for text recognition to be possible and a too low thresholdwill result in too many artifacts, again making the text recognition process impossible.Based on the observation that the body of the lines of text contains gray pixels and becausethe pixels that make the characters being of a darker shade of gray, some algorithms computeprojection proﬁles representing the sum of all the pixels values in a given direction. The methodworks well for printed documents but fails to produce good results when applied to handwrittendocuments.To address these problems, diﬀerent approaches were used: identify the local skew of thehandwritten text, calculate the accumulated space between characters, try to ﬁll the space be-tween characters or use attraction from the text pixels and repulsion from the previously detectedline trying to estimate the text line boundaries closer.3 Information EnergyThe diﬀerent pixels from an image carry diﬀerent information content. A document can beviewed as a group of low information pixels representing the space between lines of text andrespectively a group of high information pixels representing the actual lines of text. Each pixelin the energy map has a value associated with it that represents the amount of information thatthe given pixel stores in the image. If a high energy pixel is removed from the image, the resultingimage has a signiﬁcant drop in detail, whereas removing a low energy pixel results in a negligibleinformation loss.The information energy concept can be understood by trying to eliminate a continuous bandof pixels from an image representing handwritten text. This concept is illustrated in 1.Original imageRemoving low information pixels Removing high information pixelsFigure 1: Information energy exempliﬁcation10 C.A. Boiangiu, M.C. Tanase, R. IoanitescuA band with the same amount of pixels has been removed from a text document. It can beseen that when removing high information pixels there is not enough remaining information todetect the content, while when removing low information pixels, there is hardly any informationloss and the content can be usually detected entirely.An important observation is that for handwritten documents and in general text documents,the variation of pixel values represent information in itself. It can be seen that large continuousareas with similar pixel values constitute low energy areas, in particular spaces between lines oftext, while high variation pixel values constitute high energy area. Computing the energy mapof the entire document will thus provide information of the location of the high energy areaswhich represent the lines of text.4 Accounting for the Text DirectionThe algorithm begins with the calculation of the information energy for each pixel. Similarlyto [4] and [11], for each pixel, the energy value is calculated using the next formula:E(i; j) = 2  e(i; j) +mind(neighborsNumber2+ k)  E(i+ direction; j + k)(1)where:• k fulﬁlls the following condition:  neighborsNumber2 < k < neighborsNumber2• direction represents the direction of processing the energy map +1 representing left to rightprocessing and -1 the opposite direction.Using a neighborsNumber value of 3 and because k is a natural number since we work withdiscrete pixel positions, we get only the immediate neighbors. During the calculation the directioncoeﬃcient has been kept constant. As shown in images from the chapter “Test and Results”section, for documents with horizontal lines or very low skew angles we obtained good results.However, for larger skew angles the results quality decreased with wrong text line segmentationand as a result we considered accounting for the text direction during processing.To accurately follow the text, the direction of the text lines should be taken into considerationat each pixel. For each pixel, the direction coeﬃcient that accounts for the direction of the textline when that given pixel is reached, will be calculated.The calculated information energy map is presented in 2 as a result from interpolating thetwo processing directions.Computation algorithmInput: image to be processed, processing window (height*width), skewing angleFor every pixel in the image to be processed:1. For every processing window skewing anglei. Sum up the pixels in the skewed windowii. If the sum is less than the current minimum thena) Update the current minimumb) Save the window variation2. With the minimum variation saved in step 1 ii:i. Update the direction coeﬃcient for the minimum variationii. Rescale the minimum window variationGood results were obtained experimentally with processing window sizes that were at leastthe width and height of the average character in the processed text and for values of a few orderHandwritten Documents Text Line Segmentation based on Information Energy 11Original Image Computed information energy map Figure 2: The information energy map of an imageof magnitude larger. If the processing windows has a smaller size then the algorithm attempts tosegment the characters speciﬁc structures as separate lines of text leading to erroneous results.Because the size of the processing text is usually roughly known and since the results are goodwith larger sizes for the processing window, this aspect does not represent a limitation. Similarly,the minimum and maximum variation angles can be set to 20 degrees. These variation anglesrepresent the maximum skewing angles and combined cover a 40 degrees area that is suﬃcientfor most documents.The size of the processing window is ﬁxed upfront, the calculations at each step are ﬁxed.The total number of operations depends on the direction coeﬃcient and height of the initialimage since the algorithm is repeated for every pixel. For each pixel and for each variation ofthe skewing angle, the information energy level is computed.Line identiﬁcation algorithmInput: computed information energy for each pixel in the image to be processing, neigh-borsNumberOutput: the number and layout of the identiﬁed lines1. For each line (1..h)2. For each column (1..w)3. Select the minimum cost pixel on the right not found further away than neighborsNumberi. If the selected pixel is not included in any line include itii. Else move to the next lineThe algorithm looks through all the information energy levels to locate the minimal values.The values with the minimum energy level represent the blank pixel regions that separate thelines of text.5 Tests and ResultsTo test the algorithm, a number of images of handwritten documents have been used. Thetest data ﬁles consisted of about 500 diﬀerent types of documents representing old letters, libraryindex ﬁles, patents, receipts and various printed documents. The documents showed pronouncedskew angles and their layout was not trivial. The database was considered to be relevant, althoughthe number of documents is not large, because of their variety which allowed the testing of thealgorithm on a wide set of conditions.Diﬀerent methods for the computation of the information energy are used as examples toshow how the algorithm depends on this type of variation and to show the general application12 C.A. Boiangiu, M.C. Tanase, R. Ioanitescuof the concept. The results of the algorithm are discussed in the conclusions section and futurepossible solutions are presented with an explanation of the associated computational costs.Original Image Line segmentation resultFigure 3: Gaussian ﬁrst derivative energy computation Test 1Original Image Line segmentation resultFigure 4: Gaussian ﬁrst derivative energy computation Test 2Original Image Line Segmentation result Figure 5: Magnitude of the gradient energy computation Test 1Handwritten Documents Text Line Segmentation based on Information Energy 13Original Image Line segmentation resultFigure 6: Magnitude of the gradient energy computation Test 2Original Image Line segmentation resultFigure 7: Inverse Distance Transform energy computation 1Original Image Line segmentation resultFigure 8: Inverse Distance Transform energy computation Test 214 C.A. Boiangiu, M.C. Tanase, R. Ioanitescu6 Conclusions and Future WorksThis paper describes a text line segmentation algorithm that shows good results even forhandwritten documents. The algorithm is based on the concept of information energy which isused to estimate the text lines in the processed document.Experimental results showed that using constant direction coeﬃcients when calculating theinformation energy levels produces bad results for inputs that show high levels of skew. Thealgorithm addresses this problem by updating the direction coeﬃcients with a window processingalgorithm. These direction coeﬃcients account for the local text direction and for variations of theskew angles that are not uncommon in handwritten documents. By using direction coeﬃcients,pixels from the same line have higher probability of selection when calculating the minimumvalues in the information energy map.The algorithm that computes the direction coeﬃcient has a large computation complexity.Since the computation time depends on the size of the processing window, one solution is touse a smaller size. Experimental testing showed that at minimum, the height and width of theaverage character should be used for the size of this window. Alternatively the skew angle couldbe assumed to be smaller eliminating parts of the computation cases. Another change could bethe limitation of the possible line curvature angle for the detected document lines to a lowerinterval. By constraining the line curvature angle. This limitation would also address cases inwhich a detected line would follow the space between two words and reach the previous or thenext line which would represent a wrong line detection.A possible future direction is the evaluation of the robustness of the algorithm on largerimages datasets and with a variation of the document types which would allow to more extensivelyevaluate the accuracy of the handwritten text segmentation algorithm.The work presented in this paper is a building block of a much bigger project: a complete,modular, fully automatic content conversion system developed for educational purposes. In thenear future, with the completion of the system and the running in automatic batch processingof large image databases of all kind of skewed documents (containing handwriting or not) thealgorithm will be fully evaluated in order to assess its real potential as a preprocessing phase forOCR applied on handwritten documents.AcknowledgementThe work presented in this paper was funded by the Sectorial Operational Programme Hu-man Resources Development 2007-2013 of the Romanian Ministry of Labour, Family and SocialProtection through the ﬁnancial agreement POSDRU/89/1.5/S/62557.Bibliography[1] dos Santos, R.P. et al, Text Line Segmentation Based on Morphology and Histogram Pro-jection, Document Analysis and Recognition (ICDAR), 651- 655, 2009.[2] Saha, S. et al, A Hough Transform based Technique for Text Segmentation, Journal ofComputing, 2(2):135-140, 2010.[3] Arivazhagan, M. et al, A Statistical approach to line segmentation in handwritten docu-ments, Proceedings of SPIE, 2007.[4] Strand, L. et al, Minimal Cost-Path for Path-Based Distances, Image and Signal Processingand Analysis, 379-384, 2007.Handwritten Documents Text Line Segmentation based on Information Energy 15[5] Avidan, S. et al, Seam Carving for Content-Aware Image Resizing, ACM Siggraph, article10, 2007.[6] Saabni, S. et al, Language-Independent Text Lines Extraction Using Seam Carving, Docu-ment Analysis and Recognition (ICDAR), pp. 563-568, 2001.[7] Papavassiliou, V. et al , Handwritten document image segmentation into text lines andwords, Pattern Recognition, 43(1):369-377, 2010..[8] Du, X. et al, Text Line Segmentation in Handwritten Documents Using Mumford-ShahModel, Pattern Recognition, 42(12):3136-3145, 2009.[9] T ripathy, N.; Pal, U., Handwriting segmentation of unconstrained Oriya text, Frontiers inHandwriting Recognition, 306-311. 2004.[10] Kennard, D.J., Barrett, W.A., Separating Lines of Text in Free-Form Handwritten HistoricalDocuments, Document Image Analysis for Libraries, 12-23, 2006.[11] Asi, A. et al, Text Line Segmentation for Gray Scale Historical Document Images, Proceed-ings of the 2011 Workshop on Historical Document Imaging and Processing, pp. 120-126,2011.[12] Bar-Yosef, I., Input sensitive thresholding for ancient Hebrew manuscript, Pattern Recogni-tion Letters, 26(8):1168-1173, 2005.[13] Bar-Yosef, I. et al, Line segmentation for degraded handwritten historical documents, Doc-ument Analysis and Recognition, 1161-1165, 2009.

Handwritten Documents Text Line Segmentation based on Information Energy

http://univagora.ro/jour/index.php/ijccc/article/view/160/pdf_87

Handwritten Documents Text Line Segmentation based on Information Energy

Abstract

Similar works

Full text

Available Versions

Agora University Editing House: Journals