In this paper, we present an efficient segmentation technique that extracts piecewise linear patterns from hand-drawn maps. The user is only required to place the starting and end points and the method is capable of extracting the route that connects the two, which closely colocates with the hand-drawn map. It provides an effective approach to interactively process and understand those historical maps. The proposed method employs supervised learning to evaluate at every pixel location the probability that such a lineage pattern exists, followed by shortest path segmentation to extract the border of interest

Matthew Stevens

Xianghua Xie

Cronfa at Swansea University

 Cronfa -  Swansea University Open Access Repository   _____________________________________________________________   This is an author produced version of a paper published in:Image Analysis and Recognition                                              Cronfa URL for this paper:http://cronfa.swan.ac.uk/Record/cronfa49176_____________________________________________________________ Conference contribution :Essa, E., Xie, X., Turner, R., Stevens, M. & Power, D. (2016).  Extracting Lineage Information from Hand-DrawnAncient Maps. Image Analysis and Recognition, (pp. 268-275). Springer International Publishing.http://dx.doi.org/10.1007/978-3-319-41501-7_30       _____________________________________________________________  This item is brought to you by Swansea University. Any person downloading material is agreeing to abide by the termsof the repository licence. Copies of full text items may be used or reproduced in any format or medium, without priorpermission for personal research or study, educational or non-commercial purposes only. The copyright for any workremains with the original author unless otherwise specified. The full-text must not be sold in any format or mediumwithout the formal permission of the copyright holder. Permission for multiple reproductions should be obtained from the original author. Authors are personally responsible for adhering to copyright and publisher restrictions when uploading content to therepository. http://www.swansea.ac.uk/library/researchsupport/ris-support/  Extracting Lineage Information FromHand-Drawn Ancient MapsEhab Essa1, Xianghua Xie2, Richard Turner3, Matthew Stevens3, DanielPower31Department of Computer Science, Mansoura University, Egypt2Department of Computer Science &3Department of History and Classics,Swansea University, UKhttp://csvision.swan.ac.ukAbstract. In this paper, we present an efficient segmentation techniquethat extracts piecewise linear patterns from hand-drawn maps. The useris only required to place the starting and end points and the methodis capable of extracting the route that connects the two, which closelycolocates with the hand-drawn map. It provides an effective approach tointeractively process and understand those historical maps. The proposedmethod employs supervised learning to evaluate at every pixel locationthe probability that such a lineage pattern exists, followed by shortestpath segmentation to extract the border of interest.1 IntroductionThis work is concerned with processing ancient hand-drawn Welsh Marcher lord-ships maps to extract piecewise lineage patterns that corresponds to lordshipboundaries. This system of landownership was established by the Norman kingsafter their conquest of England in 1066 to protect their borders with Wales. Itreached its full extent after the English conquest of Wales in 1282. The basisfor any historical or geographical study of the Welsh Marches relies on WilliamRees’ four map sheets of South Wales and the Border in the Fourteenth Century(Ordnance Survey, 1933). The challenge is to use these maps as the basis fora geographical information systems, in which the extent and the ownership ofeach Marcher lordship can be depicted throughout their individual histories.Manual segmentation of the historical hand-drawn maps is a time-consuming,tedious process, particularly when dealing with large maps. The maps are richlyannotated and textured, e.g. various map keys, texts and colouring, see Fig. 1 foran example. The delineation and separation of individual elements play an im-portant role in automated analysis and understanding of these maps. However,automated segmentation of these maps is a non-trivial task. Image segmenta-tion can be largely classified into automatic and user-aided or semi-automaticapproaches. In automatic segmentation methods, the segmented boundaries arefound without any user interaction. These methods are typically working byexploiting visual cues from the image, such as colour, texture, edge, and/or(a) (b)Fig. 1. An example of the south Wales lordship map. (a) original image. (b) finalborder of interest highlighted by blue. The two red dots are placed by the user and thepath of interest is automated extracted.statistical distributions of those features as in [10, 3, 14, 16, 12, 13]. However, inmany cases they require high level knowledge to constrain the segmentation inorder to cope with ambiguities in appearance, texture and shape. Semi-automaticsegmentation methods introduce user interaction as an effective and efficient ap-proach to transfer domain knowledge into the segmentation process. User canfor example determine the region of interest, initialise the contour location [15,17], place some seed points on the object boundary [8, 9] to guide the segmen-tation, or draw strokes inside and/or outside object of interest [1]. Generalisingpriors using learning techniques have been shown effective in many segmentationapplications, e.g. [6].In this paper, a semi-automatic segmentation method, with minimal userinteraction, for extracting lineage patterns from ancient maps is proposed. Theproposed method combines a machine learning method with a shortest pathapproach to delineate the boundaries of domains of marcher lords. Randomforest classifier is used to generate a probability map of the potential locationof the boundaries. A shortest path method based on the Dijkstra’s algorithmis used to refine the segmentation between two seed points identified by theuser. The proposed method is tested on the south Wales lordship maps in thefourteenth century as shown in Figure 1 (a) and achieves a promising results asin Figure 1 (b).2 Proposed MethodBriefly, the proposed method first caries out pre-processing and feature extrac-tion using histograms of orientated gradients (HoG) and histograms of intensi-ties to highlight meaningful features for the boundary of interest. Next, randomforest classifier is used to generate a probability map for the boundaries andsuppress the background noise. The segmentation is then refined by finding theshortest path between user input seed points to obtain the final border.2.1 Pre-processing and Feature ExtractionThe images contains various features that are of no interest to us for extractingthose lordship boundaries. The borders of interest are piecewise linear and havereasonably good contrast. We thus devise a simple pre-processing step to reducecertain amount of background ambiguities, while maintaining as much as possi-ble for the boundaries. We use a thresholded map to enhance the original imageand to produce a greyscale image for further processing. The map images arefirst binarised in order to reduce the interference from other symbols and topog-raphy details with the boundaries. The binary image is obtained by applying athresholding on the three colour channels of the image (red, green and blue).Since the value of each colour channel varies between 0 to 255, we retrieve theindex of image pixels between 70 and 170 of each channel and combine the resultto produce the binary map. Figure 2 shows an example of such binary image onthe left and the equivalent greyscale image on the right. The greyscale image isobtained by multiplying the binary image with the original image and used toextract the histogram of image intensity features which is a set of pixel greyscalevalue quantised into a finite number of bins.The boundaries of interest are made of short line segments, i.e. piecewiselinear. We employ histogram of oriented gradients (HoG) as the second set ofvisual features to represent and capture those patterns. HoG is one of the mostsuccessful image features in object detection e.g. human detection [4]. It mea-sures the distribution of edge directions in local segments of an image. The HoGdescriptor is achieved by dividing the image into non-overlapping small regionsknown as cells, to compute the histogram of gradient orientations for each re-gion. The concatenation of these histograms are then produced to represent theimage features. Each local histogram is also normalised within a larger region(with overlapping) to enhance the invariance to contrast and illumination vari-ations. Here, to compute the direction of edges, image gradient is computed byconvolving the image with derivative filters. Each pixel within the cell votes forone of the 9 histogram bins. The contrast normalisation is carried out based onL2-norm.2.2 Boundary EvaluationNext, we train a supervised classifier to evaluate every each pixel location for thepossibility of containing one of those lineage patterns. Various machine learning(a) (b)Fig. 2. Image pre-processing. (a) binary image. (b) greyscale image generated fromthe binary image and the original image. Feature extraction is carried out on the newgreyscale image.methods may be considered. Random Forest (RF) [2] is an ensemble-based classi-fier that combines a collection of decision trees to reduce the effect of overfittingand alleviate outlier problems to improve accuracy. RF is similar to baggingmethod in that it randomly samples training examples to train each decisiontree separately. In addition, RF chooses a random subset of features attributesat each node to find the optimal tree splitting. In this work, we classify imagepixels into two classes: boundary of the marcher lordship domains, and non-boundary.In RF classification, a set of tress r ∈ {1, · · · , R} is trained separately on sub-sampling of the training examples. The testing of unseen parts of the map datais accomplished by introducing the unseen sample t to each decision tree startingfrom its root until it reaches the corresponding leaves and finally combines thepredication result of each tree by computing the average predication of each classc:P (c|v) = 1RR∑r=1pr(c|t). (1)Here, the number of trees R of the RF classifier sets to R = 500 and thenumber of attributes f randomly selected at each split is defined as f =√l,where l is the feature vector length.The RF is trained on two features as aforementioned in Section 2.1, thehistogram of the image intensity and HoG. These features extracted from ascanning window of size 128×128. An example binary classification result of the(a) (b)Fig. 3. The output of the RF classifier. (a) a binary classification result (highlightedby blue) superimposed on the original map. (b) the corresponding probabilistic mapshowing the possibility of each pixel in the map being considered as a pixel on theboundary of interest. The brighter the pixel the higher its probability.RF is shown in Figure 3 (a). The classifier can effectively highlight the borderof interest. However, there are still some symbols that are misclassified, e.g.handwritten characters, which is inevitable.2.3 Refinement Process based on Shortest PathGiven that the boundaries of interest only contains a very small percentage ofeach entire map, eliminating the false positives while retaining high true positiverate is non-trivial. and may not be entirely plausible. In order to solve this prob-lem, we proposed to use a shortest path algorithm based on Dijkstra’s algorithm[5]. The proposed method finds the optimal path with minimum cost betweentwo user points to refine the segmentation result. The cost function is definedusing the probability values produced by the RF classifier. An example of the RFprobability map is displayed in Figure 3 (b), where the brightness correspondsto the probability of the local region to be on the boundaries (i.e. bright whiterefers to high probability).Shortest path is a combinatorial optimisation problem that working by find-ing the less costly solution of the problem from a finite set of all feasible solutions.The image is discretised to a graph G(V,E), where V is a set of vertices repre-sent image pixels, and E is the set of edges. These edges are connecting verticesand also, carrying a weight value that reflects the cost of using this path.There are several methods available using the shortest path algorithm, e.g.Intelligent Scissors [11] and Live Wire [7] which segment the image on the flyby following the user clicks. The costs of edges are usually defined based on thegradient magnitudes and orientation. However these methods are sensitive tobackground noise.In this work, the Dijkstra’s algorithm is used to find the path between twovertices in a graph where the total associated edge costs are minimised. Dijkstra’salgorithm is a single-source algorithm that locates the shortest path from thesource vertex to all other vertices in the graph with non-negative edge weights.The algorithm creates a shortest path tree starting from the source vertex anditeratively adding the vertex that has the lowest distance to the source. Eachvertex is exploring its unvisited vertex neighbours and updating its assign dis-tance value. As long as the algorithm does not reach the destination vertex, itkeeps picking up unvisited vertex with the lowest distance value to be the newcurrent vertex and it repeat these steps. The cost of each edge is defined as thenegative log of the RF’s probability value. Figure 1 (b) shows the result of usingthe shortest path algorithm, cf. Figure 3. This approach allows user to efficientlyextract boundary of interest with just placing two points at a time.3 ResultsHere, we briefly summarise the evaluation that took place and the results we haveachieved. The proposed method was tested on the south Wales marcher lordshipmap in the fourteenth century. The dimension of the image is 24401 × 16610with resolution 600 dpi and bit depth 24. The map was recreated by WilliamRees and published by the Ordnance Survey in 1932. Due to the large size ofthe map, it is segmented to relatively small blocks for testing. The training ofthe RF classifier carries out on two blocks of the map of size 2000× 2000.Figure 4 shows some examples of the segmentation result. The boundariesof the domain marcher lord are drawn as a dash line. As illustrated in Figure4 (b), the binary image removes some of the unrelated details that have thecolour distribution differ from that of those boundaries. RF probability mapsand the final border segmentation superimposed on the original maps are shownin columns (c) and (d). The proposed method successfully highlights the bound-aries of interest. A full quantitative evaluation requires manual annotation ofall the borders which is very time consuming. However, the qualitative resultsdemonstrate the potential of the method to be applied to effectively extract lin-eages from those historic maps. This method serves as an important step in fullydissecting and tokenising those maps.4 Conclusions and Future WorkWe presented a semi-automatic segmentation method for Welsh marcher lord-ships map in order to delineate the boundaries of the marcher lord domains. Theproposed method extracts local features based HoG to train an RF classifier inorder to produce a probability map of the boundary location. The segmentationis refined by using a shortest path algorithm based on Dijkstra’s method to high-light the final border. The experimental results show promising performance ofthe proposed method. In the future work, we will extend the method to segmentthe map keys such as chief and lesser castles, and the other types of features.References1. Boykov, Y., Funka-Lea, G.: Graph cuts and efficient N-D image segmentation.IJCV 70(1), 109–131 (2006)2. Breiman, L.: Random forests. Machine Learning 5(1), 5–32 (2001)3. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space anal-ysis. IEEE T-PAMI 24(5), 603–619 (2002)4. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In:CVPR. vol. 1, pp. 886–893 (June 2005)5. Dijkstra, E.: A note on two problems in connexion with graphs. Numerische Math-ematik 1(1), 269–271 (1959)6. Essa, E., Xie, X., Sazonov, I., Nithiarasu, P., Smith, D.: Shape prior model formedia-adventitia border segmentation in ivus using graph cut. In: MCV-MICCAI,pp. 114–123 (2013)7. Falca˜o, A.X., et al.: User-steered image segmentation paradigms: Live wire andlive lane. Graph. Models Image Process. 60(4), 233–260 (1998)8. Jones, J.L., Essa, E., Xie, X., Smith, D.: Interactive segmentation of media-adventitia border in ivus. In: CAIP, pp. 466–474 (2013)9. Jones, J.L., Xie, X., Essa, E.: Combining region-based and imprecise boundary-based cues for interactive medical image segmentation. IJNMBE 30(12), 1649–1666(2014)10. Malik, J., Belongie, S., Leung, T., Shi, J.: Contour and texture analysis for imagesegmentation. IJCV 43(1), 7–27 (2001)11. Mortensen, E.N., Barrett, W.A.: Interactive segmentation with intelligent scissors.Graphical Models and Image Processing 60(5), 349–384 (1998)12. Rotem, O., Greenspan, H., Goldberger, J.: Combining region and edge cues forimage segmentation in a probabilistic gaussian mixture framework. In: CVPR. pp.1–8 (2007)13. Shen, W., Wang, X., Wang, Y., Bai, X., Zhang, Z.: Deepcontour: A deep convo-lutional feature learned by positive-sharing loss for contour detection. In: CVPR.pp. 3982–3991 (2015)14. Xie, X., Mirmehdi, M.: Texture exemplars for defect detection on random textures.In: Pattern Recognition and Image Analysis. pp. 404–413 (2005)15. Xie, X., Mirmehdi, M.: Magnetostatic field for the active contour model: A studyin convergence. In: BMVC. pp. 127–136 (2006)16. Xie, X., Mirmehdi, M.: Texems: random texture representation and analysis. In:Handbook of Texture Analysis, chap. 4 (2008)17. Yeo, S., Xie, X., Sazonov, I., Mirmehdi, M.: Geometric potential force for thedeformable model. In: BMVC (2009)(a) (b) (c) (d)Fig. 4. Segmentation of the south Wales marcher lordship map. (a) original image. (b)binary image. (c) RF probability map. (d) the final border (blue) between two userpoints (red) superimposed on the original map.

Extracting Lineage Information from Hand-Drawn Ancient Maps

https://cronfa.swan.ac.uk/Record/cronfa49176/Download/0049176-12092019151812.pdf

Extracting Lineage Information from Hand-Drawn Ancient Maps

Abstract

Similar works

Full text

Available Versions

Cronfa at Swansea University