Analysis and Interpretation of Graphical Documents

International audienceThis chapter is dedicated to the analysis and the interpretation of graphical documents, and as such, builds upon many of the topics covered in other parts of this handbook. It will therefore not focus on any of the technical issues related to graphical documents, such as low level filtering and binarization, primitive extraction and vectorization as developed in Chapters 2.1 and 5.1 or symbol recognition, for instance, as developed in Chapter 5.2. These tools are put in a broader framework and threaded together in complex pipelines to solve interpretation questions. This chapter provides an overview of how analysis strategies have contributed to constructing these pipelines, how specific domain knowledge is integrated in these analyses, and which interpretation contexts have been contributed to successful approaches

A graphical user interface for Boolean query specification

On-line information repositories commonly provide keyword search facilities via textual query languages based on Boolean logic. However, there is evidence to suggest that the syntactical demands of such languages can lead to user errors and adversely affect the time that it takes users to form queries. Users also face difficulties because of the conflict in semantics between AND and OR when used in Boolean logic and English language. We suggest that graphical query languages, in particular Venn-like diagrams, can alleviate the problems that users experience when forming Boolean expressions with textual languages. We describe Vquery, a Venn-diagram based user interface to the New Zealand Digital Library (NZDL). The design of Vquery has been partly motivated by analysis of NZDL usage. We found that few queries contain more than three terms, use of the intersection operator dominates and that query refinement is common. A study of the utility of Venn diagrams for query specification indicates that with little or no training users can interpret and form Venn-like diagrams which accurately correspond to Boolean expressions. The utility of Vquery is considered and directions for future work are proposed

Visualising Discourse Coherence in Non-Linear Documents

To produce coherent linear documents, Natural Language Generation systems have traditionally exploited the structuring role of textual discourse markers such as relational and referential phrases. These coherence markers of the traditional notion of text, however, do not work in non-linear documents: a new set of graphical devices is needed together with formation rules to govern their usage, supported by sound theoretical frameworks. If in linear documents graphical devices such as layout and formatting complement textual devices in the expression of discourse coherence, in non-linear documents they play a more important role. In this paper, we present our theoretical and empirical work in progress, which explores new possibilities for expressing coherence in the generation of hypertext documents

Efficient Analysis of Complex Diagrams using Constraint-Based Parsing

This paper describes substantial advances in the analysis (parsing) of diagrams using constraint grammars. The addition of set types to the grammar and spatial indexing of the data make it possible to efficiently parse real diagrams of substantial complexity. The system is probably the first to demonstrate efficient diagram parsing using grammars that easily be retargeted to other domains. The work assumes that the diagrams are available as a flat collection of graphics primitives: lines, polygons, circles, Bezier curves and text. This is appropriate for future electronic documents or for vectorized diagrams converted from scanned images. The classes of diagrams that we have analyzed include x,y data graphs and genetic diagrams drawn from the biological literature, as well as finite state automata diagrams (states and arcs). As an example, parsing a four-part data graph composed of 133 primitives required 35 sec using Macintosh Common Lisp on a Macintosh Quadra 700.Comment: 9 pages, Postscript, no fonts, compressed, uuencoded. Composed in MSWord 5.1a for the Mac. To appear in ICDAR '95. Other versions at ftp://ftp.ccs.neu.edu/pub/people/futrell

Interpretación constructiva de la fábrica de tapia de tierra del castillo de Serón de Nágima (Soria)

The article analyzes the particular construction history of the Castle of Serón de Nágima (Soria, Spain). Its constructive technique is the rammed-earth and it has not any singular or stylistic element which could be taken as chronological: for this reason the castle has been traditionally understood as a hispanic-muslim building. However, the analysis and constructive lecture and interpretation of the rammed-earth walls, putting them in comparison with others contemporaries and the study of the documentation can be used as a method to know the date of construction. The town of Serón is mentioned several times during the frontier wars between the Crowns of Castile and Aragon, but the castle is only mentioned in written documents since the 15th Century. The constructive characteristics of the rammed-earth walls are very different to the hispanic-muslim rammed-earth whereas similar to those of the Late Middle Age castles. The article also shows the constructive process of these rammed-earth walls through graphical methods

Deep Temporal-Recurrent-Replicated-Softmax for Topical Trends over Time

Dynamic topic modeling facilitates the identification of topical trends over time in temporal collections of unstructured documents. We introduce a novel unsupervised neural dynamic topic model named as Recurrent Neural Network-Replicated Softmax Model (RNNRSM), where the discovered topics at each time influence the topic discovery in the subsequent time steps. We account for the temporal ordering of documents by explicitly modeling a joint distribution of latent topical dependencies over time, using distributional estimators with temporal recurrent connections. Applying RNN-RSM to 19 years of articles on NLP research, we demonstrate that compared to state-of-the art topic models, RNNRSM shows better generalization, topic interpretation, evolution and trends. We also introduce a metric (named as SPAN) to quantify the capability of dynamic topic model to capture word evolution in topics over time.Comment: In Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2018

Sense resolution properties of logical imaging

The evaluation of an implication by Imaging is a logical technique developed in the framework of modal logic. Its interpretation in the context of a “possible worlds” semantics is very appealing for IR. In 1994, Crestani and Van Rijsbergen proposed an interpretation of Imaging in the context of IR based on the assumption that “a term is a possibleworld”. This approach enables the exploitation of term– term relationshipswhich are estimated using an information theoretic measure. Recent analysis of the probability kinematics of Logical Imaging in IR have suggested that this technique has some interesting sense resolution properties. In this paper we will present this new line of research and we will relate it to more classical research into word senses

Relational models for visual understanding of graphical documents. Application to architectural drawings

Els documents gráfics són documents que expressen continguts semántics utilitzant majoritáriament un llenguatge visual. Aquest llenguatge está format per un vocabulari (símbols) i una sintaxi (relacions estructurals entre els símbols) que conjuntament manifesten certs conceptes en un context determinat. Per tant, la interpretació dun document gráfic per part dun ordinador implica tres fases. (1) Ha de ser capadçe detectar automáticament els símbols del document. (2) Ha de ser capadç extreure les relacions estructurals entre aquests símbols. I (3), ha de tenir un model del domini per tal poder extreure la semántica. Exemples de documents gráfics de diferents dominis són els planells darquitectural i d'enginyeria, mapes, diagrames de flux, etc. El Reconeixement de Gráfics, dintre de lárea de recerca de Análisi de Documents, neix de la necessitat de la indústria dinterpretar la gran quantitat de documents gráfics digitalitzats a partir de laparició de lescáner. Tot i que molts anys han passat daquests inicis, el problema de la interpretació automática de documents sembla encara estar lluny de ser solucionat. Básicament, aquest procés sha alentit per una raó principal: la majoria dels sistemes dinterpretació que han estat presentats per la comunitat són molt centrats en una problemática específica, en el que el domini del document marca clarament la implementació del mètode. Per tant, aquests mètodes són difícils de ser reutilitzats en daltres dades i marcs daplicació, estancant així la seva adopció i evolució en favor del progrés. En aquesta tesi afrontem el problema de la interpretació automática de documents gráfics a partir dun seguit de models relacionals que treballen a tots els nivells del problema, i que han estat dissenyats des dun punt de vista genèric per tal de que puguin ser adaptats a diferents dominis. Per una part, presentem 3 mètodes diferents per a lextracció dels símbols en un document. El primer tracta el problema des dun punt de vista estructural, en el que el coneixement general de lestructura dels símbols permet trobar-los independentment de la seva aparença. El segon és un mètode estad ístic que aprèn laparença dels símbols automáticament i que, per tant, sadapta a la gran variabilitat del problema. Finalment, el tercer mètode és una combinació dambdós, heretant els beneficis de cadascun dels mètodes. Aquesta tercera implementaci ó no necessita de un aprenentatge previ i a més sadapta fácilment a múltiples notacions gráfiques. D'altra banda, presentem dos mètodes per a la extracció del context visuals. El primer mètode segueix una estratègia bottom-up que cerca les relacions estructurals en una representació de graf mitjançant algorismes dintel_ligència artificial. La segona en canvi, és un mètode basat en una gramática que mitjançant un model probabilístic aprèn automáticament lestructura dels planells. Aquest model guia la interpretació del document amb certa independència de la implementació algorísmica. Finalment, hem definit una base del coneixement fent confluir una definició ontol'ogica del domini amb dades reals. Aquest model ens permet raonar les dades des dun punt de vista contextual i trobar inconsistències semántiques entre les dades. Leficiència daquetes contribucions han estat provades en la interpretació de planells darquitectura. Aquest documents no tenen un estándard establert i la seva notació gráfica i inclusió dinformació varia de planell a planell. Per tant, és un marc rellevant del problema de reconeixement gráfic. A més, per tal de promoure la recerca en termes de interpretació de documents gráfics, fem públics tant les dades, leina per generar les dades i els evaluadors del rendiment.Graphical documents express complex concepts using a visual language. This language consists of a vocabulary (symbols) and a syntax (structural relations among symbols) that articulate a semantic meaning in a certain context. Therefore, the automatic interpretation of these sort of documents by computers entails three main steps: the detection of the symbols, the extraction of the structural relations among these symbols, and the modeling of the knowledge that permits the extraction of the semantics. Different domains in graphical documents include: architectural and engineering drawings, maps, flowcharts, etc. Graphics Recognition in particular and Document Image Analysis in general are born from the industrial need of interpreting a massive amount of digitalized documents after the emergence of the scanner. Although many years have passed, the graphical document understanding problem still seems to be far from being solved. The main reason is that the vast majority of the systems in the literature focus on a very specific problems, where the domain of the document dictates the implementation of the interpretation. As a result, it is difficult to reuse these strategies on different data and on different contexts, hindering thus the natural progress in the field. In this thesis, we face the graphical document understanding problem by proposing several relational models at different levels that are designed from a generic perspective. Firstly, we introduce three different strategies for the detection of symbols. The first method tackles the problem structurally, wherein general knowledge of the domain guides the detection. The second is a statistical method that learns the graphical appearance of the symbols and easily adapts to the big variability of the problem. The third method is a combination of the previous two inheriting their respective strengths, i.e. copes the big variability and does not need of annotated data. Secondly, we present two relational strategies that tackle the problem of the visual context extraction. The first one is a full bottom up method that heuristically searches in a graph representation the contextual relations among symbols. Contrarily, the second is syntactic method that models probabilistically the structure of the documents. It automatically learns the model, which guides the inference algorithm to counter the best structural representation for a given input. Finally, we construct a knowledge-based model consisting of an ontological definition of the domain and real data. This model permits to perform contextual reasoning and to detect semantic inconsistencies within the data. We evaluate the suitability of the proposed contributions in the framework of floor plan interpretation. Since there is no standard in the modeling of these documents, there exists an enormous notation variability and the sort of information included in the documents also varies from plan to plan. Therefore, floor plan understanding is a relevant task in the graphical document understanding problem. It is also worth to mention that, we make freely available all the resources used in this thesis (the data, the tool used to generate the data, and the evaluation scripts) aiming at fostering the research in graphical document understanding task

Hyper-Document structure: maintaining discourse coherence in non-linear documents

The passage from linear text to hypertext poses the challenge of expressing discourse coherence in non-linear text, where linguistic discourse markers no longer work. While hypertext introduces new possibilities for discourse organisation, it also requires the use of new devices which can support the expression of coherence by exploiting the technical characteristics and expressive capabilities of the medium. In this paper we show how in hypertext the notion of abstract document structure encompasses animated graphics as a form of meta-language for discourse construction