ANNIS: a linguistic database for exploring information structure

In this paper, we discuss the design and implementation of our first version of the database "ANNIS" (ANNotation of Information Structure). For research based on empirical data, ANNIS provides a uniform environment for storing this data together with its linguistic annotations. A central database promotes standardized annotation, which facilitates interpretation and comparison of the data. ANNIS is used through a standard web browser and offers tier-based visualization of data and annotations, as well as search facilities that allow for cross-level and cross-sentential queries. The paper motivates the design of the system, characterizes its user interface, and provides an initial technical evaluation of ANNIS with respect to data size and query processing

Querying and updating treebanks: A critical survey and requirements analysis

Language technology makes extensive use of hierarchically annotated text and speech data. These databases are stored in flat files and manipulated using corpus-specific query tools or special-purpose scripts. While the size of these databases and the range of applications has grown rapidly in recent years, neither method for managing the data has led to reusable, scalable software. The formal properties of the query languages are not well understood. Hence established methods for indexing tree data and optimizing tree queries cannot be employed. We analyze a range of existing linguistic query languages, and adduce a set of requirements for a reusable, scalable linguistic query language.

Corpus-consulting probabilistic approach to parsing: the CCPX parser and its complementary components

Corpus linguistics is now a major field in the study of language. In recent years corpora that are syntactically analysed have become available to researchers, and these clearly have great potential for use in the field of parsing natural language. This thesis describes a project that exploits this possibility. It makes four distinct contributions to these two fields. The first is an updated version of a corpus that is (a) analysed in terms of the rich syntax of Systemic Functional Grammar (SFG), and (b) annotated using the extensible Mark-up Language (XML). The second contribution is a native XML corpus database, and the third is a sophisticated corpus query tool for accessing it. The fourth contribution is a new type of parser that is both corpus-consulting and probabilistic. It draws its knowledge of syntactic probabilities from the corpus database, and it stores its working data within the database, so that it is strongly database-oriented. SFG has been widely used in natural language generation for approaching two decades, but it has been used far less frequently in parsing (the first stage in natural language understanding). Previous SFG corpus-based parsers have utilised traditional parsing algorithms, but they have experienced problems of efficiency and coverage, due to (a) the richness of the syntax and (b) the challenge of parsing unrestricted spoken and written texts. The present research overcomes these problems by introducing a new type of parsing algorithm that is 'semi-deterministic' (as human readers are), and utilises its knowledge of the rules—including probabilities—of English syntax. A language, however, is constantly evolving. New words and uses are added, while others become less frequent and drop out altogether. The new parsing system seeks to replicate this. As new sentences are parsed they are added to the corpus, and this slowly changes the frequencies of the words and the syntactic patterns. The corpus is in this sense dynamic, and so simulates a human's changing knowledge of words and syntax

Netgraph-A Tool for Searching in the Prague Dependency Treebank 2.0

Three sides existed whose connection is solved in this thesis. First, it was the Prague Dependency Treebank 2.0, one of the most advanced treebanks in the linguistic world. Second, there existed a very limited but extremely intuitive search tool - Netgraph 1.0. Third, there were users longing for such a simple and intuitive tool that would be powerful enough to search in the Prague Dependency Treebank. In the thesis, we study the annotation of the Prague Dependency Treebank 2.0, especially on the tectogrammatical layer, which is by far the most complex layer of the treebank, and assemble a list of requirements on a query language that would allow searching for and studying all linguistic phenomena annotated in the treebank. We propose an extension to the query language of the existing search tool Netgraph 1.0 and show that the extended query language satisfies the list of requirements. We also show how all principal linguistic phenomena annotated in the treebank can be searched for with the query language. The proposed query language has also been implemented - we present the search tool as well and talk about the data format for the tool. An attached CD-ROM contains the installation of the tool.Tato práce se zabývá spojením tří existujících stran. Na straně jedné byl Pražský závislostní korpus 2.0, jeden z nejvyspělejších korpusů lingvistického světa. Na straně druhé existoval omezený, ale velmi intuitivní vyhledávací nástroj Netgraph 1.0. A na straně třetí byli uživatelé toužící po takovém jednoduchém nástroji, který by však byl dostatečně silný pro vyhledávání v Pražském závislostním korpusu. V této práci zkoumáme anotaci Pražského závislostního korpusu 2.0, obzvláště tektogramatické roviny, jež je zdaleka nejsložitější rovinou tohoto korpusu, a vytváříme seznam požadavků na dotazovací jazyk, který by umožnil vyhledávání a studium všech lingvistických jevů v korpusu anotovaných. Navrhujeme rozšíření dotazovacího jazyka existujícího vyhledávacího nástroje Netgraphu 1.0 a ukazujeme, že tento rozšířený dotazovací jazyk vyhovuje formulovanému seznamu požadavků. Ukazujeme rovněž, jak pomocí tohoto dotazovacího jazyka mohou být vyhledány všechny podstatné lingvistické jevy anotované v korpusu. Navržený dotazovací jazyk byl rovněž implementován - zmiňujeme se tedy i o vyhledávacím nástroji a hovoříme o datech pro tento nástroj. Nástroj je možno nainstalovat z přiloženého CD-ROMu.Institute of Formal and Applied LinguisticsÚstav formální a aplikované lingvistikyFaculty of Mathematics and PhysicsMatematicko-fyzikální fakult

VIQTORYA - A Visual Query Tool for Syntactically Annotated Corpora

This paper presents a query tool for syntactically annotated corpora. The query tool is developed to search the T ubingen Treebanks annotated at the University of T ubingen. However, in principle it also can be adapted to other corpora. The tool uses a query language that allows to search for tokens, syntactic categories, grammatical functions and binary relations of (immediate) dominance and linear precedence between nodes. The overall idea is to extract in an initializing phase the relevant information from the corpus and store it in a compact way in a relational database. An incoming query is then translated into a corresponding SQL query that is evaluated on the database. A graphical user interface allows to specify queries in a user-friendly way