One Parser to Rule Them All

Despite the long history of research in parsing, constructing parsers for real programming languages remains a difficult and painful task. In the last decades, different parser generators emerged to allow the construction of parsers from a BNF-like specification. However, still today, many parsers are handwritten, or are only partly generated, and include various hacks to deal with different peculiarities in programming languages. The main problem is that current declarative syntax definition techniques are based on pure context-free grammars, while many constructs found in programming languages require context information. In this paper we propose a parsing framework that embraces context information in its core. Our framework is based on data-dependent grammars, which extend context-free grammars with arbitrary computation, variable binding and constraints. We present an implementation of our framework on top of the Generalized LL (GLL) parsing algorithm, and show how common idioms in syntax of programming languages such as (1) lexical disambiguation filters, (2) operator precedence, (3) indentation-sensitive rules, and (4) conditional preprocessor directives can be mapped to data-dependent grammars. We demonstrate the initial experience with our framework, by parsing more than 20000 Java, C#, Haskell, and OCaml source files

On the learning of vague languages for syntactic pattern recognition

The method of the learning of vague languages which represent distorted/ambiguous patterns is proposed in the paper. The goal of the method is to infer the quasi-context-sensitive string grammar which is used in our model as the generator of patterns. The method is an important component of the multi-derivational model of the parsing of vague languages used for syntactic pattern recognition

A Cubic Time Extension of Context-Free Grammars

Context-free grammars and cubic time parsing are so related in people's minds that they often think that parsing any extension of context-free grammars must need some extra time. Of course, this is not true and this paper presents a generalization of context-free grammars which keeps a cubic time complexity. This extension, which defines a sub-class of context-sensitive languages, has both a theoretical and a practical interest. The class of languages defined by these grammars is closed under both intersection and complementation (in fact it is the class containing the intersection and the complementation of context-free languages). On the other hand, these grammars can be considered as being mildly context-sensitive and can therefore be used in natural language processing

Parsing Based on Regulated Grammars

Tato práce se zabývá regulovanými gramatikami. Popisuje vlastnosti různých druhů regulovaných gramatik a srovnává je. Ke srovnání využívá příklady postavené na kontextových jazycích. Práce se nejvíce zaměřuje na programované gramatiky. Modifikuje metody syntaktické analýzy a využívá je pro programované gramatiky. Práce aplikuje navrženou metodu syntaktické analýzy na kontextové jazyky a zkoumá jejich sílu.This thesis deals with regulated grammars. It describes their behavior and it compares them using different examples of context-sensitive languages. The thesis focuses mostly on programmed grammars. It modifies the methods of parsing and it uses them for programmed grammars. This thesis applies the developed method of parsing on context-sensitive languages and examines their power.

Exploring the N-th Dimension of Language

This paper is aimed at exploring the hidden fundamental\ud computational property of natural language that has been so elusive that it has made all attempts to characterize its real computational property ultimately fail. Earlier natural language was thought to be context-free. However, it was gradually realized that this does not hold much water given that a range of natural language phenomena have been found as being of non-context-free character that they have almost scuttled plans to brand natural language contextfree. So it has been suggested that natural language is mildly context-sensitive and to some extent context-free. In all, it seems that the issue over the exact computational property has not yet been solved. Against this background it will be proposed that this exact computational property of natural language is perhaps the N-th dimension of language, if what we mean by dimension is\ud nothing but universal (computational) property of natural language

Connectionist natural language parsing

The key developments of two decades of connectionist parsing are reviewed. Connectionist parsers are assessed according to their ability to learn to represent syntactic structures from examples automatically, without being presented with symbolic grammar rules. This review also considers the extent to which connectionist parsers offer computational models of human sentence processing and provide plausible accounts of psycholinguistic data. In considering these issues, special attention is paid to the level of realism, the nature of the modularity, and the type of processing that is to be found in a wide range of parsers

Towards Robustness in Parsing - Fuzzifying Context-Free Language Recognition

We discuss the concept of robustness with respect to parsing a context-free language. Our approach is based on the notions of fuzzy language, (generalized) fuzzy context-free grammar and parser / recognizer for fuzzy languages. As concrete examples we consider a robust version of Cocke-Younger-Kasami's algorithm and a robust kind of recursive descent recognizer