Parsing Expression Grammars Made Practical

Parsing Expression Grammars (PEGs) define languages by specifying recursive-descent parser that recognises them. The PEG formalism exhibits desirable properties, such as closure under composition, built-in disambiguation, unification of syntactic and lexical concerns, and closely matching programmer intuition. Unfortunately, state of the art PEG parsers struggle with left-recursive grammar rules, which are not supported by the original definition of the formalism and can lead to infinite recursion under naive implementations. Likewise, support for associativity and explicit precedence is spotty. To remedy these issues, we introduce Autumn, a general purpose PEG library that supports left-recursion, left and right associativity and precedence rules, and does so efficiently. Furthermore, we identify infix and postfix operators as a major source of inefficiency in left-recursive PEG parsers and show how to tackle this problem. We also explore the extensibility of the PEG paradigm by showing how one can easily introduce new parsing operators and how our parser accommodates custom memoization and error handling strategies. We compare our parser to both state of the art and battle-tested PEG and CFG parsers, such as Rats!, Parboiled and ANTLR.Comment: "Proceedings of the International Conference on Software Language Engineering (SLE 2015)" - 167-172 (ISBN : 978-1-4503-3686-4

Unifying parsing and reflective printing for fully disambiguated grammars

Language designers usually need to implement parsers and printers. Despite being two closely related programs, in practice they are often designed separately, and then need to be revised and kept consistent as the language evolves. It will be more convenient if the parser and printer can be unified and developed in a single program, with their consistency guaranteed automatically. Furthermore, in certain scenarios (like showing compiler optimisation results to the programmer), it is desirable to have a more powerful reflective printer that, when an abstract syntax tree corresponding to a piece of program text is modified, can propagate the modification to the program text while preserving layouts, comments, and syntactic sugar. To address these needs, we propose a domain-specific language BiYacc, whose programs denote both a parser and a reflective printer for a fully disambiguated context-free grammar. BiYacc is based on the theory of bidirectional transformations, which helps to guarantee by construction that the generated pairs of parsers and reflective printers are consistent. Handling grammatical ambiguity is particularly challenging: we propose an approach based on generalised parsing and disambiguation filters, which produce all the parse results and (try to) select the only correct one in the parsing direction; the filters are carefully bidirectionalised so that they also work in the printing direction and do not break the consistency between the parsers and reflective printers. We show that BiYacc is capable of facilitating many tasks such as Pombrio and Krishnamurthi's 'resugaring', simple refactoring, and language evolution.We thank the reviewers and the editor for their selflessness and effort spent on reviewing our paper, a quite long one. With their help, the readability of the paper is much improved, especially regarding how several case studies are structured, how theorems for the basic BiYacc and theorems for the extended version handling ambiguous grammars are related, and how look-alike notions are `disambiguated'. This work is partially supported by the Japan Society for the Promotion of Science (JSPS) Grant-in-Aid for Scientific Research (S) No. 17H06099; in particular, most of the second author's contributions were made when he worked at the National Institute of Informatics and funded by the Grant

A Tool for the Formal Analysis of Symmetric Primitives

Tänapäeva maailmas on krüpograafia laialt kasutusel, et turvata elektroonilist sidet. Seega on oluline, et vastavad krüptograafilised algoritmid oleksid tõestatavalt turvalised. Krüptograafiliste protokollide ehitamiseks kasutatakse krüptograafilisi primitiive ja terve protokolli turvalisus tõestatakse vastavate primitiivide turvalisusle tuginedes. Sümmeetrilised primitiivid kasutavad ühte salajast võtit nii krüpteerimiseks kui ka dekrüpteermiseks ja on kiiremad kui asümmeetrilised primitiivid. Seetõttu kasutatakse sümmeetrilisi primitiive paljudes protokollides ja seega on vaja leida vastavatele primitiividele turvatõestused. Üks võimalus turvatõestuste kirjutamiseks on kasutada krüptograafiliste mängude põhist tõestamist. Krüptograafilised mängud modelleerivad primitiive kindlas keskkonnas, kus leidub vastane. Mängude põhise tõestamise abil kirjutatakse esialgne mäng ümber nii, et primitiivi turvalisust oleks lihtsam tõestada. Selline tõestamise meetod on keeruline, kuna see nõuab palju ümberkirjutamist. Lisaks sellele võib mängude ümberkirjutamisel tekkida tõestusse vigu. Antud olukorra lahendamiseks kasutatakse tööriistu, mis abistavad tõestuse läbiviimist. Antud magistritöö teemaks on tööriista arendamine, mis aitab mängude põhiseid tõestusi läbi viia sümmeetriliste primitiivide jaoks. Vastava tööriista nimi on ProveIt ja see lubab modifitseerida krüptograafilisi mänge, kasutades kindlaid tõestuse skeeme. Vastava tõestusskeemi rakendamist nimetatakse transformatsiooniks. Minu eesmärgiks oli lisada antud programmis kasutatavale keelele semantika ja kasutada seda semantikat erinevate transformatsioonide lisamiseks programmi ProveIt. Magistritöö käigus implementeerisin ProveIt jaoks järgnevad transformatisoonid: Dead Code Elimination, Statement Switching, Remove Condition, Replace Function Call, Wrap ja Random Function Simulation. Surnud koodi eemaldamiseks oli vaja implementeerida elusate muutujate analüüs. Enne transformatsiooni implementeerimist tuleb tõestada vastav turvalisuse garantii ning seetõttu tõestasin magistritöös käsitletud trasformatsioonidele vastavad turvalisuse teoreemid. Edasise tööna tuleb ProveIt lisada keerukamaid transformatisoone, et võimaldada keerukamate turvatõestuste läbiviimist.The master thesis is about improving a tool that is used for analyzing the security of symmetric primitives. This tool is named ProveIt and it can help researchers to verify the correctness and security of protocols that use symmetric primitives. Symmetric primitives are low-level symmetric-key al- gorithms. When given such protocol, the probability of breaking its security can be found by doing a finite number of reductions. However, it is not convenient to do such proofs on paper and therefore a tool can assist the researcher. ProveIt lets the user to choose which reductions to do on a given protocol. After parsing a protocol into an abstract syntax tree, ProveIt lets the user to transform the initial protocol one step at a time in order to find the probability of a successful attack against it. After a finite number of reductions the proof is complete and the probability of a successful attack is found. However, currently ProveIt is able to do only a few reductions and this limits the usability of the tool. The goal of this thesis is to add new reductions to ProveIt so that it would be possible to work with this tool using common techniques for proving the security of symmetric primitives. In order to add reductions to ProveIt it is necessary to analyze different symmetric primitives to find out which reductions are required. In the thesis we describe the necessary subset of transformations that are required for doing reduction based proofs. We show where different transformations can be applied and define how they change the security game. We prove for each transformation how they change the security game. After giving a proof we can describe how we implemented the transformations in ProveIt. As a result of this thesis ProveIt will have the basic functionalities re- quired for doing the security proofs of symmetric primitives. Besides that, all the implemented reductions have been analyzed and their effect to the security game has been proved

Guppy: Process-Oriented Programming on Embedded Devices

Guppy is a new and experimental process-oriented programming language, taking much inspiration (and some code-base) from the existing occam-pi language. This paper reports on a variety of aspects related to this, specifically language, compiler and run-time system development, enabling Guppy programs to run on desktop and embedded systems. A native code-generation approach is taken, using C as the intermediate language, and with stack-space requirements determined at compile-time

Approaches to the determination of parallelism in computer programs

Approaches to the determination of parallelism in computer program

Phobos: A front-end approach to extensible compilers (long version)

This paper describes a practical approach for implementing certain types of domain-specific languages with extensible compilers. Given a compiler with one or more front-end languages, we introduce the idea of a "generic" front-end that allows the syntactic and semantic specification of domain-specific languages. Phobos, our generic front-end, offers modular language specification, allowing the programmer to define new syntax and semantics incrementally

Parallel parsing made practical

The property of local parsability allows to parse inputs through inspecting only a bounded-length string around the current token. This in turn enables the construction of a scalable, data-parallel parsing algorithm, which is presented in this work. Such an algorithm is easily amenable to be automatically generated via a parser generator tool, which was realized, and is also presented in the following. Furthermore, to complete the framework of a parallel input analysis, a parallel scanner can also combined with the parser. To prove the practicality of a parallel lexing and parsing approach, we report the results of the adaptation of JSON and Lua to a form fit for parallel parsing (i.e. an operator-precedence grammar) through simple grammar changes and scanning transformations. The approach is validated with performance figures from both high performance and embedded multicore platforms, obtained analyzing real-world inputs as a test-bench. The results show that our approach matches or dominates the performances of production-grade LR parsers in sequential execution, and achieves significant speedups and good scaling on multi-core machines. The work is concluded by a broad and critical survey of the past work on parallel parsing and future directions on the integration with semantic analysis and incremental parsing

Type Inference for a Cryptographic Protocol Prover Tool

Krüptograafia uurimisrühm Tartus arendab mängupõhiste krüptograafiliste protokollide tõestajat nimega ProveIt. Idee ProveIt’i taga on lihtsustada tõestamise protsessi. Tegemist ei ole automaatse tõestajaga, vaid abivahendiga krüptograafidele. ProveIt võimaldab teadlastel teha muudatusi koodina üleskirjutatud tõenäosuslikele mängudele ja teeb koodi ümberkirjutamise automaatselt. See aitab garanteerida korrektsuse, hoida kokku aega ja samas jätta tõestuse arusaadavaks. Üks võimalus kirjutada üles krüptograafilisi protokolle on kirjeldada neid programmeerimiskeelele sarnase pseudokoodiga, mida nimetatakse mänguks. Mängu muutmisel samm-sammult saame mõnikord jõuda teise mänguni, mille turvalisuse hinnangut me teame. Nende mängude jada ongi üks võimalik viis, kuidas saab anda protokolli turvalisuse tõestuse - seda nimetatakse mängupõhiseks tõestamiseks. Mängupõhine tõestamine on populaarne viis analüüsimaks protokollide turvalisust, sest see on võrdlemisi intuitiivne. Tavaliselt kasutatakse paberit ja pliiatsit, et kirjutada üles mänge, kuid kuna krüptograafiliste protokollide tõestamine ei ole just kõige lihtsam asi, mida teha, siis inimesed sageli lõpetavad suure kuhja paberitega igal pool, sealhulgas mitmete tupikute ja vigadega. Meie eesmärgiks on seda muuta ja anda krüptograafile võimalus mängu ümber kirjutamise asemel keskenduda järgmise sammu väljamõtlemisele. Kuigi krüptograafial on oluline roll selle bakalaureusetöö juures, siis põhi fookus lasub siiski tüübiteoorial. Nimelt on selle töö eesmärgiks luua tüübikontroll ProveIt’i jaoks. Mängupõhiseid tõestusi saab edukalt kirjutada ilma tüüpidest mi- dagi teadmata, aga tüüpide kasutamine ProveIt’is võimaldab tagada kindlamalt korrektsuse ja kasutajat paremini aidata. Tüübikontrolli abil saame näiteks veenduda, et krüptograaf ei ürita omavahel liita numbreid ja pesukarusid. Selle fakti ja vea asukoha teadmine võib kokku hoida palju kasutaja aega. Lihtne lahendus tüübikontrolli jaoks oleks nõuda kasutajalt kõigi muutujate tüüpide defineerimist, kuid see nõuaks temalt lisatööd, mis on aga vastuolus ProveIt’i eesmärgiga. Järelikult tuleb lisaks tüübikontrollile realiseerida ka tüübituletus ehk programm, mis üritab ise kõigi muutujate tüüpe määrata. Selleks, et teha tüübituletust defineerisime me kõigepealt tüübisüsteemi ProveIt’i keele jaoks. Andsime tüübireeglid aritmeetiliste avaldiste, juhuslikult hulgast valimise ja omistamise jaoks. Funktsioonide ja loogiliste avaldiste tüüpimine ei ole käesoleva bakalaureusetöö skoobis, kuid need on vajalikud osad ProveIt’ile täieliku tüüpija lisamiseks. Lisaks tüübisüsteemi ja tüübireeglite kirjeldamisele kuulus käesoleva bakalaureuse töö juurde aritmeetiliste avaldiste tüüpimise realiseerimine ja ProveIt’ile lisamine. Selle jaoks on töös antud kuus algoritmi. Arendusvahenditena valisime Qt raamistiku ja programmeerimiskeele C++. Otsustasime keele C++ kasuks, sest ProveIt on selles keeles kirjutatud, ning see võimaldab ühildamise lihtsamaks muuta.This thesis introduces types into the ProveIt language, making it possible to perform type inference and type checking. The ProveIt language is a is a probabilistic program language used in ProveIt, a tool that helps cryptographers prove cryptographic protocols in a game-based way in the computational model. The work begins by explaining the necessary background knowledge: the computational model, game-based proving, the ProveIt prover tool and the ProveIt language. How they work and why they are relevant is also described. Introducing types into ProveIt allows further guarantee the correctness and providing better help to the user. Knowing that the user is tying to find the square root of a banana and where, might save a lot of his or her time. Although it would be easier to demand the user to state all the types for all the variables and only perform type checking, we decide not to do so and instead infer the types. This is done because our goal is to make the proving process as convenient for the user as possible. To do type inference, the work starts with defining the formal type system for the ProveIt language. Type rules for all the arithmetic operations available in ProveIt are provided and also type rules for the uniform choice statement and the assignment statement. Adding rules for typing functions and logical expressions is not in the scope of this thesis and is considered future work. In addition to the type system description, type checking and type inference were implemented. To do that, the thesis first deals with the scopes of the variables in ProveIt, as it has both local and global variables. Secondly, the relations between types are covered, including the need and means to find lowest common ancestors in directed acyclic graphs. This is necessary to be able to find the minimal common supertype if an arithmetic operation was performed between variables of different types. Next, the concepts of a minimum and a maximum type are introduced to make type inference easier and also be able to output better error messages. Finally six algorithms are given to complete the type inference and type checking