This document proposes a new serialization format for typed syntax trees of Scala programs. It aims to be ● compact: All numbers and references are length encoded, trees are inlined by default but may be shared. ● lazy: Trees can be explored from the top, and one can suspend reading subtrees at any time. The only section that needs to be scanned in full is the name table. Laziness is important for the compiler frontend as most of the information in a TASTY file is not needed then. ● extensible: New sections can be defined at will. Additional elements always carry their length, so can be safely skipped in older readers. ● precise: The format given here is in essence a serialized abstract syntax tree for typed Scala. Some of the main classifications,e.g. between types, terms and paths are reflected in the grammar. Picklers and unpicklers for the format have been implemented in https://github.com/lampepfl/dotty/pull/394. The present document gives a syntax summary of the format. More explanations are found in the TASTY reference manual

Burmako, Eugene

Odersky, Martin

Petrashko, Dmytro

This document proposes a new serialization format for typed syntax trees of Scala programs. It aims to be ● compact: All numbers and references are length encoded, trees are inlined by default but may be shared. ● lazy: Trees can be explored from the top, and one can suspend reading subtrees at any time. The only section that needs to be scanned in full is the name table. Laziness is important for the compiler frontend as most of the information in a TASTY file is not needed then. ● extensible: New sections can be defined at will. Additional elements always carry their length, so can be safely skipped in older readers. ● precise: The format given here is in essence a serialized abstract syntax tree for typed Scala. Some of the main classifications,e.g. between types, terms and paths are reflected in the grammar. Picklers and unpicklers for the format have been implemented in https://github.com/lampepfl/dotty/pull/394. The present document gives a syntax summary of the format. More explanations are found in the TASTY reference manual.LAM

Infoscience - École polytechnique fédérale de Lausanne

A TASTY Alternative Version 0.07 22 February 2017 Martin Odersky Eugene Burmako Dmytro Petrashko  This document proposes a new serialization format for typed syntax trees of Scala programs. It aims to be  ● compact: All numbers and references are length encoded, trees are inlined by default but may be shared. ● lazy: Trees can be explored from the top, and one can suspend reading subtrees at any time. The only section that needs to be scanned in full is the name table. Laziness is important for the compiler frontend as most of the information in a TASTY file is not needed then. ● extensible: New sections can be defined at will. Additional elements always carry their length, so can be safely skipped in older readers. ● precise: The format given here is in essence a serialized abstract syntax tree for typed Scala. Some of the main classifications,e.g. between types, terms and paths are reflected in the grammar.  Picklers and unpicklers for the format have been implemented in https://github.com/lampepfl/dotty/pull/394​.   The present document gives a syntax summary of the format. More explanations are found in the ​TASTY reference manual.  Notation:  We use BNF notation. Terminal symbols start with at least two consecutive upper case letters. Each terminal is represented as a single byte tag. Non-terminals are mixed case. Prefixes of the form​ ​lower case letter*_​  are for explanation of semantic content only, they can be dropped without changing the grammar.  Micro-syntax:    LongInt       = Digit* StopDigit        ​// big endian 2’s complement, fits in a Long w/o overflow   Int           = LongInt                 ​// big endian 2s complement, fits in an Int w/o overflow   Nat           = LongInt                 ​// non-negative value, fits in an Int without overflow   Digit         = 0 | ... | 127   StopDigit     = 128 | ... | 255         ​// value = digit - 128  Macro-format:    File          = Header ​majorVersion_​ Nat ​minorVersion_​ Nat UUID                    ​nameTable_​ Length Name* Section*   Header        = 0x5C 0xA1 0xAB 0x1F   UUID          = Byte*16                ​// random UUID     Section       = NameRef Length Bytes   Length        = Nat                    ​// length of rest of entry in bytes    Name          = UTF8       Length UTF8-CodePoint*                   QUALIFIED     Length ​qualified_​ NameRef ​selector_​ NameRef                   SIGNED        Length ​original_​ NameRef ​resultSig_​ NameRef ​paramSig_​ NameRef*                   EXPANDED      Length ​original_​ NameRef                   OBJECTCLASS   Length ​object_​ NameRef                   SUPERACCESSOR Length ​accessed_​ NameRef                   DEFAULTGETTER Length ​method_N​ ameRef ​paramNumber_​ Nat                   SHADOWED      Length ​original_​ NameRef                   MANGLED       Length ​mangle_​ NameRef ​name_​ NameRef                   ...     NameRef       = Nat                    ​// ordinal number of name in name table, starting from 1.  Note: Unqualified names in the name table are strings. The context decides whether a name is  a type-name or a term-name. The same string can represent both.  Standard-Section: ​"ASTs" TopLevelStat*    TopLevelStat  = PACKAGE        Length Path TopLevelStat*                   Stat    Stat          = Term                   VALDEF         Length NameRef Type ​rhs_​ Term Modifier*                   DEFDEF         Length NameRef TypeParam* Params* ​return_​ Type ​rhs_​ Term                                        Modifier*                   TYPEDEF        Length NameRef (Type | Template) Modifier*                   IMPORT         Length ​qual_​ Term Selector*   Selector      = IMPORTED              ​name_​ NameRef                   RENAMED        Length ​from_​ NameRef ​to_​ NameRef // Imports are for scala.meta, they are not used in the backend   TypeParam     = TYPEPARAM      Length NameRef Type Modifier*   Params        = PARAMS         Length Param*   Param         = PARAM          Length NameRef Type ​rhs_​ Term? Modifier*  // rhs_Term is present in the case of an aliased class parameter.   Template      = TEMPLATE       Length TypeParam*​ ​ Param*​ ​ Parent*​ ​ Self?​ ​ Stat*  // Stat* always starts with the primary constructor.   Parent        = Application                   Type   Self          = SELFDEF               ​selfName_​ NameRef ​selfType_​ Type    Term          = Path                   Application                   IDENT                 NameRef Type ​// used when ident’s type is not a TermRef                   SELECT                ​possiblySigned_​ NameRef ​qual_​ Term                    NEW                   ​cls_​ Type                   SUPER          Length ​this​ _Term ​mixinTrait​ _Type?                   PAIR           Length ​left_​ Term ​right_​ Term                   TYPED          Length ​expr_​ Term ​ascription_​ Type                   NAMEDARG       Length ​paramName_​ NameRef ​arg_​ Term                   ASSIGN         Length ​lhs_​ Term ​rhs_​ Term                   BLOCK          Length ​expr_​ Term Stat*                   IF             Length ​cond_​ Term ​then_​ Term ​else_​ Term                   CLOSURE        Length ​meth_​ Term ​target_​ Type ​env_​ Term*                   MATCH          Length ​sel_​ Term CaseDef*                   RETURN         Length ​meth_​ ASTRef ​expr_​ Term?                   TRY            Length ​expr_​ Term CaseDef* ​finalizer_​ Term?                   REPEATED       Length ​elem_​ Type ​elem_​ Term*                   BIND           Length ​boundName_​ NameRef ​patType_​ Type ​pat_​ Term                   ALTERNATIVE    Length ​alt_​ Term*                   UNAPPLY        Length ​fun_​ Term ImplicitArg* ​pat_​ Type ​pat_​ Term*                   SHARED         term_ASTRef   Application   = APPLY          Length ​fn_Term arg_​ Term*                   TYPEAPPLY      Length ​fn_​ Term ​arg_​ Type*   CaseDef       = CASEDEF        Length ​pat_​ Tree ​rhs_​ Term ​guard_​ Term?   ImplicitArg   = IMPLICITArg           ​arg_​ Term   ASTRef        = Nat // byte position in AST payload    Path          = Constant                   TERMREFdirect        ​sym_​ ASTRef                   TERMREFsymbol        ​sym_​ ASTRef ​qual_​ Type                    TERMREFpkg           ​fullyQualified​ _NameRef                   TERMREF              ​possiblySigned_​ NameRef ​qual_​ Type                    THIS                 ​clsRef_Type                   SKOLEMtype           ​refinedType_​ ASTRef                   SHARED               ​path_​ ASTRef      Constant      = UNITconst                   FALSEconst                   TRUEconst                   BYTEconst            Int                   SHORTconst           Int                   CHARconst            Nat                   INTconst             Int                   LONGconst            LongInt                   FLOATconst           Int                   DOUBLEconst          LongInt                   STRINGconst          NameRef                   NULLconst                   CLASSconst           Type                   ENUMconst            Path    Type          = Path                   TYPEREFdirect        ​sym_​ ASTRef                   RecThis              type_ASTRef                   TYPEREFsymbol        ​sym_​ ASTRef ​qual_​ Type                   TYPEREFpkg           ​fullyQualified​ _NameRef                   TYPEREF              ​possiblySigned_​ NameRef ​qual_​ Type                   SUPERtype     Length ​this_​ Type ​underlying_​ Type                   REFINEDtype   Length ​underlying_​ Type ​refinement_​ NameRef ​info_​ Type                   APPLIEDtype   Length ​tycon_​ Type ​arg_​ Type*                   TYPEBOUNDS    Length ​low_​ Type ​high_​ Type                   TYPEALIAS     Length ​alias_​ Type (COVARIANT | CONTRAVARIANT)?                   ANNOTATED     Length ​fullAnnotation_​ Term ​underlying_​ Type                   ANDtype       Length ​left_​ Type ​right_​ Type                   ORtype        Length ​left_​ Type ​right_​ Type                   BIND          Length ​boundName_​ NameRef ​bounds_​ Type                                          ​// for type-variables defined in a type pattern                   BYNAMEtype    Length ​underlying_​ Type                   POLYtype      Length ​result​ _Type NamesTypes      ​// needed for refinements                   METHODtype    Length ​result​ _Type NamesTypes      ​// needed for refinements                   PARAMtype     Length ​binde​ r_ASTref ​paramNum​ _Nat ​// needed for refinements                   NOTYPE                   SHARED               ​type_​ ASTRef     NamesTypes    = ParamType*   NameType      = ​paramName​ _NameRef ​typeOrBounds​ _ASTRef    Modifier      = PRIVATE                   INTERNAL // package private                   PROTECTED                   PRIVATEqualified          ​qualifier_​ Type // will be dropped                   PROTECTEDqualified        ​qualifier_​ Type // will be dropped                   ABSTRACT                    FINAL                   SEALED                   CASE                    IMPLICIT                   LAZY                   OVERRIDE                   INLINE // macro                   ABSOVERRIDE // abstract override                   STATIC // mapped to static Java member                   OBJECT // an object or its class                   TRAIT // a trait                   LOCAL // private[this] or protected[this]                   SYNTHETIC // generated by Scala compiler                   ARTIFACT // to be tagged Java Synthetic                    MUTABLE // a var                   LABEL // method generated as a label                   FIELDaccessor // getter or setter                   CASEaccessor // getter for case class param                   COVARIANT // type param marked “+”                   CONTRAVARIANT // type param marked “-”                   SCALA2X // Imported from Scala2.x                   DEFAULTparameterized // Method with default params                   INSUPERCALL               ​// defined in the argument of a constructor supercall                   Annotation   Annotation    = ANNOTATION     Length ​tycon_​ Type ​fullAnnotation_​ Term  Note: Tree tags are grouped into 5 categories that determine what follows, and thus allow to compute the size of the tagged tree in a generic way.  Category 1 (tags 0-63): ​tag Category 2 (tags 64-95): ​tag Nat Category 3 (tags 96-111): ​tag AST  Category 4 (tags 112-127): ​tag Nat AST Category 5 (tags 128-255): ​tag Length <payload>  Standard Section:​ "Positions" ​sourceLength_​ Nat Assoc*    Assoc         = ​addr_​ Delta ​offset_​ Delta ​offset_​ Delta?                       ​// addr_Delta      :                                             //    Difference of address to last recorded node.                                             //    All but the first addr_Deltas are > 0, the first is >= 0.                                             // 2nd offset_Delta:                                             //    Difference of end offset of addressed node vs parent node. Always <= 0                                             // 1st offset Delta, if delta >= 0 or 2nd offset delta exists:                                             //    Difference of start offset of addressed node vs parent node.                                             // 1st offset Delta, if delta < 0 and 2nd offset delta does not exist:                                             //    Difference of end offset of addressed node vs parent node.                                             // Offsets and addresses are difference encoded.                                             // Nodes which have the same positions as their parents are omitted.   Delta         = Int // Difference between consecutive offsets / tree addresses,  

A TASTY Alternative

http://infoscience.epfl.ch/record/226194/files/ATASTYAlternative.pdf

A TASTY Alternative

Abstract

Similar works

Full text

Available Versions

Infoscience - École polytechnique fédérale de Lausanne