The last few years have seen the fast and ubiquitous diffusion of JSON as one of the most widely used formats for publishing and interchanging data, as it combines the flexibility of semistructured data models with well-known data structures like records and arrays. The user willing to effectively manage JSON data collections can rely on several schema languages, like JSON Schema, JSound, and Joi, or on the type abstractions offered by modern programming languages like Swift or TypeScript. The main aim of this tutorial is to provide the audience with the basic notions for enjoying all the benefits that schemas and types can offer while processing and manipulating JSON data. This tutorial focuses on four main aspects of the relation between JSON and schemas: (1) we survey existing schema language proposals and discuss their prominent features; (2) we review how modern programming languages support JSON data as first-class citizens; (3) we analyze tools that can infer schemas from data, or that exploit schema information for improving data parsing and management; and (4) we discuss some open research challenges and opportunities related to JSON data

Baazizi, M. -A.

Colazzo, D.

Ghelli, G.

Sartiani, C.

International audienceThe last few years have seen the fast and ubiquitous diffusion of JSON as one of the most widely used formats for publishing and interchanging data, as it combines the flexibility of semistructured data models with well-known data structures like records and arrays. The user willing to effectively manage JSON data collections can rely on several schema languages, like JSON Schema, JSound, and Joi, or on the type abstractions offered by modern programming languages like Swift or TypeScript

Baazizi, Mohamed-Amine

Colazzo, Dario

Ghelli, Giorgio

Sartiani, Carlo

Archive Ouverte en Sciences de l'Information et de la Communication

English

Schemas And Types For JSON Data

Baazizi M. -A.

Colazzo D.

Ghelli G.

Sartiani C.

Archivio della Ricerca - Università di Pisa

Schemas and types for JSON data

Archivio della Ricerca - Università della Basilicata

Schemas And Types For JSON DataMohamed-Amine BaaziziSorbonne Université, LIP6 UMR 7606Francebaazizi@ia.lip6.frDario ColazzoUniversité Paris-Dauphine, PSL Research UniversityFrancedario.colazzo@dauphine.frGiorgio GhelliDipartimento di InformaticaUniversità di PisaPisa, Italyghelli@di.unipi.itCarlo SartianiDIMIEUniversità della BasilicataPotenza, Italysartiani@gmail.comABSTRACTThe last few years have seen the fast and ubiquitous diffusion ofJSON as one of the most widely used formats for publishing andinterchanging data, as it combines the flexibility of semistruc-tured data models with well-known data structures like recordsand arrays. The user willing to effectively manage JSON data col-lections can rely on several schema languages, like JSON Schema,JSound, and Joi, or on the type abstractions offered by modernprogramming languages like Swift or TypeScript.The main aim of this tutorial is to provide the audience withthe basic notions for enjoying all the benefits that schemas andtypes can offer while processing and manipulating JSON data.This tutorial focuses on four main aspects of the relation betweenJSON and schemas: (1) we survey existing schema language pro-posals and discuss their prominent features; (2) we review howmodern programming languages support JSON data as first-classcitizens; (3) we analyze tools that can infer schemas from data, orthat exploit schema information for improving data parsing andmanagement; and (4) we discuss some open research challengesand opportunities related to JSON data.1 INTRODUCTIONThe last two decades have seen a dramatic change in the data pro-cessing landscape. While at the end of the last century data wereusually very structured and managed inside relational DBMSs,nowadays they have very different characteristics: they are big,usually semistructured or even unstructured, without a rigid andpredefined schema, and hosted and produced in data process-ing platforms that do not embrace the relational model. In thisnew scenario, where data come without a schema, and multipledata models coexist, JSON is affirming as a useful format forpublishing and exchanging data, as it combines the flexibilityof XML with well-known data structures like records and ar-rays. JSON is currently employed for publishing and sharing datain many application fields and for many different purposes: forinstance, JSON is used as the result format for many web siteAPIs (e.g., Twitter, New York Times), as a common format for theremote interaction of modern web applications (e.g., Facebook’sGraphQL is entirely based on JSON), as a common format for ex-changing scientific data as well as public open data (e.g., the U.S.Government’s open data platform: https://www.data.gov).© 2019 Copyright held by the owner/author(s). Published in Proceedings of the22nd International Conference on Extending Database Technology (EDBT), March26-29, 2019, ISBN 978-3-89318-081-3 on OpenProceedings.org.Distribution of this paper is permitted under the terms of the Creative Commonslicense CC-by-nc-nd 4.0.Given the wide diffusion of JSON and its use in scientific aswell as mainstream applications, the need to directly manipulateJSON data inside applications rapidly emerged. To this aim, aschema language, specifically designed for JSON, has been in-troduced, but its adoption is not growing at a fast pace, sinceits specification is somewhat complex and many modern pro-gramming languages, like Swift and TypeScript, directly supportJSON data through their own, simple type systems; furthermore,Walmart Labs endowed JavaScript, which is inherently untyped,with a powerful schema language for JSON objects by means ofJavaScript function calls.In this tutorial proposal, we will present and discuss existingschema and type languages for JSON data, and compare theirfeatures; we will also discuss several schema-related tools, witha particular focus on approaches for schema inference. The mainaim of this tutorial is to provide the audience and developers withthe basic notions for enjoying all the benefits that schemas andtypes can offer while processing, analyzing, and manipulatingJSON data.Outline. This 1.5-hour tutorial is split into five main parts:(1) JSON primer (∼ 10 min.). In this very introductory partof the tutorial, we review the basic notions about JSONtogether with its JavaScript legacy, and present a few ex-amples, coming from publicly available datasets, that wewill use throughout the remaining parts of the tutorial.(2) Schema languages (∼ 20 min.). In this part of the tuto-rial we focus on existing schema languages for JSON datacollections and discuss their most prominent features.(3) Types in Programming Languages (∼ 15min.). In thispart of the tutorial we review how modern programminglanguages support JSON data as first class citizens. In par-ticular, we focus on programming and scripting languagesfor web and/or mobile applications, where JSON data in-terchange is a crucial task.(4) Schema Tools (∼ 30 min.). In this part of the tutorialwe analyze tools that exploit schema information for im-proving JSON data processing. We focus on the problem ofinferring a meaningful schema for schemaless JSON collec-tions, as well as on the exploitation of schema informationfor improving data parsing and management.(5) Future Opportunities (∼ 10 min.). Finally, we outlineopen research problems as potential directions for newresearch in this area.In what follows we describe at a very high level the technicalcontent covered in each of the last four aforementioned parts.Tutorial  Series ISSN: 2367-2005 437 10.5441/002/edbt.2019.392 SCHEMA LANGUAGESIn this part of the tutorial we will focus our attention on severalschema languages for JSON data, with particular emphasis onJSON Schema [4] and Walmart Labs Joi [6].JSON Schema emerged in the academic community and hasbeen developed without a specific programming or scripting lan-guage in mind. JSON Schema allows the programmer to specifya schema for any kind of JSON values, and supports traditionaltype constructors, like union and concatenation, as well as verypowerful constructors like negation types.JSON Schema has already been studied. Indeed, in [21], moti-vated by the need of laying the formal foundations for the JSONSchema language [4], Pezoa et al. present the formal semanticsof that language, as well as a theoretical study of its expressivepower and validation problem. Along the lines of [21], Bourhis etal. [15] have recently laid the foundations for a logical character-ization and formal study for JSON schema and query languages.On the contrary, Joi has been developed by Walmart as a toolfor (i) creating schemas for JSON objects and (ii) ensuring thevalidation of objects inside an untyped scripting language likeJavaScript; furthermore, to the best of our knowledge, Joi hasnot been studied so far. Joi only allows the designer to describethe schema for JSON objects, but it still provides the ability tospecify co-occurrence and mutual exclusion constraints on fields,as well as union and value-dependent types.We will analyze the most prominent features of these lan-guages, and compare their capabilities in a few scenarios. Wewill also briefly discuss JSound [5], an alternative, but quite re-strictive, schema language, as well as a few other schema-relatedproposals, such that described in [24], where Wang et al. presenta framework for efficiently managing a schema repository forJSON document stores. The proposed approach relies on a no-tion of JSON schema called skeleton. In a nutshell, a skeleton is acollection of trees describing structures that frequently appear inthe objects of a JSON data collection. In particular, the skeletonmay totally miss information about paths that can be traversedin some of the JSON objects.3 TYPES IN PROGRAMMING LANGUAGESUnlike XML, which found no space as a first class citizen in pro-gramming languages, with the obvious and notable exception ofXQuery, JSON has been designed starting from the object lan-guage of an existing scripting language. Therefore, given its wideuse in web and mainstream application development, JSON sup-port has been introduced in several strongly typed programmingand/or scripting languages.To directly and naturally manage JSON data a programminglanguage should incorporate the ability to express record types,sequence types, and union types. While record and sequencetypes can be easily found inmany programming languages, uniontypes are quite rare and usually confined to functional languagesonly.In this part of the tutorial we will discuss the support for JSONobjects inside the type systems of TypeScript [9] and Swift [8].TypeScript is a typed extension of JavaScript, while Swift is anApple-backed programming language that is rapidly becomingthe language of choice for developing applications in the Appleecosystem (iOS + macOS). These languages show similar features,but also very significant differences in the treatment of JSONobjects.We will also compare the features offered by these languageswith those of the schema languages we presented in the secondpart of the tutorial.4 SCHEMA TOOLSIn this part of the tutorial we will present several schema-relatedtools for JSON data. We will first discuss existing approachesfor inferring a schema starting from a dataset and then move toparsing tools that are able to exploit dynamic type informationto speed-up data parsing.4.1 Schema InferenceSeveral schema inference approaches for JSON data collectionshave been proposed in the past. In [10–12] authors describe adistributed, parametric schema inference approach capable ofinferring schemas at different levels of abstraction. In the contextof Spark, the Spark Dataframe schema extraction [7] is a veryinteresting tool for the automated extraction of a schema fromJSON datasets; this tool infers schemas in a distributed fashion,but, unlike the technique described in [10–12], its inference ap-proach is quite imprecise, since the type language lacks uniontypes, and the inference algorithm resorts to Str on stronglyheterogeneous collections of data. Other systems, like Jaql [13],exploit schema information for inferring the output schema of aquery, but still require an externally supplied schema for inputdata, and perform output schema inference only locally on asingle machine.There are also a few inference tools for data stored in NoSQLsystems and RDBMSs. Indeed, in the context of NoSQL systems(e.g. MongoDB), recent efforts have been dedicated to the problemof implementing tools for JSON schema inference. A JavaScriptlibrary for JSON, called mongodb-schema, is presented in [22].This tool analyzes JSON objects pulled from MongoDB, and pro-cesses them in a streaming fashion; it is able to return quiteconcise schemas, but it cannot infer information describing fieldcorrelation. Studio 3T [19] is a commercial front-end for Mon-goDB that offers a very simple schema inference and analysisfeature, but it is not able to merge similar types, and the resultingschemas can have a huge size, which is comparable to that of theinput data. In [23], a python-based tool is described, called Skin-fer, which infers JSON Schemas from a collection of JSON objects.Skinfer exploits two different functions for inferring a schemafrom an object and for merging two schemas; schema merging islimited to record types only, and cannot be recursively applied toobjects nested inside arrays. Couchbase, finally, is endowed witha schema discovery module which classifies the objects of a JSONcollection based on both structural and semantic information [3].This module is meant to facilitate query formulation and selectrelevant indexes for optimizing query workloads.When moving to RDBMSs, in [16] Abadi and al. deal withthe problem of automatically transforming denormalised, nestedJSON data into normalised relational data that can be stored ina RDBMS; this is achieved by means of a schema generationalgorithm that learns the normalised, relational schema from data.This approach ignores the original structure of the JSON inputdataset and, instead, depends on patterns in the attribute datavalues (functional dependencies) to guide its schema generation.4384.2 ParsingThere are a few novel parsing tools for JSON data that take intoaccount dynamic type information for improving the efficiencyof the applications relying on them.In a recent work [20], Li et al. present streaming techniques forefficiently parsing and importing JSON data for analytics tasks;these techniques are then used in a novel C++ JSON parser, calledMison, that exploits AVX instructions to speed up data parsingand discarding unused objects. To this end, it infers structuralinformation of data on the fly in order to detect and prune partsof the data that are not needed by a given analytics task.In [14], Bonetta and Brantner present Fad.js, a speculative,JIT-based JSON encoder and decoder designed for the OracleGraal.js JavaScript runtime. It exploits data access patterns tooptimize both encoding and decoding: indeed, Fad.js relies onthe assumption that most applications never use all the fieldsof input objects, and, for instance, skips unneeded object fieldsduring JSON object parsing.5 FUTURE OPPORTUNITIESWe finally discuss several open challenges and opportunitiesrelated to JSON schemas, including the following ones.Schema Inference and ML. While all schema inference ap-proaches covered in the previous part of the tutorial are basedon traditional techniques, a recent work by Gallinucci et al. [17]shows the potential benefits of ML approaches in schema infer-ence; furthermore, ML-based inference techniques have alreadybeen used for non-JSON data, as shown by Halevy et al. in [18].Hence, a promising research direction is to understand how thesemethods can be efficiently applied to large collections of dataand whether they can overcome some limitations of previousapproaches.Schema-Based Data Translation. While JSON is very frequentlyused for exchanging and publishing data, it is hardly used asinternal data format in Big Data management tools, that, instead,usually rely on formats like Avro [1] and Parquet [2]. When inputdatasets are heterogeneous, schemas can improve the efficiencyand the effectiveness of data format conversion. Therefore, amajor opportunity is to design schema-aware data translationalgorithms that are driven by schema information and use it toimprove the quality of the translation.6 INTENDED AUDIENCE AND COVERAGEOur goal is to present a coherent starting point for EDBT atten-dees who are interested in understanding the foundations andapplications of schemas and types for JSON data processing. Wewill not assume any background in JSON schema languages, butwill introduce them starting from the roots, giving broad cover-age of many of the key ideas, making it appropriate for graduatestudents seeking new areas to study and researchers active inthe field alike.7 BIOGRAPHICAL SKETCHESMohamed-Amine Baazizi (Ph.D.) is an assistant professor atSorbonne Université. He received his PhD from Université ofParis-Sud and completed his postdoctoral studies in TélécomParistech. His research focuses on exploiting schema informationfor optimizing the processing of semi-structured data.Dario Colazzo (Ph.D.) is Full Professor in Computer Scienceat LAMSADE - Université Paris-Dauphine. He received his PhDfrom Università di Pisa, and he completed his postdoctoral stud-ies at Università di Venezia and Université Paris Sud. His mainresearch activities focus on static analysis techniques for largescale data management.Giorgio Ghelli (Ph.D.) is Full Professor in Computer Science,at Università di Pisa. He was Visiting Professor at École NormaleSupérieure Paris, at Microsoft Research Center, Cambridge (UK),and at Microsoft Co. (Redmond, USA), member of the W3C XMLQuery Working Group, member of the board of the EAPLS. Heworked on database programming languages and type systemsfor these languages, especially in the fields of object oriented andXML data models.Carlo Sartiani (Ph.D.) is an assistant professor at Universitàdella Basilicata. He received his PhD from Università di Pisa,and he completed his postdoctoral studies at Università di Pisa.He worked on database programming languages and data in-tegration systems, and his current research activities focus onsemistructured and big data.REFERENCES[1] Apache Avro. https://avro.apache.org.[2] Apache Parquet. https://parquet.apache.org.[3] Couchbase auto-schema discovery. https://blog.couchbase.com/auto-schema-discovery/.[4] JSON Schema language. http://json-schema.org.[5] JSound schema definition language. http://www.jsoniq.org/docs/JSound/html-single/index.html.[6] Object schema description language and validator for JavaScript objects.https://github.com/hapijs/joi.[7] Spark Dataframe. https://spark.apache.org/docs/latest/sql-programming-guide.html.[8] Swift. https://swift.org.[9] TypeScript. https://www.typescriptlang.org.[10] MohamedAmine Baazizi, HoussemBen Lahmar, Dario Colazzo, Giorgio Ghelli,and Carlo Sartiani. 2017. Schema Inference for Massive JSON Datasets. InEDBT ’17.[11] Mohamed Amine Baazizi, Dario Colazzo, Giorgio Ghelli, and Carlo Sartiani.2017. Counting types for massive JSON datasets. In Proceedings of The 16thInternational Symposium on Database Programming Languages, DBPL 2017,Munich, Germany, September 1, 2017. 9:1–9:12.[12] Mohamed-Amine Baazizi, Dario Colazzo, Giorgio Ghelli, and Carlo Sartiani.2019. Parametric schema inference for massive JSON datasets. The VLDBJournal (2019). https://doi.org/10.1007/s00778-018-0532-7[13] Kevin S. Beyer, Vuk Ercegovac, Rainer Gemulla, Andrey Balmin, Mohamed Y.Eltabakh, Carl-Christian Kanne, Fatma Özcan, and Eugene J. Shekita. 2011.Jaql: A Scripting Language for Large Scale Semistructured Data Analysis.PVLDB 4, 12 (2011), 1272–1283.[14] Daniele Bonetta and Matthias Brantner. 2017. FAD.js: Fast JSON Data AccessUsing JIT-based Speculative Optimizations. PVLDB 10, 12 (2017), 1778–1789.http://www.vldb.org/pvldb/vol10/p1778-bonetta.pdf[15] Pierre Bourhis, Juan L. Reutter, Fernando Suárez, and Domagoj Vrgoc. 2017.JSON: Data model, Query languages and Schema specification. In PODS ’17.123–135.[16] Michael DiScala and Daniel J. Abadi. 2016. Automatic Generation of Nor-malized Relational Schemas from Nested Key-Value Data. In SIGMOD ’16.295–310.[17] Enrico Gallinucci, Matteo Golfarelli, and Stefano Rizzi. 2018. Schema profilingof document-oriented databases. Inf. Syst. 75 (2018), 13–25.[18] Alon Y. Halevy, Flip Korn, Natalya Fridman Noy, Christopher Olston, NeoklisPolyzotis, Sudip Roy, and Steven Euijong Whang. 2016. Goods: OrganizingGoogle’s Datasets. In Proceedings of the 2016 International Conference on Man-agement of Data, SIGMOD Conference 2016, San Francisco, CA, USA, June 26 -July 01, 2016. 795–806.[19] 3T Software Labs. 2017. Studio 3T. https://studio3t.com.[20] Yinan Li, Nikos R. Katsipoulakis, Badrish Chandramouli, Jonathan Goldstein,and Donald Kossmann. 2017. Mison: A Fast JSON Parser for Data Analytics.PVLDB 10, 10 (2017), 1118–1129.[21] Felipe Pezoa, Juan L. Reutter, Fernando Suarez, Martín Ugarte, and DomagojVrgoč. 2016. Foundations of JSON Schema. In WWW ’16. 263–273.[22] Peter Schmidt. 2017. mongodb-schema.https://github.com/mongodb-js/mongodb-schema.[23] scrapinghub. 2015. Skinfer. https://github.com/scrapinghub/skinfer.[24] Lanjun Wang, Shuo Zhang, Juwei Shi, Limei Jiao, Oktie Hassanzadeh, Jia Zou,and Chen Wangz. 2015. Schema Management for Document Stores. Proc.VLDB Endow. 8, 9 (May 2015), 922–933.439

https://hal.archives-ouvertes.fr/hal-02301878/file/EDBT_tutorial.pdf

Schemas and types for JSON data

Abstract

Similar works

Full text

Available Versions

Archive Ouverte en Sciences de l'Information et de la Communication

Archivio della Ricerca - Università di Pisa

Archivio della Ricerca - Università della Basilicata