Journal ArticleWe describe ShreX, a freely-available system for  shredding, loading and querying XML documents  in relational databases. ShreX supports all mapping  strategies proposed in the literature as well  as strategies available in commercial RDBMSs.  It provides generic (mapping-independent) functions  for loading shredded documents into relations  and for translating XML queries into SQL.  ShreX is portable and can be used with any relational  database backend

Du, Fang

Freire, Juliana

English

The University of Utah: J. Willard Marriott Digital Library

ShreX: Managing XML Documents in Relational DatabasesFang DuOGI/OHSUfangdu@cse.ogi.eduSihem Amer-YahiaAT&T Labs – Researchsihem@research.att.comJuliana FreireOGI/OHSUjuliana@cse.ogi.eduAbstractWe describe ShreX, a freely-available system forshredding, loading and querying XML documentsin relational databases. ShreX supports all map-ping strategies proposed in the literature as wellas strategies available in commercial RDBMSs.It provides generic (mapping-independent) func-tions for loading shredded documents into rela-tions and for translating XML queries into SQL.ShreX is portable and can be used with any rela-tional database backend.1 IntroductionAs applications manipulate an increasing volume of XMLdata, there is a growing need for reliable systems to storeand provide efficient access to these data. The use of rela-tional database systems for this purpose has attracted con-siderable interest with a view to leveraging their powerfuland reliable data management services.In order to store an XML document in a relationaldatabase, the tree-structure of the XML document mustfirst be mapped into an equivalent, flat, relational schema.XML documents are then shredded and loaded into themapped tables. Finally, at runtime, XML queries are trans-lated into SQL, submitted to the RDMBS, and the resultsare then translated into XML.There is a rich literature addressing the issue of stor-ing XML documents in relational backends. Several map-ping strategies (e.g., [3, 5, 6, 10, 9]) and query transla-tion algorithms (see [7] for a survey) have been proposed.In addition, support for XML storage is already availablein most commercial RDBMSs. Unfortunately, existingXML-to-relational mapping solutions suffer from severaldrawbacks. None of these solutions addresses all the stor-age problems in a single framework. For example, worksPermission to copy without fee all or part of this material is granted pro-vided that the copies are not made or distributed for direct commercialadvantage, the VLDB copyright notice and the title of the publication andits date appear, and notice is given that copying is by permission of theVery Large Data Base Endowment. To copy otherwise, or to republish,requires a fee and/or special permission from the Endowment.Proceedings of the 30th VLDB Conference,Toronto, Canada, 2004on mapping strategies often have little or no details aboutquery translation [7]. Although it has been shown thatthe efficiency of a mapping depends on the data and onthe requirements of the applications that use the data [3],many of the available mapping solutions hard-code map-ping choices; and whereas some of the solutions proposedby relational vendors do provide flexible mechanisms todefine mappings, they are proprietary, i.e., tied to one rela-tional backend, making the shredding and query translationalgorithms system- and mapping-dependent.ShreX (Shredding XML) is a freely available system1that addresses many of the limitations of existing map-ping systems. To the best of our knowledge, ShreX isthe first system to provide a comprehensive solution to therelational storage of XML data. In ShreX, an XML-to-relational mapping is specified through annotations overan XML Schema, making the mapping easy to define aswell as validate. By combining different annotations, awide range of mappings can be expressed, including allmapping strategies proposed in the literature as well asstrategies supported by database vendors. ShreX also pro-vides generic (mapping-independent) functions for docu-ment shredding and query translation. This is made possi-ble by an API which provides access to the mapping infor-mation.In what follows, we give an overview of the key featuresof ShreX. In Section 2, we describe the architecture of thesystem including the mapping interface, the shredder andthe query translator. The demonstration is described in Sec-tion 3. We review related work in Section 4 and concludein Section 5.2 The SystemThe main components of the ShreX system are shown inFigure 1. In ShreX, a mapping is defined by adding annota-tions to an XML Schema which indicate how elements andattributes should be stored in a relational database (e.g., ascolumns, as tables). The annotation processor parses anannotated XML Schema, checks the validity of the map-ping and creates the corresponding relational schema. Inaddition, the mapping information is made persistent in themapping repository. The document shredder accepts as in-1ShreX is available at http://www.cse.ogi.edu/˜ fangdu/shreX.put a document and uses the mapping API to access theinformation in the mapping repository to generate the tu-ples and populate the tables in the relational schema. Themapping repository is also accessed by the query transla-tor, which generates SQL queries from XML queries.RelationalDatabaseMappingRepositoryMapping APIValidating ParserAndShredderAnnotationProcessorQueryTranslatorAnnotatedXML SchemaXMLDocumentXML answersXML queryTuples SQLCREATE TABLEstatementsTuplesDefault RulesFigure 1: ShreX ArchitectureBelow, we give a brief overview of the mapping defini-tion framework, and of the various modules of ShreX. Thereader is referred to [1] for more details.2.1 Mapping Definition FrameworkA ShreX mapping is expressed by annotating an XMLSchema. This not only makes the mapping definitionportable, i.e., independent from the underlying relationaldatabase, but also expressive and extensible. Mappingspecifications also enable useful analyses, for example, toensure that a mapping is valid (Section 2.2).Annotations can be associated to attributes, elementsand groups in the input XML Schema. Their syntax corre-sponds to adding attributes from a namespace called shrexto a given XML Schema. The attributes supported by ShreXare shown in Table 1. Figure 2 illustrates the use of some ofthe annotations (shown in boldface). The sample schemadescribes information about shows, where a SHOW has aTITLE, a YEAR, zero or more REVIEWs, and zero or morealternative titles (AKAs).Mapping document structure. An important aspect ofa mapping is how it captures element identity, documentstructure and order. In ShreX, the choice of structure map-ping can be specified through the structurescheme at-tribute (see Table 1). For example, in Figure 2, the structurescheme selected for the document is Dewey2 (see annota-tion in the root element). Other supported schemes include:key-foreign-key for parent-child relationships and ordinalfor siblings (“KFO”) [3]; and interval encoding [8]. Theability to define multiple document structure schemes is afeature that is unique to ShreX.Outline, tablename, columnname, sqltype. Annotationsare also used to specify how individual elements and at-2http://www.oclc.org/dewey/about/about the ddc.htm.tributes in a document are represented in the relationalschema. Figure 3 shows the relational configuration forthe annotated schema of Figure 2. The annotation out-line=“true” in the element TITLE indicates it should bemapped to a separate table; and the annotation tablenamespecifies that this table should be named Showtitle. Theelement YEAR, on the other hand, has its outline attributeset to false, consequently it is inlined in the table corre-sponding to its parent element, SHOW. The annotations sql-type and columnname in the YEAR element specify that itshould be mapped to a column named Showyear and SQLtype NUMBER(4). Although not illustrated in the exam-ple, an element can also be mapped into a CLOB, using theannotation maptoclob.<element name="SHOW">shrex:structurescheme="Dewey" /><sequence><element name="TITLE" type="string"shrex:outline="true"shrex:tablename="Showtitle"/><element name="YEAR" type="integer"shrex:outline="false"shrex:columnname="Showyear"shrex:sqltype="NUMBER(4)"/><element name="REVIEW" type="ANYTYPE"minOccurs="0" maxOccurs="unbounded"shrex:edgemapping="true"/><element name="AKA" type="string"minOccurs="0" maxOccurs="unbounded"/></element>Figure 2: Annotated movie schemaTABLE SHOW( ID VARCHAR(128),Showyear NUMBER(4))TABLE Showtitle( ID VARCHAR(128),ParentID VARCHAR(128), TITLE VARCHAR(512))TABLE REVIEW( ParentID VARCHAR(128),source VARCHAR(128),ordinal VARCHAR(128),attrname VARCHAR(128),flag VARCHAR(128),value VARCHAR(128))TABLE AKA( ID VARCHAR(128),ParentID VARCHAR(128), AKA VARCHAR(512))Figure 3: Relational configuration for movie schemaMapping schemaless documents and mixing strategies.The use of annotated schemata in ShreX does not precludethe system from expressing generic (schemaless) map-pings. For example, in Figure 2, the annotation edgemap-ping=“true” in the element REVIEW indicates that REVIEWand its descendants are mapped using Edge mapping [6],i.e., a single table to store all the REVIEW elements and con-tents. This functionality is specially useful to map elementswhose structures are not known in advance, such as for ex-ample, elements of type ANYTYPE.Annotations naturally allow the definition of mappingsthat combine different mapping strategies. Note that in thisexample, part of the document is mapped using Edge, andpart is mapped using KFO.Transformation-based mappings. Additional mappingstrategies are supported by combining ShreX annotationswith the schema transformations proposed in [3]. For ex-ample, if repetition split is applied to AKA in the originalschema, i.e., AKA* → AKA?, AKA*, the first occurrence ofAKA could be inlined in the table SHOW:TABLE SHOW( ID VARCHAR(128),Showyear NUMBER(4),AKA VARCHAR(512))2.2 Annotation ProcessorThis module is in charge of parsing an annotated XMLSchema, checking the validity of a mapping, generating amapping repository, and producing the CREATE TABLEstatements necessary to construct the relational schema. Inorder to check the validity of a mapping, the annotationprocessor validates the input (annotated) schema against anXML Schema for annotations [1]. Validity checks includeverifying whether annotations are attached to the appropri-ate elements, and if table names are unique in the mappingdefinition. Additional checks, such as verifying whether amapping is lossless, are also possible.Writing an annotation for every element and attributedefinition in an XML Schema can be tedious, especially forlarge schemata. ShreX provides a set of default rules thatis used to complete mapping specifications. In fact, usingthese default rules, the system is able to automatically mapan XML Schema without any user input. It is worthy ofnote that users can add to or override these rules.2.3 Mapping Repository and APIMapping information is processed and stored in a database.By making this information persistent, ShreX avoids theneed to re-parse a mapping specification each time a doc-ument is loaded into the target database or that a queryneeds to be translated into SQL. ShreX provides an APIto the mapping repository that allows access to informa-tion such as, how elements and attributes are mapped(isInlined(ElemName|AttName)), which mapping is usedto capture document structure (getStructureScheme()),and which tables are available in the relational schema(getTableInfo( TableName)) (see [1] for details). ThisAPI allows users to write mapping-independent code whichis not tied to specific features of a particular mapping.The API also contains functions that expose informa-tion about the schema being mapped. These functions areuseful both during shredding and query translation. For ex-ample, in order to translate a descendent step //t, the querytranslator needs to determine all paths from the root to tagt – in ShreX, this can be done through a call to pathToTag.2.4 Document ShredderThe shredder is in charge of generating tuples, field valuesand CLOBs from an input document. It was designed to begeneric and independent from the mapping specification: ituses the mapping API to retrieve information about how aparticular element or attribute is mapped. Since mappingannotations are specified using attributes from a differentnamespace, the document shredder can validate the inputXML document against the annotated Schema. Tuples aregenerated while the document is parsed, using a standardXML parser. In our implementation, we use the SAX in-terface of Xerces [11], which is both efficient and scalable.For example, ShreX is able to shred and load a 1GB docu-ment into DB2 in less than 30 minutes. It is worth pointingout that even significantly smaller files cannot be loaded incommercial RDBMSs using their XML extensions. Con-sistently with what has been reported in [12], we werenot able to load documents larger than 10MB using DB2’sXML Extender.Users can set various parameters for the shredder, e.g.,target database system, login information, bulk loading op-tion. These parameters can be set either through the com-mand line or through a configuration file.2.5 Query TranslatorIn the current implementation, the query translator supportsa subset of XPath that includes child and descendant axes;position-based predicates [position()=n]; and simple pathpredicates, to SQL.3 Similar to the document shredder, thequery translator does not hard-code mapping choices, in-stead it uses the information provided by the mapping APIto dynamically decide how to perform the translation.3 Demonstration OverviewWe will demonstrate the various features of ShreX and itsutility for building applications that need to store and queryXML data in relational databases.Specifying mappings. We will show how different stor-age mapping strategies can be represented using our map-ping specification. Users will be able to define mappings bychoosing from a variety of XML schemata or creating theirown schema. Using ShreX, they will annotate a schema,validate the corresponding mapping, and create the cor-responding relational schemata. We will show how thisprocess is simplified by the ShreX graphical user interface(GUI), which allows users to browse and select tables andfields in the relational schema and visually see the corre-sponding XML elements and attributes, and vice versa.Shredding and loading. After a relational schema is cre-ated, users will be able to select a target RDBMS and in-struct the system to shred and load a document into the tar-get RDBMS. Shredding and loading can be done throughthe command-line, or from the GUI. Users can load theXML documents directly into the relational tables (i.e.,they can be bulk-loaded), or generate loading commandsand tuples.Querying. Users will be able to input XPath queries andsee the corresponding (translated) SQL queries as well ashave these queries executed against the relational backend.3An XQuery translator is currently under development, and will beavailable in the next version of ShreX.Annotation attributes Target Value Actionoutline attribute or element true, false If value is true, a relational table is created for theattribute or element. Otherwise, the attribute or el-ement is mapped to one or multiple columns in itscontaining table (i.e., inlined).tablename attribute, element orgroupstring The string is used as the table name.columnname attribute or element ofsimple typestring The string is used as the column name.sqltype attribute or element ofsimple typestring The string overrides the SQL type of a column.structurescheme root element KFO, Interval, Dewey Specifies structure mapping.edgemapping element true, false If value is true, the element and its descendants areshredded according to Edge mapping [6].maptoclob attribute or element true, false If value is true, the element or attribute is mapped toa CLOB column.Table 1: Annotation Attributes. Each row in the table contains an annotation attribute, its target (i.e., element, attribute,and group to which it applies), its possible values and its action depending on its value.4 Related WorkBourret et al [4] developed XML-DBMS, a generic tool forloading XML documents into relational tables. Althoughsimilar to ShreX in motivation, the mappings supported bythis tool are limited to the basic, shared, and hybrid tech-niques described in [10]. In addition, XML-DBMS has nosupport for query translation.MXM [2] has been proposed as a declarative mecha-nism to express XML-to-relational mappings. Our map-ping specification shares the flexibility of MXM while hav-ing the advantage of using an XML Schema syntax andproviding a comprehensive set of tools.Although XML support in commercial relational en-gines is improving rapidly, there is a wide variation in thesupported features. Some practical problems include pro-prietary solutions, lack of flexibility and scalability. To de-fine a storage strategy, the IBM DB2 XML Extender re-quires users to write a Document Access Definition specifi-cation; consequently, developers must learn a new languagein order to use DB2 (and only DB2) as a backend. Themapping facilities provided by Oracle 9iR2 are not flexibleenough to specify many useful strategies, for example, it isnot possible to specify that part of the data is to be storedusing a generic mapping such as Edge [6]. SQLServer’sOpenXML requires that documents be compiled into an in-ternal DOM representation, which greatly limits its scala-bility.5 ConclusionTo the best of our knowledge, ShreX is the first compre-hensive system for mapping, loading and querying XMLdocuments. ShreX has many novel features including theability to mix mapping strategies and to specify a documentstructure scheme. We designed ShreX to be modular andextensible. And by making the source code available, wehope ShreX will serve as a platform to develop and evaluatenew mapping strategies, query translation and optimizationalgorithms.Acknowledgments. The National Science Foundation par-tially supports Juliana Freire under grant EIA-0323604.References[1] S. Amer-Yahia, F. Du, and J. Freire. A generic and flexi-ble framework for mapping XML documents into relations.Technical report, OGI/OHSU, 2004.[2] S. Amer-Yahia and D. Srivastava. A mapping scheme andinterface for XML stores. In Proc. of WIDM, 2002.[3] P. Bohannon, J. Freire, P. Roy, and J. Sime´on. From XMLschema to relations: A cost-based approach to XML storage.In Proc. of ICDE, pages 64–75, 2002.[4] R. Bourret, C. Bornhvd, and A. P. Buchmann. A genericload/extract utility for data transfer between XML docu-ments and relational databases. In WECWIS, pages 134–143, 2000.[5] A. Deutsch, M. Fernandez, and D. Suciu. Storing semi-structured data with STORED. In Proc. of SIGMOD, pages431–442, 1999.[6] D. Florescu and D. Kossman. Storing and querying XMLdata using an RDMBS. IEEE Data Engineering Bulletin,22(3):27–34, 1999.[7] R. Krishnamurthy, R. Kaushik, and J. F. Naughton. XML-SQL query translation literature: The state of the art andopen problems. In Proc. XSym, 2003.[8] S. Paparizos and et al. Timber: A native system for queryingXML. In Proc. of SIGMOD, page 672, 2003. Demonstra-tion.[9] A. Schmidt, M. Kersten, M. Windhouwer, and F. Waas. Ef-ficient relational storage and retrieval of XML documents.In Proc. of WebDB, pages 47–52, 2000.[10] J. Shanmugasundaram, K. Tufte, G. He, C. Zhang, D. De-Witt, and J. Naughton. Relational databases for queryingXML documents: Limitations and opportunities. In Proc. ofVLDB, pages 302–314, 1999.[11] Xerces Java parser 1.4.3. http://xml.apache.org/xerces-j.[12] B. B. Yao, M. T. ¨Ozsu, and N. Khandelwal. XBench bench-mark and performance testing of XML DBMSs. In Proc. ofICDE, pages 621–632, 2004.

ShreX: managing XML documents in relational databases

https://collections.lib.utah.edu/dl_files/7d/c9/7dc9d0421ac0c9d203b1d0e61c874c6817faff14.pdf

ShreX: managing XML documents in relational databases

Abstract

Similar works

Full text

Available Versions

The University of Utah: J. Willard Marriott Digital Library