Relational query processors are probably the best understood (as well as the best engineered) query engines available today. Although carefully tuned to process instances of the relational model (tables of tuples), these processors can also provide a foundation for the evaluation of "alien" (non-relational) query languages: if a relational encoding of the alien data model and its associated query language is given, the RDBMS may act like a special-purpose processor for the new language

Boncz, Peter

Grust, Torsten

Keulen, Maurice van

Manegold, Stefan

Rittinger, Jan

Teubner, Jens

English

van Keulen, Maurice

University of Twente Research Information

Pathfinder: XQuery—The Relational WayPeter Boncz1 Torsten Grust2 Maurice van Keulen3Stefan Manegold1 Jan Rittinger1,4,† Jens Teubner21CWI Amsterdam, The Netherlands 2Technische Universität München, Germany{boncz,manegold}@cwi.nl {torsten.grust,jens.teubner}@in.tum.de3University of Twente, The Netherlands 4University of Konstanz, Germanym.vankeulen@utwente.nl rittinge@inf.uni-konstanz.de1 IntroductionRelational query processors are probably the best un-derstood (as well as the best engineered) query enginesavailable today. Although carefully tuned to processinstances of the relational model (tables of tuples),these processors can also provide a foundation for theevaluation of “alien” (non-relational) query languages:if a relational encoding of the alien data model and itsassociated query language is given, the RDBMS mayact like a special-purpose processor for the new lan-guage.This demonstration features our XQuery compilerPathfinder, the continuation of our earlier work on apurely relational XPath and XQuery processing stack[4, 6, 7] in which we developed relational encodings andprocessing strategies for the tree-shaped XML datamodel. The Pathfinder project is an exploration ofhow far we can push the idea of using mature RDBMStechnology to design and build a full-fledged XQueryimplementation. The demonstration will show thatthis line of research was and still is worth to be fol-lowed: based on the extensible relational database ker-nel MonetDB [2], Pathfinder provides highly efficientand scalable XQuery technology that scales beyond10 GB XML input instances on commodity hardware.Pathfinder requires only local extensions to the un-derlying DBMS’s kernel, such as the staircase join op-erator [7, 9]. A join recognition logic in our com-piler, as well as a careful consideration of order prop-erties of relational operators [3], allow for effective op-timizations that turn MonetDB into a highly efficientXQuery engine.Permission to copy without fee all or part of this material isgranted provided that the copies are not made or distributed fordirect commercial advantage, the VLDB copyright notice andthe title of the publication and its date appear, and notice isgiven that copying is by permission of the Very Large Data BaseEndowment. To copy otherwise, or to republish, requires a feeand/or special permission from the Endowment.Proceedings of the 31st VLDB Conference,Trondheim, Norway, 2005†This work was supported by the DFG Research Train-ing Group GK-1042 “Explorative Analysis and Visualization oflarge Information Spaces”.RDBMStree encoding (XPath accel.)XPath axes (staircase join)sequenceencodingloop-liftingXQueryFigure 1: XQuery development stack.We will briefly recite these ideas in the upcomingsection. Section 3 will provide experimental evidenceof the viability of our approach, before we sketch thesetup for this demonstration of Pathfinder.2 Pathfinder XQuery Engine OverviewTo bridge the apparent gap between the relational pro-cessing model and the XQuery data model, Pathfinderfollows the step-by-step development stack of Figure 1.Tree encoding. Relational XML processing starts offwith a suitable relational encoding for XML documenttrees. We based our system on the XPath Accelerator[4], a numbering scheme in which each node v is as-signed a pre-order rank pre(v) and a post-order rankpost(v). These numbers efficiently characterize XPathlocation steps as regions in a two-dimensional plane,e.g., v′ is a descendant of v ⇔ pre(v) < pre(v′) ∧post(v′) < post(v). Pathfinder employs an equivalentencoding variant that represents the location of a nodeas the 3-tuple 〈pre(v), size(v), level(v)〉 with the num-ber of nodes in the subtree below v, size(v), and v’sdistance from the tree root, level(v). These tuples aremaintained in a table with schema pre|size|level.XPath axes. This encoding scheme, turns XPathstep evaluation into a relational range selection.The selection predicate describes region queries onthe 〈pre(v), size(v), level(v)〉 space, where the regionqueried associates with the XPath axis [4]. Theserange selections are well supported by existing accessand index structures.Yet, the RDBMS gives away significant opportuni-ties for optimization since the system is unaware of the1322π column projection, renamingσ row selection.∪,\ disjoint union, differenceδ duplicate eliminationon, × equi-join, Cartesian product% row-numberingstaircase joinε, τ element/text node construction~ arithmetic/comparison operator ∗Table 1: The relational algebra used by Pathfinder.atomic literals document order (e1 << e2)sequences (e1, e2) node identity (e1 is e2)variables ($v) arithmetics (+, -, . . . )let $v := e1 return e2 comparisons (eq, lt, . . . )for $v in e1 return e2 Boolean operators (and, or, . . . )if e1 then e2 else e3 fn:doc(e)typeswitch clauses fn:root(e)element { e1 } { e2 } fn:data(e)text { e } fs:distinct-doc-order(e)e1 order by e2,...,en fn:count(e), fn:sum(e)XPath (e/α::ν) fn:empty(e)user defined functions fn:position(), fn:last()Table 2: XQuery dialect supported by Pathfinder. Ex-pressions may be composed arbitrarily; α denotes anXPath axis, ν a node test.isomorphism between the tree-shaped XML data andits relational encoding. Injecting such tree-awareness,however, is possible in terms of a special join opera-tor, staircase join , that helps to turn RDBMSs intohighly efficient XPath processors [7].Relational XQuery evaluation. Pathfinder com-piles XQuery expressions into a purely relational queryplan. This target algebra solely uses standard op-erators, available in off-the-shelf RDBMSs (Table 1).Note that the extra operators , ε, and τ are justshort-hands for efficient implementations of equivalentalgebraic expressions. A row-numbering operator % isprovided by many existing RDBMSs, e.g., in terms ofMonetDB’s mark operator, or the DENSE RANK() func-tion in SQL:1999. The algebra still suffices to expressthe XQuery dialect listed in Table 2, a subset that,e.g., suffices to express all XMark queries [10]. Fig-ure 5 in Section 3 depicts the query plan of a typicalXQuery FLWOR clause.Our query plans use a very explicit, “assembly-style” algebra. The exploitation of specific restric-tions which hold for this algebra (e.g., all joins areequi-joins, π does not imply duplicate elimination,all unions are disjoint) enables additional, quite far-reaching optimizations. All in all, this makes this par-ticular variant of relational algebra very efficiently im-plementable on any relational DBMS. Query plans canbecome quite large (XMark query Q8 [10], e.g., priorto optimization, compiles to a plan DAG of 120 oper-ators). This complexity may significantly be reducediter pos item1 1 101 2 20(a) (10,20) in s0.iter pos item1 1 102 1 20(b) $v in scope s1.iter pos item1 1 102 1 103 1 204 1 20(c) $v in scope s2.iter pos item1 1 1002 1 2003 1 1004 1 200(d) $w in scope s2.iter pos item1 1 1102 1 2103 1 1204 1 220(e) $v + $w in s2.inner outer1 12 13 24 2(f) map(s1,s2).iter pos item1 1 1101 2 2101 3 1201 4 220(g) Result (scope s0).Figure 3: Intermediate results in evaluation offor $v in (10,20), $w in (100,200) return $v + $w.by peep-hole style optimization [5], though.pos item1 52 "x"3 <a/>4 "x"Fig. 2: Sequenceencoding of (5,"x",<a/>,"x").Sequence encoding. The XQuerydata model is based on the sequence,an ordered (possibly heterogeneous)list of items. We implement sequenceorder by means of an explicit poscolumn, as depicted on the right.The polymorphic item column is ef-ficiently implemented with help ofMonetDB’s mposjoin operator.Loop lifting. A crucial part in XQuery evaluation isthe efficient implementation of its iteration primitive,the FLWOR clause. In [6], we developed a database style,bulk-oriented way of loop processing. We sketch thebasic idea here.The compilation from the iterative FLWOR clauses tobulk-oriented table manipulation is achieved throughloop-lifting : a column iter added to the encoding inFigure 2 accounts for the different bindings a variableassumes when it iterates over a sequence. To illustrate,Figure 3 shows some intermediate results for the rela-tional query plan that evaluatess0for $v in (10,20)s1{for $w in (100,200)s2 {return $v + $w .The loop-lifting concept makes any relational se-quence encoding depend on its associated iterationscope si. The initial expression (10,20) is evaluatedin the top-level scope s0, with a constant iter value of 1(Figure 3(a)). The for loop introduces new iter valuesfor each tuple to represent $v in s1 (Figure 3(b)): forexample, in the second iteration in scope s1, variable$v will be bound to the single item 20. The returnexpression in scope s2 is to be evaluated four times,represented by four different iter values for expressions$v, $w, and $v + $w (Figures 3(c)–(e)).The semantics of nested iteration implies that dur-ing the first two iterations of scope s2, scope s1 is stillin its first iteration, while for iterations 3 and 4 ofscope s2, s1 is in its second iteration. We capture this1323semantics by a relation map (Figure 3(f)) that relatesiter values between scopes s1 and s2.Back-mapping the encoding in Figure 3(e) to thetop-level scope finally forms the overall expression re-sult in Figure 3(g). Observe that our bulk-orientedway of evaluation is quite different from other XQueryengines, which in a sense only do nested loop, i.e.,recursive, processing. For details on the loop-liftingconcept, refer to [6].MonetDB. We designed Pathfinder as an XQuerycompiler : XQuery expressions are translated into re-lational query plans. Their execution on a databaseback-end evaluates the input expression by means ofa single algebraic query. A simple post-processor thenserializes the relational result to form a response interms of the XQuery data model.The compiler currently targets MonetDB, an ex-tensible relational database system, available in opensource [2]. Its main-memory optimized implementa-tion and features like virtual object identifiers (that,e.g., make the row-numbering operator % a no-cost op-erator) make MonetDB particularly suitable for ourobjective: scalable XQuery processing. The use ofalternative back-ends (e.g., SQL) is current work inprogress.3 Scalability and PerformanceOur implementation is meant to assess the viability ofusing relational back-ends for XQuery. RDBMSs areknown to scale well with increasing data volumes—aninevitable feature if XQuery systems are to supportlarge amounts of XML data.We used the XML generator XMLgen from theXMark benchmark set [10] to create XML instancesof sizes from 11 MB to 11 GB (50,000 to 500 millionnodes, respectively). Documents were loaded into ourMonetDB back-end to run the 20 XMark queries.3.1 Storage OverheadPathfinder stores the structural part of XML docu-ments in the pre|size|level encoding, as sketched in Sec-tion 2. A prop column stores surrogates of XML nodeproperties. Actual property values (tag names, textnode content, etc.) are maintained in separate prop-erty BATs and kept unique therein. These node prop-erties are identified by their surrogates, where nodeswith identical properties share the same surrogate.This surrogate sharing not only avoids expensivestring comparisons at query execution time, but alsoreduces space consumption on secondary storage. Forthe XMark benchmark set, disk space requirementsrange between 147 % (11 MB instance) and 125 %(110 MB instance) of the original XML document.11For larger instances, XMark documents tend to degrade:more and more duplicate text nodes bring down disk space usagebelow 80 % of the original XML instance.11 MB 110 MB 1.1 GB 11 GBQ X-Hive PF X-Hive PF X-Hive PF PF1 0.37 0.05 1.29 0.17 9.9 1.2 132 0.45 0.07 1.75 0.30 33.0 2.4 253 0.65 0.28 5.66 1.51 25.1 12.5 1264 0.10 0.08 1.00 0.45 18.1 3.8 365 0.13 0.05 0.90 0.16 20.7 1.2 116 1.07 0.02 10.17 0.05 178.1 0.3 37 1.57 0.03 24.84 0.07 278.4 0.4 48 0.85 0.14 3.51 0.75 49.1 10.4 2089 32.25 0.20 12280.66 0.87 DNF 12.9 28910 5.28 0.80 442.37 5.31 DNF 55.0 188211 98.91 0.18 19927.29 3.48 DNF 960.9 DNF12 23.39 0.14 5100.19 1.66 DNF 431.3 DNF13 0.10 0.07 1.03 0.22 12.9 1.3 1314 0.72 0.26 11.16 2.20 110.2 21.3 646315 0.03 0.09 0.49 0.28 10.6 1.7 1616 0.03 0.11 0.52 0.26 10.9 1.8 1817 0.09 0.07 0.85 0.30 11.8 2.8 2618 0.08 0.04 0.64 0.13 14.8 0.9 919 0.67 0.11 12.15 0.55 254.5 5.3 8820 0.11 0.24 1.40 0.62 24.6 4.9 50Table 3: Query evaluation times (in seconds) for dif-ferent XMark instance sizes (scale factors 0.1 to 100).3.2 Experimental SetupWe tested Pathfinder’s performance on a 1.6 GHzAMD Opteron system, equipped with 8 GB RAM. Forcomparison, we installed X-Hive/DB on the same sys-tem [8]. X-Hive appears to be one of the faster XMLdatabase systems and copes with large XML instancesquite well. We tuned the X-Hive database with valueindices on the buyer/@person and profile/@incomepaths. These indices significantly reduced executiontimes of some computationally intensive XMark joinqueries (e.g., Q8).3.3 Pathfinder PerformanceThe execution times, shown in Table 3, confirm thestrengths of our purely relational approach to XQueryevaluation. Pathfinder handles simple path queries(Q1–Q5, Q13–Q20) 2 to 20 times faster than X-Hive.If paths include recursive axes, we benefit from ourstaircase join implementation that outperforms X-Hiveby two orders of magnitude (Q6 and Q7).It is not surprising that the join queries (Q8–Q12)benefit most from our relational back-end. While X-Hive almost competes with Pathfinder for the simplejoin query Q8, its performance strongly degrades if in-termediate query results are to be joined. Pathfindercompiles these queries into join plans [3] and takes ad-vantage of efficient join implementations in our back-end. It is thus able to execute queries in reasonabletime which we were not able to run on our X-Hiveinstallation (Q9–Q12).3.4 ScalabilityTo assess the scalability of our approach, we printedXMark execution times normalized to the elapsedtimes for the 110 MB XML instance. As shown in Fig-ure 4, Pathfinder scales linearly for most of the queries.Even the join queries (Q8–Q10) are of almost linearcomplexity.1324 0.01 0.1 1 10 100 1000Q20Q19Q18Q17Q16Q15Q14Q13Q12Q11Q10Q9Q8Q7Q6Q5Q4Q3Q2Q1normalized execution times11 MB 110 MB 1.1 GB 11 GBFigure 4: Pathfinder scalability: Execution times forthe XMark benchmark set, normalized to timings forthe 110 MB instance.The only two exceptions are queries Q11/Q12.Their quadratic scaling stems from a theta-join (pred-icate >) that produces intermediate results of about120 k, 12 M, and 1.2 G tuples for the 11 MB, 110 MB,and 1.1 GB instances, respectively. Note that this con-cerns the query result, whose computation cannot beavoided. Thus, any XQuery implementation will facethis complexity.4 Demonstration SetupOur system operates in a front-end/back-end fashion.The front-end, the Pathfinder compiler, parses incom-ing XQuery expressions and translates them into a re-lational algebra expression tree, represented in termsof a MIL2 program. The code is shipped to a Mon-etDB server that then executes the entire query andserializes the encoding of the result.The demonstration features both of these compo-nents, with various interesting hooks to trace the exe-cution of XQuery expressions on our relational back-end. Our system implements the W3C October 2004working drafts [1], including schema import, static typ-ing, and full axis features.The compiler provides facilities to look “under thehood” of relational XQuery compilation. An outputof type-annotated XQuery Core expression equivalentsdemonstrates optimization techniques applied to theincoming XQuery code before compiling it into rela-tional algebra. A graphical output of relational queryplans at different compilation stages (much like theplan in Figure 5) illustrates the loop-lifting concept.Relational plans may be traced to reveal the resultcomputed for any subexpression.Our demonstration system will be pre-loaded withXMark [10] instances of different sizes up to 1.1 GB.The query texts for the 20 benchmark queries will beready to run, but users may as well state their own adhoc queries.The Pathfinder compiler is available in open sourceas part of the MonetDB/XQuery implementation viahttp://pathfinder-xquery.org/.2MIL stands for MonetDB Interpreter Languageπiter:outer,pos:pos1,item%pos1:iter,pos/outeroniter=innerπiter,pos,item:resCCCCC⊕res:(item,item1)oniter=iter1πiter1:iter,item1:itemCCCCC×CCCCpos item1 100CCCπiter:inner{{{{{{{{{{×{{{{pos1{{{πiter:inner,itemCCCC%inner:iter,positer pos item1 1 101 2 20mmmmmmmmmπinner,outer:iterVVVVVVVVVVFigure 5: Relational query plan that evaluates thequery for $v in (10,20) return $v + 100.References[1] S. Boag, D. Chamberlin, M. F. Fernández, D. Flo-rescu, J. Robie, and J. Siméon. XQuery 1.0: An XMLQuery Language. World Wide Web Consortium, Oc-tober 2004. http://www.w3.org/TR/xquery/.[2] P. Boncz. Monet: A Next-Generation DBMS KernelFor Query-Intensive Applications. PhD thesis, Uni-versity of Amsterdam, The Netherlands, May 2002.[3] P. Boncz, T. Grust, S. Manegold, J. Rittinger, andJ. Teubner. Pathfinder: Relational XQuery OverMulti-Gigabyte XML Inputs In Interactive Time.Technical Report INS-E0503, CWI, Amsterdam, TheNetherlands, March 2005.[4] T. Grust. Accelerating XPath Location Steps. InProc. of the 21st Int’l ACM SIGMOD Conf. on Man-agement of Data, pages 109–120, Madison, Wisconsin,USA, June 2002.[5] T. Grust. Purely Relational FLWORs. In Proc. ofthe ACM SIGMOD/PODS 2nd Int’l Workshop onXQuery Implementation, Experience and Perspectives(XIME-P), Baltimore, MD, USA, June 2005.[6] T. Grust, S. Sakr, and J. Teubner. XQuery on SQLHosts. In Proc. of the 30th Int’l Conference on VeryLarge Data Bases, Toronto, Canada, August 2004.[7] T. Grust, M. van Keulen, and J. Teubner. StaircaseJoin: Teach a Relational DBMS to Watch its AxisSteps. In Proc. of the 29th Int’l Conference on VeryLarge Databases, Berlin, Germany, September 2003.[8] X-Hive/DB. http://www.x-hive.com/.[9] S. Mayer, T. Grust, M. van Keulen, and J. Teub-ner. An Injection of Tree Awareness: Adding StaircaseJoin to PostgreSQL. In Proc. of the 30th Int’l Con-ference on Very Large Data Bases, pages 1305–1308,Toronto, Canada, August 2004.[10] A. Schmidt, F. Waas, M. Kersten, M. J. Carey,I. Manolescu, and R. Busse. XMark: A Benchmarkfor XML Data Management. In Proc. of the 28th Int’lConference on Very Large Databases, pages 974–985,Hong Kong, China, August 2002.1325

Pathfinder: XQuery - The Relational Way

Boncz, P.A. (Peter)

Grust, T.

Keulen, M. van

Manegold, S. (Stefan)

Rittinger, J.

Teubner, J. (Jens)

CWI's Institutional Repository

Pathfinder: XQuery—The Relational WayPeter Boncz1 Torsten Grust2 Maurice van Keulen3Stefan Manegold1 Jan Rittinger1,4,† Jens Teubner21CWI Amsterdam, The Netherlands 2Technische Universita¨t Mu¨nchen, Germany{boncz,manegold}@cwi.nl {torsten.grust,jens.teubner}@in.tum.de3University of Twente, The Netherlands 4University of Konstanz, Germanym.vankeulen@utwente.nl rittinge@inf.uni-konstanz.de1 IntroductionRelational query processors are probably the best un-derstood (as well as the best engineered) query enginesavailable today. Although carefully tuned to processinstances of the relational model (tables of tuples),these processors can also provide a foundation for theevaluation of “alien” (non-relational) query languages:if a relational encoding of the alien data model and itsassociated query language is given, the RDBMS mayact like a special-purpose processor for the new lan-guage.This demonstration features our XQuery compilerPathfinder, the continuation of our earlier work on apurely relational XPath and XQuery processing stack[4, 6, 7] in which we developed relational encodings andprocessing strategies for the tree-shaped XML datamodel. The Pathfinder project is an exploration ofhow far we can push the idea of using mature RDBMStechnology to design and build a full-fledged XQueryimplementation. The demonstration will show thatthis line of research was and still is worth to be fol-lowed: based on the extensible relational database ker-nel MonetDB [2], Pathfinder provides highly efficientand scalable XQuery technology that scales beyond10 GB XML input instances on commodity hardware.Pathfinder requires only local extensions to the un-derlying DBMS’s kernel, such as the staircase join op-erator [7, 9]. A join recognition logic in our com-piler, as well as a careful consideration of order prop-erties of relational operators [3], allow for effective op-timizations that turn MonetDB into a highly efficientXQuery engine.Permission to copy without fee all or part of this material isgranted provided that the copies are not made or distributed fordirect commercial advantage, the VLDB copyright notice andthe title of the publication and its date appear, and notice isgiven that copying is by permission of the Very Large Data BaseEndowment. To copy otherwise, or to republish, requires a feeand/or special permission from the Endowment.Proceedings of the 31st VLDB Conference,Trondheim, Norway, 2005†This work was supported by the DFG Research Train-ing Group GK-1042 “Explorative Analysis and Visualization oflarge Information Spaces”.RDBMStree encoding (XPath accel.)XPath axes (staircase join)sequenceencodingloop-liftingXQueryFigure 1: XQuery development stack.We will briefly recite these ideas in the upcomingsection. Section 3 will provide experimental evidenceof the viability of our approach, before we sketch thesetup for this demonstration of Pathfinder.2 Pathfinder XQuery Engine OverviewTo bridge the apparent gap between the relational pro-cessing model and the XQuery data model, Pathfinderfollows the step-by-step development stack of Figure 1.Tree encoding. Relational XML processing starts offwith a suitable relational encoding for XML documenttrees. We based our system on the XPath Accelerator[4], a numbering scheme in which each node v is as-signed a pre-order rank pre(v) and a post-order rankpost(v). These numbers efficiently characterize XPathlocation steps as regions in a two-dimensional plane,e.g., v′ is a descendant of v ⇔ pre(v) < pre(v′) ∧post(v′) < post(v). Pathfinder employs an equivalentencoding variant that represents the location of a nodeas the 3-tuple 〈pre(v), size(v), level(v)〉 with the num-ber of nodes in the subtree below v, size(v), and v’sdistance from the tree root, level(v). These tuples aremaintained in a table with schema pre|size|level.XPath axes. This encoding scheme, turns XPathstep evaluation into a relational range selection.The selection predicate describes region queries onthe 〈pre(v), size(v), level(v)〉 space, where the regionqueried associates with the XPath axis [4]. Theserange selections are well supported by existing accessand index structures.Yet, the RDBMS gives away significant opportuni-ties for optimization since the system is unaware of thepi column projection, renamingσ row selection.∪,\ disjoint union, differenceδ duplicate eliminationon, × equi-join, Cartesian product% row-numberingstaircase joinε, τ element/text node construction~ arithmetic/comparison operator ∗Table 1: The relational algebra used by Pathfinder.atomic literals document order (e1 << e2)sequences (e1, e2) node identity (e1 is e2)variables ($v) arithmetics (+, -, . . . )let $v := e1 return e2 comparisons (eq, lt, . . . )for $v in e1 return e2 Boolean operators (and, or, . . . )if e1 then e2 else e3 fn:doc(e)typeswitch clauses fn:root(e)element { e1 } { e2 } fn:data(e)text { e } fs:distinct-doc-order(e)e1 order by e2,...,en fn:count(e), fn:sum(e)XPath (e/α::ν) fn:empty(e)user defined functions fn:position(), fn:last()Table 2: XQuery dialect supported by Pathfinder. Ex-pressions may be composed arbitrarily; α denotes anXPath axis, ν a node test.isomorphism between the tree-shaped XML data andits relational encoding. Injecting such tree-awareness,however, is possible in terms of a special join opera-tor, staircase join , that helps to turn RDBMSs intohighly efficient XPath processors [7].Relational XQuery evaluation. Pathfinder com-piles XQuery expressions into a purely relational queryplan. This target algebra solely uses standard op-erators, available in off-the-shelf RDBMSs (Table 1).Note that the extra operators , ε, and τ are justshort-hands for efficient implementations of equivalentalgebraic expressions. A row-numbering operator % isprovided by many existing RDBMSs, e.g., in terms ofMonetDB’s mark operator, or the DENSE RANK() func-tion in SQL:1999. The algebra still suffices to expressthe XQuery dialect listed in Table 2, a subset that,e.g., suffices to express all XMark queries [10]. Fig-ure 5 in Section 3 depicts the query plan of a typicalXQuery FLWOR clause.Our query plans use a very explicit, “assembly-style” algebra. The exploitation of specific restric-tions which hold for this algebra (e.g., all joins areequi-joins, pi does not imply duplicate elimination,all unions are disjoint) enables additional, quite far-reaching optimizations. All in all, this makes this par-ticular variant of relational algebra very efficiently im-plementable on any relational DBMS. Query plans canbecome quite large (XMark query Q8 [10], e.g., priorto optimization, compiles to a plan DAG of 120 oper-ators). This complexity may significantly be reducediter pos item1 1 101 2 20(a) (10,20) in s0.iter pos item1 1 102 1 20(b) $v in scope s1.iter pos item1 1 102 1 103 1 204 1 20(c) $v in scope s2.iter pos item1 1 1002 1 2003 1 1004 1 200(d) $w in scope s2.iter pos item1 1 1102 1 2103 1 1204 1 220(e) $v + $w in s2.inner outer1 12 13 24 2(f) map(s1,s2).iter pos item1 1 1101 2 2101 3 1201 4 220(g) Result (scope s0).Figure 3: Intermediate results in evaluation offor $v in (10,20), $w in (100,200) return $v + $w.by peep-hole style optimization [5], though.pos item1 52 "x"3 <a/>4 "x"Fig. 2: Sequenceencoding of (5,"x",<a/>,"x").Sequence encoding. The XQuerydata model is based on the sequence,an ordered (possibly heterogeneous)list of items. We implement sequenceorder by means of an explicit poscolumn, as depicted on the right.The polymorphic item column is ef-ficiently implemented with help ofMonetDB’s mposjoin operator.Loop lifting. A crucial part in XQuery evaluation isthe efficient implementation of its iteration primitive,the FLWOR clause. In [6], we developed a database style,bulk-oriented way of loop processing. We sketch thebasic idea here.The compilation from the iterative FLWOR clauses tobulk-oriented table manipulation is achieved throughloop-lifting : a column iter added to the encoding inFigure 2 accounts for the different bindings a variableassumes when it iterates over a sequence. To illustrate,Figure 3 shows some intermediate results for the rela-tional query plan that evaluatess0for $v in (10,20)s1{for $w in (100,200)s2 {return $v + $w .The loop-lifting concept makes any relational se-quence encoding depend on its associated iterationscope si. The initial expression (10,20) is evaluatedin the top-level scope s0, with a constant iter value of 1(Figure 3(a)). The for loop introduces new iter valuesfor each tuple to represent $v in s1 (Figure 3(b)): forexample, in the second iteration in scope s1, variable$v will be bound to the single item 20. The returnexpression in scope s2 is to be evaluated four times,represented by four different iter values for expressions$v, $w, and $v + $w (Figures 3(c)–(e)).The semantics of nested iteration implies that dur-ing the first two iterations of scope s2, scope s1 is stillin its first iteration, while for iterations 3 and 4 ofscope s2, s1 is in its second iteration. We capture thissemantics by a relation map (Figure 3(f)) that relatesiter values between scopes s1 and s2.Back-mapping the encoding in Figure 3(e) to thetop-level scope finally forms the overall expression re-sult in Figure 3(g). Observe that our bulk-orientedway of evaluation is quite different from other XQueryengines, which in a sense only do nested loop, i.e.,recursive, processing. For details on the loop-liftingconcept, refer to [6].MonetDB. We designed Pathfinder as an XQuerycompiler : XQuery expressions are translated into re-lational query plans. Their execution on a databaseback-end evaluates the input expression by means ofa single algebraic query. A simple post-processor thenserializes the relational result to form a response interms of the XQuery data model.The compiler currently targets MonetDB, an ex-tensible relational database system, available in opensource [2]. Its main-memory optimized implementa-tion and features like virtual object identifiers (that,e.g., make the row-numbering operator % a no-cost op-erator) make MonetDB particularly suitable for ourobjective: scalable XQuery processing. The use ofalternative back-ends (e.g., SQL) is current work inprogress.3 Scalability and PerformanceOur implementation is meant to assess the viability ofusing relational back-ends for XQuery. RDBMSs areknown to scale well with increasing data volumes—aninevitable feature if XQuery systems are to supportlarge amounts of XML data.We used the XML generator XMLgen from theXMark benchmark set [10] to create XML instancesof sizes from 11 MB to 11 GB (50,000 to 500 millionnodes, respectively). Documents were loaded into ourMonetDB back-end to run the 20 XMark queries.3.1 Storage OverheadPathfinder stores the structural part of XML docu-ments in the pre|size|level encoding, as sketched in Sec-tion 2. A prop column stores surrogates of XML nodeproperties. Actual property values (tag names, textnode content, etc.) are maintained in separate prop-erty BATs and kept unique therein. These node prop-erties are identified by their surrogates, where nodeswith identical properties share the same surrogate.This surrogate sharing not only avoids expensivestring comparisons at query execution time, but alsoreduces space consumption on secondary storage. Forthe XMark benchmark set, disk space requirementsrange between 147 % (11 MB instance) and 125 %(110 MB instance) of the original XML document.11For larger instances, XMark documents tend to degrade:more and more duplicate text nodes bring down disk space usagebelow 80 % of the original XML instance.11 MB 110 MB 1.1 GB 11 GBQ X-Hive PF X-Hive PF X-Hive PF PF1 0.37 0.05 1.29 0.17 9.9 1.2 132 0.45 0.07 1.75 0.30 33.0 2.4 253 0.65 0.28 5.66 1.51 25.1 12.5 1264 0.10 0.08 1.00 0.45 18.1 3.8 365 0.13 0.05 0.90 0.16 20.7 1.2 116 1.07 0.02 10.17 0.05 178.1 0.3 37 1.57 0.03 24.84 0.07 278.4 0.4 48 0.85 0.14 3.51 0.75 49.1 10.4 2089 32.25 0.20 12280.66 0.87 DNF 12.9 28910 5.28 0.80 442.37 5.31 DNF 55.0 188211 98.91 0.18 19927.29 3.48 DNF 960.9 DNF12 23.39 0.14 5100.19 1.66 DNF 431.3 DNF13 0.10 0.07 1.03 0.22 12.9 1.3 1314 0.72 0.26 11.16 2.20 110.2 21.3 646315 0.03 0.09 0.49 0.28 10.6 1.7 1616 0.03 0.11 0.52 0.26 10.9 1.8 1817 0.09 0.07 0.85 0.30 11.8 2.8 2618 0.08 0.04 0.64 0.13 14.8 0.9 919 0.67 0.11 12.15 0.55 254.5 5.3 8820 0.11 0.24 1.40 0.62 24.6 4.9 50Table 3: Query evaluation times (in seconds) for dif-ferent XMark instance sizes (scale factors 0.1 to 100).3.2 Experimental SetupWe tested Pathfinder’s performance on a 1.6 GHzAMD Opteron system, equipped with 8 GB RAM. Forcomparison, we installed X-Hive/DB on the same sys-tem [8]. X-Hive appears to be one of the faster XMLdatabase systems and copes with large XML instancesquite well. We tuned the X-Hive database with valueindices on the buyer/@person and profile/@incomepaths. These indices significantly reduced executiontimes of some computationally intensive XMark joinqueries (e.g., Q8).3.3 Pathfinder PerformanceThe execution times, shown in Table 3, confirm thestrengths of our purely relational approach to XQueryevaluation. Pathfinder handles simple path queries(Q1–Q5, Q13–Q20) 2 to 20 times faster than X-Hive.If paths include recursive axes, we benefit from ourstaircase join implementation that outperforms X-Hiveby two orders of magnitude (Q6 and Q7).It is not surprising that the join queries (Q8–Q12)benefit most from our relational back-end. While X-Hive almost competes with Pathfinder for the simplejoin query Q8, its performance strongly degrades if in-termediate query results are to be joined. Pathfindercompiles these queries into join plans [3] and takes ad-vantage of efficient join implementations in our back-end. It is thus able to execute queries in reasonabletime which we were not able to run on our X-Hiveinstallation (Q9–Q12).3.4 ScalabilityTo assess the scalability of our approach, we printedXMark execution times normalized to the elapsedtimes for the 110 MB XML instance. As shown in Fig-ure 4, Pathfinder scales linearly for most of the queries.Even the join queries (Q8–Q10) are of almost linearcomplexity. 0.01 0.1 1 10 100 1000Q20Q19Q18Q17Q16Q15Q14Q13Q12Q11Q10Q9Q8Q7Q6Q5Q4Q3Q2Q1normalized execution times11 MB 110 MB 1.1 GB 11 GBFigure 4: Pathfinder scalability: Execution times forthe XMark benchmark set, normalized to timings forthe 110 MB instance.The only two exceptions are queries Q11/Q12.Their quadratic scaling stems from a theta-join (pred-icate >) that produces intermediate results of about120 k, 12 M, and 1.2 G tuples for the 11 MB, 110 MB,and 1.1 GB instances, respectively. Note that this con-cerns the query result, whose computation cannot beavoided. Thus, any XQuery implementation will facethis complexity.4 Demonstration SetupOur system operates in a front-end/back-end fashion.The front-end, the Pathfinder compiler, parses incom-ing XQuery expressions and translates them into a re-lational algebra expression tree, represented in termsof a MIL2 program. The code is shipped to a Mon-etDB server that then executes the entire query andserializes the encoding of the result.The demonstration features both of these compo-nents, with various interesting hooks to trace the exe-cution of XQuery expressions on our relational back-end. Our system implements the W3C October 2004working drafts [1], including schema import, static typ-ing, and full axis features.The compiler provides facilities to look “under thehood” of relational XQuery compilation. An outputof type-annotated XQuery Core expression equivalentsdemonstrates optimization techniques applied to theincoming XQuery code before compiling it into rela-tional algebra. A graphical output of relational queryplans at different compilation stages (much like theplan in Figure 5) illustrates the loop-lifting concept.Relational plans may be traced to reveal the resultcomputed for any subexpression.Our demonstration system will be pre-loaded withXMark [10] instances of different sizes up to 1.1 GB.The query texts for the 20 benchmark queries will beready to run, but users may as well state their own adhoc queries.The Pathfinder compiler is available in open sourceas part of the MonetDB/XQuery implementation viahttp://pathfinder-xquery.org/.2MIL stands for MonetDB Interpreter Languagepiiter:outer,pos:pos1,item%pos1:iter,pos/outeroniter=innerpiiter,pos,item:resCCCCC⊕res:(item,item1)oniter=iter1piiter1:iter,item1:itemCCCCC×CCCCpos item1 100CCCpiiter:inner{{{{{{{{{{×{{{{pos1{{{piiter:inner,itemCCCC%inner:iter,positer pos item1 1 101 2 20mmmmmmmmmpiinner,outer:iterVVVVVVVVVVFigure 5: Relational query plan that evaluates thequery for $v in (10,20) return $v + 100.References[1] S. Boag, D. Chamberlin, M. F. Ferna´ndez, D. Flo-rescu, J. Robie, and J. Sime´on. XQuery 1.0: An XMLQuery Language. World Wide Web Consortium, Oc-tober 2004. http://www.w3.org/TR/xquery/.[2] P. Boncz. Monet: A Next-Generation DBMS KernelFor Query-Intensive Applications. PhD thesis, Uni-versity of Amsterdam, The Netherlands, May 2002.[3] P. Boncz, T. Grust, S. Manegold, J. Rittinger, andJ. Teubner. Pathfinder: Relational XQuery OverMulti-Gigabyte XML Inputs In Interactive Time.Technical Report INS-E0503, CWI, Amsterdam, TheNetherlands, March 2005.[4] T. Grust. Accelerating XPath Location Steps. InProc. of the 21st Int’l ACM SIGMOD Conf. on Man-agement of Data, pages 109–120, Madison, Wisconsin,USA, June 2002.[5] T. Grust. Purely Relational FLWORs. In Proc. ofthe ACM SIGMOD/PODS 2nd Int’l Workshop onXQuery Implementation, Experience and Perspectives(XIME-P), Baltimore, MD, USA, June 2005.[6] T. Grust, S. Sakr, and J. Teubner. XQuery on SQLHosts. In Proc. of the 30th Int’l Conference on VeryLarge Data Bases, Toronto, Canada, August 2004.[7] T. Grust, M. van Keulen, and J. Teubner. StaircaseJoin: Teach a Relational DBMS to Watch its AxisSteps. In Proc. of the 29th Int’l Conference on VeryLarge Databases, Berlin, Germany, September 2003.[8] X-Hive/DB. http://www.x-hive.com/.[9] S. Mayer, T. Grust, M. van Keulen, and J. Teub-ner. An Injection of Tree Awareness: Adding StaircaseJoin to PostgreSQL. In Proc. of the 30th Int’l Con-ference on Very Large Data Bases, pages 1305–1308,Toronto, Canada, August 2004.[10] A. Schmidt, F. Waas, M. Kersten, M. J. Carey,I. Manolescu, and R. Busse. XMark: A Benchmarkfor XML Data Management. In Proc. of the 28th Int’lConference on Very Large Databases, pages 974–985,Hong Kong, China, August 2002.

Pathfinder:XQuery-The Relational Way

Keulen, M.

NARCIS 

Accelerating XPath Location Steps.

An Injection of Tree Awareness: Adding Staircase Join to PostgreSQL.

Monet: A Next-Generation DBMS Kernel For Query-Intensive Applications.

Path¯nder: Relational XQuery Over Multi-Gigabyte XML Inputs In Interactive Time.

Pathfinder: XQuery - The Relational Way

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information

CWI's Institutional Repository

NARCIS