In this paper we introduce a method for generating interactive documents which exploits the visual features of hypertext to represent discourse structure. We explore the consistent and principled use of graphics and animation to support navigation and comprehension of non-linear text, where textual discourse markers do not always work effectively

Busemann, Stephan

Mancini, Clara

Pietsch, Christian 

Scott, Donia

English

Clara Mancini

Christian Pietsch

Donia Scott

Crossref

Visualising discourse structure in interactive documents

Pietsch, Christian

Open Research Online (Open)

Open Research OnlineThe Open University’s repository of research publicationsand other research outputsVisualising Discourse Structure in InteractiveDocumentsConference or Workshop ItemHow to cite:Mancini, Clara; Pietsch, Christian and Scott, Donia (2007). Visualising Discourse Structure in InteractiveDocuments. In: 11th European Workshop on Natural Language Generation (ENLG), 17-20 Jun 2007, SchlossDagstuhl, Germany.For guidance on citations see FAQs.c© [not recorded]Version: [not recorded]Link(s) to article on publisher’s website:http://enlg07.dfki.de/programm.htmlCopyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.oro.open.ac.ukVisualising Discourse Structure in Interactive DocumentsClara Mancini Christian PietschCentre for Research in ComputingThe Open UniversityWalton Hall, Milton Keynes, MK7 6AA, UK{c.mancini,c.pietsch,d.scott}@open.ac.ukDonia ScottAbstractIn this paper we introduce a method for generat-ing interactive documents which exploits the visualfeatures of hypertext to represent discourse struc-ture. We explore the consistent and principled useof graphics and animation to support navigation andcomprehension of non-linear text, where textual dis-course markers do not always work effectively.1 IntroductionThere is a long and well-established literature with-in theoretical, computational and psycho-linguisticson textual devices that function to signal the coher-ence structure of a discourse to the reader. Thiswork has addressed the traditional conceptualisa-tion of text: a two-dimensional array on a physicalpage, traversed in a set pattern (e.g., left to right,top to bottom in the Western tradition). As textsare increasingly being read on a computer screen,the act of reading is progressively becoming one ofhypertext navigation. However, hypertext presentsa strikingly different conceptualisation of text, andbrings with it new challenges for conveying dis-course structure.Hypertext is distinctive in that it is interactiveand non-linear, with several reading paths availablethrough the document. It is organised around nodesand links; the reader moves from node to node bymouse-clicking on links. A node can be the equiv-alent of a traditional text page or can contain just afew sentences; links can be words or graphical ele-ments. Since nodes typically contain more than onelink, the author can only partially control the orderin which the reader will access them.With hypertext, then, a new conceptualisation oftext has emerged as a three-dimensional array ona computer screen, which can be traversed in anynumber of ways. One of the challenges this posesfor text research is that coherence markers of the tra-ditional notion of text often do not work for this newmedium. We are exploring new possibilities for sig-nalling coherence in non-linear documents, exploit-ing the graphical features of a visually rich mediumyet to be systematically exploited as a dimensionof signification. This work is set in the context ofthe textual presentation of medical records from arepository of data-encoded medical histories.2 Structure representation in non-lineartextAs we discuss elsewhere (Mancini et al., 2007), dis-course markers such as adverbials, pronouns andconnectives cannot reliably be used to signal thediscourse relation between hypertext nodes, sincenodes can be accessible in more than one way, viapaths that reflect different relevancies. This restric-tion is less likely to apply to graphical features, be-cause they are visual and work in space. Owingto its technical characteristics, hypertext is a spatialmedium (Carter, 2000) as well as a temporal one(Luesebrink, 1998), in which spatial structures havea temporal dimension and realisation: both spaceand time can be exploited in hypertext to expressdiscourse coherence through space-temporal con-figurations in a three-dimensional space.At present, most hypertexts (especially on theWeb) make no use of graphical features to signaldiscourse relations between nodes, and nodes oftenconsist of long text pages with a few links target-ing other pages, from where the source page canno longer be seen. We take a different approachwhereby the text is made more readable in twoways: by making hypertext nodes much smaller andby using graphical features to signal the coherencerelations between nodes. We use the screen as avisual field across which the text can dynamicallydistribute, as links are clicked and new nodes ap-pear, composing meaningful patterns. The presen-tation and distribution of the nodes are intended tosignify the rhetorical role that their content playswithin the discourse. To achieve this, coherencerelations are used as document structuring princi-ples during discourse construction to define hyper-text links. These are then dynamically renderedduring navigation through the consistent and con-current use of the medium’s spatial and temporalgraphic features connoting the nodes. We refer tothis paradigm as cinehypertext.Having established a parallel between textual andvisual processing, research informed by Gestalt the-ory has proposed relevant principles of documentdesign (Campbell, 1995; Riley and Parker, 1998).Additionally, a number of representational rules forvisually expressing discourse relations between hy-pertext nodes can be derived from the semiology ofgraphics, according to which graphic features canbe employed to express conceptual relationships ofsimilarity, difference, order and proportion, exploit-ing the properties of the visual image (Koch, 2001).Following these principles and rules, we have de-signed and begun testing a series of prototype vi-sual patterns expressing coherence relations in non-linear discourse (Mancini et al., 2007).3 Graphics devices to visualise interactivetext structureOur empirical work so far shows that graphics canindeed be used to express abstract relational con-cepts (Power et al., 2003; Mancini, 2005). Butcan graphics usefully support the expression of dis-course structure in hypertext navigation? Can visualdiscourse markers support comprehension, substi-tuting or complementing textual discourse markersin non-linear documents? If so, how do these tex-tual and graphical features interact: can they expressthe same semantic meaning by following the sameprinciples or do they, due to their different semi-otic characteristics, function in different ways to thesame end?Our starting point is two NLG systems that gen-erate, as linear text, summaries of a cancer patient’smedical history (respectively generated to be readby clinicians and medical researchers, or by pa-tients) from a repository of data-encoded medicalreports (Hallett et al., 2006; Williams et al., 2007).We are extending these systems to generate interac-tive text on the one hand, and its animated graphicalpresentation, on the other. Since textual and graph-ical discourse markers will interact with each other,You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination and you found that you had a lump in your right breast. A self examination is an examination of the breasts by running your hand over each breastand up under your arms and checking for changes to their size, shape or feel. On October 4th you did another self examination and you found that you had a lump in your right breast. On October 11th you had a radical mastectomy to treat cancer in your right breast. A radical mastectomy is an operation to remove the breast, along with the lymph glands under the arm and the muscles of the chest wall. Cancer is a tumour that tends to spread, both locally and to other parts of the body. Figure 1: Example section of a linear report gener-ated for a patient.our interactive documents may not present some ofthe textual features that occur in the correspondinglinear form. We aim to provide a principled accountof this interaction, and start by studying the effectof using visual markers on an interactive version ofthe linear reports already produced by our genera-tors. Figure 1 shows an example section of a lineartextual report.At present, we transform a linear medical reportinto an interactive one by dividing the text into re-lated chunks within text nodes whose graphics andanimation features signal the relations holding be-tween the chunks. In our example, the relationsholding between the nodes are SEQUENCE, ELAB-ORATION, MOTIVATION and RESULT. We gener-ate graphical presentations of these discourse rela-tions, using motion trajectory, distribution, colourvalue and dimensions of text windows. Red arrowsand bold fonts in the windows signal active links,whereas followed links are signalled by dark greyarrows and bold fonts.Specifically, the SEQUENCE of events describedin the report is expressed by the vertical alignmentof text windows appearing one under the other andhaving the same width and background colour (Fig-ure 2).Causal relations, like RESULT and MOTIVATION,holding between events are expressed by the hori-zontal alignment of text windows having the sameheight, where the one representing the result slidesout, moving from left to right, from behind the onerepresenting the cause, having a darker background(Figure 3)The expression of MOTIVATION is similar to thatof RESULT, with the difference that this time thewindow representing the motivation moves fromYou had a consultation with your doctor on September 20th 1993.You had a consultation with your doctor on September 20th 1993.On September 27th you did a self examination.A BFigure 2: Representation of SEQUENCE: the red ar-row link in the first node (A) is activated and thesecond node appears underneath it (B).You found that you had a lump in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.You found that you had a lum p in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.ABFigure 3: Representation of RESULT: the result-node slides out from behind the cause-node (A)placing itself next to it on the right hand side (B).right to left and has a lighter background (Figure 4).Finally, the definitions for which an explanationis available constitute links whose activation trig-gers an ELABORATION node, which appears slightlyoverlapped along a virtual third dimension. Whilethe background of the other nodes is grey, the back-ground of elaborative nodes is a lighter tone of red(Figure 5).We have been evaluating the significance of theseand other visual patterns, while exploring new de-signs to express discourse relations in interactivedocuments of different formats. We are also imple-menting a prototype system that employs a selectionof patterns to carry out more empirical studies. Thearchitecture of the prototype is described below.4 System architectureFew existing text generators which produce HTMLoutput actually use the properties of non-linear textfully. Sometimes HTML is used as a simple text for-matting language. Where hyperlinks are provided,The radical mastectomy was done to treat cancer in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.On October 4th you did another self examination.On October 11th you had a radi l mastectomy.You found that you had a lump in your right breast.The radical mastectomy was done to treat cancer in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.On October 4th you did another self examination.On October 11th you had a radical mastectomy.You found that you had a lump in your right breast.ABFigure 4: Representation of MOTIVATION: themotivation-node slides out from behind theconsequence-node (A) placing itself to its left (B).The radical mastectomy was done to treat cancer in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.On October 4th you did another self examination.On October 11th you had a radical mastectomy.A radical mastectomy is an operation to remove the breast, along with the lymph glands under the arm and the muscles of the chest wall.You found that you had a lump in your right breast.Cancer is a tumour that tends to spread, both locally and to other parts of the body. Figure 5: Representation of ELABORATION: theelaboration-node appears overlapped to the nodecontaining the definition that is being elaborated on.the rhetorical, semantic or pragmatic relation of thelink target to the anchor is rarely made explicit. Ourapproach is a radical departure from this practise.We propose an additional layer of abstraction em-bodied in a well-defined data format we call XCH(XML for CineHypertext), and describe a prototypearchitecture that extends previous approaches byproviding a principled way of hyperlinking and ani-mating content, thus encouraging the user to interactwith the information dynamically presented. In factor DHTML)web browser(running Flashreport generatorcinehypertext.lzxreport.lzxOpenLaszloEPRdatabasereport.xchFigure 6: Prototype architecture. Solid arrows sig-nify online data flow, dashed arrows signify dataflow which occurred earlier.we are crossing the border from document genera-tion to application generation because the user inter-acts with a Rich Internet Application (RIA) which isvery much data-driven.Our XCH format is an instance of XML defined bytwo XML schema files. One is domain-independentand captures the range of semantic/pragmatic rela-tions that can be encoded between chunks of infor-mation. The other is domain-specific and definesthe document structure of (in the current prototype)a medical report.Often, XML is processed using XSLT. Indeeda simple XSLT style sheet can be used to convertXCH to linear text. However, to transform XCHinto the interactive and animated Web applicationwe are aiming for, we prefer to use a Web toolkit.We chose OpenLaszlo which since version 4 is nolonger restricted to one particular run-time platformfor content delivery (Flash) but can also compilesource code into AJAX-like DHTML code which isnatively supported by most current browsers. Open-Laszlo also makes it very easy to animate any of itsvisual components (widgets). Its XML-based pro-gramming language (LZX) is as declarative as XSLTbut adds object-orientation and supports creating re-usable class libraries.Figure 6 shows an overview of the current sys-tem architecture. Its upper half depicts an exist-ing NLG system modified to produce XCH output.Its lower half outlines the new presentation modulewhich takes XCH as input. Here, report.lxz is thedomain-specific part of the presentation module im-plementation, and cinehypertext.lzx is the domain-independent, library part. We expect that our emerg-ing cinehypertext library and format will prove use-ful in other domains and systems.5 ConclusionThis work aims to identify ways of presenting hy-pertext discourse which employ graphics to signaldiscourse structure in a systematic and principledway, by making articulate use of the space-temporaldimensions of the electronic medium. The work ispart of a larger effort in natural language genera-tion, aimed at producing different renditions of thesame semantic content for different purposes andfor different media. One of the novel aspects ofour work is that we are generating ‘paraphrases’ thatvary not just along the traditional dimensions (dis-course, syntax, lexicalisation) but also in terms ofgraphical presentation (e.g., as textual reports in dif-ferent styles – including linear vs. non-linear – or asslides for a presentation).ReferencesK. S. Campbell. 1995. Coherence, continuity, and co-hesion. Theoretical foundations for document design.Erlbaum, Hillsdale, NJ.L. M. Carter. 2000. Arguments in hypertext. A rhetori-cal approach. In Proceedings 11th ACM Conferenceon Hypertext and Hypermedia, pages 87–91, NewYork. ACM.C. Hallett, R. Power, and D. Scott. 2006. Summarisationand visualisation of e-health data repositories. In UKE-Science All-Hands Meeting, Nottingham, UK.W. G. Koch. 2001. Jaques Bertin’s theory of graphicsand its development and influence on multimedia car-tography. Information Design Journal, 10:37–43.M. C. Luesebrink. 1998. The moment in hypertext. InProceedings 9th ACM conference on hypertext andhypermedia, pages 106–112, New York. ACM.C. Mancini, D. Scott, and S. Buckingham Shum. 2007.Visualising discourse coherence in non-linear docu-ments. Traitement Automatique des Langues, SpecialIssue on Computational Approaches to Document andDiscourse, 47(1). To appear.C. Mancini. 2005. Cinematic hypertext. Investigating anew paradigm. IOS, Amsterdam.R. Power, D. Scott, and N. Bouayad-Agha. 2003. Doc-ument structure. Computational Linguistics, 29:211–260.K. Riley and F. Parker. 1998. Parallels between visualand textual processing. IEEE Transactions on Profes-sional Communication, 41:175–185.S. Williams, P. Piwek, and R. Power. 2007. Generatingmonologue and dialogue to present personalised med-ical information to patients. In European Workshopon Natural Language Generation, Prague. Submitted.

Mancini C, Pietsch C, Scott D. Visualising Discourse Structure in Interactive Documents. In: Busemann S, ed. DFKI Document D-07-01. Saarbrücken: DFKI GmbH;  2007: 89-92.In this paper we introduce a method for generating interactive documents which exploits the visual features of hypertext to represent discourse structure. We explore the consistent and principled use of graphics and animation to support navigation and comprehension of non-linear text, where textual discourse markers do not always work effectively

Publications at Bielefeld University

Visualising Discourse Structure in Interactive DocumentsClara Mancini Christian PietschCentre for Research in ComputingThe Open UniversityWalton Hall, Milton Keynes, MK7 6AA, UK{c.mancini,c.pietsch,d.scott}@open.ac.ukDonia ScottAbstractIn this paper we introduce a method for generat-ing interactive documents which exploits the visualfeatures of hypertext to represent discourse struc-ture. We explore the consistent and principled useof graphics and animation to support navigation andcomprehension of non-linear text, where textual dis-course markers do not always work effectively.1 IntroductionThere is a long and well-established literature with-in theoretical, computational and psycho-linguisticson textual devices that function to signal the coher-ence structure of a discourse to the reader. Thiswork has addressed the traditional conceptualisa-tion of text: a two-dimensional array on a physicalpage, traversed in a set pattern (e.g., left to right,top to bottom in the Western tradition). As textsare increasingly being read on a computer screen,the act of reading is progressively becoming one ofhypertext navigation. However, hypertext presentsa strikingly different conceptualisation of text, andbrings with it new challenges for conveying dis-course structure.Hypertext is distinctive in that it is interactiveand non-linear, with several reading paths availablethrough the document. It is organised around nodesand links; the reader moves from node to node bymouse-clicking on links. A node can be the equiv-alent of a traditional text page or can contain just afew sentences; links can be words or graphical ele-ments. Since nodes typically contain more than onelink, the author can only partially control the orderin which the reader will access them.With hypertext, then, a new conceptualisation oftext has emerged as a three-dimensional array ona computer screen, which can be traversed in anynumber of ways. One of the challenges this posesfor text research is that coherence markers of the tra-ditional notion of text often do not work for this newmedium. We are exploring new possibilities for sig-nalling coherence in non-linear documents, exploit-ing the graphical features of a visually rich mediumyet to be systematically exploited as a dimensionof signification. This work is set in the context ofthe textual presentation of medical records from arepository of data-encoded medical histories.2 Structure representation in non-lineartextAs we discuss elsewhere (Mancini et al., 2007), dis-course markers such as adverbials, pronouns andconnectives cannot reliably be used to signal thediscourse relation between hypertext nodes, sincenodes can be accessible in more than one way, viapaths that reflect different relevancies. This restric-tion is less likely to apply to graphical features, be-cause they are visual and work in space. Owingto its technical characteristics, hypertext is a spatialmedium (Carter, 2000) as well as a temporal one(Luesebrink, 1998), in which spatial structures havea temporal dimension and realisation: both spaceand time can be exploited in hypertext to expressdiscourse coherence through space-temporal con-figurations in a three-dimensional space.At present, most hypertexts (especially on theWeb) make no use of graphical features to signaldiscourse relations between nodes, and nodes oftenconsist of long text pages with a few links target-ing other pages, from where the source page canno longer be seen. We take a different approachwhereby the text is made more readable in twoways: by making hypertext nodes much smaller andby using graphical features to signal the coherencerelations between nodes. We use the screen as avisual field across which the text can dynamicallydistribute, as links are clicked and new nodes ap-pear, composing meaningful patterns. The presen-tation and distribution of the nodes are intended tosignify the rhetorical role that their content playswithin the discourse. To achieve this, coherencerelations are used as document structuring princi-ples during discourse construction to define hyper-text links. These are then dynamically renderedduring navigation through the consistent and con-current use of the medium’s spatial and temporalgraphic features connoting the nodes. We refer tothis paradigm as cinehypertext.Having established a parallel between textual andvisual processing, research informed by Gestalt the-ory has proposed relevant principles of documentdesign (Campbell, 1995; Riley and Parker, 1998).Additionally, a number of representational rules forvisually expressing discourse relations between hy-pertext nodes can be derived from the semiology ofgraphics, according to which graphic features canbe employed to express conceptual relationships ofsimilarity, difference, order and proportion, exploit-ing the properties of the visual image (Koch, 2001).Following these principles and rules, we have de-signed and begun testing a series of prototype vi-sual patterns expressing coherence relations in non-linear discourse (Mancini et al., 2007).3 Graphics devices to visualise interactivetext structureOur empirical work so far shows that graphics canindeed be used to express abstract relational con-cepts (Power et al., 2003; Mancini, 2005). Butcan graphics usefully support the expression of dis-course structure in hypertext navigation? Can visualdiscourse markers support comprehension, substi-tuting or complementing textual discourse markersin non-linear documents? If so, how do these tex-tual and graphical features interact: can they expressthe same semantic meaning by following the sameprinciples or do they, due to their different semi-otic characteristics, function in different ways to thesame end?Our starting point is two NLG systems that gen-erate, as linear text, summaries of a cancer patient’smedical history (respectively generated to be readby clinicians and medical researchers, or by pa-tients) from a repository of data-encoded medicalreports (Hallett et al., 2006; Williams et al., 2007).We are extending these systems to generate interac-tive text on the one hand, and its animated graphicalpresentation, on the other. Since textual and graph-ical discourse markers will interact with each other,You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination and you found that you had a lump in your right breast. A self examination is an examination of the breasts by running your hand over each breastand up under your arms and checking for changes to their size, shape or feel. On October 4th you did another self examination and you found that you had a lump in your right breast. On October 11th you had a radical mastectomy to treat cancer in your right breast. A radical mastectomy is an operation to remove the breast, along with the lymph glands under the arm and the muscles of the chest wall. Cancer is a tumour that tends to spread, both locally and to other parts of the body. Figure 1: Example section of a linear report gener-ated for a patient.our interactive documents may not present some ofthe textual features that occur in the correspondinglinear form. We aim to provide a principled accountof this interaction, and start by studying the effectof using visual markers on an interactive version ofthe linear reports already produced by our genera-tors. Figure 1 shows an example section of a lineartextual report.At present, we transform a linear medical reportinto an interactive one by dividing the text into re-lated chunks within text nodes whose graphics andanimation features signal the relations holding be-tween the chunks. In our example, the relationsholding between the nodes are SEQUENCE, ELAB-ORATION, MOTIVATION and RESULT. We gener-ate graphical presentations of these discourse rela-tions, using motion trajectory, distribution, colourvalue and dimensions of text windows. Red arrowsand bold fonts in the windows signal active links,whereas followed links are signalled by dark greyarrows and bold fonts.Specifically, the SEQUENCE of events describedin the report is expressed by the vertical alignmentof text windows appearing one under the other andhaving the same width and background colour (Fig-ure 2).Causal relations, like RESULT and MOTIVATION,holding between events are expressed by the hori-zontal alignment of text windows having the sameheight, where the one representing the result slidesout, moving from left to right, from behind the onerepresenting the cause, having a darker background(Figure 3)The expression of MOTIVATION is similar to thatof RESULT, with the difference that this time thewindow representing the motivation moves fromYou had a consultation with your doctor on September 20th 1993.You had a consultation with your doctor on September 20th 1993.On September 27th you did a self examination.A BFigure 2: Representation of SEQUENCE: the red ar-row link in the first node (A) is activated and thesecond node appears underneath it (B).You found that you had a lump in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.You found that you had a lum p in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.ABFigure 3: Representation of RESULT: the result-node slides out from behind the cause-node (A)placing itself next to it on the right hand side (B).right to left and has a lighter background (Figure 4).Finally, the definitions for which an explanationis available constitute links whose activation trig-gers an ELABORATION node, which appears slightlyoverlapped along a virtual third dimension. Whilethe background of the other nodes is grey, the back-ground of elaborative nodes is a lighter tone of red(Figure 5).We have been evaluating the significance of theseand other visual patterns, while exploring new de-signs to express discourse relations in interactivedocuments of different formats. We are also imple-menting a prototype system that employs a selectionof patterns to carry out more empirical studies. Thearchitecture of the prototype is described below.4 System architectureFew existing text generators which produce HTMLoutput actually use the properties of non-linear textfully. Sometimes HTML is used as a simple text for-matting language. Where hyperlinks are provided,The radical mastectomy was done to treat cancer in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.On October 4th you did another self examination.On October 11th you had a radi l mastectomy.You found that you had a lump in your right breast.The radical mastectomy was done to treat cancer in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.On October 4th you did another self examination.On October 11th you had a radical mastectomy.You found that you had a lump in your right breast.ABFigure 4: Representation of MOTIVATION: themotivation-node slides out from behind theconsequence-node (A) placing itself to its left (B).The radical mastectomy was done to treat cancer in your right breast.You had a consultation with your doctor on September 20th 1993. On September 27th you did a self examination.On October 4th you did another self examination.On October 11th you had a radical mastectomy.A radical mastectomy is an operation to remove the breast, along with the lymph glands under the arm and the muscles of the chest wall.You found that you had a lump in your right breast.Cancer is a tumour that tends to spread, both locally and to other parts of the body. Figure 5: Representation of ELABORATION: theelaboration-node appears overlapped to the nodecontaining the definition that is being elaborated on.the rhetorical, semantic or pragmatic relation of thelink target to the anchor is rarely made explicit. Ourapproach is a radical departure from this practise.We propose an additional layer of abstraction em-bodied in a well-defined data format we call XCH(XML for CineHypertext), and describe a prototypearchitecture that extends previous approaches byproviding a principled way of hyperlinking and ani-mating content, thus encouraging the user to interactwith the information dynamically presented. In factor DHTML)web browser(running Flashreport generatorcinehypertext.lzxreport.lzxOpenLaszloEPRdatabasereport.xchFigure 6: Prototype architecture. Solid arrows sig-nify online data flow, dashed arrows signify dataflow which occurred earlier.we are crossing the border from document genera-tion to application generation because the user inter-acts with a Rich Internet Application (RIA) which isvery much data-driven.Our XCH format is an instance of XML defined bytwo XML schema files. One is domain-independentand captures the range of semantic/pragmatic rela-tions that can be encoded between chunks of infor-mation. The other is domain-specific and definesthe document structure of (in the current prototype)a medical report.Often, XML is processed using XSLT. Indeeda simple XSLT style sheet can be used to convertXCH to linear text. However, to transform XCHinto the interactive and animated Web applicationwe are aiming for, we prefer to use a Web toolkit.We chose OpenLaszlo which since version 4 is nolonger restricted to one particular run-time platformfor content delivery (Flash) but can also compilesource code into AJAX-like DHTML code which isnatively supported by most current browsers. Open-Laszlo also makes it very easy to animate any of itsvisual components (widgets). Its XML-based pro-gramming language (LZX) is as declarative as XSLTbut adds object-orientation and supports creating re-usable class libraries.Figure 6 shows an overview of the current sys-tem architecture. Its upper half depicts an exist-ing NLG system modified to produce XCH output.Its lower half outlines the new presentation modulewhich takes XCH as input. Here, report.lxz is thedomain-specific part of the presentation module im-plementation, and cinehypertext.lzx is the domain-independent, library part. We expect that our emerg-ing cinehypertext library and format will prove use-ful in other domains and systems.5 ConclusionThis work aims to identify ways of presenting hy-pertext discourse which employ graphics to signaldiscourse structure in a systematic and principledway, by making articulate use of the space-temporaldimensions of the electronic medium. The work ispart of a larger effort in natural language genera-tion, aimed at producing different renditions of thesame semantic content for different purposes andfor different media. One of the novel aspects ofour work is that we are generating ‘paraphrases’ thatvary not just along the traditional dimensions (dis-course, syntax, lexicalisation) but also in terms ofgraphical presentation (e.g., as textual reports in dif-ferent styles – including linear vs. non-linear – or asslides for a presentation).ReferencesK. S. Campbell. 1995. Coherence, continuity, and co-hesion. Theoretical foundations for document design.Erlbaum, Hillsdale, NJ.L. M. Carter. 2000. Arguments in hypertext. A rhetori-cal approach. In Proceedings 11th ACM Conferenceon Hypertext and Hypermedia, pages 87–91, NewYork. ACM.C. Hallett, R. Power, and D. Scott. 2006. Summarisationand visualisation of e-health data repositories. In UKE-Science All-Hands Meeting, Nottingham, UK.W. G. Koch. 2001. Jaques Bertin’s theory of graphicsand its development and influence on multimedia car-tography. Information Design Journal, 10:37–43.M. C. Luesebrink. 1998. The moment in hypertext. InProceedings 9th ACM conference on hypertext andhypermedia, pages 106–112, New York. ACM.C. Mancini, D. Scott, and S. Buckingham Shum. 2007.Visualising discourse coherence in non-linear docu-ments. Traitement Automatique des Langues, SpecialIssue on Computational Approaches to Document andDiscourse, 47(1). To appear.C. Mancini. 2005. Cinematic hypertext. Investigating anew paradigm. IOS, Amsterdam.R. Power, D. Scott, and N. Bouayad-Agha. 2003. Doc-ument structure. Computational Linguistics, 29:211–260.K. Riley and F. Parker. 1998. Parallels between visualand textual processing. IEEE Transactions on Profes-sional Communication, 41:175–185.S. Williams, P. Piwek, and R. Power. 2007. Generatingmonologue and dialogue to present personalised med-ical information to patients. In European Workshopon Natural Language Generation, Prague. Submitted.

Arguments in hypertext. A rhetorical approach.

Cinematic hypertext. Investigating a new paradigm. IOS,

Coherence, continuity, and cohesion. Theoretical foundations for document design. Erlbaum,

Generating monologueand dialogue to present personalised medical information to patients.

Jaques Bertin’s theory of graphics and its development and inﬂuence on multimedia cartography.

Visualising Discourse Structure in Interactive Documents

Abstract

Similar works

Full text

Available Versions

Crossref

Open Research Online (Open)

Publications at Bielefeld University