Amabili, Lorenzo

Hoesel, Frans H.J. van

Klaver, Tom

Kosinka, Jiri

Mendrik, Adrienne

Roerdink, J.B.T.M.

Svetachov, Pjotr

van Meersbergen, Maarten

van Ooijen, Peter

Yu, Lingyun

English

NARCIS 

Visual Storytelling for Informed Decision-Making in Medicine

ARTS repository - University of Groningen

   University of GroningenVisual Storytelling for Informed Decision-Making in MedicineAmabili, Lorenzo; Hoesel, Frans H.J. van; Klaver, Tom; Kosinka, Jiri; van Meersbergen,Maarten; Mendrik, Adrienne; van Ooijen, Peter; Roerdink, J.B.T.M.; Svetachov, Pjotr; Yu,LingyunIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Publication date:2018Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Amabili, L., Hoesel, F. H. J. V., Klaver, T., Kosinka, J., van Meersbergen, M., Mendrik, A., van Ooijen, P.,Roerdink, J. B. T. M., Svetachov, P., & Yu, L. (2018). Visual Storytelling for Informed Decision-Making inMedicine.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 31-10-2023Visual Storytelling for Informed Decision-Making in MedicineL. Amabili1 F.H.J. van Hoesel4 T. Klaver3 J. Kosinka1 M.A.J. van Meersbergen3A. Mendrik3 P.M.A. van Ooijen2 J.B.T.M. Roerdink1,2 P. Svetachov4 L. Yu11 JBI, University of Groningen 2 University Medical Center Groningen3 Netherlands eScience Center 4 Center for Information Technology, RUGIntroductionI n healthcare, big complex imaging data are processed and analyzed regularly.Afterwards, the findings are communicated to inform educated decisions. In thisprocedure, human errors or misunderstandings can occur, especially when morethan one person is involved, or due to the use of text reports, thus lengthening thetime devoted to each case. For this reason, a more effective way for communicatingand presenting findings is required.In our project, we focus on answering the following research questions:• How to communicate results effectively to enhance collaborative work.• How to reproduce findings in decision-making circumstances.• How to substitute text reports without loss of information.To this end, we aim to develop an IT tool based on the principle of visual storytellingwhich will let users present data clearly and facilitate collaboration with experts indifferent fields, and medicine in particular. As a consequence, we expect that thisnovel approach will improve the overall diagnostic process.A New Approach to Analyzing Medical DataT he main innovative aspect of this project is the use of visual storytelling inthe decision-making process. In addition, provenance data are recorded dur-ing data exploration and showed in an interactive graph which allows the user(s) tonavigate the analysis process, extending the study or authoring it with annotations,measurements or statistics (see Figure 1). This procedure can be repeated andadjusted as many times as needed, even after having presented the results [1].Figure 1: A short visual story composed by the medical input data (left), a framefrom one-user analysis (center) and a frame from multiple-user analysis (right).Figure 2: The diagnosis process in healthcare based on the traditional workflow andthe (visual) storytelling approach, which consists of a three-stage non-linear process(i.e., Exploration, Authoring, Presentation).According to our vision, a (visual) storytelling approach can be expected to successfully achieve the substitution of text reports with visual data stories which are moreinformative and less ambivalent, thus reducing human errors, improving collaborative work and increasing diagnosis understanding in a broader audience (see Figure 2).A Visual Storytelling Tool to Support Medical Decision MakingO ur project is aimed at improving medical data analysis via enhancing collaborative work and making the communication of results more effective by using a visual storytellingtool (see Figure 3). Composed of an intuitive interface to explore complex imaging data and an authoring tool to create visual data stories based on provenance data topresent findings, it provides users with increased flexibility [2]. We envision that this text-free approach will refine and facilitate the current medical work-flow, by decreasing therisks of misunderstanding, reducing the time needed for each analysis, and simplifying collaborations among both experts and non-experts. Furthermore, the reproducibility ofanalysis will boost transparency of the decision-making process [3].Figure 3: An example of an imaging data analysis made by using a prototype built upon the Phovea framework by the Caleydo team [1] (left). After exploration of brain imagedata, a visual data story is created (center). Then, the results can be reproduced and the data exploration can be extended (right).AcknowledgementsThis project has been funded by the Netherlands eScience Center(NLeSC), projectnumber DTEC.2016.015, under the call "Joint eScience and Data Science acrossTop Sectors, DTEC 2016: Disruptive Technologies", organized in collaboration withNetherlands Organisation for Scientific Research Physical Sciences (NWO) andDutch Digital Delta.ContactsProf. Dr. Jos B. T. M. Roerdink, chairJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenMSc Lorenzo Amabili, PhD StudentJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenReferences[1] S. Gratzl, A. Lex, N. Gehlenborg, N. Cosgrove, M. Streit. From Visual Exploration to Storytelling and Back Again. Comput. Graph. Forum 35(3): 491-500, 2016.[2] M. Wohlfart. Story Telling Aspects in Medical Applications. Central European Seminar for Computer Graphics, 2006.[3] W. Jorritsma, F. Cnossen, P.M.A. van Ooijen. Improving the radiologist-CAD interaction: designing for appropriate trust. Clinical Radiology, 70(2):115-22, 2014.

Dissertations of the University of Groningen

   University of GroningenVisual Storytelling for Informed Decision-Making in MedicineAmabili, Lorenzo; Hoesel, Frans H.J. van; Klaver, Tom; Kosinka, Jiri; van Meersbergen,Maarten; Mendrik, Adrienne; van Ooijen, Peter; Roerdink, J.B.T.M.; Svetachov, Pjotr; Yu,LingyunIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Publication date:2018Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Amabili, L., Hoesel, F. H. J. V., Klaver, T., Kosinka, J., van Meersbergen, M., Mendrik, A., van Ooijen, P.,Roerdink, J. B. T. M., Svetachov, P., & Yu, L. (2018). Visual Storytelling for Informed Decision-Making inMedicine.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 05-06-2022Visual Storytelling for Informed Decision-Making in MedicineL. Amabili1 F.H.J. van Hoesel4 T. Klaver3 J. Kosinka1 M.A.J. van Meersbergen3A. Mendrik3 P.M.A. van Ooijen2 J.B.T.M. Roerdink1,2 P. Svetachov4 L. Yu11 JBI, University of Groningen 2 University Medical Center Groningen3 Netherlands eScience Center 4 Center for Information Technology, RUGIntroductionI n healthcare, big complex imaging data are processed and analyzed regularly.Afterwards, the findings are communicated to inform educated decisions. In thisprocedure, human errors or misunderstandings can occur, especially when morethan one person is involved, or due to the use of text reports, thus lengthening thetime devoted to each case. For this reason, a more effective way for communicatingand presenting findings is required.In our project, we focus on answering the following research questions:• How to communicate results effectively to enhance collaborative work.• How to reproduce findings in decision-making circumstances.• How to substitute text reports without loss of information.To this end, we aim to develop an IT tool based on the principle of visual storytellingwhich will let users present data clearly and facilitate collaboration with experts indifferent fields, and medicine in particular. As a consequence, we expect that thisnovel approach will improve the overall diagnostic process.A New Approach to Analyzing Medical DataT he main innovative aspect of this project is the use of visual storytelling inthe decision-making process. In addition, provenance data are recorded dur-ing data exploration and showed in an interactive graph which allows the user(s) tonavigate the analysis process, extending the study or authoring it with annotations,measurements or statistics (see Figure 1). This procedure can be repeated andadjusted as many times as needed, even after having presented the results [1].Figure 1: A short visual story composed by the medical input data (left), a framefrom one-user analysis (center) and a frame from multiple-user analysis (right).Figure 2: The diagnosis process in healthcare based on the traditional workflow andthe (visual) storytelling approach, which consists of a three-stage non-linear process(i.e., Exploration, Authoring, Presentation).According to our vision, a (visual) storytelling approach can be expected to successfully achieve the substitution of text reports with visual data stories which are moreinformative and less ambivalent, thus reducing human errors, improving collaborative work and increasing diagnosis understanding in a broader audience (see Figure 2).A Visual Storytelling Tool to Support Medical Decision MakingO ur project is aimed at improving medical data analysis via enhancing collaborative work and making the communication of results more effective by using a visual storytellingtool (see Figure 3). Composed of an intuitive interface to explore complex imaging data and an authoring tool to create visual data stories based on provenance data topresent findings, it provides users with increased flexibility [2]. We envision that this text-free approach will refine and facilitate the current medical work-flow, by decreasing therisks of misunderstanding, reducing the time needed for each analysis, and simplifying collaborations among both experts and non-experts. Furthermore, the reproducibility ofanalysis will boost transparency of the decision-making process [3].Figure 3: An example of an imaging data analysis made by using a prototype built upon the Phovea framework by the Caleydo team [1] (left). After exploration of brain imagedata, a visual data story is created (center). Then, the results can be reproduced and the data exploration can be extended (right).AcknowledgementsThis project has been funded by the Netherlands eScience Center(NLeSC), projectnumber DTEC.2016.015, under the call "Joint eScience and Data Science acrossTop Sectors, DTEC 2016: Disruptive Technologies", organized in collaboration withNetherlands Organisation for Scientific Research Physical Sciences (NWO) andDutch Digital Delta.ContactsProf. Dr. Jos B. T. M. Roerdink, chairJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenMSc Lorenzo Amabili, PhD StudentJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenReferences[1] S. Gratzl, A. Lex, N. Gehlenborg, N. Cosgrove, M. Streit. From Visual Exploration to Storytelling and Back Again. Comput. Graph. Forum 35(3): 491-500, 2016.[2] M. Wohlfart. Story Telling Aspects in Medical Applications. Central European Seminar for Computer Graphics, 2006.[3] W. Jorritsma, F. Cnossen, P.M.A. van Ooijen. Improving the radiologist-CAD interaction: designing for appropriate trust. Clinical Radiology, 70(2):115-22, 2014.

Proceedings - University of Groningen

   University of GroningenVisual Storytelling for Informed Decision-Making in MedicineAmabili, Lorenzo; Hoesel, Frans H.J. van; Klaver, Tom; Kosinka, Jiri; van Meersbergen,Maarten; Mendrik, Adrienne; van Ooijen, Peter; Roerdink, J.B.T.M.; Svetachov, Pjotr; Yu,LingyunIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Publication date:2018Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Amabili, L., Hoesel, F. H. J. V., Klaver, T., Kosinka, J., van Meersbergen, M., Mendrik, A., van Ooijen, P.,Roerdink, J. B. T. M., Svetachov, P., & Yu, L. (2018). Visual Storytelling for Informed Decision-Making inMedicine.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 04-06-2022Visual Storytelling for Informed Decision-Making in MedicineL. Amabili1 F.H.J. van Hoesel4 T. Klaver3 J. Kosinka1 M.A.J. van Meersbergen3A. Mendrik3 P.M.A. van Ooijen2 J.B.T.M. Roerdink1,2 P. Svetachov4 L. Yu11 JBI, University of Groningen 2 University Medical Center Groningen3 Netherlands eScience Center 4 Center for Information Technology, RUGIntroductionI n healthcare, big complex imaging data are processed and analyzed regularly.Afterwards, the findings are communicated to inform educated decisions. In thisprocedure, human errors or misunderstandings can occur, especially when morethan one person is involved, or due to the use of text reports, thus lengthening thetime devoted to each case. For this reason, a more effective way for communicatingand presenting findings is required.In our project, we focus on answering the following research questions:• How to communicate results effectively to enhance collaborative work.• How to reproduce findings in decision-making circumstances.• How to substitute text reports without loss of information.To this end, we aim to develop an IT tool based on the principle of visual storytellingwhich will let users present data clearly and facilitate collaboration with experts indifferent fields, and medicine in particular. As a consequence, we expect that thisnovel approach will improve the overall diagnostic process.A New Approach to Analyzing Medical DataT he main innovative aspect of this project is the use of visual storytelling inthe decision-making process. In addition, provenance data are recorded dur-ing data exploration and showed in an interactive graph which allows the user(s) tonavigate the analysis process, extending the study or authoring it with annotations,measurements or statistics (see Figure 1). This procedure can be repeated andadjusted as many times as needed, even after having presented the results [1].Figure 1: A short visual story composed by the medical input data (left), a framefrom one-user analysis (center) and a frame from multiple-user analysis (right).Figure 2: The diagnosis process in healthcare based on the traditional workflow andthe (visual) storytelling approach, which consists of a three-stage non-linear process(i.e., Exploration, Authoring, Presentation).According to our vision, a (visual) storytelling approach can be expected to successfully achieve the substitution of text reports with visual data stories which are moreinformative and less ambivalent, thus reducing human errors, improving collaborative work and increasing diagnosis understanding in a broader audience (see Figure 2).A Visual Storytelling Tool to Support Medical Decision MakingO ur project is aimed at improving medical data analysis via enhancing collaborative work and making the communication of results more effective by using a visual storytellingtool (see Figure 3). Composed of an intuitive interface to explore complex imaging data and an authoring tool to create visual data stories based on provenance data topresent findings, it provides users with increased flexibility [2]. We envision that this text-free approach will refine and facilitate the current medical work-flow, by decreasing therisks of misunderstanding, reducing the time needed for each analysis, and simplifying collaborations among both experts and non-experts. Furthermore, the reproducibility ofanalysis will boost transparency of the decision-making process [3].Figure 3: An example of an imaging data analysis made by using a prototype built upon the Phovea framework by the Caleydo team [1] (left). After exploration of brain imagedata, a visual data story is created (center). Then, the results can be reproduced and the data exploration can be extended (right).AcknowledgementsThis project has been funded by the Netherlands eScience Center(NLeSC), projectnumber DTEC.2016.015, under the call "Joint eScience and Data Science acrossTop Sectors, DTEC 2016: Disruptive Technologies", organized in collaboration withNetherlands Organisation for Scientific Research Physical Sciences (NWO) andDutch Digital Delta.ContactsProf. Dr. Jos B. T. M. Roerdink, chairJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenMSc Lorenzo Amabili, PhD StudentJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenReferences[1] S. Gratzl, A. Lex, N. Gehlenborg, N. Cosgrove, M. Streit. From Visual Exploration to Storytelling and Back Again. Comput. Graph. Forum 35(3): 491-500, 2016.[2] M. Wohlfart. Story Telling Aspects in Medical Applications. Central European Seminar for Computer Graphics, 2006.[3] W. Jorritsma, F. Cnossen, P.M.A. van Ooijen. Improving the radiologist-CAD interaction: designing for appropriate trust. Clinical Radiology, 70(2):115-22, 2014.

   University of GroningenVisual Storytelling for Informed Decision-Making in MedicineAmabili, Lorenzo; Hoesel, Frans H.J. van; Klaver, Tom; Kosinka, Jiri; van Meersbergen,Maarten; Mendrik, Adrienne; van Ooijen, Peter; Roerdink, J.B.T.M.; Svetachov, Pjotr; Yu,LingyunIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Publication date:2018Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Amabili, L., Hoesel, F. H. J. V., Klaver, T., Kosinka, J., van Meersbergen, M., Mendrik, A., van Ooijen, P.,Roerdink, J. B. T. M., Svetachov, P., & Yu, L. (2018). Visual Storytelling for Informed Decision-Making inMedicine.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 13-02-2023Visual Storytelling for Informed Decision-Making in MedicineL. Amabili1 F.H.J. van Hoesel4 T. Klaver3 J. Kosinka1 M.A.J. van Meersbergen3A. Mendrik3 P.M.A. van Ooijen2 J.B.T.M. Roerdink1,2 P. Svetachov4 L. Yu11 JBI, University of Groningen 2 University Medical Center Groningen3 Netherlands eScience Center 4 Center for Information Technology, RUGIntroductionI n healthcare, big complex imaging data are processed and analyzed regularly.Afterwards, the findings are communicated to inform educated decisions. In thisprocedure, human errors or misunderstandings can occur, especially when morethan one person is involved, or due to the use of text reports, thus lengthening thetime devoted to each case. For this reason, a more effective way for communicatingand presenting findings is required.In our project, we focus on answering the following research questions:• How to communicate results effectively to enhance collaborative work.• How to reproduce findings in decision-making circumstances.• How to substitute text reports without loss of information.To this end, we aim to develop an IT tool based on the principle of visual storytellingwhich will let users present data clearly and facilitate collaboration with experts indifferent fields, and medicine in particular. As a consequence, we expect that thisnovel approach will improve the overall diagnostic process.A New Approach to Analyzing Medical DataT he main innovative aspect of this project is the use of visual storytelling inthe decision-making process. In addition, provenance data are recorded dur-ing data exploration and showed in an interactive graph which allows the user(s) tonavigate the analysis process, extending the study or authoring it with annotations,measurements or statistics (see Figure 1). This procedure can be repeated andadjusted as many times as needed, even after having presented the results [1].Figure 1: A short visual story composed by the medical input data (left), a framefrom one-user analysis (center) and a frame from multiple-user analysis (right).Figure 2: The diagnosis process in healthcare based on the traditional workflow andthe (visual) storytelling approach, which consists of a three-stage non-linear process(i.e., Exploration, Authoring, Presentation).According to our vision, a (visual) storytelling approach can be expected to successfully achieve the substitution of text reports with visual data stories which are moreinformative and less ambivalent, thus reducing human errors, improving collaborative work and increasing diagnosis understanding in a broader audience (see Figure 2).A Visual Storytelling Tool to Support Medical Decision MakingO ur project is aimed at improving medical data analysis via enhancing collaborative work and making the communication of results more effective by using a visual storytellingtool (see Figure 3). Composed of an intuitive interface to explore complex imaging data and an authoring tool to create visual data stories based on provenance data topresent findings, it provides users with increased flexibility [2]. We envision that this text-free approach will refine and facilitate the current medical work-flow, by decreasing therisks of misunderstanding, reducing the time needed for each analysis, and simplifying collaborations among both experts and non-experts. Furthermore, the reproducibility ofanalysis will boost transparency of the decision-making process [3].Figure 3: An example of an imaging data analysis made by using a prototype built upon the Phovea framework by the Caleydo team [1] (left). After exploration of brain imagedata, a visual data story is created (center). Then, the results can be reproduced and the data exploration can be extended (right).AcknowledgementsThis project has been funded by the Netherlands eScience Center(NLeSC), projectnumber DTEC.2016.015, under the call "Joint eScience and Data Science acrossTop Sectors, DTEC 2016: Disruptive Technologies", organized in collaboration withNetherlands Organisation for Scientific Research Physical Sciences (NWO) andDutch Digital Delta.ContactsProf. Dr. Jos B. T. M. Roerdink, chairJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenMSc Lorenzo Amabili, PhD StudentJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenReferences[1] S. Gratzl, A. Lex, N. Gehlenborg, N. Cosgrove, M. Streit. From Visual Exploration to Storytelling and Back Again. Comput. Graph. Forum 35(3): 491-500, 2016.[2] M. Wohlfart. Story Telling Aspects in Medical Applications. Central European Seminar for Computer Graphics, 2006.[3] W. Jorritsma, F. Cnossen, P.M.A. van Ooijen. Improving the radiologist-CAD interaction: designing for appropriate trust. Clinical Radiology, 70(2):115-22, 2014.

University of Groningen

   University of GroningenVisual Storytelling for Informed Decision-Making in MedicineAmabili, Lorenzo; Hoesel, Frans H.J. van; Klaver, Tom; Kosinka, Jiri; van Meersbergen,Maarten; Mendrik, Adrienne; van Ooijen, Peter; Roerdink, J.B.T.M.; Svetachov, Pjotr; Yu,LingyunIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Publication date:2018Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Amabili, L., Hoesel, F. H. J. V., Klaver, T., Kosinka, J., van Meersbergen, M., Mendrik, A., van Ooijen, P.,Roerdink, J. B. T. M., Svetachov, P., & Yu, L. (2018). Visual Storytelling for Informed Decision-Making inMedicine.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 20-11-2022Visual Storytelling for Informed Decision-Making in MedicineL. Amabili1 F.H.J. van Hoesel4 T. Klaver3 J. Kosinka1 M.A.J. van Meersbergen3A. Mendrik3 P.M.A. van Ooijen2 J.B.T.M. Roerdink1,2 P. Svetachov4 L. Yu11 JBI, University of Groningen 2 University Medical Center Groningen3 Netherlands eScience Center 4 Center for Information Technology, RUGIntroductionI n healthcare, big complex imaging data are processed and analyzed regularly.Afterwards, the findings are communicated to inform educated decisions. In thisprocedure, human errors or misunderstandings can occur, especially when morethan one person is involved, or due to the use of text reports, thus lengthening thetime devoted to each case. For this reason, a more effective way for communicatingand presenting findings is required.In our project, we focus on answering the following research questions:• How to communicate results effectively to enhance collaborative work.• How to reproduce findings in decision-making circumstances.• How to substitute text reports without loss of information.To this end, we aim to develop an IT tool based on the principle of visual storytellingwhich will let users present data clearly and facilitate collaboration with experts indifferent fields, and medicine in particular. As a consequence, we expect that thisnovel approach will improve the overall diagnostic process.A New Approach to Analyzing Medical DataT he main innovative aspect of this project is the use of visual storytelling inthe decision-making process. In addition, provenance data are recorded dur-ing data exploration and showed in an interactive graph which allows the user(s) tonavigate the analysis process, extending the study or authoring it with annotations,measurements or statistics (see Figure 1). This procedure can be repeated andadjusted as many times as needed, even after having presented the results [1].Figure 1: A short visual story composed by the medical input data (left), a framefrom one-user analysis (center) and a frame from multiple-user analysis (right).Figure 2: The diagnosis process in healthcare based on the traditional workflow andthe (visual) storytelling approach, which consists of a three-stage non-linear process(i.e., Exploration, Authoring, Presentation).According to our vision, a (visual) storytelling approach can be expected to successfully achieve the substitution of text reports with visual data stories which are moreinformative and less ambivalent, thus reducing human errors, improving collaborative work and increasing diagnosis understanding in a broader audience (see Figure 2).A Visual Storytelling Tool to Support Medical Decision MakingO ur project is aimed at improving medical data analysis via enhancing collaborative work and making the communication of results more effective by using a visual storytellingtool (see Figure 3). Composed of an intuitive interface to explore complex imaging data and an authoring tool to create visual data stories based on provenance data topresent findings, it provides users with increased flexibility [2]. We envision that this text-free approach will refine and facilitate the current medical work-flow, by decreasing therisks of misunderstanding, reducing the time needed for each analysis, and simplifying collaborations among both experts and non-experts. Furthermore, the reproducibility ofanalysis will boost transparency of the decision-making process [3].Figure 3: An example of an imaging data analysis made by using a prototype built upon the Phovea framework by the Caleydo team [1] (left). After exploration of brain imagedata, a visual data story is created (center). Then, the results can be reproduced and the data exploration can be extended (right).AcknowledgementsThis project has been funded by the Netherlands eScience Center(NLeSC), projectnumber DTEC.2016.015, under the call "Joint eScience and Data Science acrossTop Sectors, DTEC 2016: Disruptive Technologies", organized in collaboration withNetherlands Organisation for Scientific Research Physical Sciences (NWO) andDutch Digital Delta.ContactsProf. Dr. Jos B. T. M. Roerdink, chairJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenMSc Lorenzo Amabili, PhD StudentJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenReferences[1] S. Gratzl, A. Lex, N. Gehlenborg, N. Cosgrove, M. Streit. From Visual Exploration to Storytelling and Back Again. Comput. Graph. Forum 35(3): 491-500, 2016.[2] M. Wohlfart. Story Telling Aspects in Medical Applications. Central European Seminar for Computer Graphics, 2006.[3] W. Jorritsma, F. Cnossen, P.M.A. van Ooijen. Improving the radiologist-CAD interaction: designing for appropriate trust. Clinical Radiology, 70(2):115-22, 2014.

   University of GroningenVisual Storytelling for Informed Decision-Making in MedicineAmabili, Lorenzo; Hoesel, Frans H.J. van; Klaver, Tom; Kosinka, Jiri; van Meersbergen,Maarten; Mendrik, Adrienne; van Ooijen, Peter; Roerdink, J.B.T.M.; Svetachov, Pjotr; Yu,LingyunIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Publication date:2018Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Amabili, L., Hoesel, F. H. J. V., Klaver, T., Kosinka, J., van Meersbergen, M., Mendrik, A., van Ooijen, P.,Roerdink, J. B. T. M., Svetachov, P., & Yu, L. (2018). Visual Storytelling for Informed Decision-Making inMedicine.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 10-09-2023Visual Storytelling for Informed Decision-Making in MedicineL. Amabili1 F.H.J. van Hoesel4 T. Klaver3 J. Kosinka1 M.A.J. van Meersbergen3A. Mendrik3 P.M.A. van Ooijen2 J.B.T.M. Roerdink1,2 P. Svetachov4 L. Yu11 JBI, University of Groningen 2 University Medical Center Groningen3 Netherlands eScience Center 4 Center for Information Technology, RUGIntroductionI n healthcare, big complex imaging data are processed and analyzed regularly.Afterwards, the findings are communicated to inform educated decisions. In thisprocedure, human errors or misunderstandings can occur, especially when morethan one person is involved, or due to the use of text reports, thus lengthening thetime devoted to each case. For this reason, a more effective way for communicatingand presenting findings is required.In our project, we focus on answering the following research questions:• How to communicate results effectively to enhance collaborative work.• How to reproduce findings in decision-making circumstances.• How to substitute text reports without loss of information.To this end, we aim to develop an IT tool based on the principle of visual storytellingwhich will let users present data clearly and facilitate collaboration with experts indifferent fields, and medicine in particular. As a consequence, we expect that thisnovel approach will improve the overall diagnostic process.A New Approach to Analyzing Medical DataT he main innovative aspect of this project is the use of visual storytelling inthe decision-making process. In addition, provenance data are recorded dur-ing data exploration and showed in an interactive graph which allows the user(s) tonavigate the analysis process, extending the study or authoring it with annotations,measurements or statistics (see Figure 1). This procedure can be repeated andadjusted as many times as needed, even after having presented the results [1].Figure 1: A short visual story composed by the medical input data (left), a framefrom one-user analysis (center) and a frame from multiple-user analysis (right).Figure 2: The diagnosis process in healthcare based on the traditional workflow andthe (visual) storytelling approach, which consists of a three-stage non-linear process(i.e., Exploration, Authoring, Presentation).According to our vision, a (visual) storytelling approach can be expected to successfully achieve the substitution of text reports with visual data stories which are moreinformative and less ambivalent, thus reducing human errors, improving collaborative work and increasing diagnosis understanding in a broader audience (see Figure 2).A Visual Storytelling Tool to Support Medical Decision MakingO ur project is aimed at improving medical data analysis via enhancing collaborative work and making the communication of results more effective by using a visual storytellingtool (see Figure 3). Composed of an intuitive interface to explore complex imaging data and an authoring tool to create visual data stories based on provenance data topresent findings, it provides users with increased flexibility [2]. We envision that this text-free approach will refine and facilitate the current medical work-flow, by decreasing therisks of misunderstanding, reducing the time needed for each analysis, and simplifying collaborations among both experts and non-experts. Furthermore, the reproducibility ofanalysis will boost transparency of the decision-making process [3].Figure 3: An example of an imaging data analysis made by using a prototype built upon the Phovea framework by the Caleydo team [1] (left). After exploration of brain imagedata, a visual data story is created (center). Then, the results can be reproduced and the data exploration can be extended (right).AcknowledgementsThis project has been funded by the Netherlands eScience Center(NLeSC), projectnumber DTEC.2016.015, under the call "Joint eScience and Data Science acrossTop Sectors, DTEC 2016: Disruptive Technologies", organized in collaboration withNetherlands Organisation for Scientific Research Physical Sciences (NWO) andDutch Digital Delta.ContactsProf. Dr. Jos B. T. M. Roerdink, chairJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenMSc Lorenzo Amabili, PhD StudentJBI, Institute for Mathematics andComputing ScienceUniversity of GroningenNijenborgh 99747 AG GroningenReferences[1] S. Gratzl, A. Lex, N. Gehlenborg, N. Cosgrove, M. Streit. From Visual Exploration to Storytelling and Back Again. Comput. Graph. Forum 35(3): 491-500, 2016.[2] M. Wohlfart. Story Telling Aspects in Medical Applications. Central European Seminar for Computer Graphics, 2006.[3] W. Jorritsma, F. Cnossen, P.M.A. van Ooijen. Improving the radiologist-CAD interaction: designing for appropriate trust. Clinical Radiology, 70(2):115-22, 2014.

https://pure.rug.nl/ws/files/171342405/Visual_Storytelling_for_Informed_Decision_Making_in_Medicine.pdf

Visual Storytelling for Informed Decision-Making in Medicine

Abstract

Similar works

Full text

Available Versions

NARCIS

ARTS repository - University of Groningen

Dissertations of the University of Groningen

Proceedings - University of Groningen

ARTS repository - University of Groningen

University of Groningen

Proceedings - University of Groningen

Dissertations of the University of Groningen