Grbich, S

Lam, C

Meinert, E

Milne-Ives, M

Surodina, S

van Velthoven, M

Plymouth Electronic Archive and Research Library

Authors’ Response to Peer ReviewsAuthors’ Response to Peer Reviews of “Machine Learning forRisk Group Identification and User Data Collection in a HerpesSimplex Virus Patient Registry: Algorithm Development andValidation Study”Svitlana Surodina1,2, BSc, MBA; Ching Lam3, MEng; Svetislav Grbich1, MSc; Madison Milne-Ives4, BAS, MSc;Michelle van Velthoven5, BSc, MSc, PhD; Edward Meinert4,6, MA, MSc, MBA, MPA, PhD, CEng FBCS EUR ING1Skein Ltd, London, United Kingdom2Department of Informatics, King’s College London, London, United Kingdom3Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, United Kingdom4Centre for Health Technology, University of Plymouth, Plymouth, United Kingdom5Nuffield Department of Primary Health Sciences, Medical Sciences Division, University of Oxford, Oxford, United Kingdom6Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, United KingdomCorresponding Author:Edward Meinert, MA, MSc, MBA, MPA, PhD, CEng FBCS EUR INGCentre for Health TechnologyUniversity of Plymouth6 Kirkby Place, Room 2Plymouth, PL4 6DNUnited KingdomPhone: 44 1752600600Email: edward.meinert@plymouth.ac.ukRelated Articles:Preprint: http://preprints.jmir.org/preprint/25560Peer-Review Report by William Nowell (Reviewer Z): https://med.jmirx.org/2021/2/e28919/Peer-Review Report by José Alberto Benítez-Andrades (Reviewer BK): https://med.jmirx.org/2021/2/e28922/Published Article: https://med.jmirx.org/2021/2/e25560/(JMIRx Med 2021;2(2):e28917) doi: 10.2196/28917KEYWORDSdata collection; herpes simplex virus; registries; machine learning; risk assessment; artificial intelligence; medical informationsystem; user-centered design; predictor; riskThis is the authors’ response to peer-review reports for"Machine Learning for Risk Group Identification and UserData Collection in a Herpes Simplex Virus Patient Registry:Algorithm Development and Validation Study."Round 1Thank you for your consideration of our manuscript [1] forpublication in JMIR Medical Informatics. We have madeamendments to the manuscript reflecting the valuable reviewcomments [2,3] forwarded to us and feel the paper is nowacceptable for publication. Please let us know if there is anythingwe can do further to improve our paper.Reviewer ZGeneral CommentsWe thank the reviewer [2] for their time and consideration inthe review of our manuscript.1. Thank you for your feedback; the issues raised have beenaddressed as advised by reworking the Introduction, Challenges,and Discussion sections in order to improve the focus and clarityof the paper.2. We agree that there was a lack of precision in specifying thechallenges and overlap in the problems described. We haveedited the list and the objectives to focus on data collection anduser experience in the Aims and Objectives section.JMIRx Med 2021 | vol. 2 | iss. 2 | e28917 | p. 1https://xmed.jmir.org/2021/2/e28917(page number not for citation purposes)Surodina et alJMIRx MedXSL•FORenderXThe privacy and security considerations, while remaining acritical element of the registry development, are not directlyaddressed in our study and have been set aside for future work.Control over one’s own data is linked to the amount of discloseddata (perceived and objective), so it is related to the objectivesaddressed by the algorithm.3. Thank you for the comment. As this paragraph lacked clarityon the fact that the goal of using the ArthritisPower examplewas to illustrate the variety of approaches and the specificity ofregistry contexts, design approaches, and purpose, we havehighlighted the differences and limitations of design studies forregistries such as ArthritisPower.4. In the Introduction, we have added the section Review of PastStudies, which narrows down the focus of our study and betterplaces it in the context of past research.5. In using the US National Health and Nutrition ExaminationSurvey (NHANES) data set, we were guided by the NHANESdata disclosure terms, which indicate that all data sets complywith the anonymization requirements and have been approvedby the National Center for Health Statistics Research EthicsReview Board (previously the NHANES Institutional ReviewBoard [IRB]; protocol #2018-01). Hence, since we onlyanalyzed anonymized data without identification, no directcommunication with the IRB was made. The guidance termsand conditions used can be found online [4].We have added a corresponding comment to the Database Usedsubsection. The sentence was expanded.Minor Comments6. Pg 3, lines 3,4: The word “data” has been added.7. Pg 3, line 7: We have updated the Introduction section witha few examples and a reference to our previous study, whichfocused on the unique challenges that a herpes simplex virus(HSV) registry design poses [5]. Paragraphs in the Introductionand Discussion sections have been expanded8. Pg 6: Thank you for the comment; the criteria in the list havebeen edited to clearly differentiate between each point.Clarifications were also added. Bullet points were edited andexpanded upon.9. Pg 6, line 20: As JMIR uses the American MedicalAssociation style guide, we edited the spelling to be consistentwith American spelling. Spelling was changed in 17 instances.10. Pg 7, lines 16,17: This ratio was chosen in order to keep thevariance low and to leave enough data for training, and it isused as a standard split. A smaller training set was tested;however, it resulted in poorer performance. The text wasclarified to add the rationale.11. Pg 8: A sentence has been added to clarify the process andthe scope of the paper.12. Pg 9, lines 24-26: We have edited the paragraph to includethe indication of the maximum possible number of questions(n=62) and predicted time to complete the full questionnaire.Moreover, we additionally reviewed the literature to estimatethe expected improvement in drop-off rates and added this tothe text. The maximum number of questions (n=62) has beenadded to the text, as well as an estimate of an average reductionin time that is needed to answer the questionnaire to generatea high-reliability risk group prediction.13. Pg 10, line 1: The edit was made accordingly.14. Pg 11, lines 13,14: We primarily relied on the result of ourprevious study [5], where semistructured reviews indicated thesechallenges and such links. In terms of quantitative evidencefrom the literature, it was hard to estimate the degree of possibleimprovements; the existing research highlights such complexitydue to the multifactorial nature of “response burden,” butmultiple research papers showed evidence of an improvementwhen using shorter questionnaires, with some cases of betterretention among particular groups [6,7]. One future directionwould be to test the actual change in real data once the modelis trialed. Moreover, some of the sensitive questions (that areusually the best indications for HSV type 2) were removed atthe data preprocessing stage, still obtaining a high accuracy forthe model. The Data Set Preparation section has been expanded.15. Pg 11, line 21: This has been changed to reflect thesuggestion.16. Pg 12, lines 5,6: Although the users are based in the UnitedKingdom and the study focused on the UK context, for thepurposes of this study, we used a US data set due to the freeaccess, sufficient size, and presence of extensive variables. Ourresearch aims to lay the groundwork that would be applicablefor patient data collection systems both among UK and USusers. The Intended User Journey subsection has been edited.17. Pg 12, line 22: A more detailed description has been added.Reviewer BKGeneral CommentsWe thank the reviewer [3] for their time and consideration inthe review of our manuscript. The issues raised have beenaddressed in view of the peer-review feedback provided.Specific CommentsMajor Comments1. We have reviewed the guidelines and applied the followingchanges:• Structure• Changed Background to Introduction• Heading format (removed numbering)• Added titles for the multimedia appendices• Shorten the paper by moving a figure into the MultimediaAppendices section• Removed author-made abbreviations• Edited title• Edited the order of sections2. In the Introduction, we have added the section Review of PastStudies, which narrows down the focus of our study and reviewsstudies applied to the classification problem in the context underconsideration.JMIRx Med 2021 | vol. 2 | iss. 2 | e28917 | p. 2https://xmed.jmir.org/2021/2/e28917(page number not for citation purposes)Surodina et alJMIRx MedXSL•FORenderX3. We have added the clarification that the split into the trainand test subsets was done at random to ensure the data in bothdata sets were evenly distributed. We thank you for highlightingthe missing details on preprocessing; this was separated into asubsection and clarified. Cross-validation: thank you, this indeedwasn't described in the text. We have now added a sectiondescribing these steps.4. A matrix of confusion has been added.5. Thank you for the suggestion; we have used GridSearchCVbut did not mention it in the previous version of the paper. Wehave added a section describing these steps.6. In the Review of Past Studies section, we have comparedexisting models with the proposed approach.7. The code has been added to an open repository on GitHuband is now available online [8].Round 2Reviewer ZSpecific CommentsMajor CommentsThank you for your valuable review comments [2]. We havenow worked on a copyediting review, addressing the UK Englishspelling instances among other issues.Minor Comments1-3. The text has been edited as suggested.4. The labels have been added.5-9. The text has been edited as suggested.References1. Surodina S, Lam C, Grbich S, Milne-Ives M, van Velthoven M, Meinert E. Machine Learning for Risk Group Identificationand User Data Collection in a Herpes Simplex Virus Patient Registry: Algorithm Development and Validation Study.JMIRx Med 2021 Jun 10;2(2):e25560 [FREE Full text] [doi: 10.2196/25560]2. Nowell WB. Peer Review of "Machine Learning for Risk Group Identification and User Data Collection in a Herpes SimplexVirus Patient Registry: Algorithm Development and Validation Study". JMIRx Med 2021 Jun 10;2(2):e28919 [FREE Fulltext] [doi: 10.2196/28919]3. Benítez Andrades JA. Peer Review of "Machine Learning for Risk Group Identification and User Data Collection in aHerpes Simplex Virus Patient Registry: Algorithm Development and Validation Study". JMIRx Med 2021 Jun 10;2(2):e28922[FREE Full text] [doi: 10.2196/28922]4. Data User Agreement. National Center for Health Statistics, Centers for Disease Control and Prevention. URL: https://www.cdc.gov/nchs/data_access/restrictions.htm [accessed 2021-05-31]5. van Velthoven MH, Lam C, de Cock C, Stenfors T, Chaudhury H, Meinert E. Development of an Innovative Real-WorldEvidence Registry for the Herpes Simplex Virus: Case Study. JMIR Dermatol 2020 Mar 12;3(1):e16933. [doi: 10.2196/16933]6. Brueton VC, Tierney J, Stenning S, Harding S, Meredith S, Nazareth I, et al. Strategies to improve retention in randomisedtrials. Cochrane Database Syst Rev 2013 Dec 03(12):MR000032 [FREE Full text] [doi: 10.1002/14651858.MR000032.pub2][Medline: 24297482]7. Rolstad S, Adler J, Rydén A. Response burden and questionnaire length: is shorter better? A review and meta-analysis.Value Health 2011 Dec;14(8):1101-1108 [FREE Full text] [doi: 10.1016/j.jval.2011.06.003] [Medline: 22152180]8. Denvert/HSV_Alpha_Public. GitHub. URL: https://github.com/Denvert/HSV_Alpha_Public [accessed 2021-05-31]Edited by G Eysenbach; this is a non–peer-reviewed article. Submitted 18.03.21; accepted 18.03.21; published 11.06.21.Please cite as:Surodina S, Lam C, Grbich S, Milne-Ives M, van Velthoven M, Meinert EAuthors’Response to Peer Reviews of “Machine Learning for Risk Group Identification and User Data Collection in a Herpes SimplexVirus Patient Registry: Algorithm Development and Validation Study”JMIRx Med 2021;2(2):e28917URL: https://xmed.jmir.org/2021/2/e28917doi: 10.2196/28917PMID:©Svitlana Surodina, Ching Lam, Svetislav Grbich, Madison Milne-Ives, Michelle van Velthoven, Edward Meinert. Originallypublished in JMIRx Med (https://med.jmirx.org), 11.06.2021. This is an open-access article distributed under the terms of theCreative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution,and reproduction in any medium, provided the original work, first published in JMIRx Med, is properly cited. The completebibliographic information, a link to the original publication on https://med.jmirx.org/, as well as this copyright and licenseinformation must be included.JMIRx Med 2021 | vol. 2 | iss. 2 | e28917 | p. 3https://xmed.jmir.org/2021/2/e28917(page number not for citation purposes)Surodina et alJMIRx MedXSL•FORenderX

Authors’ Response to Peer Reviews of “Machine Learning for Risk Group Identification and User Data Collection in a Herpes Simplex Virus Patient Registry: Algorithm Development and Validation Study”

Crossref

Surodina, Svitlana

Lam, Ching

Grbich, Svetislav

Milne-Ives, Madison

van Velthoven, Michelle

Meinert, Edward

Oxford University Research Archive

Authors’ response to peer reviews of “Machine learning for risk group identification and user data collection in a herpes simplex virus patient registry: algorithm development and validation study”

https://pearl.plymouth.ac.uk/bitstream/10026.1/18918/1/PDF.pdf

Authors’ Response to Peer Reviews of “Machine Learning for Risk Group Identification and User Data Collection in a Herpes Simplex Virus Patient Registry: Algorithm Development and Validation Study”

Abstract

Similar works

Full text

Available Versions

Plymouth Electronic Archive and Research Library

Crossref

Oxford University Research Archive