Aprahamian, I.

Azzolino, D.

Calvani, R.

Canevelli, M.

Cesari, Matteo

de Souto Barreto, P.

Fielding, R. A.

Inzitari, M.

Marzetti, E.

Vanacore, N.

English

Cesari, M.

Barreto de Souto, Philipe

Fielding, R.A.

HAL Université de Toulouse, et Toulouse INP

On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical Appraisal

Matteo Cesari

R. Calvani

M. Canevelli

I. Aprahamian

P. de Souto Barreto

D. Azzolino

R. A. Fielding

N. Vanacore

M. Inzitari

E. Marzetti

AIR Universita degli studi di Milano

On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic : A Critical Appraisal

ARTS repository - University of Groningen

   University of GroningenOn Schrodinger's Cat and Evaluation of Trials Disrupted by the Covid19 PandemicCesari, Matteo; Calvani, R.; Canevelli, M.; Aprahamian, I.; de Souto Barreto, P.; Azzolino, D.;Fielding, R. A.; Vanacore, N.; Inzitari, M.; Marzetti, E.Published in:Journal of frailty & agingDOI:10.14283/jfa.2021.23IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2021Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Cesari, M., Calvani, R., Canevelli, M., Aprahamian, I., de Souto Barreto, P., Azzolino, D., Fielding, R. A.,Vanacore, N., Inzitari, M., & Marzetti, E. (2021). On Schrodinger's Cat and Evaluation of Trials Disrupted bythe Covid19 Pandemic: A Critical Appraisal. Journal of frailty & aging, 10(4), 310-312.https://doi.org/10.14283/jfa.2021.23CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 15-10-2024310Received May 5, 2021Accepted for publication May 7, 2021EditorialKey words: Research, methodology, publishing, medicine, science.From the beginning of 2020, the world has been fighting the SARS-Cov-2 outbreak. The life of each one of us has profoundly changed. Unavoidably, our clinical routine has drastically modified in its priorities and methodologies (1). The COVID-19 pandemic has also raised significant issues in the field of research. The investigators’ responsibility has increased with the need to thoughtfully weigh the risk-benefit ratio for each protocol in an emergency and evolving scenario (2). Many projects that had started in the pre-COVID-19 era were halted in the past months, mainly because of 1) the restrictions that governments put in place to limit the spread of the Coronavirus, 2) the inadequacy/inappropriateness of the research activities as planned before the pandemic, and/or 3) the participants’ reluctancy at attending the scheduled visits in facilities usually located within the hospital premises. Some investigators managed to redesign their study methods and make them compatible with the new situation. Someone also proposed possible solutions to tackle the most common issues of the “new” research routine (3). Others faced challenging (if not insurmountable) stalls to solve and eventually decided to stop their studies. Indeed, it was not rare the closure of research activities in some countries, especially during the early period of the pandemic.How should we judge the findings generated by studies that suffered the effects of the COVID-19 pandemic? In principle, the modification of a protocol “on the way” or the premature ending of a study are deviations that, in “normal” times, would be flagged as major methodological flaws. Purists of research methodology may indeed find these deviations hard to accept. But, can we judge today’s research activities without considering the new context of COVID-19 and the global consequences we all have been experiencing? What degree of flexibility should we apply when reviewing an article containing issues clearly due to the pandemic? In this context, it is also essential to consider the increasing interest around pragmatic research for achieving a wise compromise between the potentially decontextualized findings from rigid methodologies and the heterogenous “real life” scenarios where the study results will be applied (4). Perhaps, the pandemic may lead us to introduce sufficient doses of pragmatism and creativity as core elements of future research. For sure, scientific projects will have to include backup plans considering the possibility of postponing/modifying recruitment strategies, maintaining protocol adherence despite environmental changes, and designing alternative strategies for intervention delivery. In this issue of The Journal of Frailty & Aging, Tavoian and colleagues (5) present the results of a pilot study initiated before the COVID-19 outbreak. The trial was focused on exploring whether stationary bicycle high-intensity interval training was a more efficient standalone exercise strategy to improve cardiovascular and muscular function compared with resistance or aerobic training in older adults. Unfortunately, the restrictions imposed by the emergent pandemic situation led to a premature stop of the program, precluding the achievement of the expected sample size and leaving the researchers with partial results. The data could appear insufficient to convey a complete and clear message (at least coherent with the original study aims). For example, the unbalanced representation of women in the study sample (11 vs. only 3 men) might already render the findings of difficult interpretation. On the other hand, we might say that something is always better than nothing! Those partial results could still feed the discussion in the scientific community and serve to support further steps in the field (6). The Chinese philosopher Lao Tzu once said: “The journey of a thousand miles begins with one step”. The study by Tavoian and colleagues was well-presented and conducted rigorously, methodologically speaking. It also had its record on clinicaltrials.gov, having been registered before the © Serdi and Springer Nature Switzerland AG 2021On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical AppraisalM. Cesari1,2, R. Calvani3,4, M. Canevelli5,6, I. Aprahamian7,8, P. de Souto Barreto9,10, D. Azzolino1,2, R.A. Fielding11, N. Vanacore5, M. Inzitari12,13, E. Marzetti3,141. Department of Clinical Sciences and Community Health, University of Milan, Milan, Italy; 2. Healthy Aging Lab, IRCCS Istituti Clinici Scientifici Maugeri, Milan, Italy; 3. Fondazione Policlinico Universitario «Agostino Gemelli» IRCCS, Rome, Italy; 4. Aging Research Center, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet and Stockholm University, Stockholm, Sweden; 5. National Centre for Disease Prevention and Health Promotion, Italian National Institute of Health, Rome, Italy; 6. Department of Human Neuroscience, Sapienza University of Rome, Rome, Italy; 7. Geriatrics Division, Internal Medicine Department, Jundiaí Medical School, Jundiaí, Brazil; 8. University of Groningen, University Medical Center Groningen, Groningen, The Netherlands; 9. Gérontopôle de Toulouse, Institut du Vieillissement, Centre Hospitalo-Universitaire de Toulouse, Toulouse, France; 10. CERPOP INSERM 1295, University of Toulouse III, INSERM, UPS, Toulouse, France; 11. Nutrition, Exercise Physiology, and Sarcopenia Laboratory, Jean Mayer USDA Human Nutrition Research Center on Aging, Tufts University, Boston, MA, USA; 12. RE-FIT Barcelona research group, Parc Sanitari Pere Virgili and Vall d’Hebron Institute of Research (VHIR), Barcelona, Spain; 13. Department of Health Sciences, Universitat Oberta de Catalunya (UOC), Barcelona, Spain; 14. Università Cattolica del Sacro Cuore, Institute of Internal Medicine and Geriatrics, Rome, Italy.Corresponding Author: Matteo Cesari, MD, PhD. IRCCS Istituti Clinici Scientifici Maugeri; Via Camaldoli 64, 20138 Milan, Italy. Email: macesari@gmail.com; Twitter: @macesariJ Frailty Aging 2021;10(4)310-312Published online May 24, 2021, http://dx.doi.org/10.14283/jfa.2021.23 311JFA  - Volume 10, Number 4, 2021initiation of recruitment. The topic was of interest and novel. Last but not least, the trial was framed as a pilot study with the primary aim of collecting preliminary data for future larger-scale activities. Under these premises, it could have been unfair to reject it simply because an external event (i.e., the COVID-19 pandemic) had disrupted the researchers’ original plans. Are we sure that what is (partially) presented by Tavoian and colleagues may not be sufficient to address questions raised by other researchers? After all, when a study is well-conducted, it is always important to publish it, independently of issues that might have affected its conduct. Of course, the expected impact of an unforeseen event disrupting a pilot study is different from that occurring in a phase 3 trial. This implies that a case-by-case approach is crucial but needs to be structured on specific criteria. Otherwise, the risk could be to fall into a dangerous level of subjectivity or to reject everything in search of a utopic perfection. No study is perfect, and every research has an own value. It is a matter of clearly and fairly presenting the experience, constructively discussing the strengths and weaknesses. At the same time, the readers should be transparently informed of the deviations imposed by the COVID-19 outbreak and provided with all the information necessary to judge the value of findings and discern potentially relevant results from “background noise”. The need for clarity and details is important for discriminating vague justifications (e.g., “due to the pandemic”) from founded decisions based on political, sanitary, and/or scientific choices (2). It is also noteworthy that the numbers we report in our tables are persons who have trusted in our research and devoted part of their time to it. The restitution of the study results has to be considered an ethical duty of the researcher in front of the participant. Consistently, many investigations benefit from funds allocated by private or public institutions for specific research purposes and potentially taken from other priorities. It is, thus, also a matter of financial clarity and accountability behavior.With the aim of providing the correct legitimacy to those valuable research activities critically affected by the COVID-19 pandemic, it has been felt that a set of standardizing criteria could be helpful. Table 1 presents aspects that might support clinicians, researchers and policymakers in better judging the quality of the information provided by partial scientific results.The post-COVID-19 world we live in is putting us in the position of reconsidering many aspects of our (professional) lives. Different priorities are today modifying our decisions. How we conduct research today is not the same as before COVID-19. The population we are studying is not the same either, and its values and needs might be different (7). Some adaptations of research to the new scenario (e.g., more extensive use of technologies, higher involvement of participants in collecting their data) are necessary to continue advancing in research and avoiding that what was initiated before COVID-19 could go lost.  Conflicts of interest: MC has received honoraria from Nestlè Health Sciences for presentations at scientific meetings and serving as a member of Expert Advisory Boards. MI has received honoraria from Nestlé Health Sciences for serving as an expert advisor. EM has received honoraria from Abbott, Nestlè, Nutricia, and Thermofisher for presentations at scientific meetings. RAF received grants and personal fees from Nestlé. RAF also reports grants from National Institutes of Health (National Institute on Aging), during the conduct of the study; grants, personal fees and other from Axcella Health, other from Inside Tracker, grants and personal fees from Biophytis, grants and personal fees from Astellas, personal fees from Cytokinetics, personal fees from Amazentis, personal fees from Glaxo Smith Kline, personal fees from Juvicell, outside the submitted work. No conflict of interest reported by the other authors.Table 1. Specific criteria to additionally consider when judging if an article is still worth being published despite COVID-19-related deviations from original plansCriterion How to potentially verifyA solid and innovative rationale for the study • The introduction section should present the scientific rationale and provide sufficient evidence to support the studyCertified design of the study • When applicable and as appropriate, the study protocol is registered (e.g., a trial has a clinicatriasl.gov ID, a systematic review has its record in PROSPERO)• The study was originally of adequate statistical power to answer the research question(s)• The main methodological and logistical complexities encountered in the study and the adopted countermeasures are discussed. This information could be helpful for researchers preparing to conduct similar studiesThe methodological deviations due to COVID-19 are presented and justified • The Methods and Discussion sections should clearly describe the difference(s) between the original plans and the final output of the study• The deviation from the protocol due to the COVID-19 pandemic is adequately justified (e.g., remote assessments were conducted to limit attrition)• An interim statistical analysis for futility or efficacy (also adopting less stringent alpha and beta values) is performed• The Discussion section should clearly and transparently explain how the deviations from the original protocol might have affected the study findings• The study findings are presented and discussed concisely (e.g., in the form of brief communications/short reports rather than as full research papers) and cautiously (i.e., as hypothesis-generating rather than as confirmatory evidence)“Point of no return” • No possibility of temporary suspension of the protocol (i.e., modification of the study timeline)• No possibility of adapting the planned activities to the emergent COVID-19 scenario (i.e., adaptation of the methodology)312ON SCHRÖDINGER’S CAT AND EVALUATION OF TRIALS DISRUPTED BY THE COVID19 PANDEMICAcknowledgments: RAF is partially supported by the US Department of Agriculture (USDA), under agreement No. 58-8050-9-004 and by NIH Boston Claude D Pepper Center (OAIC; 1P30AG031679). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the USDA.References 1. Astrone P, Cesari M. Integrated Care and Geriatrics: A Call to Renovation from the COVID-19 Pandemic. J Frailty Aging. 2021;10(2):182-183. doi: 10.14283/jfa.2020.592.  Fleming TR, Labriola D, Wittes J. Conducting Clinical Research During the COVID-19 Pandemic: Protecting Scientific Integrity. JAMA. 2020;324(1):33-34. doi:10.1001/jama.2020.92863.  Kroenke K, Bair MJ, Sachs GA. Continuing Research During a Crisis. J Gen Intern Med. 2021;36(4):1086-1088. doi:10.1007/s11606-021-06636-54.  Ford I, Norrie J. Pragmatic Trials. N Engl J Med. 2016;375(5):454-463. doi:10.1056/NEJMra15100595.  Tavoian D, Russ DW, Law TD, et al. Effects of three different exercise strategies for optimizing aerobic capacity and skeletal muscle performance in older adults: a pilot study. J Frailty Aging. doi:10.14283/jfa.2021.216.  Karmakar S, Dhar R, Jee B. Covid-19: research methods must be flexible in a crisis. BMJ. 2020:m2668. doi:10.1136/bmj.m26687.  Palmer K, Monaco A, Kivipelto M, et al. The potential long-term impact of the COVID-19 outbreak on patients with non-communicable diseases in Europe: consequences for healthy ageing. Aging Clin Exp Res. 2020;32(7):1189-1194. doi:10.1007/s40520-020-01601-4How to cite this article: M. Cesari, R. Calvani, M. Canevelli, et al. On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical Appraisal. J Frailty Aging 2021;10(4)310-312; http://dx.doi.org/10.14283/jfa.2021.23

On Schrodinger's Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic:A Critical Appraisal

   University of GroningenOn Schrodinger's Cat and Evaluation of Trials Disrupted by the Covid19 PandemicCesari, Matteo; Calvani, R.; Canevelli, M.; Aprahamian, I.; de Souto Barreto, P.; Azzolino, D.;Fielding, R. A.; Vanacore, N.; Inzitari, M.; Marzetti, E.Published in:Journal of frailty & agingDOI:10.14283/jfa.2021.23IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2021Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Cesari, M., Calvani, R., Canevelli, M., Aprahamian, I., de Souto Barreto, P., Azzolino, D., Fielding, R. A.,Vanacore, N., Inzitari, M., & Marzetti, E. (2021). On Schrodinger's Cat and Evaluation of Trials Disrupted bythe Covid19 Pandemic: A Critical Appraisal. Journal of frailty & aging, 10(4), 310-312.https://doi.org/10.14283/jfa.2021.23CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 21-06-2022310Received May 5, 2021Accepted for publication May 7, 2021EditorialKey words: Research, methodology, publishing, medicine, science.From the beginning of 2020, the world has been fighting the SARS-Cov-2 outbreak. The life of each one of us has profoundly changed. Unavoidably, our clinical routine has drastically modified in its priorities and methodologies (1). The COVID-19 pandemic has also raised significant issues in the field of research. The investigators’ responsibility has increased with the need to thoughtfully weigh the risk-benefit ratio for each protocol in an emergency and evolving scenario (2). Many projects that had started in the pre-COVID-19 era were halted in the past months, mainly because of 1) the restrictions that governments put in place to limit the spread of the Coronavirus, 2) the inadequacy/inappropriateness of the research activities as planned before the pandemic, and/or 3) the participants’ reluctancy at attending the scheduled visits in facilities usually located within the hospital premises. Some investigators managed to redesign their study methods and make them compatible with the new situation. Someone also proposed possible solutions to tackle the most common issues of the “new” research routine (3). Others faced challenging (if not insurmountable) stalls to solve and eventually decided to stop their studies. Indeed, it was not rare the closure of research activities in some countries, especially during the early period of the pandemic.How should we judge the findings generated by studies that suffered the effects of the COVID-19 pandemic? In principle, the modification of a protocol “on the way” or the premature ending of a study are deviations that, in “normal” times, would be flagged as major methodological flaws. Purists of research methodology may indeed find these deviations hard to accept. But, can we judge today’s research activities without considering the new context of COVID-19 and the global consequences we all have been experiencing? What degree of flexibility should we apply when reviewing an article containing issues clearly due to the pandemic? In this context, it is also essential to consider the increasing interest around pragmatic research for achieving a wise compromise between the potentially decontextualized findings from rigid methodologies and the heterogenous “real life” scenarios where the study results will be applied (4). Perhaps, the pandemic may lead us to introduce sufficient doses of pragmatism and creativity as core elements of future research. For sure, scientific projects will have to include backup plans considering the possibility of postponing/modifying recruitment strategies, maintaining protocol adherence despite environmental changes, and designing alternative strategies for intervention delivery. In this issue of The Journal of Frailty & Aging, Tavoian and colleagues (5) present the results of a pilot study initiated before the COVID-19 outbreak. The trial was focused on exploring whether stationary bicycle high-intensity interval training was a more efficient standalone exercise strategy to improve cardiovascular and muscular function compared with resistance or aerobic training in older adults. Unfortunately, the restrictions imposed by the emergent pandemic situation led to a premature stop of the program, precluding the achievement of the expected sample size and leaving the researchers with partial results. The data could appear insufficient to convey a complete and clear message (at least coherent with the original study aims). For example, the unbalanced representation of women in the study sample (11 vs. only 3 men) might already render the findings of difficult interpretation. On the other hand, we might say that something is always better than nothing! Those partial results could still feed the discussion in the scientific community and serve to support further steps in the field (6). The Chinese philosopher Lao Tzu once said: “The journey of a thousand miles begins with one step”. The study by Tavoian and colleagues was well-presented and conducted rigorously, methodologically speaking. It also had its record on clinicaltrials.gov, having been registered before the © Serdi and Springer Nature Switzerland AG 2021On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical AppraisalM. Cesari1,2, R. Calvani3,4, M. Canevelli5,6, I. Aprahamian7,8, P. de Souto Barreto9,10, D. Azzolino1,2, R.A. Fielding11, N. Vanacore5, M. Inzitari12,13, E. Marzetti3,141. Department of Clinical Sciences and Community Health, University of Milan, Milan, Italy; 2. Healthy Aging Lab, IRCCS Istituti Clinici Scientifici Maugeri, Milan, Italy; 3. Fondazione Policlinico Universitario «Agostino Gemelli» IRCCS, Rome, Italy; 4. Aging Research Center, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet and Stockholm University, Stockholm, Sweden; 5. National Centre for Disease Prevention and Health Promotion, Italian National Institute of Health, Rome, Italy; 6. Department of Human Neuroscience, Sapienza University of Rome, Rome, Italy; 7. Geriatrics Division, Internal Medicine Department, Jundiaí Medical School, Jundiaí, Brazil; 8. University of Groningen, University Medical Center Groningen, Groningen, The Netherlands; 9. Gérontopôle de Toulouse, Institut du Vieillissement, Centre Hospitalo-Universitaire de Toulouse, Toulouse, France; 10. CERPOP INSERM 1295, University of Toulouse III, INSERM, UPS, Toulouse, France; 11. Nutrition, Exercise Physiology, and Sarcopenia Laboratory, Jean Mayer USDA Human Nutrition Research Center on Aging, Tufts University, Boston, MA, USA; 12. RE-FIT Barcelona research group, Parc Sanitari Pere Virgili and Vall d’Hebron Institute of Research (VHIR), Barcelona, Spain; 13. Department of Health Sciences, Universitat Oberta de Catalunya (UOC), Barcelona, Spain; 14. Università Cattolica del Sacro Cuore, Institute of Internal Medicine and Geriatrics, Rome, Italy.Corresponding Author: Matteo Cesari, MD, PhD. IRCCS Istituti Clinici Scientifici Maugeri; Via Camaldoli 64, 20138 Milan, Italy. Email: macesari@gmail.com; Twitter: @macesariJ Frailty Aging 2021;10(4)310-312Published online May 24, 2021, http://dx.doi.org/10.14283/jfa.2021.23 311JFA  - Volume 10, Number 4, 2021initiation of recruitment. The topic was of interest and novel. Last but not least, the trial was framed as a pilot study with the primary aim of collecting preliminary data for future larger-scale activities. Under these premises, it could have been unfair to reject it simply because an external event (i.e., the COVID-19 pandemic) had disrupted the researchers’ original plans. Are we sure that what is (partially) presented by Tavoian and colleagues may not be sufficient to address questions raised by other researchers? After all, when a study is well-conducted, it is always important to publish it, independently of issues that might have affected its conduct. Of course, the expected impact of an unforeseen event disrupting a pilot study is different from that occurring in a phase 3 trial. This implies that a case-by-case approach is crucial but needs to be structured on specific criteria. Otherwise, the risk could be to fall into a dangerous level of subjectivity or to reject everything in search of a utopic perfection. No study is perfect, and every research has an own value. It is a matter of clearly and fairly presenting the experience, constructively discussing the strengths and weaknesses. At the same time, the readers should be transparently informed of the deviations imposed by the COVID-19 outbreak and provided with all the information necessary to judge the value of findings and discern potentially relevant results from “background noise”. The need for clarity and details is important for discriminating vague justifications (e.g., “due to the pandemic”) from founded decisions based on political, sanitary, and/or scientific choices (2). It is also noteworthy that the numbers we report in our tables are persons who have trusted in our research and devoted part of their time to it. The restitution of the study results has to be considered an ethical duty of the researcher in front of the participant. Consistently, many investigations benefit from funds allocated by private or public institutions for specific research purposes and potentially taken from other priorities. It is, thus, also a matter of financial clarity and accountability behavior.With the aim of providing the correct legitimacy to those valuable research activities critically affected by the COVID-19 pandemic, it has been felt that a set of standardizing criteria could be helpful. Table 1 presents aspects that might support clinicians, researchers and policymakers in better judging the quality of the information provided by partial scientific results.The post-COVID-19 world we live in is putting us in the position of reconsidering many aspects of our (professional) lives. Different priorities are today modifying our decisions. How we conduct research today is not the same as before COVID-19. The population we are studying is not the same either, and its values and needs might be different (7). Some adaptations of research to the new scenario (e.g., more extensive use of technologies, higher involvement of participants in collecting their data) are necessary to continue advancing in research and avoiding that what was initiated before COVID-19 could go lost.  Conflicts of interest: MC has received honoraria from Nestlè Health Sciences for presentations at scientific meetings and serving as a member of Expert Advisory Boards. MI has received honoraria from Nestlé Health Sciences for serving as an expert advisor. EM has received honoraria from Abbott, Nestlè, Nutricia, and Thermofisher for presentations at scientific meetings. RAF received grants and personal fees from Nestlé. RAF also reports grants from National Institutes of Health (National Institute on Aging), during the conduct of the study; grants, personal fees and other from Axcella Health, other from Inside Tracker, grants and personal fees from Biophytis, grants and personal fees from Astellas, personal fees from Cytokinetics, personal fees from Amazentis, personal fees from Glaxo Smith Kline, personal fees from Juvicell, outside the submitted work. No conflict of interest reported by the other authors.Table 1. Specific criteria to additionally consider when judging if an article is still worth being published despite COVID-19-related deviations from original plansCriterion How to potentially verifyA solid and innovative rationale for the study • The introduction section should present the scientific rationale and provide sufficient evidence to support the studyCertified design of the study • When applicable and as appropriate, the study protocol is registered (e.g., a trial has a clinicatriasl.gov ID, a systematic review has its record in PROSPERO)• The study was originally of adequate statistical power to answer the research question(s)• The main methodological and logistical complexities encountered in the study and the adopted countermeasures are discussed. This information could be helpful for researchers preparing to conduct similar studiesThe methodological deviations due to COVID-19 are presented and justified • The Methods and Discussion sections should clearly describe the difference(s) between the original plans and the final output of the study• The deviation from the protocol due to the COVID-19 pandemic is adequately justified (e.g., remote assessments were conducted to limit attrition)• An interim statistical analysis for futility or efficacy (also adopting less stringent alpha and beta values) is performed• The Discussion section should clearly and transparently explain how the deviations from the original protocol might have affected the study findings• The study findings are presented and discussed concisely (e.g., in the form of brief communications/short reports rather than as full research papers) and cautiously (i.e., as hypothesis-generating rather than as confirmatory evidence)“Point of no return” • No possibility of temporary suspension of the protocol (i.e., modification of the study timeline)• No possibility of adapting the planned activities to the emergent COVID-19 scenario (i.e., adaptation of the methodology)312ON SCHRÖDINGER’S CAT AND EVALUATION OF TRIALS DISRUPTED BY THE COVID19 PANDEMICAcknowledgments: RAF is partially supported by the US Department of Agriculture (USDA), under agreement No. 58-8050-9-004 and by NIH Boston Claude D Pepper Center (OAIC; 1P30AG031679). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the USDA.References 1. Astrone P, Cesari M. Integrated Care and Geriatrics: A Call to Renovation from the COVID-19 Pandemic. J Frailty Aging. 2021;10(2):182-183. doi: 10.14283/jfa.2020.592.  Fleming TR, Labriola D, Wittes J. Conducting Clinical Research During the COVID-19 Pandemic: Protecting Scientific Integrity. JAMA. 2020;324(1):33-34. doi:10.1001/jama.2020.92863.  Kroenke K, Bair MJ, Sachs GA. Continuing Research During a Crisis. J Gen Intern Med. 2021;36(4):1086-1088. doi:10.1007/s11606-021-06636-54.  Ford I, Norrie J. Pragmatic Trials. N Engl J Med. 2016;375(5):454-463. doi:10.1056/NEJMra15100595.  Tavoian D, Russ DW, Law TD, et al. Effects of three different exercise strategies for optimizing aerobic capacity and skeletal muscle performance in older adults: a pilot study. J Frailty Aging. doi:10.14283/jfa.2021.216.  Karmakar S, Dhar R, Jee B. Covid-19: research methods must be flexible in a crisis. BMJ. 2020:m2668. doi:10.1136/bmj.m26687.  Palmer K, Monaco A, Kivipelto M, et al. The potential long-term impact of the COVID-19 outbreak on patients with non-communicable diseases in Europe: consequences for healthy ageing. Aging Clin Exp Res. 2020;32(7):1189-1194. doi:10.1007/s40520-020-01601-4How to cite this article: M. Cesari, R. Calvani, M. Canevelli, et al. On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical Appraisal. J Frailty Aging 2021;10(4)310-312; http://dx.doi.org/10.14283/jfa.2021.23

NARCIS 

On Schrodinger's Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical Appraisal

University of Groningen

   University of GroningenOn Schrodinger's Cat and Evaluation of Trials Disrupted by the Covid19 PandemicCesari, Matteo; Calvani, R.; Canevelli, M.; Aprahamian, I.; de Souto Barreto, P.; Azzolino, D.;Fielding, R. A.; Vanacore, N.; Inzitari, M.; Marzetti, E.Published in:Journal of frailty & agingDOI:10.14283/jfa.2021.23IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2021Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Cesari, M., Calvani, R., Canevelli, M., Aprahamian, I., de Souto Barreto, P., Azzolino, D., Fielding, R. A.,Vanacore, N., Inzitari, M., & Marzetti, E. (2021). On Schrodinger's Cat and Evaluation of Trials Disrupted bythe Covid19 Pandemic: A Critical Appraisal. Journal of frailty & aging, 10(4), 310-312.https://doi.org/10.14283/jfa.2021.23CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 20-11-2022310Received May 5, 2021Accepted for publication May 7, 2021EditorialKey words: Research, methodology, publishing, medicine, science.From the beginning of 2020, the world has been fighting the SARS-Cov-2 outbreak. The life of each one of us has profoundly changed. Unavoidably, our clinical routine has drastically modified in its priorities and methodologies (1). The COVID-19 pandemic has also raised significant issues in the field of research. The investigators’ responsibility has increased with the need to thoughtfully weigh the risk-benefit ratio for each protocol in an emergency and evolving scenario (2). Many projects that had started in the pre-COVID-19 era were halted in the past months, mainly because of 1) the restrictions that governments put in place to limit the spread of the Coronavirus, 2) the inadequacy/inappropriateness of the research activities as planned before the pandemic, and/or 3) the participants’ reluctancy at attending the scheduled visits in facilities usually located within the hospital premises. Some investigators managed to redesign their study methods and make them compatible with the new situation. Someone also proposed possible solutions to tackle the most common issues of the “new” research routine (3). Others faced challenging (if not insurmountable) stalls to solve and eventually decided to stop their studies. Indeed, it was not rare the closure of research activities in some countries, especially during the early period of the pandemic.How should we judge the findings generated by studies that suffered the effects of the COVID-19 pandemic? In principle, the modification of a protocol “on the way” or the premature ending of a study are deviations that, in “normal” times, would be flagged as major methodological flaws. Purists of research methodology may indeed find these deviations hard to accept. But, can we judge today’s research activities without considering the new context of COVID-19 and the global consequences we all have been experiencing? What degree of flexibility should we apply when reviewing an article containing issues clearly due to the pandemic? In this context, it is also essential to consider the increasing interest around pragmatic research for achieving a wise compromise between the potentially decontextualized findings from rigid methodologies and the heterogenous “real life” scenarios where the study results will be applied (4). Perhaps, the pandemic may lead us to introduce sufficient doses of pragmatism and creativity as core elements of future research. For sure, scientific projects will have to include backup plans considering the possibility of postponing/modifying recruitment strategies, maintaining protocol adherence despite environmental changes, and designing alternative strategies for intervention delivery. In this issue of The Journal of Frailty & Aging, Tavoian and colleagues (5) present the results of a pilot study initiated before the COVID-19 outbreak. The trial was focused on exploring whether stationary bicycle high-intensity interval training was a more efficient standalone exercise strategy to improve cardiovascular and muscular function compared with resistance or aerobic training in older adults. Unfortunately, the restrictions imposed by the emergent pandemic situation led to a premature stop of the program, precluding the achievement of the expected sample size and leaving the researchers with partial results. The data could appear insufficient to convey a complete and clear message (at least coherent with the original study aims). For example, the unbalanced representation of women in the study sample (11 vs. only 3 men) might already render the findings of difficult interpretation. On the other hand, we might say that something is always better than nothing! Those partial results could still feed the discussion in the scientific community and serve to support further steps in the field (6). The Chinese philosopher Lao Tzu once said: “The journey of a thousand miles begins with one step”. The study by Tavoian and colleagues was well-presented and conducted rigorously, methodologically speaking. It also had its record on clinicaltrials.gov, having been registered before the © Serdi and Springer Nature Switzerland AG 2021On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical AppraisalM. Cesari1,2, R. Calvani3,4, M. Canevelli5,6, I. Aprahamian7,8, P. de Souto Barreto9,10, D. Azzolino1,2, R.A. Fielding11, N. Vanacore5, M. Inzitari12,13, E. Marzetti3,141. Department of Clinical Sciences and Community Health, University of Milan, Milan, Italy; 2. Healthy Aging Lab, IRCCS Istituti Clinici Scientifici Maugeri, Milan, Italy; 3. Fondazione Policlinico Universitario «Agostino Gemelli» IRCCS, Rome, Italy; 4. Aging Research Center, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet and Stockholm University, Stockholm, Sweden; 5. National Centre for Disease Prevention and Health Promotion, Italian National Institute of Health, Rome, Italy; 6. Department of Human Neuroscience, Sapienza University of Rome, Rome, Italy; 7. Geriatrics Division, Internal Medicine Department, Jundiaí Medical School, Jundiaí, Brazil; 8. University of Groningen, University Medical Center Groningen, Groningen, The Netherlands; 9. Gérontopôle de Toulouse, Institut du Vieillissement, Centre Hospitalo-Universitaire de Toulouse, Toulouse, France; 10. CERPOP INSERM 1295, University of Toulouse III, INSERM, UPS, Toulouse, France; 11. Nutrition, Exercise Physiology, and Sarcopenia Laboratory, Jean Mayer USDA Human Nutrition Research Center on Aging, Tufts University, Boston, MA, USA; 12. RE-FIT Barcelona research group, Parc Sanitari Pere Virgili and Vall d’Hebron Institute of Research (VHIR), Barcelona, Spain; 13. Department of Health Sciences, Universitat Oberta de Catalunya (UOC), Barcelona, Spain; 14. Università Cattolica del Sacro Cuore, Institute of Internal Medicine and Geriatrics, Rome, Italy.Corresponding Author: Matteo Cesari, MD, PhD. IRCCS Istituti Clinici Scientifici Maugeri; Via Camaldoli 64, 20138 Milan, Italy. Email: macesari@gmail.com; Twitter: @macesariJ Frailty Aging 2021;10(4)310-312Published online May 24, 2021, http://dx.doi.org/10.14283/jfa.2021.23 311JFA  - Volume 10, Number 4, 2021initiation of recruitment. The topic was of interest and novel. Last but not least, the trial was framed as a pilot study with the primary aim of collecting preliminary data for future larger-scale activities. Under these premises, it could have been unfair to reject it simply because an external event (i.e., the COVID-19 pandemic) had disrupted the researchers’ original plans. Are we sure that what is (partially) presented by Tavoian and colleagues may not be sufficient to address questions raised by other researchers? After all, when a study is well-conducted, it is always important to publish it, independently of issues that might have affected its conduct. Of course, the expected impact of an unforeseen event disrupting a pilot study is different from that occurring in a phase 3 trial. This implies that a case-by-case approach is crucial but needs to be structured on specific criteria. Otherwise, the risk could be to fall into a dangerous level of subjectivity or to reject everything in search of a utopic perfection. No study is perfect, and every research has an own value. It is a matter of clearly and fairly presenting the experience, constructively discussing the strengths and weaknesses. At the same time, the readers should be transparently informed of the deviations imposed by the COVID-19 outbreak and provided with all the information necessary to judge the value of findings and discern potentially relevant results from “background noise”. The need for clarity and details is important for discriminating vague justifications (e.g., “due to the pandemic”) from founded decisions based on political, sanitary, and/or scientific choices (2). It is also noteworthy that the numbers we report in our tables are persons who have trusted in our research and devoted part of their time to it. The restitution of the study results has to be considered an ethical duty of the researcher in front of the participant. Consistently, many investigations benefit from funds allocated by private or public institutions for specific research purposes and potentially taken from other priorities. It is, thus, also a matter of financial clarity and accountability behavior.With the aim of providing the correct legitimacy to those valuable research activities critically affected by the COVID-19 pandemic, it has been felt that a set of standardizing criteria could be helpful. Table 1 presents aspects that might support clinicians, researchers and policymakers in better judging the quality of the information provided by partial scientific results.The post-COVID-19 world we live in is putting us in the position of reconsidering many aspects of our (professional) lives. Different priorities are today modifying our decisions. How we conduct research today is not the same as before COVID-19. The population we are studying is not the same either, and its values and needs might be different (7). Some adaptations of research to the new scenario (e.g., more extensive use of technologies, higher involvement of participants in collecting their data) are necessary to continue advancing in research and avoiding that what was initiated before COVID-19 could go lost.  Conflicts of interest: MC has received honoraria from Nestlè Health Sciences for presentations at scientific meetings and serving as a member of Expert Advisory Boards. MI has received honoraria from Nestlé Health Sciences for serving as an expert advisor. EM has received honoraria from Abbott, Nestlè, Nutricia, and Thermofisher for presentations at scientific meetings. RAF received grants and personal fees from Nestlé. RAF also reports grants from National Institutes of Health (National Institute on Aging), during the conduct of the study; grants, personal fees and other from Axcella Health, other from Inside Tracker, grants and personal fees from Biophytis, grants and personal fees from Astellas, personal fees from Cytokinetics, personal fees from Amazentis, personal fees from Glaxo Smith Kline, personal fees from Juvicell, outside the submitted work. No conflict of interest reported by the other authors.Table 1. Specific criteria to additionally consider when judging if an article is still worth being published despite COVID-19-related deviations from original plansCriterion How to potentially verifyA solid and innovative rationale for the study • The introduction section should present the scientific rationale and provide sufficient evidence to support the studyCertified design of the study • When applicable and as appropriate, the study protocol is registered (e.g., a trial has a clinicatriasl.gov ID, a systematic review has its record in PROSPERO)• The study was originally of adequate statistical power to answer the research question(s)• The main methodological and logistical complexities encountered in the study and the adopted countermeasures are discussed. This information could be helpful for researchers preparing to conduct similar studiesThe methodological deviations due to COVID-19 are presented and justified • The Methods and Discussion sections should clearly describe the difference(s) between the original plans and the final output of the study• The deviation from the protocol due to the COVID-19 pandemic is adequately justified (e.g., remote assessments were conducted to limit attrition)• An interim statistical analysis for futility or efficacy (also adopting less stringent alpha and beta values) is performed• The Discussion section should clearly and transparently explain how the deviations from the original protocol might have affected the study findings• The study findings are presented and discussed concisely (e.g., in the form of brief communications/short reports rather than as full research papers) and cautiously (i.e., as hypothesis-generating rather than as confirmatory evidence)“Point of no return” • No possibility of temporary suspension of the protocol (i.e., modification of the study timeline)• No possibility of adapting the planned activities to the emergent COVID-19 scenario (i.e., adaptation of the methodology)312ON SCHRÖDINGER’S CAT AND EVALUATION OF TRIALS DISRUPTED BY THE COVID19 PANDEMICAcknowledgments: RAF is partially supported by the US Department of Agriculture (USDA), under agreement No. 58-8050-9-004 and by NIH Boston Claude D Pepper Center (OAIC; 1P30AG031679). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the USDA.References 1. Astrone P, Cesari M. Integrated Care and Geriatrics: A Call to Renovation from the COVID-19 Pandemic. J Frailty Aging. 2021;10(2):182-183. doi: 10.14283/jfa.2020.592.  Fleming TR, Labriola D, Wittes J. Conducting Clinical Research During the COVID-19 Pandemic: Protecting Scientific Integrity. JAMA. 2020;324(1):33-34. doi:10.1001/jama.2020.92863.  Kroenke K, Bair MJ, Sachs GA. Continuing Research During a Crisis. J Gen Intern Med. 2021;36(4):1086-1088. doi:10.1007/s11606-021-06636-54.  Ford I, Norrie J. Pragmatic Trials. N Engl J Med. 2016;375(5):454-463. doi:10.1056/NEJMra15100595.  Tavoian D, Russ DW, Law TD, et al. Effects of three different exercise strategies for optimizing aerobic capacity and skeletal muscle performance in older adults: a pilot study. J Frailty Aging. doi:10.14283/jfa.2021.216.  Karmakar S, Dhar R, Jee B. Covid-19: research methods must be flexible in a crisis. BMJ. 2020:m2668. doi:10.1136/bmj.m26687.  Palmer K, Monaco A, Kivipelto M, et al. The potential long-term impact of the COVID-19 outbreak on patients with non-communicable diseases in Europe: consequences for healthy ageing. Aging Clin Exp Res. 2020;32(7):1189-1194. doi:10.1007/s40520-020-01601-4How to cite this article: M. Cesari, R. Calvani, M. Canevelli, et al. On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical Appraisal. J Frailty Aging 2021;10(4)310-312; http://dx.doi.org/10.14283/jfa.2021.23

   University of GroningenOn Schrodinger's Cat and Evaluation of Trials Disrupted by the Covid19 PandemicCesari, Matteo; Calvani, R.; Canevelli, M.; Aprahamian, I.; de Souto Barreto, P.; Azzolino, D.;Fielding, R. A.; Vanacore, N.; Inzitari, M.; Marzetti, E.Published in:Journal of frailty & agingDOI:10.14283/jfa.2021.23IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2021Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Cesari, M., Calvani, R., Canevelli, M., Aprahamian, I., de Souto Barreto, P., Azzolino, D., Fielding, R. A.,Vanacore, N., Inzitari, M., & Marzetti, E. (2021). On Schrodinger's Cat and Evaluation of Trials Disrupted bythe Covid19 Pandemic: A Critical Appraisal. Journal of frailty & aging, 10(4), 310-312.https://doi.org/10.14283/jfa.2021.23CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 29-04-2023310Received May 5, 2021Accepted for publication May 7, 2021EditorialKey words: Research, methodology, publishing, medicine, science.From the beginning of 2020, the world has been fighting the SARS-Cov-2 outbreak. The life of each one of us has profoundly changed. Unavoidably, our clinical routine has drastically modified in its priorities and methodologies (1). The COVID-19 pandemic has also raised significant issues in the field of research. The investigators’ responsibility has increased with the need to thoughtfully weigh the risk-benefit ratio for each protocol in an emergency and evolving scenario (2). Many projects that had started in the pre-COVID-19 era were halted in the past months, mainly because of 1) the restrictions that governments put in place to limit the spread of the Coronavirus, 2) the inadequacy/inappropriateness of the research activities as planned before the pandemic, and/or 3) the participants’ reluctancy at attending the scheduled visits in facilities usually located within the hospital premises. Some investigators managed to redesign their study methods and make them compatible with the new situation. Someone also proposed possible solutions to tackle the most common issues of the “new” research routine (3). Others faced challenging (if not insurmountable) stalls to solve and eventually decided to stop their studies. Indeed, it was not rare the closure of research activities in some countries, especially during the early period of the pandemic.How should we judge the findings generated by studies that suffered the effects of the COVID-19 pandemic? In principle, the modification of a protocol “on the way” or the premature ending of a study are deviations that, in “normal” times, would be flagged as major methodological flaws. Purists of research methodology may indeed find these deviations hard to accept. But, can we judge today’s research activities without considering the new context of COVID-19 and the global consequences we all have been experiencing? What degree of flexibility should we apply when reviewing an article containing issues clearly due to the pandemic? In this context, it is also essential to consider the increasing interest around pragmatic research for achieving a wise compromise between the potentially decontextualized findings from rigid methodologies and the heterogenous “real life” scenarios where the study results will be applied (4). Perhaps, the pandemic may lead us to introduce sufficient doses of pragmatism and creativity as core elements of future research. For sure, scientific projects will have to include backup plans considering the possibility of postponing/modifying recruitment strategies, maintaining protocol adherence despite environmental changes, and designing alternative strategies for intervention delivery. In this issue of The Journal of Frailty & Aging, Tavoian and colleagues (5) present the results of a pilot study initiated before the COVID-19 outbreak. The trial was focused on exploring whether stationary bicycle high-intensity interval training was a more efficient standalone exercise strategy to improve cardiovascular and muscular function compared with resistance or aerobic training in older adults. Unfortunately, the restrictions imposed by the emergent pandemic situation led to a premature stop of the program, precluding the achievement of the expected sample size and leaving the researchers with partial results. The data could appear insufficient to convey a complete and clear message (at least coherent with the original study aims). For example, the unbalanced representation of women in the study sample (11 vs. only 3 men) might already render the findings of difficult interpretation. On the other hand, we might say that something is always better than nothing! Those partial results could still feed the discussion in the scientific community and serve to support further steps in the field (6). The Chinese philosopher Lao Tzu once said: “The journey of a thousand miles begins with one step”. The study by Tavoian and colleagues was well-presented and conducted rigorously, methodologically speaking. It also had its record on clinicaltrials.gov, having been registered before the © Serdi and Springer Nature Switzerland AG 2021On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical AppraisalM. Cesari1,2, R. Calvani3,4, M. Canevelli5,6, I. Aprahamian7,8, P. de Souto Barreto9,10, D. Azzolino1,2, R.A. Fielding11, N. Vanacore5, M. Inzitari12,13, E. Marzetti3,141. Department of Clinical Sciences and Community Health, University of Milan, Milan, Italy; 2. Healthy Aging Lab, IRCCS Istituti Clinici Scientifici Maugeri, Milan, Italy; 3. Fondazione Policlinico Universitario «Agostino Gemelli» IRCCS, Rome, Italy; 4. Aging Research Center, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet and Stockholm University, Stockholm, Sweden; 5. National Centre for Disease Prevention and Health Promotion, Italian National Institute of Health, Rome, Italy; 6. Department of Human Neuroscience, Sapienza University of Rome, Rome, Italy; 7. Geriatrics Division, Internal Medicine Department, Jundiaí Medical School, Jundiaí, Brazil; 8. University of Groningen, University Medical Center Groningen, Groningen, The Netherlands; 9. Gérontopôle de Toulouse, Institut du Vieillissement, Centre Hospitalo-Universitaire de Toulouse, Toulouse, France; 10. CERPOP INSERM 1295, University of Toulouse III, INSERM, UPS, Toulouse, France; 11. Nutrition, Exercise Physiology, and Sarcopenia Laboratory, Jean Mayer USDA Human Nutrition Research Center on Aging, Tufts University, Boston, MA, USA; 12. RE-FIT Barcelona research group, Parc Sanitari Pere Virgili and Vall d’Hebron Institute of Research (VHIR), Barcelona, Spain; 13. Department of Health Sciences, Universitat Oberta de Catalunya (UOC), Barcelona, Spain; 14. Università Cattolica del Sacro Cuore, Institute of Internal Medicine and Geriatrics, Rome, Italy.Corresponding Author: Matteo Cesari, MD, PhD. IRCCS Istituti Clinici Scientifici Maugeri; Via Camaldoli 64, 20138 Milan, Italy. Email: macesari@gmail.com; Twitter: @macesariJ Frailty Aging 2021;10(4)310-312Published online May 24, 2021, http://dx.doi.org/10.14283/jfa.2021.23 311JFA  - Volume 10, Number 4, 2021initiation of recruitment. The topic was of interest and novel. Last but not least, the trial was framed as a pilot study with the primary aim of collecting preliminary data for future larger-scale activities. Under these premises, it could have been unfair to reject it simply because an external event (i.e., the COVID-19 pandemic) had disrupted the researchers’ original plans. Are we sure that what is (partially) presented by Tavoian and colleagues may not be sufficient to address questions raised by other researchers? After all, when a study is well-conducted, it is always important to publish it, independently of issues that might have affected its conduct. Of course, the expected impact of an unforeseen event disrupting a pilot study is different from that occurring in a phase 3 trial. This implies that a case-by-case approach is crucial but needs to be structured on specific criteria. Otherwise, the risk could be to fall into a dangerous level of subjectivity or to reject everything in search of a utopic perfection. No study is perfect, and every research has an own value. It is a matter of clearly and fairly presenting the experience, constructively discussing the strengths and weaknesses. At the same time, the readers should be transparently informed of the deviations imposed by the COVID-19 outbreak and provided with all the information necessary to judge the value of findings and discern potentially relevant results from “background noise”. The need for clarity and details is important for discriminating vague justifications (e.g., “due to the pandemic”) from founded decisions based on political, sanitary, and/or scientific choices (2). It is also noteworthy that the numbers we report in our tables are persons who have trusted in our research and devoted part of their time to it. The restitution of the study results has to be considered an ethical duty of the researcher in front of the participant. Consistently, many investigations benefit from funds allocated by private or public institutions for specific research purposes and potentially taken from other priorities. It is, thus, also a matter of financial clarity and accountability behavior.With the aim of providing the correct legitimacy to those valuable research activities critically affected by the COVID-19 pandemic, it has been felt that a set of standardizing criteria could be helpful. Table 1 presents aspects that might support clinicians, researchers and policymakers in better judging the quality of the information provided by partial scientific results.The post-COVID-19 world we live in is putting us in the position of reconsidering many aspects of our (professional) lives. Different priorities are today modifying our decisions. How we conduct research today is not the same as before COVID-19. The population we are studying is not the same either, and its values and needs might be different (7). Some adaptations of research to the new scenario (e.g., more extensive use of technologies, higher involvement of participants in collecting their data) are necessary to continue advancing in research and avoiding that what was initiated before COVID-19 could go lost.  Conflicts of interest: MC has received honoraria from Nestlè Health Sciences for presentations at scientific meetings and serving as a member of Expert Advisory Boards. MI has received honoraria from Nestlé Health Sciences for serving as an expert advisor. EM has received honoraria from Abbott, Nestlè, Nutricia, and Thermofisher for presentations at scientific meetings. RAF received grants and personal fees from Nestlé. RAF also reports grants from National Institutes of Health (National Institute on Aging), during the conduct of the study; grants, personal fees and other from Axcella Health, other from Inside Tracker, grants and personal fees from Biophytis, grants and personal fees from Astellas, personal fees from Cytokinetics, personal fees from Amazentis, personal fees from Glaxo Smith Kline, personal fees from Juvicell, outside the submitted work. No conflict of interest reported by the other authors.Table 1. Specific criteria to additionally consider when judging if an article is still worth being published despite COVID-19-related deviations from original plansCriterion How to potentially verifyA solid and innovative rationale for the study • The introduction section should present the scientific rationale and provide sufficient evidence to support the studyCertified design of the study • When applicable and as appropriate, the study protocol is registered (e.g., a trial has a clinicatriasl.gov ID, a systematic review has its record in PROSPERO)• The study was originally of adequate statistical power to answer the research question(s)• The main methodological and logistical complexities encountered in the study and the adopted countermeasures are discussed. This information could be helpful for researchers preparing to conduct similar studiesThe methodological deviations due to COVID-19 are presented and justified • The Methods and Discussion sections should clearly describe the difference(s) between the original plans and the final output of the study• The deviation from the protocol due to the COVID-19 pandemic is adequately justified (e.g., remote assessments were conducted to limit attrition)• An interim statistical analysis for futility or efficacy (also adopting less stringent alpha and beta values) is performed• The Discussion section should clearly and transparently explain how the deviations from the original protocol might have affected the study findings• The study findings are presented and discussed concisely (e.g., in the form of brief communications/short reports rather than as full research papers) and cautiously (i.e., as hypothesis-generating rather than as confirmatory evidence)“Point of no return” • No possibility of temporary suspension of the protocol (i.e., modification of the study timeline)• No possibility of adapting the planned activities to the emergent COVID-19 scenario (i.e., adaptation of the methodology)312ON SCHRÖDINGER’S CAT AND EVALUATION OF TRIALS DISRUPTED BY THE COVID19 PANDEMICAcknowledgments: RAF is partially supported by the US Department of Agriculture (USDA), under agreement No. 58-8050-9-004 and by NIH Boston Claude D Pepper Center (OAIC; 1P30AG031679). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the USDA.References 1. Astrone P, Cesari M. Integrated Care and Geriatrics: A Call to Renovation from the COVID-19 Pandemic. J Frailty Aging. 2021;10(2):182-183. doi: 10.14283/jfa.2020.592.  Fleming TR, Labriola D, Wittes J. Conducting Clinical Research During the COVID-19 Pandemic: Protecting Scientific Integrity. JAMA. 2020;324(1):33-34. doi:10.1001/jama.2020.92863.  Kroenke K, Bair MJ, Sachs GA. Continuing Research During a Crisis. J Gen Intern Med. 2021;36(4):1086-1088. doi:10.1007/s11606-021-06636-54.  Ford I, Norrie J. Pragmatic Trials. N Engl J Med. 2016;375(5):454-463. doi:10.1056/NEJMra15100595.  Tavoian D, Russ DW, Law TD, et al. Effects of three different exercise strategies for optimizing aerobic capacity and skeletal muscle performance in older adults: a pilot study. J Frailty Aging. doi:10.14283/jfa.2021.216.  Karmakar S, Dhar R, Jee B. Covid-19: research methods must be flexible in a crisis. BMJ. 2020:m2668. doi:10.1136/bmj.m26687.  Palmer K, Monaco A, Kivipelto M, et al. The potential long-term impact of the COVID-19 outbreak on patients with non-communicable diseases in Europe: consequences for healthy ageing. Aging Clin Exp Res. 2020;32(7):1189-1194. doi:10.1007/s40520-020-01601-4How to cite this article: M. Cesari, R. Calvani, M. Canevelli, et al. On Schrödinger’s Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic: A Critical Appraisal. J Frailty Aging 2021;10(4)310-312; http://dx.doi.org/10.14283/jfa.2021.23

HAL: Hyper Article en Ligne

https://pure.rug.nl/ws/files/196900509/Cesari2021_Article_OnSchr_dingerSCatAndEvaluation.pdf

On Schrodinger's Cat and Evaluation of Trials Disrupted by the Covid19 Pandemic:A Critical Appraisal

Abstract

Similar works

Full text

Available Versions

HAL Université de Toulouse, et Toulouse INP

AIR Universita degli studi di Milano

ARTS repository - University of Groningen

ARTS repository - University of Groningen

NARCIS

University of Groningen

University of Groningen

HAL: Hyper Article en Ligne