Voors, Adriaan A.

English

Proceedings - University of Groningen

   University of GroningenShould Enrichment With Natriuretic Peptide Levels Be Mandatory in Global Clinical Trials?Voors, Adriaan A.Published in:JACC. Heart failureDOI:10.1016/j.jchf.2020.01.007IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2020Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Voors, A. A. (2020). Should Enrichment With Natriuretic Peptide Levels Be Mandatory in Global ClinicalTrials? JACC. Heart failure, 8(5), 369-371. https://doi.org/10.1016/j.jchf.2020.01.007CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 31-10-2023J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0ª 2 0 2 0 B Y T H E AM E R I C A N C O L L E G E O F C A R D I O L O G Y F O UN DA T I O NP U B L I S H E D B Y E L S E V I E REDITORIAL COMMENTShould Enrichment WithNatriuretic Peptide LevelsBe Mandatory in Global Clinical Trials?*Adriaan A. Voors, MD, PHDT he design of large, phase III randomizedclinical drug trials in patients with heartfailure requires a careful selection of inclu-sion and exclusion criteria. These criteria should: 1)reflect the target population that will most likelybenefit from the investigational product; 2) be nottoo lenient as this allows low-risk patients and thoseout of the target population to be recruited; 3) be nottoo strict as this will limit trial recruitment and clin-ical applicability of the investigational product; 4)make it reasonably certain that the patients’ condi-tions are indeed diagnosed with heart failure (diag-nostic criteria); and 5) reflect a patient populationat (high) risk for the primary endpoint (enrichmentcriteria).SEE PAGE 359Two criteria frequently used for both diagnosticand enrichment purposes are previous heart failurehospital admission and plasma N-terminal pro–B-typenatriuretic peptide (NT-proBNP) thresholds. In thisissue of JACC: Heart Failure, Cunningham et al. (1)describe what happens when NT-proBNP criteria areadded to the heart failure hospitalization criteria onlyISSN 2213-1779/$36.00*Editorials published in JACC: Heart Failure reflect the views of theauthors and do not necessarily represent the views of JACC: Heart Failureor the American College of Cardiology.From the University of Groningen, Department of Cardiology, UniversityMedical Center Groningen, Groningen, the Netherlands. Dr. Voor is aconsultant for and has received support from Amgen, AstraZeneca,Bayer, Boehringer Ingelheim, Cytokinetics, MyoKardia, Novartis,NovoNordisk, and Roche Diagnostics.The author attests they are in compliance with human studiescommittees and animal welfare regulations of the authors’ institutionsand Food and Drug Administration guidelines, including patient consentwhere appropriate. For more information, visit the JACC: Heart Failureauthor instructions page.after one-fourth of the patients are already recruited.This happened after a protocol amendment in theCOMMANDER-HF (A Study to Assess the Effective-ness and Safety of Rivaroxaban in Reducing the Riskof Death, Myocardial Infarction or Stroke in Partici-pants With Heart Failure and Coronary ArteryDisease Following an Episode of DecompensatedHeart Failure; NCT01877915) trial.COMMANDER-HF was a double-blind, randomizedtrial of the effects of the direct factor Xa inhibitorrivaroxaban in 5,022 chronic heart failure patient witha left ventricular ejection fraction of 40% or less,coronary artery disease, and a recent (<21 days) heartfailure hospital admission (2). The trial did not show areduction of the primary outcome, which was thecomposite of death from any cause, or myocardialinfarction, or stroke.COMMANDER-HF started with only 1 of thefrequently used diagnostic/enrichment criterion (arecent heart failure hospital admission) and did notrequire an NT-proBNP threshold for inclusion. How-ever, after the enrollment of 1,155 patients (23.0%),the study leadership observed an overall even ratethat was lower than expected. By that time, the ma-jority of patients (89%) were enrolled in EasternEurope, and only 1% in North America, 4% in LatinAmerica and Asia Pacific, and 5% in Western Europeand South Africa. Surprisingly, the paper by Cun-ningham et al. (1) does not list the countries whichthey considered to be Eastern European. Variousdefinitions of Eastern Europe exist, as the definitionsare based on historic, religious, and cultural differ-ences and to a lesser extent on geographic bound-aries. For example, Latvia and Lithuania enrolledpatients in COMMANDER-HF but those are notEastern European countries. Nonetheless, because ofthe very low event rate, the steering committeehttps://doi.org/10.1016/j.jchf.2020.01.007Voors J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0Natriuretic Peptide Enrichment of Global Heart Failure Trials M A Y 2 0 2 0 : 3 6 9 – 7 1370amended the enrollment criteria to require a plasmaNT-proBNP concentration of >800 ng/l or a BNPconcentration of >200 ng/l at any time between theindex admission for decompensated HF andrandomization. The purpose of the amendment “wasto prevent further enrollment of low risk patientswho had not truly been hospitalized for decom-pensated HF, and thereby increase event rates andallow for more rapid completion of this event-driventrial” (1). The addition of NT-proBNP criteria later inthe trial provided a unique opportunity to comparepatient characteristics and event rates before andafter the protocol amendment.First, as expected, after allowing only patients withelevated NT-proBNP levels, the event rate increasedsubstantially. Second, after the amendment, thecharacteristics of the patients significantly changedas well. Although this does not come as a surprise,those designing large randomized clinical heart fail-ure trials should be aware of those changes. Forexample, adding an NT-proBNP criterion will result inan older population. This may not always be desiredas treatment and side effects may be different.Notably, in the paper by Cunningham et al. (1)bleeding events increased in the patients treated withrivaroxaban after the amendment as well. Higher NT-proBNP thresholds may also increase the number ofpatients with atrial fibrillation, but although patientswith atrial fibrillation were excluded fromCOMMANDER-HF, this was not confirmed in thepresent study.The most remarkable finding of the present studywas that recruitment slowed considerably in EasternEurope and picked up in other regions. This mighthave been related to the larger effect that theamendment had on Eastern Europe, but it should benoted that other regions, such as Asia Pacific, hadnot been initiated before the amendment. Evenmore surprising was the fact that the increase in theevent rate after the amendment was seen only inEastern Europe and not in any other region of theworld. Thus, the increase in event rate in the trialwas completely driven by the increase in EasternEurope. These findings suggest that adding NT-proBNP enrichment criteria had a much greaterimpact on Eastern Europe than it did on otherregions.How can we interpret these findings? The authorssuggest that when no objective NT-proBNP criteriaare used, the sites in Eastern Europe that recruitedpatients might not have had acute decompensatedheart failure after all. This may be related to culturaldifferences where the threshold for a heart failurehospital admission is much lower than in other partsof the world. Given these cultural differences and thesubstantial recruitment (>50%) from Eastern Europein the COMMANDER-HF trial, it is remarkable thatnone of the executive members and/or data safetymonitoring board members was from 1 of the EasternEuropean countries. In addition, if it is true “that sitesin Eastern Europe recruited patients who might nothave had acute decompensated heart failure after all(1),” why was this not picked up by the ClinicalResearch Organization (CRO). They should havechecked eligibility criteria preferably on site fromsource documents, and presumably these criteriawere not violated or patients should not have beenallowed to be included in the final analyses. Thisimplies that either the sites in Eastern Europecorrectly recruited patients that met all eligibilitycriteria or the noncompliance with these criteria wasnot discovered and reported by the CRO. Alterna-tively, the definitions of “symptoms of worseningdyspnea or fatigue, objective signs of congestion,and/or adjustment of HF medications, requiringhospital admission or unscheduled parenteraldiuretic” may be not be specific enough and patientsin Eastern Europe might have met these criteria butdid not yield the expected and required patientpopulation.In contrast, the present findings fromCOMMANDER-HF share remarkable similarities withthose from TOPCAT. TOPCAT (Treatment of Pre-served Cardiac Function Heart Failure with an Aldo-sterone Antagonist Trial) failed to demonstrate thatspironolactone reduced death from cardiovascularcauses or aborted cardiac arrest or heart failure hos-pitalizations in 3,445 patients with heart failure withpreserved ejection fraction (3). Patients could beincluded based on either a previous (<12 months)hospitalization due to heart failure or based onelevated NT-proBNP levels. The trial recruited pa-tients from North America, Eastern Europe, and SouthAmerica. Sites in Russia and the Republic of Georgia(officially not an Eastern European country) providedmost of the early enrollment, primarily based on thehospitalization criterion because NT-proBNP levelswere initially unavailable there (4). Similar to eventrates in COMMANDER-HF, the clinical event rate wasstrikingly low in Russia and Georgia. In addition, lownatriuretic peptide measurements from Russia andGeorgia, available later in the trial, suggested no heartfailure or only mild heart failure. The primary resultsshowed a significant reduction of the primaryendpoint in North and South America but not inRussia and Georgia (5). These data, together with thefinding that no spironolactone metabolite could bedemonstrated in the urine samples from a largeJ A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0 VoorsM A Y 2 0 2 0 : 3 6 9 – 7 1 Natriuretic Peptide Enrichment of Global Heart Failure Trials371proportion of active-arm patients in Russia andGeorgia, aroused concerns regarding study conduct atsome sites in these countries. Although lessons couldhave been learned from TOPCAT, COMMANDER-HFwas already well underway, and the amendmentwas applied when these results became available.What do we learn from these important data byCunningham et al. (1) First, as expected, addingNT-proBNP thresholds to the eligibility criteria ofheart failure trials will increase event rates. Second,also expected but important to realize, addingNT-proBNP criteria will substantially change thepatient characteristics and may also influencetreatment responses and side effects. Third, addingNT-proBNP criteria reduces regional heterogeneityin patient characteristics and clinical events.Finally, as the Data Safety and Monitoring Board ofTOPCAT recommended (4), “launch the trial in avariety of geographic jurisdictions, and do not allow1 or 2 geographic areas to dominate early (or late)enrollment.”ADDRESS FOR CORRESPONDENCE: Dr. Adriaan A.Voors, University of Groningen, Department ofCardiology, University Medical Center Groningen,Hanzeplein 1, 9713GZ Groningen, the Netherlands.E-mail: a.a.voors@umcg.nl.RE F E RENCE S1. Cunningham JW, Ferreira JP, Deng H, et al.Natriuretic peptide-based inclusion criteria ina heart failure clinical trial: insights fromCOMMANDER HF. J Am Coll Cardiol HF 2020;8:359–68.2. Zannad F, Anker SD, Byra WM, et al., for theCOMMANDER HF Investigators. Rivaroxaban inpatients with heart failure, sinus rhythm, andcoronary disease. N Engl J Med 2018;379:1332–42.3. Pitt B, Pfeffer MA, Assmann SF, et al., for theTOPCAT Investigators. Spironolactone for heartfailure with preserved ejection fraction. N Engl JMed 2014;370:1383–92.4. Pfeffer MA, Claggett B, Assmann SF, et al.Regional variation in patients and outcomes in theTreatment of Preserved Cardiac FunctionHeart Failure with an Aldosterone Antagonist(TOPCAT) trial. Circulation 2015;131:34–42.5. Bristow MR, Enciso JS, Gersh BJ, et al. Detec-tion and management of geographic disparities inthe TOPCAT trial: lessons learned and derivativerecommendations. J Am Coll Cardiol Basic TransScience 2016;1:180–9.KEY WORDS cardiac biomarkers, clinical trials,heart failure with reduced ejection fraction

Should Enrichment With Natriuretic Peptide Levels Be Mandatory in Global Clinical Trials?

   University of GroningenShould Enrichment With Natriuretic Peptide Levels Be Mandatory in Global Clinical Trials?Voors, Adriaan A.Published in:JACC. Heart failureDOI:10.1016/j.jchf.2020.01.007IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2020Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Voors, A. A. (2020). Should Enrichment With Natriuretic Peptide Levels Be Mandatory in Global ClinicalTrials? JACC. Heart failure, 8(5), 369-371. https://doi.org/10.1016/j.jchf.2020.01.007CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 04-06-2022J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0ª 2 0 2 0 B Y T H E AM E R I C A N C O L L E G E O F C A R D I O L O G Y F O UN DA T I O NP U B L I S H E D B Y E L S E V I E REDITORIAL COMMENTShould Enrichment WithNatriuretic Peptide LevelsBe Mandatory in Global Clinical Trials?*Adriaan A. Voors, MD, PHDT he design of large, phase III randomizedclinical drug trials in patients with heartfailure requires a careful selection of inclu-sion and exclusion criteria. These criteria should: 1)reflect the target population that will most likelybenefit from the investigational product; 2) be nottoo lenient as this allows low-risk patients and thoseout of the target population to be recruited; 3) be nottoo strict as this will limit trial recruitment and clin-ical applicability of the investigational product; 4)make it reasonably certain that the patients’ condi-tions are indeed diagnosed with heart failure (diag-nostic criteria); and 5) reflect a patient populationat (high) risk for the primary endpoint (enrichmentcriteria).SEE PAGE 359Two criteria frequently used for both diagnosticand enrichment purposes are previous heart failurehospital admission and plasma N-terminal pro–B-typenatriuretic peptide (NT-proBNP) thresholds. In thisissue of JACC: Heart Failure, Cunningham et al. (1)describe what happens when NT-proBNP criteria areadded to the heart failure hospitalization criteria onlyISSN 2213-1779/$36.00*Editorials published in JACC: Heart Failure reflect the views of theauthors and do not necessarily represent the views of JACC: Heart Failureor the American College of Cardiology.From the University of Groningen, Department of Cardiology, UniversityMedical Center Groningen, Groningen, the Netherlands. Dr. Voor is aconsultant for and has received support from Amgen, AstraZeneca,Bayer, Boehringer Ingelheim, Cytokinetics, MyoKardia, Novartis,NovoNordisk, and Roche Diagnostics.The author attests they are in compliance with human studiescommittees and animal welfare regulations of the authors’ institutionsand Food and Drug Administration guidelines, including patient consentwhere appropriate. For more information, visit the JACC: Heart Failureauthor instructions page.after one-fourth of the patients are already recruited.This happened after a protocol amendment in theCOMMANDER-HF (A Study to Assess the Effective-ness and Safety of Rivaroxaban in Reducing the Riskof Death, Myocardial Infarction or Stroke in Partici-pants With Heart Failure and Coronary ArteryDisease Following an Episode of DecompensatedHeart Failure; NCT01877915) trial.COMMANDER-HF was a double-blind, randomizedtrial of the effects of the direct factor Xa inhibitorrivaroxaban in 5,022 chronic heart failure patient witha left ventricular ejection fraction of 40% or less,coronary artery disease, and a recent (<21 days) heartfailure hospital admission (2). The trial did not show areduction of the primary outcome, which was thecomposite of death from any cause, or myocardialinfarction, or stroke.COMMANDER-HF started with only 1 of thefrequently used diagnostic/enrichment criterion (arecent heart failure hospital admission) and did notrequire an NT-proBNP threshold for inclusion. How-ever, after the enrollment of 1,155 patients (23.0%),the study leadership observed an overall even ratethat was lower than expected. By that time, the ma-jority of patients (89%) were enrolled in EasternEurope, and only 1% in North America, 4% in LatinAmerica and Asia Pacific, and 5% in Western Europeand South Africa. Surprisingly, the paper by Cun-ningham et al. (1) does not list the countries whichthey considered to be Eastern European. Variousdefinitions of Eastern Europe exist, as the definitionsare based on historic, religious, and cultural differ-ences and to a lesser extent on geographic bound-aries. For example, Latvia and Lithuania enrolledpatients in COMMANDER-HF but those are notEastern European countries. Nonetheless, because ofthe very low event rate, the steering committeehttps://doi.org/10.1016/j.jchf.2020.01.007Voors J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0Natriuretic Peptide Enrichment of Global Heart Failure Trials M A Y 2 0 2 0 : 3 6 9 – 7 1370amended the enrollment criteria to require a plasmaNT-proBNP concentration of >800 ng/l or a BNPconcentration of >200 ng/l at any time between theindex admission for decompensated HF andrandomization. The purpose of the amendment “wasto prevent further enrollment of low risk patientswho had not truly been hospitalized for decom-pensated HF, and thereby increase event rates andallow for more rapid completion of this event-driventrial” (1). The addition of NT-proBNP criteria later inthe trial provided a unique opportunity to comparepatient characteristics and event rates before andafter the protocol amendment.First, as expected, after allowing only patients withelevated NT-proBNP levels, the event rate increasedsubstantially. Second, after the amendment, thecharacteristics of the patients significantly changedas well. Although this does not come as a surprise,those designing large randomized clinical heart fail-ure trials should be aware of those changes. Forexample, adding an NT-proBNP criterion will result inan older population. This may not always be desiredas treatment and side effects may be different.Notably, in the paper by Cunningham et al. (1)bleeding events increased in the patients treated withrivaroxaban after the amendment as well. Higher NT-proBNP thresholds may also increase the number ofpatients with atrial fibrillation, but although patientswith atrial fibrillation were excluded fromCOMMANDER-HF, this was not confirmed in thepresent study.The most remarkable finding of the present studywas that recruitment slowed considerably in EasternEurope and picked up in other regions. This mighthave been related to the larger effect that theamendment had on Eastern Europe, but it should benoted that other regions, such as Asia Pacific, hadnot been initiated before the amendment. Evenmore surprising was the fact that the increase in theevent rate after the amendment was seen only inEastern Europe and not in any other region of theworld. Thus, the increase in event rate in the trialwas completely driven by the increase in EasternEurope. These findings suggest that adding NT-proBNP enrichment criteria had a much greaterimpact on Eastern Europe than it did on otherregions.How can we interpret these findings? The authorssuggest that when no objective NT-proBNP criteriaare used, the sites in Eastern Europe that recruitedpatients might not have had acute decompensatedheart failure after all. This may be related to culturaldifferences where the threshold for a heart failurehospital admission is much lower than in other partsof the world. Given these cultural differences and thesubstantial recruitment (>50%) from Eastern Europein the COMMANDER-HF trial, it is remarkable thatnone of the executive members and/or data safetymonitoring board members was from 1 of the EasternEuropean countries. In addition, if it is true “that sitesin Eastern Europe recruited patients who might nothave had acute decompensated heart failure after all(1),” why was this not picked up by the ClinicalResearch Organization (CRO). They should havechecked eligibility criteria preferably on site fromsource documents, and presumably these criteriawere not violated or patients should not have beenallowed to be included in the final analyses. Thisimplies that either the sites in Eastern Europecorrectly recruited patients that met all eligibilitycriteria or the noncompliance with these criteria wasnot discovered and reported by the CRO. Alterna-tively, the definitions of “symptoms of worseningdyspnea or fatigue, objective signs of congestion,and/or adjustment of HF medications, requiringhospital admission or unscheduled parenteraldiuretic” may be not be specific enough and patientsin Eastern Europe might have met these criteria butdid not yield the expected and required patientpopulation.In contrast, the present findings fromCOMMANDER-HF share remarkable similarities withthose from TOPCAT. TOPCAT (Treatment of Pre-served Cardiac Function Heart Failure with an Aldo-sterone Antagonist Trial) failed to demonstrate thatspironolactone reduced death from cardiovascularcauses or aborted cardiac arrest or heart failure hos-pitalizations in 3,445 patients with heart failure withpreserved ejection fraction (3). Patients could beincluded based on either a previous (<12 months)hospitalization due to heart failure or based onelevated NT-proBNP levels. The trial recruited pa-tients from North America, Eastern Europe, and SouthAmerica. Sites in Russia and the Republic of Georgia(officially not an Eastern European country) providedmost of the early enrollment, primarily based on thehospitalization criterion because NT-proBNP levelswere initially unavailable there (4). Similar to eventrates in COMMANDER-HF, the clinical event rate wasstrikingly low in Russia and Georgia. In addition, lownatriuretic peptide measurements from Russia andGeorgia, available later in the trial, suggested no heartfailure or only mild heart failure. The primary resultsshowed a significant reduction of the primaryendpoint in North and South America but not inRussia and Georgia (5). These data, together with thefinding that no spironolactone metabolite could bedemonstrated in the urine samples from a largeJ A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0 VoorsM A Y 2 0 2 0 : 3 6 9 – 7 1 Natriuretic Peptide Enrichment of Global Heart Failure Trials371proportion of active-arm patients in Russia andGeorgia, aroused concerns regarding study conduct atsome sites in these countries. Although lessons couldhave been learned from TOPCAT, COMMANDER-HFwas already well underway, and the amendmentwas applied when these results became available.What do we learn from these important data byCunningham et al. (1) First, as expected, addingNT-proBNP thresholds to the eligibility criteria ofheart failure trials will increase event rates. Second,also expected but important to realize, addingNT-proBNP criteria will substantially change thepatient characteristics and may also influencetreatment responses and side effects. Third, addingNT-proBNP criteria reduces regional heterogeneityin patient characteristics and clinical events.Finally, as the Data Safety and Monitoring Board ofTOPCAT recommended (4), “launch the trial in avariety of geographic jurisdictions, and do not allow1 or 2 geographic areas to dominate early (or late)enrollment.”ADDRESS FOR CORRESPONDENCE: Dr. Adriaan A.Voors, University of Groningen, Department ofCardiology, University Medical Center Groningen,Hanzeplein 1, 9713GZ Groningen, the Netherlands.E-mail: a.a.voors@umcg.nl.RE F E RENCE S1. Cunningham JW, Ferreira JP, Deng H, et al.Natriuretic peptide-based inclusion criteria ina heart failure clinical trial: insights fromCOMMANDER HF. J Am Coll Cardiol HF 2020;8:359–68.2. Zannad F, Anker SD, Byra WM, et al., for theCOMMANDER HF Investigators. Rivaroxaban inpatients with heart failure, sinus rhythm, andcoronary disease. N Engl J Med 2018;379:1332–42.3. Pitt B, Pfeffer MA, Assmann SF, et al., for theTOPCAT Investigators. Spironolactone for heartfailure with preserved ejection fraction. N Engl JMed 2014;370:1383–92.4. Pfeffer MA, Claggett B, Assmann SF, et al.Regional variation in patients and outcomes in theTreatment of Preserved Cardiac FunctionHeart Failure with an Aldosterone Antagonist(TOPCAT) trial. Circulation 2015;131:34–42.5. Bristow MR, Enciso JS, Gersh BJ, et al. Detec-tion and management of geographic disparities inthe TOPCAT trial: lessons learned and derivativerecommendations. J Am Coll Cardiol Basic TransScience 2016;1:180–9.KEY WORDS cardiac biomarkers, clinical trials,heart failure with reduced ejection fraction

NARCIS 

Dissertations of the University of Groningen

   University of GroningenShould Enrichment With Natriuretic Peptide Levels Be Mandatory in Global Clinical Trials?Voors, Adriaan A.Published in:JACC. Heart failureDOI:10.1016/j.jchf.2020.01.007IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2020Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Voors, A. A. (2020). Should Enrichment With Natriuretic Peptide Levels Be Mandatory in Global ClinicalTrials? JACC. Heart failure, 8(5), 369-371. https://doi.org/10.1016/j.jchf.2020.01.007CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 05-06-2022J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0ª 2 0 2 0 B Y T H E AM E R I C A N C O L L E G E O F C A R D I O L O G Y F O UN DA T I O NP U B L I S H E D B Y E L S E V I E REDITORIAL COMMENTShould Enrichment WithNatriuretic Peptide LevelsBe Mandatory in Global Clinical Trials?*Adriaan A. Voors, MD, PHDT he design of large, phase III randomizedclinical drug trials in patients with heartfailure requires a careful selection of inclu-sion and exclusion criteria. These criteria should: 1)reflect the target population that will most likelybenefit from the investigational product; 2) be nottoo lenient as this allows low-risk patients and thoseout of the target population to be recruited; 3) be nottoo strict as this will limit trial recruitment and clin-ical applicability of the investigational product; 4)make it reasonably certain that the patients’ condi-tions are indeed diagnosed with heart failure (diag-nostic criteria); and 5) reflect a patient populationat (high) risk for the primary endpoint (enrichmentcriteria).SEE PAGE 359Two criteria frequently used for both diagnosticand enrichment purposes are previous heart failurehospital admission and plasma N-terminal pro–B-typenatriuretic peptide (NT-proBNP) thresholds. In thisissue of JACC: Heart Failure, Cunningham et al. (1)describe what happens when NT-proBNP criteria areadded to the heart failure hospitalization criteria onlyISSN 2213-1779/$36.00*Editorials published in JACC: Heart Failure reflect the views of theauthors and do not necessarily represent the views of JACC: Heart Failureor the American College of Cardiology.From the University of Groningen, Department of Cardiology, UniversityMedical Center Groningen, Groningen, the Netherlands. Dr. Voor is aconsultant for and has received support from Amgen, AstraZeneca,Bayer, Boehringer Ingelheim, Cytokinetics, MyoKardia, Novartis,NovoNordisk, and Roche Diagnostics.The author attests they are in compliance with human studiescommittees and animal welfare regulations of the authors’ institutionsand Food and Drug Administration guidelines, including patient consentwhere appropriate. For more information, visit the JACC: Heart Failureauthor instructions page.after one-fourth of the patients are already recruited.This happened after a protocol amendment in theCOMMANDER-HF (A Study to Assess the Effective-ness and Safety of Rivaroxaban in Reducing the Riskof Death, Myocardial Infarction or Stroke in Partici-pants With Heart Failure and Coronary ArteryDisease Following an Episode of DecompensatedHeart Failure; NCT01877915) trial.COMMANDER-HF was a double-blind, randomizedtrial of the effects of the direct factor Xa inhibitorrivaroxaban in 5,022 chronic heart failure patient witha left ventricular ejection fraction of 40% or less,coronary artery disease, and a recent (<21 days) heartfailure hospital admission (2). The trial did not show areduction of the primary outcome, which was thecomposite of death from any cause, or myocardialinfarction, or stroke.COMMANDER-HF started with only 1 of thefrequently used diagnostic/enrichment criterion (arecent heart failure hospital admission) and did notrequire an NT-proBNP threshold for inclusion. How-ever, after the enrollment of 1,155 patients (23.0%),the study leadership observed an overall even ratethat was lower than expected. By that time, the ma-jority of patients (89%) were enrolled in EasternEurope, and only 1% in North America, 4% in LatinAmerica and Asia Pacific, and 5% in Western Europeand South Africa. Surprisingly, the paper by Cun-ningham et al. (1) does not list the countries whichthey considered to be Eastern European. Variousdefinitions of Eastern Europe exist, as the definitionsare based on historic, religious, and cultural differ-ences and to a lesser extent on geographic bound-aries. For example, Latvia and Lithuania enrolledpatients in COMMANDER-HF but those are notEastern European countries. Nonetheless, because ofthe very low event rate, the steering committeehttps://doi.org/10.1016/j.jchf.2020.01.007Voors J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0Natriuretic Peptide Enrichment of Global Heart Failure Trials M A Y 2 0 2 0 : 3 6 9 – 7 1370amended the enrollment criteria to require a plasmaNT-proBNP concentration of >800 ng/l or a BNPconcentration of >200 ng/l at any time between theindex admission for decompensated HF andrandomization. The purpose of the amendment “wasto prevent further enrollment of low risk patientswho had not truly been hospitalized for decom-pensated HF, and thereby increase event rates andallow for more rapid completion of this event-driventrial” (1). The addition of NT-proBNP criteria later inthe trial provided a unique opportunity to comparepatient characteristics and event rates before andafter the protocol amendment.First, as expected, after allowing only patients withelevated NT-proBNP levels, the event rate increasedsubstantially. Second, after the amendment, thecharacteristics of the patients significantly changedas well. Although this does not come as a surprise,those designing large randomized clinical heart fail-ure trials should be aware of those changes. Forexample, adding an NT-proBNP criterion will result inan older population. This may not always be desiredas treatment and side effects may be different.Notably, in the paper by Cunningham et al. (1)bleeding events increased in the patients treated withrivaroxaban after the amendment as well. Higher NT-proBNP thresholds may also increase the number ofpatients with atrial fibrillation, but although patientswith atrial fibrillation were excluded fromCOMMANDER-HF, this was not confirmed in thepresent study.The most remarkable finding of the present studywas that recruitment slowed considerably in EasternEurope and picked up in other regions. This mighthave been related to the larger effect that theamendment had on Eastern Europe, but it should benoted that other regions, such as Asia Pacific, hadnot been initiated before the amendment. Evenmore surprising was the fact that the increase in theevent rate after the amendment was seen only inEastern Europe and not in any other region of theworld. Thus, the increase in event rate in the trialwas completely driven by the increase in EasternEurope. These findings suggest that adding NT-proBNP enrichment criteria had a much greaterimpact on Eastern Europe than it did on otherregions.How can we interpret these findings? The authorssuggest that when no objective NT-proBNP criteriaare used, the sites in Eastern Europe that recruitedpatients might not have had acute decompensatedheart failure after all. This may be related to culturaldifferences where the threshold for a heart failurehospital admission is much lower than in other partsof the world. Given these cultural differences and thesubstantial recruitment (>50%) from Eastern Europein the COMMANDER-HF trial, it is remarkable thatnone of the executive members and/or data safetymonitoring board members was from 1 of the EasternEuropean countries. In addition, if it is true “that sitesin Eastern Europe recruited patients who might nothave had acute decompensated heart failure after all(1),” why was this not picked up by the ClinicalResearch Organization (CRO). They should havechecked eligibility criteria preferably on site fromsource documents, and presumably these criteriawere not violated or patients should not have beenallowed to be included in the final analyses. Thisimplies that either the sites in Eastern Europecorrectly recruited patients that met all eligibilitycriteria or the noncompliance with these criteria wasnot discovered and reported by the CRO. Alterna-tively, the definitions of “symptoms of worseningdyspnea or fatigue, objective signs of congestion,and/or adjustment of HF medications, requiringhospital admission or unscheduled parenteraldiuretic” may be not be specific enough and patientsin Eastern Europe might have met these criteria butdid not yield the expected and required patientpopulation.In contrast, the present findings fromCOMMANDER-HF share remarkable similarities withthose from TOPCAT. TOPCAT (Treatment of Pre-served Cardiac Function Heart Failure with an Aldo-sterone Antagonist Trial) failed to demonstrate thatspironolactone reduced death from cardiovascularcauses or aborted cardiac arrest or heart failure hos-pitalizations in 3,445 patients with heart failure withpreserved ejection fraction (3). Patients could beincluded based on either a previous (<12 months)hospitalization due to heart failure or based onelevated NT-proBNP levels. The trial recruited pa-tients from North America, Eastern Europe, and SouthAmerica. Sites in Russia and the Republic of Georgia(officially not an Eastern European country) providedmost of the early enrollment, primarily based on thehospitalization criterion because NT-proBNP levelswere initially unavailable there (4). Similar to eventrates in COMMANDER-HF, the clinical event rate wasstrikingly low in Russia and Georgia. In addition, lownatriuretic peptide measurements from Russia andGeorgia, available later in the trial, suggested no heartfailure or only mild heart failure. The primary resultsshowed a significant reduction of the primaryendpoint in North and South America but not inRussia and Georgia (5). These data, together with thefinding that no spironolactone metabolite could bedemonstrated in the urine samples from a largeJ A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0 VoorsM A Y 2 0 2 0 : 3 6 9 – 7 1 Natriuretic Peptide Enrichment of Global Heart Failure Trials371proportion of active-arm patients in Russia andGeorgia, aroused concerns regarding study conduct atsome sites in these countries. Although lessons couldhave been learned from TOPCAT, COMMANDER-HFwas already well underway, and the amendmentwas applied when these results became available.What do we learn from these important data byCunningham et al. (1) First, as expected, addingNT-proBNP thresholds to the eligibility criteria ofheart failure trials will increase event rates. Second,also expected but important to realize, addingNT-proBNP criteria will substantially change thepatient characteristics and may also influencetreatment responses and side effects. Third, addingNT-proBNP criteria reduces regional heterogeneityin patient characteristics and clinical events.Finally, as the Data Safety and Monitoring Board ofTOPCAT recommended (4), “launch the trial in avariety of geographic jurisdictions, and do not allow1 or 2 geographic areas to dominate early (or late)enrollment.”ADDRESS FOR CORRESPONDENCE: Dr. Adriaan A.Voors, University of Groningen, Department ofCardiology, University Medical Center Groningen,Hanzeplein 1, 9713GZ Groningen, the Netherlands.E-mail: a.a.voors@umcg.nl.RE F E RENCE S1. Cunningham JW, Ferreira JP, Deng H, et al.Natriuretic peptide-based inclusion criteria ina heart failure clinical trial: insights fromCOMMANDER HF. J Am Coll Cardiol HF 2020;8:359–68.2. Zannad F, Anker SD, Byra WM, et al., for theCOMMANDER HF Investigators. Rivaroxaban inpatients with heart failure, sinus rhythm, andcoronary disease. N Engl J Med 2018;379:1332–42.3. Pitt B, Pfeffer MA, Assmann SF, et al., for theTOPCAT Investigators. Spironolactone for heartfailure with preserved ejection fraction. N Engl JMed 2014;370:1383–92.4. Pfeffer MA, Claggett B, Assmann SF, et al.Regional variation in patients and outcomes in theTreatment of Preserved Cardiac FunctionHeart Failure with an Aldosterone Antagonist(TOPCAT) trial. Circulation 2015;131:34–42.5. Bristow MR, Enciso JS, Gersh BJ, et al. Detec-tion and management of geographic disparities inthe TOPCAT trial: lessons learned and derivativerecommendations. J Am Coll Cardiol Basic TransScience 2016;1:180–9.KEY WORDS cardiac biomarkers, clinical trials,heart failure with reduced ejection fraction

ARTS repository - University of Groningen

   University of GroningenShould Enrichment With Natriuretic Peptide Levels Be Mandatory in Global Clinical Trials?Voors, Adriaan A.Published in:JACC. Heart failureDOI:10.1016/j.jchf.2020.01.007IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2020Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Voors, A. A. (2020). Should Enrichment With Natriuretic Peptide Levels Be Mandatory in Global ClinicalTrials? JACC. Heart failure, 8(5), 369-371. https://doi.org/10.1016/j.jchf.2020.01.007CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 21-06-2022J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0ª 2 0 2 0 B Y T H E AM E R I C A N C O L L E G E O F C A R D I O L O G Y F O UN DA T I O NP U B L I S H E D B Y E L S E V I E REDITORIAL COMMENTShould Enrichment WithNatriuretic Peptide LevelsBe Mandatory in Global Clinical Trials?*Adriaan A. Voors, MD, PHDT he design of large, phase III randomizedclinical drug trials in patients with heartfailure requires a careful selection of inclu-sion and exclusion criteria. These criteria should: 1)reflect the target population that will most likelybenefit from the investigational product; 2) be nottoo lenient as this allows low-risk patients and thoseout of the target population to be recruited; 3) be nottoo strict as this will limit trial recruitment and clin-ical applicability of the investigational product; 4)make it reasonably certain that the patients’ condi-tions are indeed diagnosed with heart failure (diag-nostic criteria); and 5) reflect a patient populationat (high) risk for the primary endpoint (enrichmentcriteria).SEE PAGE 359Two criteria frequently used for both diagnosticand enrichment purposes are previous heart failurehospital admission and plasma N-terminal pro–B-typenatriuretic peptide (NT-proBNP) thresholds. In thisissue of JACC: Heart Failure, Cunningham et al. (1)describe what happens when NT-proBNP criteria areadded to the heart failure hospitalization criteria onlyISSN 2213-1779/$36.00*Editorials published in JACC: Heart Failure reflect the views of theauthors and do not necessarily represent the views of JACC: Heart Failureor the American College of Cardiology.From the University of Groningen, Department of Cardiology, UniversityMedical Center Groningen, Groningen, the Netherlands. Dr. Voor is aconsultant for and has received support from Amgen, AstraZeneca,Bayer, Boehringer Ingelheim, Cytokinetics, MyoKardia, Novartis,NovoNordisk, and Roche Diagnostics.The author attests they are in compliance with human studiescommittees and animal welfare regulations of the authors’ institutionsand Food and Drug Administration guidelines, including patient consentwhere appropriate. For more information, visit the JACC: Heart Failureauthor instructions page.after one-fourth of the patients are already recruited.This happened after a protocol amendment in theCOMMANDER-HF (A Study to Assess the Effective-ness and Safety of Rivaroxaban in Reducing the Riskof Death, Myocardial Infarction or Stroke in Partici-pants With Heart Failure and Coronary ArteryDisease Following an Episode of DecompensatedHeart Failure; NCT01877915) trial.COMMANDER-HF was a double-blind, randomizedtrial of the effects of the direct factor Xa inhibitorrivaroxaban in 5,022 chronic heart failure patient witha left ventricular ejection fraction of 40% or less,coronary artery disease, and a recent (<21 days) heartfailure hospital admission (2). The trial did not show areduction of the primary outcome, which was thecomposite of death from any cause, or myocardialinfarction, or stroke.COMMANDER-HF started with only 1 of thefrequently used diagnostic/enrichment criterion (arecent heart failure hospital admission) and did notrequire an NT-proBNP threshold for inclusion. How-ever, after the enrollment of 1,155 patients (23.0%),the study leadership observed an overall even ratethat was lower than expected. By that time, the ma-jority of patients (89%) were enrolled in EasternEurope, and only 1% in North America, 4% in LatinAmerica and Asia Pacific, and 5% in Western Europeand South Africa. Surprisingly, the paper by Cun-ningham et al. (1) does not list the countries whichthey considered to be Eastern European. Variousdefinitions of Eastern Europe exist, as the definitionsare based on historic, religious, and cultural differ-ences and to a lesser extent on geographic bound-aries. For example, Latvia and Lithuania enrolledpatients in COMMANDER-HF but those are notEastern European countries. Nonetheless, because ofthe very low event rate, the steering committeehttps://doi.org/10.1016/j.jchf.2020.01.007Voors J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0Natriuretic Peptide Enrichment of Global Heart Failure Trials M A Y 2 0 2 0 : 3 6 9 – 7 1370amended the enrollment criteria to require a plasmaNT-proBNP concentration of >800 ng/l or a BNPconcentration of >200 ng/l at any time between theindex admission for decompensated HF andrandomization. The purpose of the amendment “wasto prevent further enrollment of low risk patientswho had not truly been hospitalized for decom-pensated HF, and thereby increase event rates andallow for more rapid completion of this event-driventrial” (1). The addition of NT-proBNP criteria later inthe trial provided a unique opportunity to comparepatient characteristics and event rates before andafter the protocol amendment.First, as expected, after allowing only patients withelevated NT-proBNP levels, the event rate increasedsubstantially. Second, after the amendment, thecharacteristics of the patients significantly changedas well. Although this does not come as a surprise,those designing large randomized clinical heart fail-ure trials should be aware of those changes. Forexample, adding an NT-proBNP criterion will result inan older population. This may not always be desiredas treatment and side effects may be different.Notably, in the paper by Cunningham et al. (1)bleeding events increased in the patients treated withrivaroxaban after the amendment as well. Higher NT-proBNP thresholds may also increase the number ofpatients with atrial fibrillation, but although patientswith atrial fibrillation were excluded fromCOMMANDER-HF, this was not confirmed in thepresent study.The most remarkable finding of the present studywas that recruitment slowed considerably in EasternEurope and picked up in other regions. This mighthave been related to the larger effect that theamendment had on Eastern Europe, but it should benoted that other regions, such as Asia Pacific, hadnot been initiated before the amendment. Evenmore surprising was the fact that the increase in theevent rate after the amendment was seen only inEastern Europe and not in any other region of theworld. Thus, the increase in event rate in the trialwas completely driven by the increase in EasternEurope. These findings suggest that adding NT-proBNP enrichment criteria had a much greaterimpact on Eastern Europe than it did on otherregions.How can we interpret these findings? The authorssuggest that when no objective NT-proBNP criteriaare used, the sites in Eastern Europe that recruitedpatients might not have had acute decompensatedheart failure after all. This may be related to culturaldifferences where the threshold for a heart failurehospital admission is much lower than in other partsof the world. Given these cultural differences and thesubstantial recruitment (>50%) from Eastern Europein the COMMANDER-HF trial, it is remarkable thatnone of the executive members and/or data safetymonitoring board members was from 1 of the EasternEuropean countries. In addition, if it is true “that sitesin Eastern Europe recruited patients who might nothave had acute decompensated heart failure after all(1),” why was this not picked up by the ClinicalResearch Organization (CRO). They should havechecked eligibility criteria preferably on site fromsource documents, and presumably these criteriawere not violated or patients should not have beenallowed to be included in the final analyses. Thisimplies that either the sites in Eastern Europecorrectly recruited patients that met all eligibilitycriteria or the noncompliance with these criteria wasnot discovered and reported by the CRO. Alterna-tively, the definitions of “symptoms of worseningdyspnea or fatigue, objective signs of congestion,and/or adjustment of HF medications, requiringhospital admission or unscheduled parenteraldiuretic” may be not be specific enough and patientsin Eastern Europe might have met these criteria butdid not yield the expected and required patientpopulation.In contrast, the present findings fromCOMMANDER-HF share remarkable similarities withthose from TOPCAT. TOPCAT (Treatment of Pre-served Cardiac Function Heart Failure with an Aldo-sterone Antagonist Trial) failed to demonstrate thatspironolactone reduced death from cardiovascularcauses or aborted cardiac arrest or heart failure hos-pitalizations in 3,445 patients with heart failure withpreserved ejection fraction (3). Patients could beincluded based on either a previous (<12 months)hospitalization due to heart failure or based onelevated NT-proBNP levels. The trial recruited pa-tients from North America, Eastern Europe, and SouthAmerica. Sites in Russia and the Republic of Georgia(officially not an Eastern European country) providedmost of the early enrollment, primarily based on thehospitalization criterion because NT-proBNP levelswere initially unavailable there (4). Similar to eventrates in COMMANDER-HF, the clinical event rate wasstrikingly low in Russia and Georgia. In addition, lownatriuretic peptide measurements from Russia andGeorgia, available later in the trial, suggested no heartfailure or only mild heart failure. The primary resultsshowed a significant reduction of the primaryendpoint in North and South America but not inRussia and Georgia (5). These data, together with thefinding that no spironolactone metabolite could bedemonstrated in the urine samples from a largeJ A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0 VoorsM A Y 2 0 2 0 : 3 6 9 – 7 1 Natriuretic Peptide Enrichment of Global Heart Failure Trials371proportion of active-arm patients in Russia andGeorgia, aroused concerns regarding study conduct atsome sites in these countries. Although lessons couldhave been learned from TOPCAT, COMMANDER-HFwas already well underway, and the amendmentwas applied when these results became available.What do we learn from these important data byCunningham et al. (1) First, as expected, addingNT-proBNP thresholds to the eligibility criteria ofheart failure trials will increase event rates. Second,also expected but important to realize, addingNT-proBNP criteria will substantially change thepatient characteristics and may also influencetreatment responses and side effects. Third, addingNT-proBNP criteria reduces regional heterogeneityin patient characteristics and clinical events.Finally, as the Data Safety and Monitoring Board ofTOPCAT recommended (4), “launch the trial in avariety of geographic jurisdictions, and do not allow1 or 2 geographic areas to dominate early (or late)enrollment.”ADDRESS FOR CORRESPONDENCE: Dr. Adriaan A.Voors, University of Groningen, Department ofCardiology, University Medical Center Groningen,Hanzeplein 1, 9713GZ Groningen, the Netherlands.E-mail: a.a.voors@umcg.nl.RE F E RENCE S1. Cunningham JW, Ferreira JP, Deng H, et al.Natriuretic peptide-based inclusion criteria ina heart failure clinical trial: insights fromCOMMANDER HF. J Am Coll Cardiol HF 2020;8:359–68.2. Zannad F, Anker SD, Byra WM, et al., for theCOMMANDER HF Investigators. Rivaroxaban inpatients with heart failure, sinus rhythm, andcoronary disease. N Engl J Med 2018;379:1332–42.3. Pitt B, Pfeffer MA, Assmann SF, et al., for theTOPCAT Investigators. Spironolactone for heartfailure with preserved ejection fraction. N Engl JMed 2014;370:1383–92.4. Pfeffer MA, Claggett B, Assmann SF, et al.Regional variation in patients and outcomes in theTreatment of Preserved Cardiac FunctionHeart Failure with an Aldosterone Antagonist(TOPCAT) trial. Circulation 2015;131:34–42.5. Bristow MR, Enciso JS, Gersh BJ, et al. Detec-tion and management of geographic disparities inthe TOPCAT trial: lessons learned and derivativerecommendations. J Am Coll Cardiol Basic TransScience 2016;1:180–9.KEY WORDS cardiac biomarkers, clinical trials,heart failure with reduced ejection fraction

University of Groningen

   University of GroningenShould Enrichment With Natriuretic Peptide Levels Be Mandatory in Global Clinical Trials?Voors, Adriaan A.Published in:JACC. Heart failureDOI:10.1016/j.jchf.2020.01.007IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2020Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Voors, A. A. (2020). Should Enrichment With Natriuretic Peptide Levels Be Mandatory in Global ClinicalTrials? JACC. Heart failure, 8(5), 369-371. https://doi.org/10.1016/j.jchf.2020.01.007CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 19-11-2022J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0ª 2 0 2 0 B Y T H E AM E R I C A N C O L L E G E O F C A R D I O L O G Y F O UN DA T I O NP U B L I S H E D B Y E L S E V I E REDITORIAL COMMENTShould Enrichment WithNatriuretic Peptide LevelsBe Mandatory in Global Clinical Trials?*Adriaan A. Voors, MD, PHDT he design of large, phase III randomizedclinical drug trials in patients with heartfailure requires a careful selection of inclu-sion and exclusion criteria. These criteria should: 1)reflect the target population that will most likelybenefit from the investigational product; 2) be nottoo lenient as this allows low-risk patients and thoseout of the target population to be recruited; 3) be nottoo strict as this will limit trial recruitment and clin-ical applicability of the investigational product; 4)make it reasonably certain that the patients’ condi-tions are indeed diagnosed with heart failure (diag-nostic criteria); and 5) reflect a patient populationat (high) risk for the primary endpoint (enrichmentcriteria).SEE PAGE 359Two criteria frequently used for both diagnosticand enrichment purposes are previous heart failurehospital admission and plasma N-terminal pro–B-typenatriuretic peptide (NT-proBNP) thresholds. In thisissue of JACC: Heart Failure, Cunningham et al. (1)describe what happens when NT-proBNP criteria areadded to the heart failure hospitalization criteria onlyISSN 2213-1779/$36.00*Editorials published in JACC: Heart Failure reflect the views of theauthors and do not necessarily represent the views of JACC: Heart Failureor the American College of Cardiology.From the University of Groningen, Department of Cardiology, UniversityMedical Center Groningen, Groningen, the Netherlands. Dr. Voor is aconsultant for and has received support from Amgen, AstraZeneca,Bayer, Boehringer Ingelheim, Cytokinetics, MyoKardia, Novartis,NovoNordisk, and Roche Diagnostics.The author attests they are in compliance with human studiescommittees and animal welfare regulations of the authors’ institutionsand Food and Drug Administration guidelines, including patient consentwhere appropriate. For more information, visit the JACC: Heart Failureauthor instructions page.after one-fourth of the patients are already recruited.This happened after a protocol amendment in theCOMMANDER-HF (A Study to Assess the Effective-ness and Safety of Rivaroxaban in Reducing the Riskof Death, Myocardial Infarction or Stroke in Partici-pants With Heart Failure and Coronary ArteryDisease Following an Episode of DecompensatedHeart Failure; NCT01877915) trial.COMMANDER-HF was a double-blind, randomizedtrial of the effects of the direct factor Xa inhibitorrivaroxaban in 5,022 chronic heart failure patient witha left ventricular ejection fraction of 40% or less,coronary artery disease, and a recent (<21 days) heartfailure hospital admission (2). The trial did not show areduction of the primary outcome, which was thecomposite of death from any cause, or myocardialinfarction, or stroke.COMMANDER-HF started with only 1 of thefrequently used diagnostic/enrichment criterion (arecent heart failure hospital admission) and did notrequire an NT-proBNP threshold for inclusion. How-ever, after the enrollment of 1,155 patients (23.0%),the study leadership observed an overall even ratethat was lower than expected. By that time, the ma-jority of patients (89%) were enrolled in EasternEurope, and only 1% in North America, 4% in LatinAmerica and Asia Pacific, and 5% in Western Europeand South Africa. Surprisingly, the paper by Cun-ningham et al. (1) does not list the countries whichthey considered to be Eastern European. Variousdefinitions of Eastern Europe exist, as the definitionsare based on historic, religious, and cultural differ-ences and to a lesser extent on geographic bound-aries. For example, Latvia and Lithuania enrolledpatients in COMMANDER-HF but those are notEastern European countries. Nonetheless, because ofthe very low event rate, the steering committeehttps://doi.org/10.1016/j.jchf.2020.01.007Voors J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0Natriuretic Peptide Enrichment of Global Heart Failure Trials M A Y 2 0 2 0 : 3 6 9 – 7 1370amended the enrollment criteria to require a plasmaNT-proBNP concentration of >800 ng/l or a BNPconcentration of >200 ng/l at any time between theindex admission for decompensated HF andrandomization. The purpose of the amendment “wasto prevent further enrollment of low risk patientswho had not truly been hospitalized for decom-pensated HF, and thereby increase event rates andallow for more rapid completion of this event-driventrial” (1). The addition of NT-proBNP criteria later inthe trial provided a unique opportunity to comparepatient characteristics and event rates before andafter the protocol amendment.First, as expected, after allowing only patients withelevated NT-proBNP levels, the event rate increasedsubstantially. Second, after the amendment, thecharacteristics of the patients significantly changedas well. Although this does not come as a surprise,those designing large randomized clinical heart fail-ure trials should be aware of those changes. Forexample, adding an NT-proBNP criterion will result inan older population. This may not always be desiredas treatment and side effects may be different.Notably, in the paper by Cunningham et al. (1)bleeding events increased in the patients treated withrivaroxaban after the amendment as well. Higher NT-proBNP thresholds may also increase the number ofpatients with atrial fibrillation, but although patientswith atrial fibrillation were excluded fromCOMMANDER-HF, this was not confirmed in thepresent study.The most remarkable finding of the present studywas that recruitment slowed considerably in EasternEurope and picked up in other regions. This mighthave been related to the larger effect that theamendment had on Eastern Europe, but it should benoted that other regions, such as Asia Pacific, hadnot been initiated before the amendment. Evenmore surprising was the fact that the increase in theevent rate after the amendment was seen only inEastern Europe and not in any other region of theworld. Thus, the increase in event rate in the trialwas completely driven by the increase in EasternEurope. These findings suggest that adding NT-proBNP enrichment criteria had a much greaterimpact on Eastern Europe than it did on otherregions.How can we interpret these findings? The authorssuggest that when no objective NT-proBNP criteriaare used, the sites in Eastern Europe that recruitedpatients might not have had acute decompensatedheart failure after all. This may be related to culturaldifferences where the threshold for a heart failurehospital admission is much lower than in other partsof the world. Given these cultural differences and thesubstantial recruitment (>50%) from Eastern Europein the COMMANDER-HF trial, it is remarkable thatnone of the executive members and/or data safetymonitoring board members was from 1 of the EasternEuropean countries. In addition, if it is true “that sitesin Eastern Europe recruited patients who might nothave had acute decompensated heart failure after all(1),” why was this not picked up by the ClinicalResearch Organization (CRO). They should havechecked eligibility criteria preferably on site fromsource documents, and presumably these criteriawere not violated or patients should not have beenallowed to be included in the final analyses. Thisimplies that either the sites in Eastern Europecorrectly recruited patients that met all eligibilitycriteria or the noncompliance with these criteria wasnot discovered and reported by the CRO. Alterna-tively, the definitions of “symptoms of worseningdyspnea or fatigue, objective signs of congestion,and/or adjustment of HF medications, requiringhospital admission or unscheduled parenteraldiuretic” may be not be specific enough and patientsin Eastern Europe might have met these criteria butdid not yield the expected and required patientpopulation.In contrast, the present findings fromCOMMANDER-HF share remarkable similarities withthose from TOPCAT. TOPCAT (Treatment of Pre-served Cardiac Function Heart Failure with an Aldo-sterone Antagonist Trial) failed to demonstrate thatspironolactone reduced death from cardiovascularcauses or aborted cardiac arrest or heart failure hos-pitalizations in 3,445 patients with heart failure withpreserved ejection fraction (3). Patients could beincluded based on either a previous (<12 months)hospitalization due to heart failure or based onelevated NT-proBNP levels. The trial recruited pa-tients from North America, Eastern Europe, and SouthAmerica. Sites in Russia and the Republic of Georgia(officially not an Eastern European country) providedmost of the early enrollment, primarily based on thehospitalization criterion because NT-proBNP levelswere initially unavailable there (4). Similar to eventrates in COMMANDER-HF, the clinical event rate wasstrikingly low in Russia and Georgia. In addition, lownatriuretic peptide measurements from Russia andGeorgia, available later in the trial, suggested no heartfailure or only mild heart failure. The primary resultsshowed a significant reduction of the primaryendpoint in North and South America but not inRussia and Georgia (5). These data, together with thefinding that no spironolactone metabolite could bedemonstrated in the urine samples from a largeJ A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0 VoorsM A Y 2 0 2 0 : 3 6 9 – 7 1 Natriuretic Peptide Enrichment of Global Heart Failure Trials371proportion of active-arm patients in Russia andGeorgia, aroused concerns regarding study conduct atsome sites in these countries. Although lessons couldhave been learned from TOPCAT, COMMANDER-HFwas already well underway, and the amendmentwas applied when these results became available.What do we learn from these important data byCunningham et al. (1) First, as expected, addingNT-proBNP thresholds to the eligibility criteria ofheart failure trials will increase event rates. Second,also expected but important to realize, addingNT-proBNP criteria will substantially change thepatient characteristics and may also influencetreatment responses and side effects. Third, addingNT-proBNP criteria reduces regional heterogeneityin patient characteristics and clinical events.Finally, as the Data Safety and Monitoring Board ofTOPCAT recommended (4), “launch the trial in avariety of geographic jurisdictions, and do not allow1 or 2 geographic areas to dominate early (or late)enrollment.”ADDRESS FOR CORRESPONDENCE: Dr. Adriaan A.Voors, University of Groningen, Department ofCardiology, University Medical Center Groningen,Hanzeplein 1, 9713GZ Groningen, the Netherlands.E-mail: a.a.voors@umcg.nl.RE F E RENCE S1. Cunningham JW, Ferreira JP, Deng H, et al.Natriuretic peptide-based inclusion criteria ina heart failure clinical trial: insights fromCOMMANDER HF. J Am Coll Cardiol HF 2020;8:359–68.2. Zannad F, Anker SD, Byra WM, et al., for theCOMMANDER HF Investigators. Rivaroxaban inpatients with heart failure, sinus rhythm, andcoronary disease. N Engl J Med 2018;379:1332–42.3. Pitt B, Pfeffer MA, Assmann SF, et al., for theTOPCAT Investigators. Spironolactone for heartfailure with preserved ejection fraction. N Engl JMed 2014;370:1383–92.4. Pfeffer MA, Claggett B, Assmann SF, et al.Regional variation in patients and outcomes in theTreatment of Preserved Cardiac FunctionHeart Failure with an Aldosterone Antagonist(TOPCAT) trial. Circulation 2015;131:34–42.5. Bristow MR, Enciso JS, Gersh BJ, et al. Detec-tion and management of geographic disparities inthe TOPCAT trial: lessons learned and derivativerecommendations. J Am Coll Cardiol Basic TransScience 2016;1:180–9.KEY WORDS cardiac biomarkers, clinical trials,heart failure with reduced ejection fraction

   University of GroningenShould Enrichment With Natriuretic Peptide Levels Be Mandatory in Global Clinical Trials?Voors, Adriaan A.Published in:JACC. Heart failureDOI:10.1016/j.jchf.2020.01.007IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionPublisher's PDF, also known as Version of recordPublication date:2020Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Voors, A. A. (2020). Should Enrichment With Natriuretic Peptide Levels Be Mandatory in Global ClinicalTrials? JACC. Heart failure, 8(5), 369-371. https://doi.org/10.1016/j.jchf.2020.01.007CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 29-04-2023J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0ª 2 0 2 0 B Y T H E AM E R I C A N C O L L E G E O F C A R D I O L O G Y F O UN DA T I O NP U B L I S H E D B Y E L S E V I E REDITORIAL COMMENTShould Enrichment WithNatriuretic Peptide LevelsBe Mandatory in Global Clinical Trials?*Adriaan A. Voors, MD, PHDT he design of large, phase III randomizedclinical drug trials in patients with heartfailure requires a careful selection of inclu-sion and exclusion criteria. These criteria should: 1)reflect the target population that will most likelybenefit from the investigational product; 2) be nottoo lenient as this allows low-risk patients and thoseout of the target population to be recruited; 3) be nottoo strict as this will limit trial recruitment and clin-ical applicability of the investigational product; 4)make it reasonably certain that the patients’ condi-tions are indeed diagnosed with heart failure (diag-nostic criteria); and 5) reflect a patient populationat (high) risk for the primary endpoint (enrichmentcriteria).SEE PAGE 359Two criteria frequently used for both diagnosticand enrichment purposes are previous heart failurehospital admission and plasma N-terminal pro–B-typenatriuretic peptide (NT-proBNP) thresholds. In thisissue of JACC: Heart Failure, Cunningham et al. (1)describe what happens when NT-proBNP criteria areadded to the heart failure hospitalization criteria onlyISSN 2213-1779/$36.00*Editorials published in JACC: Heart Failure reflect the views of theauthors and do not necessarily represent the views of JACC: Heart Failureor the American College of Cardiology.From the University of Groningen, Department of Cardiology, UniversityMedical Center Groningen, Groningen, the Netherlands. Dr. Voor is aconsultant for and has received support from Amgen, AstraZeneca,Bayer, Boehringer Ingelheim, Cytokinetics, MyoKardia, Novartis,NovoNordisk, and Roche Diagnostics.The author attests they are in compliance with human studiescommittees and animal welfare regulations of the authors’ institutionsand Food and Drug Administration guidelines, including patient consentwhere appropriate. For more information, visit the JACC: Heart Failureauthor instructions page.after one-fourth of the patients are already recruited.This happened after a protocol amendment in theCOMMANDER-HF (A Study to Assess the Effective-ness and Safety of Rivaroxaban in Reducing the Riskof Death, Myocardial Infarction or Stroke in Partici-pants With Heart Failure and Coronary ArteryDisease Following an Episode of DecompensatedHeart Failure; NCT01877915) trial.COMMANDER-HF was a double-blind, randomizedtrial of the effects of the direct factor Xa inhibitorrivaroxaban in 5,022 chronic heart failure patient witha left ventricular ejection fraction of 40% or less,coronary artery disease, and a recent (<21 days) heartfailure hospital admission (2). The trial did not show areduction of the primary outcome, which was thecomposite of death from any cause, or myocardialinfarction, or stroke.COMMANDER-HF started with only 1 of thefrequently used diagnostic/enrichment criterion (arecent heart failure hospital admission) and did notrequire an NT-proBNP threshold for inclusion. How-ever, after the enrollment of 1,155 patients (23.0%),the study leadership observed an overall even ratethat was lower than expected. By that time, the ma-jority of patients (89%) were enrolled in EasternEurope, and only 1% in North America, 4% in LatinAmerica and Asia Pacific, and 5% in Western Europeand South Africa. Surprisingly, the paper by Cun-ningham et al. (1) does not list the countries whichthey considered to be Eastern European. Variousdefinitions of Eastern Europe exist, as the definitionsare based on historic, religious, and cultural differ-ences and to a lesser extent on geographic bound-aries. For example, Latvia and Lithuania enrolledpatients in COMMANDER-HF but those are notEastern European countries. Nonetheless, because ofthe very low event rate, the steering committeehttps://doi.org/10.1016/j.jchf.2020.01.007Voors J A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0Natriuretic Peptide Enrichment of Global Heart Failure Trials M A Y 2 0 2 0 : 3 6 9 – 7 1370amended the enrollment criteria to require a plasmaNT-proBNP concentration of >800 ng/l or a BNPconcentration of >200 ng/l at any time between theindex admission for decompensated HF andrandomization. The purpose of the amendment “wasto prevent further enrollment of low risk patientswho had not truly been hospitalized for decom-pensated HF, and thereby increase event rates andallow for more rapid completion of this event-driventrial” (1). The addition of NT-proBNP criteria later inthe trial provided a unique opportunity to comparepatient characteristics and event rates before andafter the protocol amendment.First, as expected, after allowing only patients withelevated NT-proBNP levels, the event rate increasedsubstantially. Second, after the amendment, thecharacteristics of the patients significantly changedas well. Although this does not come as a surprise,those designing large randomized clinical heart fail-ure trials should be aware of those changes. Forexample, adding an NT-proBNP criterion will result inan older population. This may not always be desiredas treatment and side effects may be different.Notably, in the paper by Cunningham et al. (1)bleeding events increased in the patients treated withrivaroxaban after the amendment as well. Higher NT-proBNP thresholds may also increase the number ofpatients with atrial fibrillation, but although patientswith atrial fibrillation were excluded fromCOMMANDER-HF, this was not confirmed in thepresent study.The most remarkable finding of the present studywas that recruitment slowed considerably in EasternEurope and picked up in other regions. This mighthave been related to the larger effect that theamendment had on Eastern Europe, but it should benoted that other regions, such as Asia Pacific, hadnot been initiated before the amendment. Evenmore surprising was the fact that the increase in theevent rate after the amendment was seen only inEastern Europe and not in any other region of theworld. Thus, the increase in event rate in the trialwas completely driven by the increase in EasternEurope. These findings suggest that adding NT-proBNP enrichment criteria had a much greaterimpact on Eastern Europe than it did on otherregions.How can we interpret these findings? The authorssuggest that when no objective NT-proBNP criteriaare used, the sites in Eastern Europe that recruitedpatients might not have had acute decompensatedheart failure after all. This may be related to culturaldifferences where the threshold for a heart failurehospital admission is much lower than in other partsof the world. Given these cultural differences and thesubstantial recruitment (>50%) from Eastern Europein the COMMANDER-HF trial, it is remarkable thatnone of the executive members and/or data safetymonitoring board members was from 1 of the EasternEuropean countries. In addition, if it is true “that sitesin Eastern Europe recruited patients who might nothave had acute decompensated heart failure after all(1),” why was this not picked up by the ClinicalResearch Organization (CRO). They should havechecked eligibility criteria preferably on site fromsource documents, and presumably these criteriawere not violated or patients should not have beenallowed to be included in the final analyses. Thisimplies that either the sites in Eastern Europecorrectly recruited patients that met all eligibilitycriteria or the noncompliance with these criteria wasnot discovered and reported by the CRO. Alterna-tively, the definitions of “symptoms of worseningdyspnea or fatigue, objective signs of congestion,and/or adjustment of HF medications, requiringhospital admission or unscheduled parenteraldiuretic” may be not be specific enough and patientsin Eastern Europe might have met these criteria butdid not yield the expected and required patientpopulation.In contrast, the present findings fromCOMMANDER-HF share remarkable similarities withthose from TOPCAT. TOPCAT (Treatment of Pre-served Cardiac Function Heart Failure with an Aldo-sterone Antagonist Trial) failed to demonstrate thatspironolactone reduced death from cardiovascularcauses or aborted cardiac arrest or heart failure hos-pitalizations in 3,445 patients with heart failure withpreserved ejection fraction (3). Patients could beincluded based on either a previous (<12 months)hospitalization due to heart failure or based onelevated NT-proBNP levels. The trial recruited pa-tients from North America, Eastern Europe, and SouthAmerica. Sites in Russia and the Republic of Georgia(officially not an Eastern European country) providedmost of the early enrollment, primarily based on thehospitalization criterion because NT-proBNP levelswere initially unavailable there (4). Similar to eventrates in COMMANDER-HF, the clinical event rate wasstrikingly low in Russia and Georgia. In addition, lownatriuretic peptide measurements from Russia andGeorgia, available later in the trial, suggested no heartfailure or only mild heart failure. The primary resultsshowed a significant reduction of the primaryendpoint in North and South America but not inRussia and Georgia (5). These data, together with thefinding that no spironolactone metabolite could bedemonstrated in the urine samples from a largeJ A C C : H E A R T F A I L U R E V O L . 8 , N O . 5 , 2 0 2 0 VoorsM A Y 2 0 2 0 : 3 6 9 – 7 1 Natriuretic Peptide Enrichment of Global Heart Failure Trials371proportion of active-arm patients in Russia andGeorgia, aroused concerns regarding study conduct atsome sites in these countries. Although lessons couldhave been learned from TOPCAT, COMMANDER-HFwas already well underway, and the amendmentwas applied when these results became available.What do we learn from these important data byCunningham et al. (1) First, as expected, addingNT-proBNP thresholds to the eligibility criteria ofheart failure trials will increase event rates. Second,also expected but important to realize, addingNT-proBNP criteria will substantially change thepatient characteristics and may also influencetreatment responses and side effects. Third, addingNT-proBNP criteria reduces regional heterogeneityin patient characteristics and clinical events.Finally, as the Data Safety and Monitoring Board ofTOPCAT recommended (4), “launch the trial in avariety of geographic jurisdictions, and do not allow1 or 2 geographic areas to dominate early (or late)enrollment.”ADDRESS FOR CORRESPONDENCE: Dr. Adriaan A.Voors, University of Groningen, Department ofCardiology, University Medical Center Groningen,Hanzeplein 1, 9713GZ Groningen, the Netherlands.E-mail: a.a.voors@umcg.nl.RE F E RENCE S1. Cunningham JW, Ferreira JP, Deng H, et al.Natriuretic peptide-based inclusion criteria ina heart failure clinical trial: insights fromCOMMANDER HF. J Am Coll Cardiol HF 2020;8:359–68.2. Zannad F, Anker SD, Byra WM, et al., for theCOMMANDER HF Investigators. Rivaroxaban inpatients with heart failure, sinus rhythm, andcoronary disease. N Engl J Med 2018;379:1332–42.3. Pitt B, Pfeffer MA, Assmann SF, et al., for theTOPCAT Investigators. Spironolactone for heartfailure with preserved ejection fraction. N Engl JMed 2014;370:1383–92.4. Pfeffer MA, Claggett B, Assmann SF, et al.Regional variation in patients and outcomes in theTreatment of Preserved Cardiac FunctionHeart Failure with an Aldosterone Antagonist(TOPCAT) trial. Circulation 2015;131:34–42.5. Bristow MR, Enciso JS, Gersh BJ, et al. Detec-tion and management of geographic disparities inthe TOPCAT trial: lessons learned and derivativerecommendations. J Am Coll Cardiol Basic TransScience 2016;1:180–9.KEY WORDS cardiac biomarkers, clinical trials,heart failure with reduced ejection fraction

https://pure.rug.nl/ws/files/149822019/1_s2.0_S2213177920300214_main.pdf

Should Enrichment With Natriuretic Peptide Levels Be Mandatory in Global Clinical Trials?

Abstract

Similar works

Full text

Available Versions

Proceedings - University of Groningen

Proceedings - University of Groningen

NARCIS

Dissertations of the University of Groningen

ARTS repository - University of Groningen

University of Groningen

ARTS repository - University of Groningen

University of Groningen