Delphi study flowchart.

<p>Delphi study flowchart.</p

Six Thematic Categories Used to Generate and Structure the Delphi Survey.

<p>Six Thematic Categories Used to Generate and Structure the Delphi Survey.</p

RANK ORDER of the <i>Dependency</i> of Consensus Indices’ on the GROUP CONFORMITY INDEX in a Delphi Survey

<p>*The dependency value ranges from 0.000 to 1.000. A value of “0.000” shows complete independence of the Consensus Index from the Delphi survey characteristic examined (e.g., the Group Conformity Index) whereas a value of “1.000” shows complete dependence. The dependency value is the maximum numeric difference observed for each consensus index when the Group Conformity Index in a simulated Delphi survey varied from 0.0 to 1.0.</p><p>All Delphi consensus indices (the <i>left column</i>) typically take a value ranging from 0.000 to 1.000, except the Interquartile Range (IQR). For example, in the case of the Fleiss’ Kappa, a maximum difference of 0.504 can be anticipated when the Group Conformity Index varies from 0.0 to 1.0. For the IQR, the dependency data are normalized by dividing the difference observed in simulations by the maximum possible difference (9.000), i.e., the length of the Likert scale from 1 to 10 used in the simulations.</p><p>RANK ORDER of the <i>Dependency</i> of Consensus Indices’ on the GROUP CONFORMITY INDEX in a Delphi Survey</p

RANK ORDER of the <i>Dependency</i> of Consensus Indices’ on the NUMBER OF QUESTIONS (6–40) in a Delphi Survey

<p>*The dependency value ranges from 0.000 to 1.000. A value of “0.000” shows complete independence of the Consensus Index from the Delphi survey characteristic examined (e.g., the number of questions) whereas a value of “1.000” shows complete dependence. The dependency value is the maximum numeric difference observed for each consensus index when the number of questions in a simulated Delphi survey varied from 6 to 40.</p><p>All Delphi consensus indices (the <i>left column</i>) typically take a value ranging from 0.000 to 1.000, except the Interquartile Range (IQR). For example, in the case of Fleiss’ Kappa, a maximum difference of 0.025 can be anticipated when the number of Delphi survey questions vary from 6 to 40.</p><p>For the Interquartile Range, the dependency data were normalized by dividing the difference observed in simulations by the maximum possible difference (9.000), i.e., the length of the Likert scale from 1 to 10 used in the simulations.</p><p>RANK ORDER of the <i>Dependency</i> of Consensus Indices’ on the NUMBER OF QUESTIONS (6–40) in a Delphi Survey</p

RANK ORDER of the <i>Dependency</i> of Consensus Indices’ on the NUMBER OF EXPERTS (Sample Size) (6–50) in a Delphi Survey

<p>*The dependency value ranges from 0.000 to 1.000. A value of “0.000” shows complete independence of the Consensus Index from the Delphi survey characteristic examined (e.g., the sample size) whereas a value of “1.000” shows complete dependence. The dependency value is the maximum numeric difference observed for each consensus index when the number of experts in a simulated Delphi survey varied from 6 to 50.</p><p>All Delphi consensus indices (the <i>left column</i>) typically take a value ranging from 0.000 to 1.000, except the Interquartile Range (IQR). For example, in the case of Clustered Mode, a maximum difference of 0.130 can be anticipated when the sample size varies from 6 to 50. For the IQR, the dependency data are normalized by dividing the difference observed in simulations by the maximum possible difference (9.000), i.e., the length of the Likert scale from 1 to 10 used in the simulations. Accordingly, the IQR can vary by a value of 0.396 when the Delphi sample size varies within the above range.</p><p>RANK ORDER of the <i>Dependency</i> of Consensus Indices’ on the NUMBER OF EXPERTS (Sample Size) (6–50) in a Delphi Survey</p

Evidence as the focus of foresight research.

<p>Situated conceptually between novel technology <i>emergence</i> and <i>adoption future(s)</i>, examination of the attitudes towards co-production of technology-related evidence by the innovation ecosystem constituents (e.g., scientists and policymakers) can help build strategic foresight on the innovation trajectory.</p

Radar chart for Delphi survey round 2 and round 3 responses.

<p>Distribution of the standard deviations for each of the 107 Delphi survey questions (from 1 to 107, in the clockwise direction) in round 2 (<i>blue line</i>) and round 3 (<i>red line</i>). Note that the distribution of the standard deviations across the 107 survey questions is dampened in round 3 (<i>red line</i>) as anticipated in Delphi surveys.</p

Ranking of the Issues with DISSENSUS Among Experts in Regards to Metagenomic Tests for Water Safety.

<p><i>Dissensus</i> is observed for a survey question if on the 7-point Likert scale, the bottom 3 points as a cluster (a score of one, two or three), <i>and</i> the top 3 points as a cluster (a score of five, six or seven), <i>each</i> gathered at least 33% of ratings. <i>Dissensus</i> is the sum of the ratings gathered from the top and bottom 3 point clusters on the 7-point Likert scale (Columns A + B above).</p><p>Ranking of the Issues with DISSENSUS Among Experts in Regards to Metagenomic Tests for Water Safety.</p

Ranking of the Issues with CONSENSUS Among Experts in Regards to Metagenomic Tests for Water Safety.

<p>*Tied ranking</p><p>Ranking of the Issues with CONSENSUS Among Experts in Regards to Metagenomic Tests for Water Safety.</p

Word cloud generated from the Delphi round 1 responses.

<p>Word cloud generated from the Delphi round 1 responses.</p