Impact of experimental designs with increased amounts of data.

Whole pp-plot graph classification results and average normalized distances from the pp-plot graphs to the diagonal for three illustrative benchmark models. The outcome for the originally published design , and two artificial designs and with increased temporal sampling is shown. The respective number of estimated parameters, total number of data points and their ratio is given below.</p

Average distance of pp-plot graphs from the diagonal.

Distances are normalized by the maximal possible distance value, i.e. a graph lying on the upper x-axis at pemp = 1 for all pemp in the pp-plot. Models are ordered according to the ranking of the classification results from Fig 4A. The histogram on the right shows the distribution of average distances from the diagonal summarized for all parameters.</p

Classification results of whole pp-plots graphs compared to the -distribution.

Results are sorted by the percentage of non-problematic, i.e. conservative and perfect consensus cases. The anti-conservative and alternating fraction of cases is indicated by a negative sign. (A) Overall results for all model parameters and (B-E) results separately for each model parameter type.</p

All pp-plots graphs of the estimated model parameters, separated by their identifiability status.

(A) identifiable parameters and (B) practically non-identifiable based on their profile likelihoods. All graphs are classified in perfect (red), conservative (yellow), anti-conservative (blue) or alternating (purple) cases.</p

Parametric bootstrapping procedure for empirical likelihood ratios.

Parametric bootstrapping procedure for empirical likelihood ratios.</p

Data-space representation of four different scenarios using the nodal curve model manifold example.

(A,D,G,J) Different shapes of the model manifold (black solid line) in the data space with a choice of data realisations (N = 25 out of 200, gray crosses) drawn from a n-ball around the true parameter (blue cross) and fit onto the model manifold (gray solid line and red circles). Distances between the true parameter and the fits (N = 200) correspond to the empirical likelihood ratio. (B,E,H,K) Corresponding histograms to each scenario illustrate the distribution of the distances, i.e. likelihood ratios compared to a -distribution (red line). (C,F,I,L) The pp-plot panel allows for a detailed comparison of the empirical likelihood ratios to a -distribution (N = 200).</p

Conservativeness ratio <i>CR</i> at asymptotic and adapted thresholds.

(A) Histogram of conservativeness ratio CR for the typically utilized 95%-threshold from asymptotic theory for all analyzed parameters, separately for parameter groups and individual models as a reference. (B) Table summarizes the CR for all parameters for alternative confidence levels using asymptotic theory and for adapted thresholds or adapted confidence levels using Chebyshev’s or Cantelli’s inequality. (C, F) All blue histograms including panel (A) share the same threshold value T = 3.84, but differ in the interpretation of the confidence level. (D, G) The green histograms correspond to a confidence level of 90% with slightly different thresholds using Chebyshev’s or Cantelli’s inequality. (E, H) Likewise, purple histograms correspond to a confidence level of 95%. Vertical black solid lines in the histograms indicate the CR > 1 − α and CR > 0.95 limit, which coincides for the asymptotic case.</p

Empirical likelihood ratio classification.

A: Probability-probability plot (pp-plot) with four possible graph scenarios. B: -samples show a noticeable deviation from the diagonal in the pp-plot. The pp-plot in panel B shows graphs from 1000 samples each with Nreal = 500 random numbers drawn from a -distribution. C: Classifications plot with classification regions and classification scheme for mixed cases: the white graph is classified as conservative, although its upper part lies in the perfect consensus area. Likewise, the light gray graph is identified with the anti-conservative case, while the black graph represents an alternating case which populates all three areas.</p

Overview of benchmark model properties and identifiability status.

(A) Estimated parameters of all 19 models and distribution of parameter types. (B) Identifiability of model parameters based on the profile likelihood using the original data and a threshold of Δα=0.05 = 3.84. Either identifiable parameters with finite profile likelihood-based confidence intervals (darker colors) or practically non-identifiable parameters (lighter colors) were identified. Structural non-identifiability was not observed.</p

Bartlett correction results of pp-plot classifications at the 95% confidence level.

Heatmap colors according to pp-plot classification for each Bartlett correction factor as indicated on the y-axis. Black tiles show optimal Bartlett correction factor closest to 1 of the individual parameter in each column. Since columns are ordered by this correction factor, the x-axis corresponds to the approximative conservativeness ratio aCR. White line and black bold numbers indicate the uncorrected outcome for C = 1, whereas green line and numbers show overall optimal Bartlett correction for a CR ≈ 95%. Deviations from 0.95 on x-axis originate from binning issues. (A) Results for all 768 parameters, (B) same results grouped by parameter type and (C) identifiability status. (D) Outcome for individual models reveal that a comprehensive overall optimal correction factor is difficult to determine. (E) Overall optimal Bartlett correction factors C* from exhaustive search with resulting thresholds for the likelihood ratio statistic.</p