Uncertainty analysis in integrated assessment: the users’ perspective

Integrated Assessment (IA) models aim at providing information- and decision-support to complex problems. This paper argues that uncertainty analysis in IA models should be user-driven in order to strengthen science–policy interaction. We suggest an approach to uncertainty analysis that starts with investigating model users’ demands for uncertainty information. These demands are called “uncertainty information needs”. Identifying model users’ uncertainty information needs allows focusing the analysis on those uncertainties which users consider relevant and meaningful. As an illustrative example, we discuss the case of examining users’ uncertainty information needs in the SEAMLESS Integrated Framework (SEAMLESS-IF), an IA model chain for assessing and comparing alternative agricultural and environmental policy options. The most important user group of SEAMLESS-IF are policy experts at the European and national level. Uncertainty information needs of this user group were examined in an interactive process during the development of SEAMLESS-IF and by using a questionnaire. Results indicate that users’ information requirements differed from the uncertainty categories considered most relevant by model developers. In particular, policy experts called for addressing a broader set of uncertainty sources (e.g. model structure and technical model setup). The findings highlight that investigating users’ uncertainty information needs is an essential step towards creating confidence in an IA model and its outcomes. This alone, however, may not be sufficient for effectively implementing a user-oriented uncertainty analysis in such models. As the case study illustrates, it requires to include uncertainty analysis into user participation from the outset of the IA modelling process

The Essential Elements of a Risk Governance Framework for Current and Future Nanotechnologies

Societies worldwide are investing considerable resources into the safe development and use of nanomaterials. Although each of these protective efforts is crucial for governing the risks of nanomaterials, they are insufficient in isolation. What is missing is a more integrative governance approach that goes beyond legislation. Development of this approach must b

Choosing the best for preventing the worst: A structured analysis of the selection of risk management options in REACH restriction dossiers.

Under the European chemicals legislation REACH (Registration, Evaluation, Authorisation and restriction of CHemicals), the use of chemicals posing an unacceptable risk for humans and the environment can be restricted. This requires that regulatory authorities of EU member states, or the European Chemicals Agency on request of the Commission, submit a restriction proposal in which they suggest one or multiple risk management options (RMOs). The options are recommended to be evaluated in a socio-economic analysis (SEA) using defined criteria. This paper explores the drivers of the selection of the preferred RMO in 32 restriction dossiers. Applying principal component analysis reveals that the selection of the preferred RMO, and the evaluation of possible trade-offs between alternative RMOs, is determined by criteria characterizing a measure's effectiveness and practicality, in particular its risk reduction capacity (R) and proportionality. A logistic regression using quantitative estimates provided in SEA suggests that the probability for an RMO to be selected is the higher the higher its R and the lower the costs of the restriction. Based on our analysis we conclude that the selection process of RMOs in REACH restriction dossiers could be strengthened by defining a limited but unambiguous set of criteria, conducting a score-based evaluation as a default, and by defining transparent decision rules.</p

Socio-economic analysis in REACH restriction dossiers for chemicals management: A critical review

This paper offers a critical review of socio-economic analysis (SEA) in Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH) restriction dossiers. We examine the conceptual setup of SEA and identify the methods used for impact assessment. Moreover, we analyse the outcomes of quantitative impact assessment across restriction dossiers and substance groups. We find that impact assessment has largely focused on economic and health impacts. Environmental, social, wider economic and distributional impacts have either been evaluated qualitatively or not at all. While this can be explained by the specific scope of the proposed restriction or by lacking data, we also observe a lack of approaches for environmental and health impact assessment. This underlines the need to develop integrated methods that transform information about chemical effects and risks into impacts and, ultimately, into benefits and damages. Furthermore, to strengthen the function of SEA as decision-support tool in REACH restriction procedures, a comparative SEA of at least two alternative restriction options should be the default.</p

A tutorial for analysing the cost-effectiveness of alternative methods for assessing chemical toxicity: the case of acute oral toxicity prediction

Compared with traditional animal methods for toxicity testing, in vitro and in silico methods are widely considered to allow for more cost-effective assessment of chemicals. However, how to assess the cost-effectiveness of alternative methods has remained unclear. This paper offers a user-oriented tutorial to performing cost-effectiveness analysis (CEA) of alternative (non-animal) methods. The purpose is to demonstrate how CEA facilitates the identification of the alternative method, or the combination of methods, that offers the highest information gain per unit of cost. We illustrate how information gains and costs of single methods and method combinations can be assessed. Using acute oral toxicity as an example, we apply CEA to a set of four in silico methods (ToxSuite, TOPKAT, T.E.S.T, ADMET Predictor), one in vitro method (3T3 neutral red uptake cytotoxicity assay), and various combinations of these methods. Our results underline that in silico tools are more cost-effective than the in vitro test. Battery combinations of alternative methods, however, do not necessarily outperform single methods because information gains from a battery are easily outweighed by additional costs.JRC.I.5-Systems Toxicolog

A Critical Review of Adverse Outcome Pathway-Based Concepts and Tools for Integrating Information from Nonanimal Testing Methods: The Case of Skin Sensitization

Integrating information from in vitro, in silico, and in chemico methods into toxicity testing strategies has been widely considered the way of phasing out animal testing. At the same time, testing strategies using new approaches and methods shall provide adequate and relevant information about chemicals' hazardous properties. We reviewed objectives and requirements for guiding the process of data integration that are suggested in the scientific literature. Based on the existing approaches, we develop criteria for resource-efficient testing strategies, and we evaluate existing testing strategies for skin sensitization hazard and risk assessment under these criteria. We conclude that existing testing strategies—except two cases—still focus predominantly on maximizing toxicity information, but largely ignore resource efficiency criteria. Balancing information gained from testing strategies with its respective direct and indirect costs (including also welfare losses for society in case of unintended health or environmental damages) is a necessary condition to allow for transparent comparisons of their resource efficiency. Therefore, developing approaches for balancing information gains and costs should become an explicit part of the developmental process of nonanimal testing strategies to ensure that phasing out animal testing complies not only with regulatory information requirements but also with available resources

The impact of precision uncertainty on predictive accuracy metrics of non-animal testing methods

The ability of non-animal methods to correctly predict the outcome of in vivo testing in repeated applications is referred to as precision. Due to dichotomizing continuous read-outs into discrete “positive/negative” hazard data, non-animal methods can reveal discordant classifications if results are sufficiently close to a defined classification threshold. This paper explores the impact of precision uncertainty on the predictive accuracy of non-animal methods. Using selected non-animal methods for assessing skin sensitization hazard as case study examples, we explore the impact of precision uncertainty separately and in combination with uncertainty due to varying composition and size of experimental samples. Our results underline that discrete numbers on a non-animal method’s sensitivity, specificity, and concordance are of limited value for evaluation of its predictivity. Instead, information on the variability and the upper and lower limits of accuracy metrics should be provided to ensure a transparent assessment of a testing method’s predictivity, and to allow for a meaningful comparison of the predictivity of a non-animal method with that of an animal test.</p