A literature review on the use of expert opinion in probabilistic risk analysis

Risk assessment is part of the decision making process in many fields of discipline, such as engineering, public health, environment, program management, regulatory policy, and finance. There has been considerable debate over the philosophical and methodological treatment of risk in the past few decades, ranging from its definition and classification to methods of its assessment. Probabilistic risk analysis (PRA) specifically deals with events represented by low probabilities of occurring with high levels of unfavorable consequences. Expert judgment is often a critical source of information in PRA, since empirical data on the variables of interest are rarely available. The author reviews the literature on the use of expert opinion in PRA, in particular on the approaches to eliciting and aggregating experts'assessments. The literature suggests that the methods by which expert opinions are collected and combined have a significant effect on the resulting estimates. The author discusses two types of approaches to eliciting and aggregating expert judgments-behavioral and mathematical approaches, with the emphasis on the latter. It is generally agreed that mathematical approaches tend to yield more accurate estimates than behavioral approaches. After a short description of behavioral approaches, the author discusses mathematical approaches in detail, presenting three aggregation models: non-Bayesian axiomatic models, Bayesian models, andpsychological scaling models. She also discusses issues of stochastic dependence.Health Monitoring&Evaluation,ICT Policy and Strategies,Public Health Promotion,Enterprise Development&Reform,Statistical&Mathematical Sciences,ICT Policy and Strategies,Health Monitoring&Evaluation,Statistical&Mathematical Sciences,Science Education,Scientific Research&Science Parks

A multimodal neuroimaging classifier for alcohol dependence

With progress in magnetic resonance imaging technology and a broader dissemination of state-of-the-art imaging facilities, the acquisition of multiple neuroimaging modalities is becoming increasingly feasible. One particular hope associated with multimodal neuroimaging is the development of reliable data-driven diagnostic classifiers for psychiatric disorders, yet previous studies have often failed to find a benefit of combining multiple modalities. As a psychiatric disorder with established neurobiological effects at several levels of description, alcohol dependence is particularly well-suited for multimodal classification. To this aim, we developed a multimodal classification scheme and applied it to a rich neuroimaging battery (structural, functional task-based and functional resting-state data) collected in a matched sample of alcohol-dependent patients (N = 119) and controls (N = 97). We found that our classification scheme yielded 79.3% diagnostic accuracy, which outperformed the strongest individual modality - grey-matter density - by 2.7%. We found that this moderate benefit of multimodal classification depended on a number of critical design choices: a procedure to select optimal modality-specific classifiers, a fine-grained ensemble prediction based on cross-modal weight matrices and continuous classifier decision values. We conclude that the combination of multiple neuroimaging modalities is able to moderately improve the accuracy of machine-learning-based diagnostic classification in alcohol dependence

A multimodal neuroimaging classifier for alcohol dependence

With progress in magnetic resonance imaging technology and a broader dissemination of state-of-the-art imaging facilities, the acquisition of multiple neuroimaging modalities is becoming increasingly feasible. One particular hope associated with multimodal neuroimaging is the development of reliable data-driven diagnostic classifiers for psychiatric disorders, yet previous studies have often failed to find a benefit of combining multiple modalities. As a psychiatric disorder with established neurobiological effects at several levels of description, alcohol dependence is particularly well-suited for multimodal classification. To this aim, we developed a multimodal classification scheme and applied it to a rich neuroimaging battery (structural, functional task-based and functional resting-state data) collected in a matched sample of alcohol-dependent patients (N = 119) and controls (N = 97). We found that our classification scheme yielded 79.3% diagnostic accuracy, which outperformed the strongest individual modality - grey-matter density - by 2.7%. We found that this moderate benefit of multimodal classification depended on a number of critical design choices: a procedure to select optimal modality-specific classifiers, a fine-grained ensemble prediction based on cross-modal weight matrices and continuous classifier decision values. We conclude that the combination of multiple neuroimaging modalities is able to moderately improve the accuracy of machine-learning-based diagnostic classification in alcohol dependence

Can Neuroscience Help Predict Future Antisocial Behavior?

Part I of this Article reviews the tools currently available to predict antisocial behavior. Part II discusses legal precedent regarding the use of, and challenges to, various prediction methods. Part III introduces recent neuroscience work in this area and reviews two studies that have successfully used neuroimaging techniques to predict recidivism. Part IV discusses some criticisms that are commonly levied against the various prediction methods and highlights the disparity between the attitudes of the scientific and legal communities toward risk assessment generally and neuroscience specifically. Lastly, Part V explains why neuroscience methods will likely continue to help inform and, ideally, improve the tools we use to help assess, understand, and predict human behavior

Expert Elicitation for Reliable System Design

This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.Comment: This paper commented in: [arXiv:0708.0285], [arXiv:0708.0287], [arXiv:0708.0288]. Rejoinder in [arXiv:0708.0293]. Published at http://dx.doi.org/10.1214/088342306000000510 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Psychometrics in Practice at RCEC

A broad range of topics is dealt with in this volume: from combining the psychometric generalizability and item response theories to the ideas for an integrated formative use of data-driven decision making, assessment for learning and diagnostic testing. A number of chapters pay attention to computerized (adaptive) and classification testing. Other chapters treat the quality of testing in a general sense, but for topics like maintaining standards or the testing of writing ability, the quality of testing is dealt with more specifically.\ud All authors are connected to RCEC as researchers. They present one of their current research topics and provide some insight into the focus of RCEC. The selection of the topics and the editing intends that the book should be of special interest to educational researchers, psychometricians and practitioners in educational assessment

Psychometric Evaluation and Design of Patient-Centered Communication Measures for Cancer Care Settings

Objective To evaluate the psychometric properties of questions that assess patient perceptions of patient-provider communication and design measures of patient-centered communication (PCC). Methods Participants (adults with colon or rectal cancer living in North Carolina) completed a survey at 2 to 3 months post-diagnosis. The survey included 87 questions in six PCC Functions: Exchanging Information, Fostering Health Relationships, Making Decisions, Responding to Emotions, Enabling Patient Self-Management, and Managing Uncertainty. For each Function we conducted factor analyses, item response theory modeling, and tests for differential item functioning, and assessed reliability and construct validity. Results Participants included 501 respondents; 46% had a high school education or less. Reliability within each Function ranged from 0.90 to 0.96. The PCC-Ca-36 (36-question survey; reliability=0.94) and PCC-Ca-6 (6-question survey; reliability=0.92) measures differentiated between individuals with poor and good health (i.e., known-groups validity) and were highly correlated with the HINTS communication scale (i.e., convergent validity). Conclusion This study provides theory-grounded PCC measures found to be reliable and valid in colorectal cancer patients in North Carolina. Future work should evaluate measure validity over time and in other cancer populations. Practice implications The PCC-Ca-36 and PCC-Ca-6 measures may be used for surveillance, intervention research, and quality improvement initiatives

On the association between adolescent autonomy and psychosocial functioning: examining decisional independence from a self-determination theory perspective

In the present study, we focus on the concept of adolescent autonomy and its relation with psychosocial functioning. Specifically, we aim to differentiate between 2 prevailing conceptualizations of autonomy, that is, (a) autonomy defined as independence versus dependence and (b) autonomy defined as self-endorsed versus controlled functioning. A 2nd goal is to examine the relative contribution of each autonomy operationalization in the prediction of adolescents' adjustment (i.e., well-being, problem behavior, and intimacy). Data were gathered in a sample of 707 Belgian adolescents. Using a newly developed questionnaire, we assessed both the degree of independent decision making per se and the self-endorsed versus controlled motives underlying both independent and dependent decision making. The degree of independent decision making could clearly be differentiated from the underlying motives for doing so. Moreover, independent decision making as such showed unique associations with more problem behavior. Further, as expected, self-endorsed motives for both independent and dependent decision making generally related to an adaptive pattern of psychosocial functioning, and controlled motives were associated with maladjustment. The discussion focuses on the difference between the 2 perspectives on autonomy and on the different meaning of the motives underlying independent, relative to dependent, decision making