Value of information analysis for interventional and counterfactual Bayesian networks in forensic medical sciences

Objectives: Inspired by real-world examples from the forensic medical sciences domain, we seek to determine whether a decision about an interventional action could be subject to amendments on the basis of some incomplete information within the model, and whether it would be worthwhile for the decision maker to seek further information prior to suggesting a decision

Criminally Incompetent Academic Misinterpretation of Criminal Data-and how the Media Pushed the Fake News

On 17 Jan 2018 multiple news sources (e.g. see here, here, and here) ran a story about a new research paper ‎ that claims to expose both the inaccuracies and racial bias in one of the most common algorithms used by parole boards to predict recidivism (i.e. whether or not a defendant will re-offend). The research paper was written by the world famous computer scientist Hany Farid (along with a student Julia Dressel). But the real story here is that the paper’s accusation of racial bias (specifically that the algorithm is biased against black people) is based on a fundamental misunderstanding of causation and statistics. The algorithm is no more ‘biased’ against black people than it is biased against white single parents, ‎ old people, people living in Beattyville Kentucky, or women called ‘Amber’. In fact, as we show in this brief article, if you choose any factor that correlates with poverty you will inevitably replicate the statistical ‘bias’ claimed in the paper. And if you accept the validity of the claims in the paper then you must also accept, for example, that a charity which uses poverty as a factor to identify and help homeless people is being racist because it is biased against white people (and also, interestingly, Indian Americans). The fact that the article was published and that none of the media running the story realise that they are pushing fake news is what is most important here. Depressingly, many similar research studies involving the same kind of misinterpretation of statistics result in popular media articles that push a false narrative of one kind or another

Integrating expert knowledge with data in Bayesian networks: Preserving data-driven expectations when the expert variables remain unobserved

When developing a causal probabilistic model, i.e. a Bayesian network (BN), it is common to incorporate expert knowledge of factors that are important for decision analysis but where historical data are unavailable or difficult to obtain. This paper focuses on the problem whereby the distribution of some continuous variable in a BN is known from data, but where we wish to explicitly model the impact of some additional expert variable (for which there is expert judgment but no data). Because the statistical outcomes are already influenced by the causes an expert might identify as variables missing from the dataset, the incentive here is to add the expert factor to the model in such a way that the distribution of the data variable is preserved when the expert factor remains unobserved. We provide a method for eliciting expert judgment that ensures the expected values of a data variable are preserved under all the known conditions. We show that it is generally neither possible, nor realistic, to preserve the variance of the data variable, but we provide a method towards determining the accuracy of expertise in terms of the extent to which the variability of the revised empirical distribution is minimised. We also describe how to incorporate the assessment of extremely rare or previously unobserved events

From complex questionnaire and interviewing data to intelligent Bayesian network models for medical decision support

OBJECTIVES: 1) To develop a rigorous and repeatable method for building effective Bayesian network (BN) models for medical decision support from complex, unstructured and incomplete patient questionnaires and interviews that inevitably contain examples of repetitive, redundant and contradictory responses; 2) To exploit expert knowledge in the BN development since further data acquisition is usually not possible; 3) To ensure the BN model can be used for interventional analysis; 4) To demonstrate why using data alone to learn the model structure and parameters is often unsatisfactory even when extensive data is available. METHOD: The method is based on applying a range of recent BN developments targeted at helping experts build BNs given limited data. While most of the components of the method are based on established work, its novelty is that it provides a rigorous consolidated and generalised framework that addresses the whole life-cycle of BN model development. The method is based on two original and recent validated BN models in forensic psychiatry, known as DSVM-MSS and DSVM-P. RESULTS: When employed with the same datasets, the DSVM-MSS demonstrated competitive to superior predictive performance (AUC scores 0.708 and 0.797) against the state-of-the-art (AUC scores ranging from 0.527 to 0.705), and the DSVM-P demonstrated superior predictive performance (cross-validated AUC score of 0.78) against the state-of-the-art (AUC scores ranging from 0.665 to 0.717). More importantly, the resulting models go beyond improving predictive accuracy and into usefulness for risk management purposes through intervention, and enhanced decision support in terms of answering complex clinical questions that are based on unobserved evidence. CONCLUSIONS: This development process is applicable to any application domain which involves large-scale decision analysis based on such complex information, rather than based on data with hard facts, and in conjunction with the incorporation of expert knowledge for decision support via intervention. The novelty extends to challenging the decision scientists to reason about building models based on what information is really required for inference, rather than based on what data is available and hence, forces decision scientists to use available data in a much smarter way

Probabilistic Decision Tools for Determining Impacts of Agricultural Development Policy on Household Nutrition

Governments around the world have agreed to end hunger and food insecurity and to improve global nutrition, largely through changes to agriculture and food systems. However, they are faced with a lot of uncertainty when making policy decisions, since any agricultural changes will influence social and biophysical systems, which could yield either positive or negative nutrition outcomes. We outline a holistic probability modeling approach with Bayesian Network (BN) models for nutritional impacts resulting from agricultural development policy. The approach includes the elicitation of expert knowledge for impact model development, including sensitivity analysis and value of information calculations. It aims at a generalizable methodology that can be applied in a wide range of contexts. To showcase this approach, we develop an impact model of Vision 2040, Uganda's development strategy, which, among other objectives, seeks to transform the country's agricultural landscape from traditional systems to large-scale commercial agriculture. Model results suggest that Vision 2040 is likely to have negative outcomes for the rural livelihoods it intends to support; it may have no appreciable influence on household hunger but, by influencing preferences for and access to quality nutritional foods, may increase the prevalence of micronutrient deficiency. The results highlight the trade-offs that must be negotiated when making decisions regarding agriculture for nutrition, and the capacity of BNs to make these trade-offs explicit. The work illustrates the value of BNs for supporting evidence-based agricultural development decisions

A Comprehensive Scoping Review of Bayesian Networks in Healthcare: Past, Present and Future

No comprehensive review of Bayesian networks (BNs) in healthcare has been published in the past, making it difficult to organize the research contributions in the present and identify challenges and neglected areas that need to be addressed in the future. This unique and novel scoping review of BNs in healthcare provides an analytical framework for comprehensively characterizing the domain and its current state. The review shows that: (1) BNs in healthcare are not used to their full potential; (2) a generic BN development process is lacking; (3) limitations exists in the way BNs in healthcare are presented in the literature, which impacts understanding, consensus towards systematic methodologies, practice and adoption of BNs; and (4) a gap exists between having an accurate BN and a useful BN that impacts clinical practice. This review empowers researchers and clinicians with an analytical framework and findings that will enable understanding of the need to address the problems of restricted aims of BNs, ad hoc BN development methods, and the lack of BN adoption in practice. To map the way forward, the paper proposes future research directions and makes recommendations regarding BN development methods and adoption in practice

Decision analysis methods guide: agricultural policy for nutrition

It is often very difficult to make accurate projections about how interventions will affect the real world and to use such projections to develop effective implementation plans, monitor progress and evaluate project impacts. This is due to a variety of factors including lack of data, complex impact pathways and risks and uncertainties that are difficult to factor into intervention planning. Scientific approaches to produce reliable impact projections are rarely applied in agricultural development, but Decision Analysis techniques commonly used in other fields have the potential to improve development decisions. This working paper outlines a Decision Analysis approach that can help decision makers efficiently allocate resources to enhance the effectiveness of policy decisions. The procedures outlined in this publication feature the construction of causal models – models that describe the mechanisms through which intervention impacts will be delivered – that are codeveloped by experts, stakeholders and analysts through facilitated participatory processes. These models are then formalized as Bayesian Network (BN) models, a modelling approach that has been widely applied in a range of disciplines, including medical sciences, genetics, environmental sciences and legal reasoning. BNs allow for the formal representation of causal models, such as intervention impact pathways. They can work effectively with incomplete information, combine expert knowledge with other sources of information and allow for adequate consideration of risk. This paper illustrates the use of participatory workshops that convene experts on the systems, stakeholders involved in ongoing or prospective projects and analysts. These teams can jointly develop impact pathways for the interventions, which can be formalized into quantitative BN models. After several rounds of feedback elicitation and the inclusion of data from experts and other sources, stochastic simulations can be run to determine the likely impacts of the interventions. Results can be presented back to stakeholders for feedback. Through the tools presented in this working paper, critical uncertainties in the models of intervention impact pathways can be identified. These high-value variables can determine uncertainty about project outcomes. Further measurement or disaggregation of these variables can support decisionmaking processes. By demonstrating improved intervention decisions with little additional investment and improved tools for intervention decision modelling, we hope that this approach will be widely adopted and used to enhance the efficacy of development activitie