A scoping review of metamodeling applications and opportunities for advanced health economic analyses

Introduction: Metamodels, also known as meta-models, surrogate models, or emulators, are used in several fields of research to negate runtime issues with analyzing computational demanding simulation models. This study introduces metamodeling and presents results of a review on metamodeling applications in health economics. Areas covered: A scoping review was performed to identify studies that applied metamodeling methods in a health economic context. After search and selection, 13 publications were found to employ metamodeling methods in health economics. Metamodels were used to perform value of information analysis (n = 5, 38%), deterministic sensitivity analysis (n = 4, 31%), model calibration (n = 1, 8%), probabilistic sensitivity analysis (n = 1), or optimization (n = 1, 8%). One study was found to extrapolate a simulation model to other countries (n = 1, 8%). Applied metamodeling techniques varied considerably between studies, with linear regression being most frequently applied (n = 7, 54%). Expert commentary: Although it has great potential to enable computational demanding analyses of health economic models, metamodeling in health economics is still in its infancy, as illustrated by the limited number of applications and the relatively simple metamodeling methods applied. Comprehensive guidance specific to health economics is needed to provide modelers with the information and tools needed to utilize the full potential of metamodels

The Assessment of Patient Clinical Outcome: Advantages, Models, Features of an Ideal Model

Background: The assessment of patient clinical outcome focuses on measuring various aspects of the health status of a patient who is under healthcare intervention. Patient clinical outcome assessment is a very significant process in the clinical field as it allows health care professionals to better understand the effectiveness of their health care programs and thus for enhancing the health care quality in general. It is thus vital that a high quality, informative review of current issues regarding the assessment of patient clinical outcome should be conducted. Aims & Objectives: 1) Summarizes the advantages of the assessment of patient clinical outcome; 2) reviews some of the existing patient clinical outcome assessment models namely: Simulation, Markov, Bayesian belief networks, Bayesian statistics and Conventional statistics, and Kaplan-Meier analysis models; and 3) demonstrates the desired features that should be fulfilled by a well-established ideal patient clinical outcome assessment model. Material & Methods: An integrative review of the literature has been performed using the Google Scholar to explore the field of patient clinical outcome assessment. Conclusion: This paper will directly support researchers, clinicians and health care professionals in their understanding of developments in the domain of the assessment of patient clinical outcome, thus enabling them to propose ideal assessment models

Med Decis Making

Purpose:Metamodels are simplified approximations of more complex models that can be used as surrogates for the original models. Challenges in using metamodels for policy analysis arise when there are multiple correlated outputs of interest. We develop a framework for metamodeling with policy simulations to accommodate multivariate outcomes.Methods:We combine two algorithm adaptation methods \u2013 multi-target stacking and regression chain with maximum correlation \u2013 with different base learners including linear regression (LR), elastic net (EE) with second-order terms, Gaussian process regression (GPR), random forests (RFs), and neural networks. We optimize integrated models using variable selection and hyperparameter tuning. We compare accuracy, efficiency, and interpretability of different approaches. As an example application, we develop metamodels to emulate a microsimulation model of testing and treatment strategies for hepatitis C in correctional settings.Results:Output variables from the simulation model were correlated (average \u3c1=0.58). Without multioutput algorithm adaptation methods, in-sample fit (measured by R2) ranged from 0.881 for LR to 0.987 for GPR. The multioutput algorithm adaptation method increased R2 by an average 0.002 across base learners. Variable selection and hyperparameter tuning increased R2 by 0.009. Simpler models such as LR, EE, and RF required minimal training and prediction time. LR and EE had advantages in model interpretability, and we considered methods for improving interpretability of other models.Conclusions:In our example application, the choice of base learner had the largest impact on R2; multioutput algorithm adaptation and variable selection and hyperparameter tuning had modest impact. While advantages and disadvantages of specific learning algorithms may vary across different modeling applications, our framework for metamodeling in policy analyses with multivariate outcomes has broad applicability to decision analysis in health and medicine.R37 DA015612/DA/NIDA NIH HHSUnited States/U38 PS004644/PS/NCHHSTP CDC HHSUnited States/2022-10-01T00:00:00Z35735216PMC945245411962vault:4324

Emulator-based Bayesian calibration of the CISNET colorectal cancer models

PURPOSE: To calibrate Cancer Intervention and Surveillance Modeling Network (CISNET) 's SimCRC, MISCAN-Colon, and CRC-SPIN simulation models of the natural history colorectal cancer (CRC) with an emulator-based Bayesian algorithm and internally validate the model-predicted outcomes to calibration targets.METHODS: We used Latin hypercube sampling to sample up to 50,000 parameter sets for each CISNET-CRC model and generated the corresponding outputs. We trained multilayer perceptron artificial neural networks (ANN) as emulators using the input and output samples for each CISNET-CRC model. We selected ANN structures with corresponding hyperparameters (i.e., number of hidden layers, nodes, activation functions, epochs, and optimizer) that minimize the predicted mean square error on the validation sample. We implemented the ANN emulators in a probabilistic programming language and calibrated the input parameters with Hamiltonian Monte Carlo-based algorithms to obtain the joint posterior distributions of the CISNET-CRC models' parameters. We internally validated each calibrated emulator by comparing the model-predicted posterior outputs against the calibration targets.RESULTS: The optimal ANN for SimCRC had four hidden layers and 360 hidden nodes, MISCAN-Colon had 4 hidden layers and 114 hidden nodes, and CRC-SPIN had one hidden layer and 140 hidden nodes. The total time for training and calibrating the emulators was 7.3, 4.0, and 0.66 hours for SimCRC, MISCAN-Colon, and CRC-SPIN, respectively. The mean of the model-predicted outputs fell within the 95% confidence intervals of the calibration targets in 98 of 110 for SimCRC, 65 of 93 for MISCAN, and 31 of 41 targets for CRC-SPIN.CONCLUSIONS: Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis, like the CISNET CRC models.</p

Prediction of claims in export credit finance: a comparison of four machine learning techniques

This study evaluates four machine learning (ML) techniques (Decision Trees (DT), Random Forests (RF), Neural Networks (NN) and Probabilistic Neural Networks (PNN)) on their ability to accurately predict export credit insurance claims. Additionally, we compare the performance of the ML techniques against a simple benchmark (BM) heuristic. The analysis is based on the utilisation of a dataset provided by the Berne Union, which is the most comprehensive collection of export credit insurance data and has been used in only two scientific studies so far. All ML techniques performed relatively well in predicting whether or not claims would be incurred, and, with limitations, in predicting the order of magnitude of the claims. No satisfactory results were achieved predicting actual claim ratios. RF performed significantly better than DT, NN and PNN against all prediction tasks, and most reliably carried their validation performance forward to test performance

Models for predicting the quality of life domains on the general population through the orange data mining approach

The incidence of type 2 diabetes mellitus (DM) has been predicted to increase until 2045 in the world. Furthermore, long-term treatment and lifestyle factors affect the quality of life. This study aims to determine the models that can be used to predict the quality-of-life domains in prediabetes patients by using Artificial Intelligent (AI) devices. This is a cross-sectional design in which the inclusion criteria were individuals of age above 18 years and has never been diagnosed with diabetes mellitus (both type 1 DM and type 2 DM), fasted for at least 8 hours, and are willing to sign an informed consent after having received an explanation. Participants were asked to fill out two questionnaires, namely the Indonesian version of the Finnish Diabetes Risk Score (FINDRISC) and the EuroQoL-5 Dimensions-5 Level (EQ-5D-5L). The AI application uses Orange® machine learning with three models used in predictive analysis, such as Logistic Regression, Neural Network, and SVM. In addition, the model was evaluated using the sensitivity, precision, and accuracy of the AU-ROC parameters. The results showed that the neural network model based on the AUC value, precision, accuracy, and also the ROC analysis, was the best for predicting the utility index of domains in the EQ-5D-5L questionnaire, based on demographic data and the FINDRISC questionnaire