5 research outputs found
Recommended from our members
Comparing predictions made by a prediction model, clinical score, and physicians Pediatric asthma exacerbations in the emergency department
Background: Asthma exacerbations are one of the most common medical reasons for children to be brought to the hospital emergency department (ED). Various prediction models have been proposed to support diagnosis of exacerbations and evaluation of their severity. Objectives: First, to evaluate prediction models constructed from data using machine learning techniques and to select the best performing model. Second, to compare predictions from the selected model with predictions from the Pediatric Respiratory Assessment Measure (PRAM) score, and predictions made by ED physicians.
Design: A two-phase study conducted in the ED of an academic pediatric hospital. In phase 1 data collected prospectively using paper forms was used to construct and evaluate five prediction models, and the best performing model was selected. In phase 2, data collected prospectively using a mobile system was used to compare the predictions of the selected prediction model with those from PRAM and ED physicians.
Measurements: Area under the receiver operating characteristic curve and accuracy in phase 1; accuracy, sensitivity, specificity, positive and negative predictive values in phase 2.
Results: In phase 1 prediction models were derived from a data set of 240 patients and evaluated using 10-fold cross validation. A naive Bayes (NB) model demonstrated the best performance and it was selected for phase 2. Evaluation in phase 2 was conducted on data from 82 patients. Predictions made by the NB model were less accurate than the PRAM score and physicians (accuracy of 70.7%, 73.2% and 78.0% respectively), however, according to McNemar’s test it is not possible to conclude that the differences between predictions are statistically significant.
Conclusion: Both the PRAM score and the NB model were less accurate than physicians. The NB model can handle incomplete patient data and as such may complement the PRAM score. However, it requires further research to improve its accuracy
Recommended from our members
A Task-based Support Architecture for Developing Point-of-care Clinical Decision Support Systems for the Emergency Department
Objectives: The purpose of this study was to create a task-based support architecture for developing clinical decision support systems (CDSSs) that assist physicians in making decisions at the point-of-care in the emergency department (ED). The backbone of the proposed architecture was established by a task-based emergency workflow model for a patient-physician encounter.
Methods: The architecture was designed according to an agent-oriented paradigm. Specifically, we used the O-MaSE (Organization-based Multi-agent System Engineering) method that allows for iterative translation of functional requirements into architectural components (e.g., agents). The agent-oriented paradigm was extended with ontology-driven design to implement ontological models representing knowledge required by specific agents to operate.
Results: The task-based architecture allows for the creation of a CDSS that is aligned with the task-based emergency workflow model. It facilitates decoupling of executable components (agents) from embedded domain knowledge (ontological models), thus supporting their interoperability, sharing, and reuse. The generic architecture was implemented as a pilot system, MET3-AE – a CDSS to help with the management of pediatric asthma exacerbation in the ED. The system was evaluated in a hospital ED.
Conclusions: The architecture allows for the creation of a CDSS that integrates support for all tasks from the task-based emergency workflow model, and interacts with hospital information systems. Proposed architecture also allows for reusing and sharing system components and knowledge across disease-specific CDSSs
Recommended from our members
Expanding usability analysis with intrinsic motivation concepts to learn about CDSS adoption: a case study
Objectives
Despite many clinical decision support systems (CDSSs) being rated as highly usable, CDSSs have not been widely adopted in clinical practice. We posit that there are factors aside from usability that impact adoption of CDSSs; in particular we are interested in the role played by MDs intrinsic motivation to use computer-based support. Our research aim is to investigate the relationship between usability and intrinsic motivation in order to learn about adoption of CDSS in clinical practice.
Methods
Following the evaluation of a CDSS, 19 MDs completed a 2 part questionnaire about their intrinsic motivation to use computer-based support in general and the usability of the evaluated CDSS.
Results
The analysis of MDs motivation to use computer-based support demonstrated that MDs are comfortable using computer-based support and in general find using it quite easy (a motivation rating of 0.66 on a (0, 1) scale was computed). However MDs also reported a perceived lack of competence associated with a lack of prior experience using technology in practice, which results in pressure and tension. The considered CDSS scored highly on all usability dimensions and a usability rating of 0.74 was recorded. The examination of the relationship between motivation and usability suggested that users who were motivated to use computer-based support experienced better usability than those who reported low levels of motivation.
Conclusions
Our small case study suggests that an important factor supplementing the usability of CDSSs is intrinsic motivation to use computer-based support in general. We posit that the lack of such a measure thus far in CDSS evaluation may to some extent explain seeming MD satisfaction with CDSSs on one hand, but their limited adoption on the other. We recommend that clinical managers responsible for deploying CDSS should invest in training MDs to use technology underlying computer-based support applications instead of focusing only on the features of the specific CDSS to be deployed
Recommended from our members
Learning the preferences of physicians for the organization of result lists of medical evidence articles
Objectives
Despite many clinical decision support systems (CDSSs) being rated as highly usable, CDSSs have not been widely adopted in clinical practice. We posit that there are factors aside from usability that impact adoption of CDSSs; in particular we are interested in the role played by MDs intrinsic motivation to use computer-based support. Our research aim is to investigate the relationship between usability and intrinsic motivation in order to learn about adoption of CDSS in clinical practice.
Methods
Following the evaluation of a CDSS, 19 MDs completed a 2 part questionnaire about their intrinsic motivation to use computer-based support in general and the usability of the evaluated CDSS.
Results
The analysis of MDs motivation to use computer-based support demonstrated that MDs are comfortable using computer-based support and in general find using it quite easy (a motivation rating of 0.66 on a (0, 1) scale was computed). However MDs also reported a perceived lack of competence associated with a lack of prior experience using technology in practice, which results in pressure and tension. The considered CDSS scored highly on all usability dimensions and a usability rating of 0.74 was recorded. The examination of the relationship between motivation and usability suggested that users who were motivated to use computer-based support experienced better usability than those who reported low levels of motivation.
Conclusions
Our small case study suggests that an important factor supplementing the usability of CDSSs is intrinsic motivation to use computer-based support in general. We posit that the lack of such a measure thus far in CDSS evaluation may to some extent explain seeming MD satisfaction with CDSSs on one hand, but their limited adoption on the other. We recommend that clinical managers responsible for deploying CDSS should invest in training MDs to use technology underlying computer-based support applications instead of focusing only on the features of the specific CDSS to be deployed