MetaCAM: Ensemble-Based Class Activation Map

The need for clear, trustworthy explanations of deep learning model predictions is essential for high-criticality fields, such as medicine and biometric identification. Class Activation Maps (CAMs) are an increasingly popular category of visual explanation methods for Convolutional Neural Networks (CNNs). However, the performance of individual CAMs depends largely on experimental parameters such as the selected image, target class, and model. Here, we propose MetaCAM, an ensemble-based method for combining multiple existing CAM methods based on the consensus of the top-k% most highly activated pixels across component CAMs. We perform experiments to quantifiably determine the optimal combination of 11 CAMs for a given MetaCAM experiment. A new method denoted Cumulative Residual Effect (CRE) is proposed to summarize large-scale ensemble-based experiments. We also present adaptive thresholding and demonstrate how it can be applied to individual CAMs to improve their performance, measured using pixel perturbation method Remove and Debias (ROAD). Lastly, we show that MetaCAM outperforms existing CAMs and refines the most salient regions of images used for model predictions. In a specific example, MetaCAM improved ROAD performance to 0.393 compared to 11 individual CAMs with ranges from -0.101-0.172, demonstrating the importance of combining CAMs through an ensembling method and adaptive thresholding.Comment: 9 page

Agreement and Correlation Between Different Topical Corticosteroid Potency Classification Systems

Importance: Topical corticosteroids (TCSs) are available in multiple potencies that alter their effectiveness and safety. Pharmacoepidemiologic studies on TCSs are hampered by the absence of a universal potency classification system, limiting comparisons across studies, robust exposure classification, and clinical interpretation. Objective: To classify TCSs into 3 commonly used potency classification systems and evaluate the agreement and correlation between the 3 systems. Design, Setting, and Participants: In this classification study, a comprehensive list of TCS formulations was compiled using sources identified in the literature, the Ontario Drug Benefit Formulary, a recent Cochrane review on the use of TCSs in people with eczema, and the Anatomical Therapeutic Classification (ATC) of the World Health Organization from August 11, 2021, to January 6, 2022. Topical corticosteroid potency classifications were assigned and compared using the 7-category US classification system, a 4-category classification from a recent Cochrane review largely based on the UK formulary, and the 4-category ATC classification. To facilitate comparisons across systems, the 7-category US system was consolidated into 4 categories. Main Outcomes and Measures: Cohen weighted ? (?w) and Spearman rank correlation coefficients (r) were computed to examine agreement and correlation between the classification systems. Results: A total of 232 unique TCS formulations (ATC, n = 231; US classification, n = 232; Cochrane review, n = 89) were included. Overall, there was low-to-moderate agreement but strong correlation between the classification systems. The US classification had weak agreement with the ATC system (?w, 0.53; 95% CI, 0.45-0.60) and moderate agreement with the Cochrane review classification (?w, 0.60; 95% CI, 0.48-0.73); there was weak agreement between the ATC and Cochrane review classifications (?w, 0.58; 95% CI, 0.46-0.71). The US classification strongly correlated with the ATC system (r, 0.77; 95% CI, 0.71-0.82) and Cochrane review classification (r, 0.74; 95% CI, 0.62-0.82). There was also a strong correlation between the Cochrane review and ATC classifications (r, 0.71; 95% CI, 0.58-0.80). Conclusions and Relevance: This classification study used multiple resources to classify 232 TCS formulations into 3 potency classifications. Because these systems are often incongruent, they may yield different results in pharmacoepidemiologic studies; investigators need to be transparent in their classification approach and consider alternative potency definitions in sensitivity analyses.