LMDA Review, volume 8, issue 1

Contents include: From the President, Jayme Koszyn, Blaulapalooza in Imilwaukee, Notes from Avignon: Diary of a Francophile Theatre Junkie, 1997 Conference News, Canadian Regional News, A Letter to My Colleagues from Lynn M. Thomson, ATHR Group Meets, Proposes Panel for New \u27Turgs, Regional News: The Chicago Gang Meets Again, Member News, and Regional VP\u27s.https://soundideas.pugetsound.edu/lmdareview/1015/thumbnail.jp

Finishing the euchromatic sequence of the human genome

The sequence of the human genome encodes the genetic instructions for human physiology, as well as rich information about human evolution. In 2001, the International Human Genome Sequencing Consortium reported a draft sequence of the euchromatic portion of the human genome. Since then, the international collaboration has worked to convert this draft into a genome sequence with high accuracy and nearly complete coverage. Here, we report the result of this finishing process. The current genome sequence (Build 35) contains 2.85 billion nucleotides interrupted by only 341 gaps. It covers ∼99% of the euchromatic genome and is accurate to an error rate of ∼1 event per 100,000 bases. Many of the remaining euchromatic gaps are associated with segmental duplications and will require focused work with new methods. The near-complete sequence, the first for a vertebrate, greatly improves the precision of biological analyses of the human genome including studies of gene number, birth and death. Notably, the human enome seems to encode only 20,000-25,000 protein-coding genes. The genome sequence reported here should serve as a firm foundation for biomedical research in the decades ahead

Interpretable Active Learning

Active learning has long been a topic of study in machine learning. However, as increasingly complex and opaque models have become standard practice, the process of active learning, too, has become more opaque. There has been little investigation into interpreting what specific trends and patterns an active learning strategy may be exploring. This work expands on the Local Interpretable Model-agnostic Explanations framework (LIME) to provide explanations for active learning recommendations. We demonstrate how LIME can be used to generate locally faithful explanations for an active learning strategy, and how these explanations can be used to understand how different models and datasets explore a problem space over time. In order to quantify the per-subgroup differences in how an active learning strategy queries spatial regions, we introduce a notion of uncertainty bias (based on disparate impact) to measure the discrepancy in the confidence for a model's predictions between one subgroup and another. Using the uncertainty bias measure, we show that our query explanations accurately reflect the subgroup focus of the active learning queries, allowing for an interpretable explanation of what is being learned as points with similar sources of uncertainty have their uncertainty bias resolved. We demonstrate that this technique can be applied to track uncertainty bias over user-defined clusters or automatically generated clusters based on the source of uncertainty.Comment: 13 pages, 8 figures, presented at 2018 Conference on Fairness, Accountability, and Transparency (FAT*), New York, New York, USA. Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR 81:49-61, 201

Prevalence of Effective Audit-and-Feedback Practices in Primary Care Settings: A Qualitative Examination Within Veterans Health Administration

OBJECTIVE: The purpose of this study is to uncover and catalog the various practices for delivering and disseminating clinical performance in various Veterans Affairs (VA) locations and to evaluate their quality against evidence-based models of effective feedback as reported in the literature. BACKGROUND: Feedback can enhance clinical performance in subsequent performance episodes. However, evidence is clear that the way in which feedback is delivered determines whether performance is harmed or improved. METHOD: We purposively sampled 16 geographically dispersed VA hospitals based on high, low, consistently moderate, and moderately average highly variable performance on a set of 17 outpatient clinical performance measures. We excluded four sites due to insufficient interview data. We interviewed four key personnel from each location (n = 48) to uncover effective and ineffective audit and feedback strategies. Interviews were transcribed and analyzed qualitatively using a framework-based content analysis approach to identify emergent themes. RESULTS: We identified 102 unique strategies used to deliver feedback. Of these strategies, 64 (62.74%) have been found to be ineffective according to the audit-and-feedback research literature. Comparing features common to effective (e.g., individually tailored, computerized feedback reports) versus ineffective (e.g., large staff meetings) strategies, most ineffective strategies delivered feedback in meetings, whereas strategies receiving the highest effectiveness scores delivered feedback via visually understood reports that did not occur in a group setting. CONCLUSIONS: Findings show that current practices are leveraging largely ineffective feedback strategies. Future research should seek to identify the longitudinal impact of current feedback and audit practices on clinical performance. APPLICATION: Feedback in primary care has little standardization and does not follow available evidence for effective feedback design. Future research in this area is warranted

Disentangling Influence: Using disentangled representations to audit model predictions

Motivated by the need to audit complex and black box models, there has been extensive research on quantifying how data features influence model predictions. Feature influence can be direct (a direct influence on model outcomes) and indirect (model outcomes are influenced via proxy features). Feature influence can also be expressed in aggregate over the training or test data or locally with respect to a single point. Current research has typically focused on one of each of these dimensions. In this paper, we develop disentangled influence audits, a procedure to audit the indirect influence of features. Specifically, we show that disentangled representations provide a mechanism to identify proxy features in the dataset, while allowing an explicit computation of feature influence on either individual outcomes or aggregate-level outcomes. We show through both theory and experiments that disentangled influence audits can both detect proxy features and show, for each individual or in aggregate, which of these proxy features affects the classifier being audited the most. In this respect, our method is more powerful than existing methods for ascertaining feature influence

Mental models of audit and feedback in primary care settings

Abstract Background Audit and feedback has been shown to be instrumental in improving quality of care, particularly in outpatient settings. The mental model individuals and organizations hold regarding audit and feedback can moderate its effectiveness, yet this has received limited study in the quality improvement literature. In this study we sought to uncover patterns in mental models of current feedback practices within high- and low-performing healthcare facilities. Methods We purposively sampled 16 geographically dispersed VA hospitals based on high and low performance on a set of chronic and preventive care measures. We interviewed up to 4 personnel from each location (n = 48) to determine the facility’s receptivity to audit and feedback practices. Interview transcripts were analyzed via content and framework analysis to identify emergent themes. Results We found high variability in the mental models of audit and feedback, which we organized into positive and negative themes. We were unable to associate mental models of audit and feedback with clinical performance due to high variance in facility performance over time. Positive mental models exhibit perceived utility of audit and feedback practices in improving performance; whereas, negative mental models did not. Conclusions Results speak to the variability of mental models of feedback, highlighting how facilities perceive current audit and feedback practices. Findings are consistent with prior research  in that variability in feedback mental models is associated with lower performance.; Future research should seek to empirically link mental models revealed in this paper to high and low levels of clinical performance

Reports of unintended consequences of financial incentives to improve management of hypertension

<div><p>Background</p><p>Given the increase in financial-incentive programs nationwide, many physicians and physician groups are concerned about potential unintended consequences of providing financial incentives to improve quality of care. However, few studies examine whether actual unintended consequences result from providing financial incentives to physicians. We sought to document the extent to which the unintended consequences discussed in the literature were observable in a randomized clinical trial (RCT) of financial incentives.</p><p>Methods</p><p>We conducted a qualitative observational study nested within a larger RCT of financial incentives to improve hypertension care. We conducted 30-minute telephone interviews with primary care personnel at facilities participating in the RCT housed at12 geographically dispersed Veterans Affairs Medical Centers nationwide. Participants answered questions about unintended effects, clinic team dynamics, organizational impact on care delivery, study participation. We employed a blend of inductive and deductive qualitative techniques for analysis.</p><p>Participants</p><p>Sixty-five participants were recruited from RCT enrollees and personnel not enrolled in the larger RCT, plus one primary care leader per site.</p><p>Results</p><p>Emergent themes included possible patient harm, emphasis on documentation over improving care, reduced professional morale, and positive spillover. All discussions of unintended consequences involving patient harm were only concerns, not actual events. Several unintended consequences concerned ancillary initiatives for quality improvement (e.g., practice guidelines and performance measurement systems) rather than financial incentives.</p><p>Conclusions</p><p>Many unintended consequences of financial incentives noted were either only concerns or attributable to ancillary quality-improvement initiatives. Actual unintended consequences included improved documentation of care without necessarily improving actual care, and positive unintended consequences.</p><p>Trial registration</p><p>Clinicaltrials.gov Identifier: <a href="https://clinicaltrials.gov/ct2/show/NCT00302718" target="_blank">NCT00302718</a></p></div

Study-site characteristics.

<p>Study-site characteristics.</p