Feedback GAP : study protocol for a cluster-randomized trial of goal setting and action plans to increase the effectiveness of audit and feedback interventions in primary care

Peer reviewedPublisher PD

Using family physician Electronic Medical Record data to measure the pathways of cancer care

  Introduction Gaps in care have been identified along the disease pathway for specific cancers. However, no real-world data exists to identify wait times along these cancer pathways. Secondary use of family physician (FP) electronic medical record data (EMR) can augment existing health administrative data in measuring steps in the care pathways. Objectives and Approach We used FP EMR data to identify care pathways for lung cancer and breast cancer patients from the description of symptoms, to the initiation of investigations, referrals to specialty care and the receipt of specific treatments (surgery, chemotherapy, radiation treatment). Data from the Electronic Medical Record Administrative data Linked Database (EMRALD) held at the Institute for Clinical Evaluative Sciences (ICES) was used to identify a cohort of lung cancer and breast cancer patients. Data abstractors examined the FP EMR notes to identify pre-diagnostic symptoms, pre-diagnostic radiological test, biopsy results, oncology and surgical specialist referrals and post-diagnostic surgical and oncological consultations. Results To date, abstractors have reviewed the FP EMR notes for 300 lung cancer patient and 1200 breast cancer patients. Abstractors identified an index date where there was documentation of the first abnormal test result and/or a FP progress note documenting a “suspicious” or “concerning” sign or symptom. For both lung cancer and breast cancer patients, a pre-diagnostic index date was identified in 88.5% of FP EMR notes. For lung cancer patients 66.7% based were based on abnormal chest x-rays and for breast cancer patients 81.1% were based on abnormal mammograms. Pre-diagnostic symptoms were identified in 62.1% of FP EMR notes and 81.6% had post-diagnostic consultation notes. Wait times from the index date to seeing an oncological specialist were less than four weeks for all patients. Conclusion/Implications We are able to use information from FP EMRs linked to health administrative data to identify pre-diagnostic care received by patients prior to their cancer diagnosis. This information can be used to identify care gaps and measure wait times in receiving cancer care from a patient’s perspective

FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation

Most large language models (LLMs) are trained once and never updated; thus, they lack the ability to dynamically adapt to our ever-changing world. In this work, we perform a detailed study of the factuality of LLM-generated text in the context of answering questions that test current world knowledge. Specifically, we introduce FreshQA, a novel dynamic QA benchmark encompassing a diverse range of question and answer types, including questions that require fast-changing world knowledge as well as questions with false premises that need to be debunked. We benchmark a diverse array of both closed and open-source LLMs under a two-mode evaluation procedure that allows us to measure both correctness and hallucination. Through human evaluations involving more than 50K judgments, we shed light on limitations of these models and demonstrate significant room for improvement: for instance, all models (regardless of model size) struggle on questions that involve fast-changing knowledge and false premises. Motivated by these results, we present FreshPrompt, a simple few-shot prompting method that substantially boosts the performance of an LLM on FreshQA by incorporating relevant and up-to-date information retrieved from a search engine into the prompt. Our experiments show that FreshPrompt outperforms both competing search engine-augmented prompting methods such as Self-Ask (Press et al., 2022) as well as commercial systems such as Perplexity.AI. Further analysis of FreshPrompt reveals that both the number of retrieved evidences and their order play a key role in influencing the correctness of LLM-generated answers. Additionally, instructing the LLM to generate concise and direct answers helps reduce hallucination compared to encouraging more verbose answers. To facilitate future work, we release FreshQA at github.com/freshllms/freshqa and commit to updating it at regular intervals.Comment: Preprint, 26 pages, 10 figures, 5 tables; Added FreshEva

An administrative data validation study of the accuracy of algorithms for identifying rheumatoid arthritis: the influence of the reference standard on algorithm performance

BACKGROUND: We have previously validated administrative data algorithms to identify patients with rheumatoid arthritis (RA) using rheumatology clinic records as the reference standard. Here we reassessed the accuracy of the algorithms using primary care records as the reference standard. METHODS: We performed a retrospective chart abstraction study using a random sample of 7500 adult patients under the care of 83 family physicians contributing to the Electronic Medical Record Administrative data Linked Database (EMRALD) in Ontario, Canada. Using physician-reported diagnoses as the reference standard, we computed and compared the sensitivity, specificity, and predictive values for over 100 administrative data algorithms for RA case ascertainment. RESULTS: We identified 69 patients with RA for a lifetime RA prevalence of 0.9%. All algorithms had excellent specificity (>97%). However, sensitivity varied (75-90%) among physician billing algorithms. Despite the low prevalence of RA, most algorithms had adequate positive predictive value (PPV; 51-83%). The algorithm of “[1 hospitalization RA diagnosis code] or [3 physician RA diagnosis codes with ≥1 by a specialist over 2 years]” had a sensitivity of 78% (95% CI 69–88), specificity of 100% (95% CI 100–100), PPV of 78% (95% CI 69–88) and NPV of 100% (95% CI 100–100). CONCLUSIONS: Administrative data algorithms for detecting RA patients achieved a high degree of accuracy amongst the general population. However, results varied slightly from our previous report, which can be attributed to differences in the reference standards with respect to disease prevalence, spectrum of disease, and type of comparator group

Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners

Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning -- that may provide utility for robots, but remain prone to confidently hallucinated predictions. In this work, we present KnowNo, which is a framework for measuring and aligning the uncertainty of LLM-based planners such that they know when they don't know and ask for help when needed. KnowNo builds on the theory of conformal prediction to provide statistical guarantees on task completion while minimizing human help in complex multi-step planning settings. Experiments across a variety of simulated and real robot setups that involve tasks with different modes of ambiguity (e.g., from spatial to numeric uncertainties, from human preferences to Winograd schemas) show that KnowNo performs favorably over modern baselines (which may involve ensembles or extensive prompt tuning) in terms of improving efficiency and autonomy, while providing formal assurances. KnowNo can be used with LLMs out of the box without model-finetuning, and suggests a promising lightweight approach to modeling uncertainty that can complement and scale with the growing capabilities of foundation models. Website: https://robot-help.github.ioComment: Conference on Robot Learning (CoRL) 2023, Oral Presentatio

Changes in the mucosal barrier during acute and chronic Trichuris muris infection

The intestinal mucosal barrier, part of the innate immune defence, is responsive to the external environment and changes in response to infection. There is disparate evidence for the epithelial and goblet cell products within the intrinsic barrier being part of a response to resolve infection. We comprehensively analysed the changes of mucosal glycoconjugates during acute and chronic infection by utilising the Trichuris muris (T. muris) model. Transcription factors, atonal homolog 1 (Math-1) and SAM pointed domain containing ETS transcription factor (Spdef) were activated during acute infection, which promoted stem cell fate towards a secretory cell phenotype. The thickness of the intermediate barrier, the carbohydrate-rich glycocalyx, composed of cell surface mucins increased with exposure to T. muris, with an increase in Muc4, Muc13 and Muc17. Overall, hypersecretion of glycoproteins into the extrinsic barrier (mediated by IL-13) via the gamma amino-butyric acid-α3 receptor (GABA-α3), was observed during acute infection. Furthermore, altered glycosylation was observed during acute and chronic infection; mucins were more highly charged during acute infection than during chronic infection. This study readdresses the changes within the mucosal barrier, in particular in the cell surface and secreted mucins during acute and chronic nematode infection

Nasal lavage natural killer cell function is suppressed in smokers after live attenuated influenza virus

<p>Abstract</p> <p>Background</p> <p>Modified function of immune cells in nasal secretions may play a role in the enhanced susceptibility to respiratory viruses that is seen in smokers. Innate immune cells in nasal secretions have largely been characterized by cellular differentials using morphologic criteria alone, which have successfully identified neutrophils as a significant cell population within nasal lavage fluid (NLF) cells. However, flow cytometry may be a superior method to fully characterize NLF immune cells. We therefore characterized immune cells in NLF by flow cytometry, determined the effects of live attenuated influenza virus (LAIV) on NLF and peripheral blood immune cells, and compared responses in samples obtained from smokers and nonsmokers.</p> <p>Methods</p> <p>In a prospective observational study, we characterized immune cells in NLF of nonsmokers at baseline using flow cytometry and immunohistochemistry. Nonsmokers and smokers were inoculated with LAIV on day 0 and serial nasal lavages were collected on days 1-4 and day 9 post-LAIV. LAIV-induced changes of NLF cells were characterized using flow cytometry. Cell-free NLF was analyzed for immune mediators by bioassay. Peripheral blood natural killer (NK) cells from nonsmokers and smokers at baseline were stimulated <it>in vitro </it>with LAIV followed by flow cytometric and mediator analyses.</p> <p>Results</p> <p>CD45(+)CD56(-)CD16(+) neutrophils and CD45(+)CD56(+) NK cells comprised median 4.62% (range 0.33-14.52) and 23.27% (18.29-33.97), respectively, of non-squamous NLF cells in nonsmokers at baseline. LAIV did not induce changes in total NK cell or neutrophil percentages in either nonsmokers or smokers. Following LAIV inoculation, CD16(+) NK cell percentages and granzyme B levels increased in nonsmokers, and these effects were suppressed in smokers. LAIV inoculation enhanced expression of activating receptor NKG2D and chemokine receptor CXCR3 on peripheral blood NK cells from both nonsmokers and smokers <it>in vitro </it>but did not induce changes in CD16(+) NK cells or granzyme B activity in either group.</p> <p>Conclusions</p> <p>These data are the first to identify NK cells as a major immune cell type in the NLF cell population and demonstrate that mucosal NK cell cytotoxic function is suppressed in smokers following LAIV. Altered NK cell function in smokers suggests a potential mechanism that may enhance susceptibility to respiratory viruses.</p

Respiratory syncytial virus infection induces higher Toll-like receptor-3 expression and TNF-α production than human metapneumovirus infection

published_or_final_versio

Finishing the euchromatic sequence of the human genome

The sequence of the human genome encodes the genetic instructions for human physiology, as well as rich information about human evolution. In 2001, the International Human Genome Sequencing Consortium reported a draft sequence of the euchromatic portion of the human genome. Since then, the international collaboration has worked to convert this draft into a genome sequence with high accuracy and nearly complete coverage. Here, we report the result of this finishing process. The current genome sequence (Build 35) contains 2.85 billion nucleotides interrupted by only 341 gaps. It covers ∼99% of the euchromatic genome and is accurate to an error rate of ∼1 event per 100,000 bases. Many of the remaining euchromatic gaps are associated with segmental duplications and will require focused work with new methods. The near-complete sequence, the first for a vertebrate, greatly improves the precision of biological analyses of the human genome including studies of gene number, birth and death. Notably, the human enome seems to encode only 20,000-25,000 protein-coding genes. The genome sequence reported here should serve as a firm foundation for biomedical research in the decades ahead