ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders

Our approach, which we call Embeddings for Language/Image-aligned X-Rays, or ELIXR, leverages a language-aligned image encoder combined or grafted onto a fixed LLM, PaLM 2, to perform a broad range of tasks. We train this lightweight adapter architecture using images paired with corresponding free-text radiology reports from the MIMIC-CXR dataset. ELIXR achieved state-of-the-art performance on zero-shot chest X-ray (CXR) classification (mean AUC of 0.850 across 13 findings), data-efficient CXR classification (mean AUCs of 0.893 and 0.898 across five findings (atelectasis, cardiomegaly, consolidation, pleural effusion, and pulmonary edema) for 1% (~2,200 images) and 10% (~22,000 images) training data), and semantic search (0.76 normalized discounted cumulative gain (NDCG) across nineteen queries, including perfect retrieval on twelve of them). Compared to existing data-efficient methods including supervised contrastive learning (SupCon), ELIXR required two orders of magnitude less data to reach similar performance. ELIXR also showed promise on CXR vision-language tasks, demonstrating overall accuracies of 58.7% and 62.5% on visual question answering and report quality assurance tasks, respectively. These results suggest that ELIXR is a robust and versatile approach to CXR AI

Towards Generalist Biomedical AI

Medicine is inherently multimodal, with rich data modalities spanning text, imaging, genomics, and more. Generalist biomedical artificial intelligence (AI) systems that flexibly encode, integrate, and interpret this data at scale can potentially enable impactful applications ranging from scientific discovery to care delivery. To enable the development of these models, we first curate MultiMedBench, a new multimodal biomedical benchmark. MultiMedBench encompasses 14 diverse tasks such as medical question answering, mammography and dermatology image interpretation, radiology report generation and summarization, and genomic variant calling. We then introduce Med-PaLM Multimodal (Med-PaLM M), our proof of concept for a generalist biomedical AI system. Med-PaLM M is a large multimodal generative model that flexibly encodes and interprets biomedical data including clinical language, imaging, and genomics with the same set of model weights. Med-PaLM M reaches performance competitive with or exceeding the state of the art on all MultiMedBench tasks, often surpassing specialist models by a wide margin. We also report examples of zero-shot generalization to novel medical concepts and tasks, positive transfer learning across tasks, and emergent zero-shot medical reasoning. To further probe the capabilities and limitations of Med-PaLM M, we conduct a radiologist evaluation of model-generated (and human) chest X-ray reports and observe encouraging performance across model scales. In a side-by-side ranking on 246 retrospective chest X-rays, clinicians express a pairwise preference for Med-PaLM M reports over those produced by radiologists in up to 40.50% of cases, suggesting potential clinical utility. While considerable work is needed to validate these models in real-world use cases, our results represent a milestone towards the development of generalist biomedical AI systems

The human body at cellular resolution: the NIH Human Biomolecular Atlas Program

Abstract: Transformative technologies are enabling the construction of three-dimensional maps of tissues with unprecedented spatial and molecular resolution. Over the next seven years, the NIH Common Fund Human Biomolecular Atlas Program (HuBMAP) intends to develop a widely accessible framework for comprehensively mapping the human body at single-cell resolution by supporting technology development, data acquisition, and detailed spatial mapping. HuBMAP will integrate its efforts with other funding agencies, programs, consortia, and the biomedical research community at large towards the shared vision of a comprehensive, accessible three-dimensional molecular and cellular atlas of the human body, in health and under various disease conditions

M31N 2008-12a - the remarkable recurrent nova in M31: Pan-chromatic observations of the 2015 eruption

The Andromeda Galaxy recurrent nova M31N 2008-12a had been observed in eruption ten times, including yearly eruptions from 2008-2014. With a measured recurrence period of $P_\mathrm{rec}=351\pm13$ days (we believe the true value to be half of this) and a white dwarf very close to the Chandrasekhar limit, M31N 2008-12a has become the leading pre-explosion supernova type Ia progenitor candidate. Following multi-wavelength follow-up observations of the 2013 and 2014 eruptions, we initiated a campaign to ensure early detection of the predicted 2015 eruption, which triggered ambitious ground and space-based follow-up programs. In this paper we present the 2015 detection; visible to near-infrared photometry and visible spectroscopy; and ultraviolet and X-ray observations from the Swift observatory. The LCOGT 2m (Hawaii) discovered the 2015 eruption, estimated to have commenced at Aug. $28.28\pm0.12$ UT. The 2013-2015 eruptions are remarkably similar at all wavelengths. New early spectroscopic observations reveal short-lived emission from material with velocities $\sim13000$ km s$^{-1}$, possibly collimated outflows. Photometric and spectroscopic observations of the eruption provide strong evidence supporting a red giant donor. An apparently stochastic variability during the early super-soft X-ray phase was comparable in amplitude and duration to past eruptions, but the 2013 and 2015 eruptions show evidence of a brief flux dip during this phase. The multi-eruption Swift/XRT spectra show tentative evidence of high-ionization emission lines above a high-temperature continuum. Following Henze et al. (2015a), the updated recurrence period based on all known eruptions is $P_\mathrm{rec}=174\pm10$ d, and we expect the next eruption of M31N 2008-12a to occur around mid-Sep. 2016

Human coronavirus OC43 3CL protease and the potential of ML188 as a broad-spectrum lead compound: Homology modelling and molecular dynamic studies

BACKGROUND: The coronavirus 3 chymotrypsin-like protease (3CL(pro)) is a validated target in the design of potential anticoronavirus inhibitors. The high degree of homology within the protease’s active site and substrate conservation supports the identification of broad spectrum lead compounds. A previous study identified the compound ML188, also termed 16R, as an inhibitor of the Severe Acute Respiratory Syndrome coronavirus (SARS-CoV) 3CL(pro). This study will detail the generation of a homology model of the 3CL(pro) of the human coronavirus OC43 and determine the potential of 16R to form a broad-spectrum lead compound. MODELLER was used to generate a suitable three-dimensional model of the OC43 3CL(pro) and the Prime module of Schrӧdinger predicted the binding conformation and free energy of binding of 16R within the 3CL(pro) active site. Molecular dynamics further confirmed ligand stability and hydrogen bonding networks. RESULTS: A high quality homology model of the OC43 3CL(pro) was successfully generated in an active conformation. Further studies reproduced the binding pose of 16R within the active site of the generated model, where its free energy of binding was shown to equal that of the 3CL(pro) of SARS-CoV, a receptor it is experimentally proven to inhibit. The stability of the ligand was subsequently confirmed by molecular dynamics. CONCLUSION: The lead compound 16R may represent a broad-spectrum inhibitor of the 3CL(pro) of OC43 and potentially other coronaviruses. This study provides an atomistic structure of the 3CL(pro) of OC43 and supports further experimental validation of the inhibitory effects of 16R. These findings further confirm that the 3CL(pro) of coronaviruses can be inhibited by broad spectrum lead compounds

Real Time Power Monitoring & integration with BIM

A Real Time Power Monitoring (RTPM) System is proposed in which end-use detailed energy consumption data is provided for each load level. The data will be integrated with a BIM (Building Information Modeling) Model to create a Real-time on-line electronic BIM Model. This paper describes the RTPM system and the process leading to its creation. The basic components of the proposed system were designed and tested using a prototype board. © 2010 IEEE