BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity

Understanding the functional organization of higher visual cortex is a central focus in neuroscience. Past studies have primarily mapped the visual and semantic selectivity of neural populations using hand-selected stimuli, which may potentially bias results towards pre-existing hypotheses of visual cortex functionality. Moving beyond conventional approaches, we introduce a data-driven method that generates natural language descriptions for images predicted to maximally activate individual voxels of interest. Our method -- Semantic Captioning Using Brain Alignments ("BrainSCUBA") -- builds upon the rich embedding space learned by a contrastive vision-language model and utilizes a pre-trained large language model to generate interpretable captions. We validate our method through fine-grained voxel-level captioning across higher-order visual regions. We further perform text-conditioned image synthesis with the captions, and show that our images are semantically coherent and yield high predicted activations. Finally, to demonstrate how our method enables scientific discovery, we perform exploratory investigations on the distribution of "person" representations in the brain, and discover fine-grained semantic selectivity in body-selective areas. Unlike earlier studies that decode text, our method derives voxel-wise captions of semantic selectivity. Our results show that BrainSCUBA is a promising means for understanding functional preferences in the brain, and provides motivation for further hypothesis-driven investigation of visual cortex

Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Models

A long standing goal in neuroscience has been to elucidate the functional organization of the brain. Within higher visual cortex, functional accounts have remained relatively coarse, focusing on regions of interest (ROIs) and taking the form of selectivity for broad categories such as faces, places, bodies, food, or words. Because the identification of such ROIs has typically relied on manually assembled stimulus sets consisting of isolated objects in non-ecological contexts, exploring functional organization without robust a priori hypotheses has been challenging. To overcome these limitations, we introduce a data-driven approach in which we synthesize images predicted to activate a given brain region using paired natural images and fMRI recordings, bypassing the need for category-specific stimuli. Our approach -- Brain Diffusion for Visual Exploration ("BrainDiVE") -- builds on recent generative methods by combining large-scale diffusion models with brain-guided image synthesis. Validating our method, we demonstrate the ability to synthesize preferred images with appropriate semantic specificity for well-characterized category-selective ROIs. We then show that BrainDiVE can characterize differences between ROIs selective for the same high-level category. Finally we identify novel functional subdivisions within these ROIs, validated with behavioral data. These results advance our understanding of the fine-grained functional organization of human visual cortex, and provide well-specified constraints for further examination of cortical organization using hypothesis-driven methods.Comment: NeurIPS 2023 (Oral). Project page: https://www.cs.cmu.edu/~afluo/BrainDiVE

StableSemantics: A Synthetic Language-Vision Dataset of Semantic Representations in Naturalistic Images

Understanding the semantics of visual scenes is a fundamental challenge in Computer Vision. A key aspect of this challenge is that objects sharing similar semantic meanings or functions can exhibit striking visual differences, making accurate identification and categorization difficult. Recent advancements in text-to-image frameworks have led to models that implicitly capture natural scene statistics. These frameworks account for the visual variability of objects, as well as complex object co-occurrences and sources of noise such as diverse lighting conditions. By leveraging large-scale datasets and cross-attention conditioning, these models generate detailed and contextually rich scene representations. This capability opens new avenues for improving object recognition and scene understanding in varied and challenging environments. Our work presents StableSemantics, a dataset comprising 224 thousand human-curated prompts, processed natural language captions, over 2 million synthetic images, and 10 million attention maps corresponding to individual noun chunks. We explicitly leverage human-generated prompts that correspond to visually interesting stable diffusion generations, provide 10 generations per phrase, and extract cross-attention maps for each image. We explore the semantic distribution of generated images, examine the distribution of objects within images, and benchmark captioning and open vocabulary segmentation methods on our data. To the best of our knowledge, we are the first to release a diffusion dataset with semantic attributions. We expect our proposed dataset to catalyze advances in visual semantic understanding and provide a foundation for developing more sophisticated and effective visual models. Website: https://stablesemantics.github.io/StableSemanticsComment: Dataset website: https://stablesemantics.github.io/StableSemantic

Tuberculosis in Pediatric Antiretroviral Therapy Programs in Low- and Middle-Income Countries: Diagnosis and Screening Practices

Background The global burden of childhood tuberculosis (TB) is estimated to be 0.5 million new cases per year. Human immunodeficiency virus (HIV)-infected children are at high risk for TB. Diagnosis of TB in HIV-infected children remains a major challenge. Methods We describe TB diagnosis and screening practices of pediatric antiretroviral treatment (ART) programs in Africa, Asia, the Caribbean, and Central and South America. We used web-based questionnaires to collect data on ART programs and patients seen from March to July 2012. Forty-three ART programs treating children in 23 countries participated in the study. Results Sputum microscopy and chest Radiograph were available at all programs, mycobacterial culture in 40 (93%) sites, gastric aspiration in 27 (63%), induced sputum in 23 (54%), and Xpert MTB/RIF in 16 (37%) sites. Screening practices to exclude active TB before starting ART included contact history in 41 sites (84%), symptom screening in 38 (88%), and chest Radiograph in 34 sites (79%). The use of diagnostic tools was examined among 146 children diagnosed with TB during the study period. Chest Radiograph was used in 125 (86%) children, sputum microscopy in 76 (52%), induced sputum microscopy in 38 (26%), gastric aspirate microscopy in 35 (24%), culture in 25 (17%), and Xpert MTB/RIF in 11 (8%) children. Conclusions Induced sputum and Xpert MTB/RIF were infrequently available to diagnose childhood TB, and screening was largely based on symptom identification. There is an urgent need to improve the capacity of ART programs in low- and middle-income countries to exclude and diagnose TB in HIV-infected childre

Vascular Complications in the Era of Transcatheter Treatment of Adult Structural Heart Disease: A Single-Center Early Experience

Transcatheter treatment is becoming the mainstay treatment for structural heart diseases (SHD) in prohibitive surgical risk patients. Recently with the encouraging results, it is being offered to regular risk patients. Peripheral vascular complications (VCs) are still inherent to these procedures due to the nature of this atherosclerotic high-risk group and the profile of the devices. This is a single-center early first year experience with such events occurring after initiating a SHD program treating severe aortic stenosis, aortic regurgitation, mitral valve prolapse and regurgitation, as well as paravalvular leaks. Out of 33 patients in this time period, 5 developed PV complications which are detailed in this article with their associated risk factors and management. These include access-related complications, closure device issues, arterial rupture post device embolization, and vessel dissection. Vascular complications of those procedures take special interest since they are associated with a worse long-term prognosis. Thus, prevention with proper planning remains of essence along with multidisciplinary management. © The Author(s) 2020

Spontaneous splenic rupture in an active duty Marine upon return from Iraq: a case report

<p>Abstract</p> <p>Introduction</p> <p>Atraumatic splenic rupture is a rare event that has been associated with several infectious disease processes. In the active duty military population, potential exposure to these pathogens is significant. Here we discuss the case of an active duty Marine with spontaneous splenic rupture upon return from a six-month deployment in Iraq.</p> <p>Case presentation</p> <p>A previously healthy 30-year-old Caucasian male active duty Marine presented with abdominal pain, fever and diarrhea after deployment to Iraq in support of Operation Iraqi Freedom. Based on clinical and radiographic evidence, a diagnosis of spontaneous splenic rupture was ultimately suspected. After exploratory laparotomy with confirmation of rupture, splenectomy was performed, and the patient made a full, uneventful recovery. Histopathologic examination revealed mild splenomegaly with a ruptured capsule of undetermined cause.</p> <p>Conclusion</p> <p>Spontaneous splenic rupture is a rare event that may lead to life-threatening hemorrhage if not diagnosed and treated quickly. Although the cause of this patient's case was unknown, atraumatic splenic rupture has been associated with a variety of infectious diseases and demonstrates some risks the active duty military population may face while on deployment. Having an awareness of these pathogens and their role in splenic rupture, clinicians caring for military personnel must be prepared to recognize and treat this potentially fatal complication.</p

Major lower extremity amputations in a developing country: 10-Year experience at a tertiary medical center

Background: Lower extremity amputation (LEA) is a major surgical procedure with a high risk of significant morbidity and mortality. The objective of this study was to describe mortality and functionality outcomes following this procedure in a developing country. Methods: This is a retrospective study of all patients undergoing LEA for non-traumatic etiology between 2007 and 2017. Medical records were used to retrieve demographics, comorbidities, and perioperative complications of identified patients. Patients were contacted to follow-up on their medical, postoperative care, and ambulatory status. Mortality and postoperative functionality rates were analyzed. Results: The study included 78 patients. Median follow-up duration was 24 months. Hypertension (81%) and diabetes (79%) were the most common comorbidities. Mortality rates at 30 days, 1, and 5 years were 10.3, 29.2, and 65.5%, respectively. Mortality was significantly associated with age > 70 at amputation (p = 0.042), hypertension (p = 0.003), chronic kidney disease (p = 0.031), and perioperative sepsis (p = 0.01). Only 1.6% of patients were discharged into a specialized care center, and only 27% of patients were ambulatory postoperatively, although 90.5% were fitted with a prosthesis. Conclusions: Survival following major amputation in a developing country is currently comparable to more developed regions of the world. Major discrepancy seems to exist in ambulatory status following the procedure. Discharge placement policies should be properly set, and rehabilitation centers funding should be increased. Awareness may also be warranted to educate patients and families about the value and positive impact of rehabilitation centers. © The Author(s) 2020

Interplay between ESR1/PIK3CA codon variants, oncogenic pathway alterations and clinical phenotype in patients with metastatic breast cancer (MBC): comprehensive circulating tumor DNA (ctDNA) analysis

Background: although being central for the biology and druggability of hormone-receptor positive, HER2 negative metastatic breast cancer (MBC), ESR1 and PIK3CA mutations are simplistically dichotomized as mutated or wild type in current clinical practice. Methods: The study analyzed a multi-institutional cohort comprising 703 patients with luminal-like MBC characterized for circulating tumor DNA through next generation sequencing (NGS). Pathway classification was defined based on previous work (i.e., RTK, RAS, RAF, MEK, NRF2, ER, WNT, MYC, P53, cell cycle, notch, PI3K). Single nucleotide variations (SNVs) were annotated for their oncogenicity through OncoKB. Only pathogenic variants were included in the models. Associations among clinical characteristics, pathway classification, and ESR1/PIK3CA codon variants were explored. Results: The results showed a differential pattern of associations for ESR1 and PIK3CA codon variants in terms of co-occurring pathway alterations patterns of metastatic dissemination, and prognosis. ESR1 537 was associated with SNVs in the ER and RAF pathways, CNVs in the MYC pathway and bone metastases, while ESR1 538 with SNVs in the cell cycle pathway and liver metastases. PIK3CA 1047 and 542 were associated with CNVs in the PI3K pathway and with bone metastases. Conclusions: The study demonstrated how ESR1 and PIK3CA codon variants, together with alterations in specific oncogenic pathways, can differentially impact the biology and clinical phenotype of luminal-like MBC. As novel endocrine therapy agents such as selective estrogen receptor degraders (SERDS) and PI3K inhibitors are being developed, these results highlight the pivotal role of ctDNA NGS to describe tumor evolution and optimize clinical decision making

Discrimination between two different grades of human glioma based on blood vessel infrared spectral imaging

Gliomas are brain tumours classified into four grades with increasing malignancy from I to IV. The development and the progression of malignant glioma largely depend on the tumour vascularization. Due to their tissue heterogeneity, glioma cases can be difficult to classify into a specific grade using the gold standard of histological observation, hence the need to base classification on a quantitative and reliable analytical method for accurately grading the disease. Previous works focused specifically on vascularization study by Fourier transform infrared (FTIR) spectroscopy, proving this method to be a way forward to detect biochemical changes in the tumour tissue not detectable by visual techniques. In this project, we employed FTIR imaging using a focal plane array (FPA) detector and globar source to analyse large areas of glioma tumour tissue sections via molecular fingerprinting in view of helping to define markers of the tumour grade. Unsupervised multivariate analysis (hierarchical cluster analysis and principal component analysis) of blood vessel spectral data, retrieved from the FPA images, revealed the fine structure of the borderline between two areas identified by a pathologist as grades III and IV. Spectroscopic indicators are found capable of discriminating different areas in the tumour tissue and are proposed as biomolecular markers for potential future use of grading gliomas. Graphical Abstract Infrared imaging of glioma blood vessels provides a means to revise the pathologists' line of demarcation separating grade III (GIII) from grade IV (GIV) parts