Medical Image Understanding with Pretrained Vision Language Models: A Comprehensive Study

The large-scale pre-trained vision language models (VLM) have shown remarkable domain transfer capability on natural images. However, it remains unknown whether this capability can also apply to the medical image domain. This paper thoroughly studies the knowledge transferability of pre-trained VLMs to the medical domain, where we show that well-designed medical prompts are the key to elicit knowledge from pre-trained VLMs. We demonstrate that by prompting with expressive attributes that are shared between domains, the VLM can carry the knowledge across domains and improve its generalization. This mechanism empowers VLMs to recognize novel objects with fewer or without image samples. Furthermore, to avoid the laborious manual designing process, we develop three approaches for automatic generation of medical prompts, which can inject expert-level medical knowledge and image-specific information into the prompts for fine-grained grounding. We conduct extensive experiments on thirteen different medical datasets across various modalities, showing that our well-designed prompts greatly improve the zero-shot performance compared to the default prompts, and our fine-tuned models surpass the supervised models by a significant margin.Comment: 14 pages, 4 figures

FlowX: Towards Explainable Graph Neural Networks via Message Flows

We investigate the explainability of graph neural networks (GNNs) as a step toward elucidating their working mechanisms. While most current methods focus on explaining graph nodes, edges, or features, we argue that, as the inherent functional mechanism of GNNs, message flows are more natural for performing explainability. To this end, we propose a novel method here, known as FlowX, to explain GNNs by identifying important message flows. To quantify the importance of flows, we propose to follow the philosophy of Shapley values from cooperative game theory. To tackle the complexity of computing all coalitions' marginal contributions, we propose a flow sampling scheme to compute Shapley value approximations as initial assessments of further training. We then propose an information-controlled learning algorithm to train flow scores toward diverse explanation targets: necessary or sufficient explanations. Experimental studies on both synthetic and real-world datasets demonstrate that our proposed FlowX and its variants lead to improved explainability of GNNs. The code is available at https://github.com/divelab/DIG

ConES: Concept Embedding Search for Parameter Efficient Tuning Large Vision Language Models

Large pre-trained vision-language models have shown great prominence in transferring pre-acquired knowledge to various domains and downstream tasks with appropriate prompting or tuning. Existing prevalent tuning methods can be generally categorized into three genres: 1) prompt engineering by creating suitable prompt texts, which is time-consuming and requires domain expertise; 2) or simply fine-tuning the whole model, which is extremely inefficient; 3) prompt tuning through parameterized prompt embeddings with the text encoder. Nevertheless, all methods rely on the text encoder for bridging the modality gap between vision and language. In this work, we question the necessity of the cumbersome text encoder for a more lightweight and efficient tuning paradigm as well as more representative prompt embeddings closer to the image representations. To achieve this, we propose a Concept Embedding Search (ConES) approach by optimizing prompt embeddings -- without the need of the text encoder -- to capture the 'concept' of the image modality through a variety of task objectives. By dropping the text encoder, we are able to significantly speed up the learning process, \eg, from about an hour to just ten minutes in our experiments for personalized text-to-image generation without impairing the generation quality. Moreover, our proposed approach is orthogonal to current existing tuning methods since the searched concept embeddings can be further utilized in the next stage of fine-tuning the pre-trained large models for boosting performance. Extensive experiments show that our approach can beat the prompt tuning and textual inversion methods in a variety of downstream tasks including objection detection, instance segmentation, and image generation. Our approach also shows better generalization capability for unseen concepts in specialized domains, such as the medical domain

A Chinese Herbal Formula to Improve General Psychological Status in Posttraumatic Stress Disorder: A Randomized Placebo-Controlled Trial on Sichuan Earthquake Survivors

Introduction. Posttraumatic stress disorder (PTSD) is accompanied by poor general psychological status (GPS). In the present study, we investigated the effects of a Chinese herbal formula on GPS in earthquake survivors with PTSD. Methods. A randomized, double-blind, placebo-controlled trial compared a Chinese herbal formula, Xiao-Tan-Jie-Yu-Fang (XTJYF), to placebo in 2008 Sichuan earthquake survivors with PTSD. Patients were randomized into XTJYF (n = 123) and placebo (n = 122) groups. Baseline-to-end-point score changes in the three global indices of the Symptom Checklist-90-Revised (SCL-90-R) and rates of response in the SCL global severity index (GSI) were the primary endpoints. A subanalysis of the nine SCL factors and the sleep quality score were secondary endpoints. Results and Discussion. Compared to placebo, the XTJYF group was significantly improved in all three SCL global indices (P = 0.001~0.028). More patients in the XTJYF group reported “much improved” than the placebo group (P = 0.001). The XTJYF group performed significantly better than control in five out of nine SCL factors (somatization, obsessive-compulsive behavior, depression, anxiety, and hostility (P = 0.001~0.036)), and in sleep quality score (P < 0.001). XTJYF produced no serious adverse events. These findings suggest that XTJYF may be an effective and safe treatment option for improving GPS in patients with PTSD

Identification of PSEN1 and APP Gene Mutations in Korean Patients with Early-Onset Alzheimer's Disease

Although mutations in three genes, amyloid precursor protein (APP), presenilin 1 (PSEN1), and presenilin 2 (PSEN2), have been identified as genetic causes of early-onset Alzheimer's disease (EOAD), there has been a single report on a PSEN1 mutation in Koreans. In the present study, we performed a genetic analysis of six Korean patients with EOAD. Direct sequencing analysis of the APP, PSEN1 and PSEN2 genes revealed two different mutations of the PSEN1 gene (G206S and M233T) and one mutation of the APP gene (V715M) in three patients with age-at-onset of 34, 35, and 42 yr, respectively. In addition, two patients with age-at-onset of 55 and 62 yr, respectively, were homozygous for APOE ε4 allele. One woman had no genetic alterations. These findings suggest that PSEN1 and APP gene mutations may not be uncommon in Korean patients with EOAD and that genetic analysis should be provided to EOAD patients not only for the identification of their genetic causes but also for the appropriate genetic counseling

Insights into the metabolic profiling of Polygonati Rhizoma fermented by Lactiplantibacillus plantarum under aerobic and anaerobic conditions using a UHPLC-QE-MS/MS system

IntroductionPolygonati Rhizoma is a multi-purpose food with medicinal uses. Fermentation of Polygonati Rhizoma by lactic acid bacteria could provide new insights into the development of Polygonati Rhizoma products.MethodsIn this study, Lactiplantibacillus plantarum was fermented with Polygonati Rhizoma extracts in a bioreactor under aerobic and anaerobic conditions with pH and DO real-time detection. Metabolic profiling was determined by UHPLC-QE-MS/MS system. Principal component analysis and orthogonal partial least-squares discriminant analysis were used to perform multivariate analysis.ResultsA total of 98 differential metabolites were identified in broth after fermentation, and 36 were identified between fermentation under aerobic and anaerobic conditions. The main metabolic pathways in the fermentation process are ABC transport and amino acid biosynthesis. Most of the compounds such as L-arginine, L-aspartic acid, leucine, L-lysine, citrate, inosine, carnitine, betaine, and thiamine were significantly increased during fermentation, playing a role in enhancing food flavor. Compared with anaerobic fermentation, aerobic conditions led to a significant rise in the levels of some compounds such as valine, isoleucine, and glutamate; this increase was mainly related to branched-chain amino acid transaminase, isocitrate dehydrogenase, and glutamate dehydrogenase.DiscussionAerobic fermentation is more beneficial for the fermentation of Polygonati Rhizoma by L. plantarum to produce flavor and functional substances. This study is the first report on the fermentation of Polygonati Rhizoma by L. plantarum and provides insights that would be applicable in the development of Polygonati Rhizoma fermented products

Cerebrospinal Fluid Nitric Oxide Synthase is a Potential Mediator Between Cigarette Smoke Exposure and Sleep Disorders

Jiaying Lao,1,2,&ast; Hang Tan,3,&ast; Yuyu Wu,2,&ast; Ting Ding,4 Xinqian Liu,1 Lanrong Sun,2 Xiyi Chen,2 Chongrong Zhu,2 Yiming Kang,5 Yu-Hsin Chen,2,6 Chonghui Tang,1 Fan Wang,7 Yanlong Liu2,6 1Department of Neurosurgery, Affiliated Cixi Hospital, Wenzhou Medical University, Ningbo, People’s Republic of China; 2School of Mental Health, Wenzhou Medical University, Wenzhou, People’s Republic of China; 3Department of neurosurgery, Hangzhou Mingzhou Brain Rehabilitation Hospital, Hangzhou, People’s Republic of China; 4Department of Infectious Diseases, The Affiliated Xiangshan Hospital of Wenzhou Medical University, Ningbo, People’s Republic of China; 5Psychosomatic Medicine Research Division, Inner Mongolia Medical University, Huhhot, People’s Republic of China; 6Zhejiang Provincial Clinical Research Center for Mental Disorders, The Affiliated Wenzhou Kangning Hospital, Wenzhou Medical University, Wenzhou, People’s Republic of China; 7Beijing Hui−Long−Guan Hospital, Peking University, Beijing, People’s Republic of China&ast;These authors contributed equally to this workCorrespondence: Yanlong Liu, School of Mental Health, Medical University, Wenzhou, 325035, People’s Republic of China, Email [email protected] Chonghui Tang, Department of Neurosurgery, Affiliated Cixi Hospital, Wenzhou Medical University, Ningbo, 315300, People’s Republic of China, Email [email protected]: Cigarette smoking and low peripheral nitric oxide synthase (NOS) levels are strongly associated with sleep disorders. However, whether cerebrospinal fluid (CSF) NOS relates to sleep disorders and whether CSF NOS mediates the relationship between cigarette smoking and sleep disorders is unclear.Methods: We measured CSF levels of total NOS (tNOS) and its isoforms (inducible NOS [iNOS] and constitutive NOS [cNOS]) in 191 Chinese male subjects. We applied the Pittsburgh Sleep Quality Index (PSQI).Results: The PSQI scores of active smokers were significantly higher than those of non-smokers, while CSF tNOS, iNOS, and cNOS were significantly lower (all p < 0.001). CSF tNOS, iNOS, and cNOS were negatively associated with PSQI scores in the general population (all p < 0.001). Mediation analysis suggested that CSF tNOS, iNOS, and cNOS mediate the relationship between smoking and PSQI scores, and the indirect effect accounted for 78.93%, 66.29%, and 81.65% of the total effect, respectively.Conclusion: Cigarette smoking is associated with sleep disorders. Active smokers had significantly lower CSF levels of tNOS, iNOS, and cNOS. Furthermore, tNOS, iNOS, and cNOS mediate the relationship between cigarette smoking and sleep quality. This study provides insights into how cigarette smoke affects sleep disorders.Keywords: cerebrospinal fluid nitric oxide synthase, cigarette smoking, Pittsburgh Sleep Quality Index, sleep disorders, mediatio

Diabetic foot ulcers segmentation challenge report: benchmark and analysis

Monitoring the healing progress of diabetic foot ulcers is a challenging process. Accurate segmentation of foot ulcers can help podiatrists to quantitatively measure the size of wound regions to assist prediction of healing status. The main challenge in this field is the lack of publicly available manual delineation, which can be time consuming and laborious. Recently, methods based on deep learning have shown excellent results in automatic segmentation of medical images, however, they require large-scale datasets for training, and there is limited consensus on which methods perform the best. The 2022 Diabetic Foot Ulcers segmentation challenge was held in conjunction with the 2022 International Conference on Medical Image Computing and Computer Assisted Intervention, which sought to address these issues and stimulate progress in this research domain. A training set of 2000 images exhibiting diabetic foot ulcers was released with corresponding segmentation ground truth masks. Of the 72 (approved) requests from 47 countries, 26 teams used this data to develop fully automated systems to predict the true segmentation masks on a test set of 2000 images, with the corresponding ground truth segmentation masks kept private. Predictions from participating teams were scored and ranked according to their average Dice similarity coefficient of the ground truth masks and prediction masks. The winning team achieved a Dice of 0.7287 for diabetic foot ulcer segmentation. This challenge has now entered a live leaderboard stage where it serves as a challenging benchmark for diabetic foot ulcer segmentation

Tunable hot-carrier photodetection beyond the bandgap spectral limit

The spectral response of common optoelectronic photodetectors is restricted by a cutoff wavelength limit λ that is related to the activation energy (or bandgap) of the semiconductor structure (or material) (Δ) through the relationship λ = hc/Δ. This spectral rule dominates device design and intrinsically limits the long-wavelength response of a semiconductor photodetector. Here, we report a new, long-wavelength photodetection principle based on a hot-cold hole energy transfer mechanism that overcomes this spectral limit. Hot carriers injected into a semiconductor structure interact with cold carriers and excite them to higher energy states. This enables a very long-wavelength infrared response. In our experiments, we observe a response up to 55 μm, which is tunable by varying the degree of hot-hole injection, for a GaAs/AlGaAs sample with Δ = 0.32 eV (equivalent to 3.9 μm in wavelength)