Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learning

Recently, multi-modal vision-language foundation models have gained significant attention in the medical field. While these models offer great opportunities, they still face a number of challenges, such as the requirement for fine-grained knowledge understanding in computer-aided diagnosis and capability of utilizing very limited or no task-specific labeled data in real-world clinical applications. In this study, we present MaCo, a novel multi-modal medical foundation model that explores masked contrastive learning to achieve granular alignment and zero-shot learning for a variety of medical imaging tasks. MaCo incorporates a correlation weighting mechanism to adjust the correlation between masked image patches and their corresponding reports, thereby enhancing the representation learning capabilities. We evaluate MaCo on six well-known open-source X-ray datasets, and the experimental results show it outperforms seven state-of-the-art approaches for classification, segmentation, and zero-shot phase grounding, demonstrating its great potential to promote a wide range of medical image analysis tasks

LiSum: Open Source Software License Summarization with Multi-Task Learning

Open source software (OSS) licenses regulate the conditions under which users can reuse, modify, and distribute the software legally. However, there exist various OSS licenses in the community, written in a formal language, which are typically long and complicated to understand. In this paper, we conducted a 661-participants online survey to investigate the perspectives and practices of developers towards OSS licenses. The user study revealed an indeed need for an automated tool to facilitate license understanding. Motivated by the user study and the fast growth of licenses in the community, we propose the first study towards automated license summarization. Specifically, we released the first high quality text summarization dataset and designed two tasks, i.e., license text summarization (LTS), aiming at generating a relatively short summary for an arbitrary license, and license term classification (LTC), focusing on the attitude inference towards a predefined set of key license terms (e.g., Distribute). Aiming at the two tasks, we present LiSum, a multi-task learning method to help developers overcome the obstacles of understanding OSS licenses. Comprehensive experiments demonstrated that the proposed jointly training objective boosted the performance on both tasks, surpassing state-of-the-art baselines with gains of at least 5 points w.r.t. F1 scores of four summarization metrics and achieving 95.13% micro average F1 score for classification simultaneously. We released all the datasets, the replication package, and the questionnaires for the community

Enhancing Representation in Medical Vision-Language Foundation Models via Multi-Scale Information Extraction Techniques

The development of medical vision-language foundation models has attracted significant attention in the field of medicine and healthcare due to their promising prospect in various clinical applications. While previous studies have commonly focused on feature learning at a single learning scale, investigation on integrating multi-scale information is lacking, which may hinder the potential for mutual reinforcement among these features. This paper aims to bridge this gap by proposing a method that effectively exploits multi-scale information to enhance the performance of medical foundation models. The proposed method simultaneously exploits features at the local, instance, modality and global aspects, facilitating comprehensive representation learning within the models. We evaluate the effectiveness of the proposed method on six open-source datasets across different clinical tasks, demonstrating its ability to enhance the performance of medical foundation models

Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining

Accurate medical image segmentation demands the integration of multi-scale information, spanning from local features to global dependencies. However, it is challenging for existing methods to model long-range global information, where convolutional neural networks (CNNs) are constrained by their local receptive fields, and vision transformers (ViTs) suffer from high quadratic complexity of their attention mechanism. Recently, Mamba-based models have gained great attention for their impressive ability in long sequence modeling. Several studies have demonstrated that these models can outperform popular vision models in various tasks, offering higher accuracy, lower memory consumption, and less computational burden. However, existing Mamba-based models are mostly trained from scratch and do not explore the power of pretraining, which has been proven to be quite effective for data-efficient medical image analysis. This paper introduces a novel Mamba-based model, Swin-UMamba, designed specifically for medical image segmentation tasks, leveraging the advantages of ImageNet-based pretraining. Our experimental results reveal the vital role of ImageNet-based training in enhancing the performance of Mamba-based models. Swin-UMamba demonstrates superior performance with a large margin compared to CNNs, ViTs, and latest Mamba-based models. Notably, on AbdomenMRI, Encoscopy, and Microscopy datasets, Swin-UMamba outperforms its closest counterpart U-Mamba_Enc by an average score of 2.72%.Comment: Code and models of Swin-UMamba are publicly available at: https://github.com/JiarunLiu/Swin-UMamb

OsteoporosAtlas: a human osteoporosis-related gene database

Background Osteoporosis is a common, complex disease of bone with a strong heritable component, characterized by low bone mineral density, microarchitectural deterioration of bone tissue and an increased risk of fracture. Due to limited drug selection for osteoporosis and increasing morbidity, mortality of osteoporotic fractures, osteoporosis has become a major health burden in aging societies. Current researches for identifying specific loci or genes involved in osteoporosis contribute to a greater understanding of the pathogenesis of osteoporosis and the development of better diagnosis, prevention and treatment strategies. However, little is known about how most causal genes work and interact to influence osteoporosis. Therefore, it is greatly significant to collect and analyze the studies involved in osteoporosis-related genes. Unfortunately, the information about all these osteoporosis-related genes is scattered in a large amount of extensive literature. Currently, there is no specialized database for easily accessing relevant information about osteoporosis-related genes and miRNAs. Methods We extracted data from literature abstracts in PubMed by text-mining and manual curation. Moreover, a local MySQL database containing all the data was developed with PHP on a Windows server. Results OsteoporosAtlas (http://biokb.ncpsb.org/osteoporosis/), the first specialized database for easily accessing relevant information such as osteoporosis-related genes and miRNAs, was constructed and served for researchers. OsteoporosAtlas enables users to retrieve, browse and download osteoporosis-related genes and miRNAs. Gene ontology and pathway analyses were integrated into OsteoporosAtlas. It currently includes 617 human encoding genes, 131 human non-coding miRNAs, and 128 functional roles. We think that OsteoporosAtlas will be an important bioinformatics resource to facilitate a better understanding of the pathogenesis of osteoporosis and developing better diagnosis, prevention and treatment strategies

The role of inflammatory biomarkers in the development and progression of pre-eclampsia: a systematic review and meta-analysis

BackgroundPre-eclampsia (PE) is a pregnancy complication associated with maternal and fetal morbidity and mortality. Among the potential pathogenesis discussed, inflammation is considered an essential initiator of PE. Previous studies have compared the levels of various inflammatory biomarkers that indicate the existence of PE; however, the relative levels of pro-inflammatory and anti-inflammatory biomarkers and their dynamic changes during PE progression remain unclear. This knowledge is essential to explain the occurrence and progression of the disease.ObjectiveWe aimed to identify the relationship between inflammatory status and PE using inflammatory biomarkers as indicators. We also discussed the underlying mechanism by which inflammatory imbalance contributes to PE by comparing the relative levels of pro-inflammatory and anti-inflammatory biomarkers. Furthermore, we identified additional risk factors for PE.MethodsWe reviewed PubMed, Embase, and the Cochrane Library for articles published until 15th September 2022. Original articles that investigated inflammatory biomarkers in PE and normal pregnancy were included. We selected healthy pregnant women as controls. The inflammatory biomarkers in the case and control groups were expressed as standardized mean differences and 95% confidence intervals using a random-effects model. Study quality was assessed using the Newcastle-Ottawa Scale. Publication bias was assessed using Egger’s test.ResultsThirteen articles that investigated 2,549 participants were included in this meta-analysis. Patients with PE had significantly higher levels of C-reactive protein (CRP), interleukin (IL)-4, IL-6, IL-8, IL-10, and tumor necrosis factor (TNF) than the controls. CRP and pro-inflammatory cytokine levels were higher than those of anti-inflammatory cytokines. Patients with gestational age > 34 weeks had significantly higher IL-6 and TNF levels. Patients with higher systolic blood pressure had significantly higher IL-8, IL-10, and CRP levels.ConclusionInflammatory imbalance is an independent risk factor for PE development. Impairment of the anti-inflammatory system is a crucial initiating factor for PE development. Failed autoregulation, manifested as prolonged exposure to pro-inflammatory cytokines, leads to PE progression. Higher levels of inflammatory biomarkers suggest more severe symptoms, and pregnant women after 34 weeks of gestation are more susceptible to PE

AllTogether: Investigating the Efficacy of Spliced Prompt for Web Navigation using Large Language Models

Large Language Models (LLMs) have emerged as promising agents for web navigation tasks, interpreting objectives and interacting with web pages. However, the efficiency of spliced prompts for such tasks remains underexplored. We introduces AllTogether, a standardized prompt template that enhances task context representation, thereby improving LLMs' performance in HTML-based web navigation. We evaluate the efficacy of this approach through prompt learning and instruction finetuning based on open-source Llama-2 and API-accessible GPT models. Our results reveal that models like GPT-4 outperform smaller models in web navigation tasks. Additionally, we find that the length of HTML snippet and history trajectory significantly influence performance, and prior step-by-step instructions prove less effective than real-time environmental feedback. Overall, we believe our work provides valuable insights for future research in LLM-driven web agents.Comment: Include wrong information in comment. Should be 7 pages and not published ye