Core-Periphery Principle Guided Redesign of Self-Attention in Transformers

Designing more efficient, reliable, and explainable neural network architectures is critical to studies that are based on artificial intelligence (AI) techniques. Previous studies, by post-hoc analysis, have found that the best-performing ANNs surprisingly resemble biological neural networks (BNN), which indicates that ANNs and BNNs may share some common principles to achieve optimal performance in either machine learning or cognitive/behavior tasks. Inspired by this phenomenon, we proactively instill organizational principles of BNNs to guide the redesign of ANNs. We leverage the Core-Periphery (CP) organization, which is widely found in human brain networks, to guide the information communication mechanism in the self-attention of vision transformer (ViT) and name this novel framework as CP-ViT. In CP-ViT, the attention operation between nodes is defined by a sparse graph with a Core-Periphery structure (CP graph), where the core nodes are redesigned and reorganized to play an integrative role and serve as a center for other periphery nodes to exchange information. We evaluated the proposed CP-ViT on multiple public datasets, including medical image datasets (INbreast) and natural image datasets. Interestingly, by incorporating the BNN-derived principle (CP structure) into the redesign of ViT, our CP-ViT outperforms other state-of-the-art ANNs. In general, our work advances the state of the art in three aspects: 1) This work provides novel insights for brain-inspired AI: we can utilize the principles found in BNNs to guide and improve our ANN architecture design; 2) We show that there exist sweet spots of CP graphs that lead to CP-ViTs with significantly improved performance; and 3) The core nodes in CP-ViT correspond to task-related meaningful and important image patches, which can significantly enhance the interpretability of the trained deep model.Comment: Core-periphery, functional brain networks, Vi

Clinical characteristics of respiratory tract infection caused by Klebsiella pneumoniae in immunocompromised patients: a retrospective cohort study

PurposeWith advancements in medical technology and the growth of an aging society, the number of immunocompromised patients has increased progressively. Klebsiella pneumoniae (K. pneumoniae) is one of the most common opportunistic pathogens, causing a severe disease burden. We aimed to further clarify the differences in respiratory tract K. pneumoniae infections between immunocompromised and immunocompetent populations.MethodsWe retrospectively compared cases of respiratory tract K. pneumoniae infection in immunocompromised and immunocompetent patients admitted to Ruijin Hospital in Shanghai between January 2019 and August 2020 to clarify the differences between the two groups.ResultsWe enrolled 400 immunocompromised patients and 386 immunocompetent patients. Compared to the immunocompetent group, immunocompromised patients were more likely to develop bacteremia and shock and to require mechanical ventilation support during hospitalization. Immunocompromised patients also had a greater probability of polymicrobial infection and a higher rate of antibacterial resistance to carbapenem, which resulted in a higher intensive care unit admission rate, 30-day case fatality rate (CFR), and 6-month CFR. Multivariate analysis indicated that immunocompromised patients with respiratory diseases (odds ratio [OR], 2.189; 95% confidence interval [CI], 1.103-4.344; P = 0.025) and cardiovascular diseases (OR, 2.008; 95% CI, 1.055-3.822; P = 0.034), using mechanical ventilation (OR, 3.982; 95% CI, 2.053-7.722; P = 0.000), or infected with multidrug-resistant K. pneumoniae (OR, 3.870; 95%, 1.577-9.498; P = 0.003) were more likely to have a higher 30-day CFR.ConclusionThe disease burden of K. pneumoniae infection in immunocompromised patients is high. Immunocompromised patients who presented with respiratory diseases and cardiovascular diseases, used mechanical ventilation, or were infected with multidrug-resistant K. pneumoniae experienced a higher 30-day mortality rate

Hierarchical Semantic Tree Concept Whitening for Interpretable Image Classification

With the popularity of deep neural networks (DNNs), model interpretability is becoming a critical concern. Many approaches have been developed to tackle the problem through post-hoc analysis, such as explaining how predictions are made or understanding the meaning of neurons in middle layers. Nevertheless, these methods can only discover the patterns or rules that naturally exist in models. In this work, rather than relying on post-hoc schemes, we proactively instill knowledge to alter the representation of human-understandable concepts in hidden layers. Specifically, we use a hierarchical tree of semantic concepts to store the knowledge, which is leveraged to regularize the representations of image data instances while training deep models. The axes of the latent space are aligned with the semantic concepts, where the hierarchical relations between concepts are also preserved. Experiments on real-world image datasets show that our method improves model interpretability, showing better disentanglement of semantic concepts, without negatively affecting model classification performance

Patterns of Lymph Node Metastasis and Optimal Surgical Strategy in Small (≤20 mm) Gastroenteropancreatic Neuroendocrine Tumors

BackgroundRegional lymph node metastasis (LNM) is crucial for planning additional lymphadenectomy, and is directly correlated with poor prognosis in gastroenteropancreatic neuroendocrine tumors (GEP-NETs). However, the patterns of LNM for small (≤20 mm) GEP-NETs remain unclear. This population-based study aimed at evaluating LNM patterns and identifying optimal surgical strategies from the standpoint of lymph node dissemination.MethodsThis retrospective cohort study retrieved data from the Surveillance, Epidemiology, and End Results (SEER) 18 registries database for 17,308 patients diagnosed as having localized well-differentiated GEP-NETs ≤ 20 mm between January 1, 2004, and December 31, 2017. The patterns of LNM were characterized in 6,622 patients who underwent extended resection for adequate lymph node harvest.ResultsOf 6,622 patients with localized small GEP-NETs in the current study, 2,380 (36%) presented with LNM after regional lymphadenectomy. Nodal involvement was observed in approximately 7.4%, 49.1%, 13.6%, 53.7%, 13.8%, 7.8%, and 15.4% of gastric (g-), small intestinal (si-), appendiceal (a-), colonic (c-), rectal (r-), non-functional pancreatic (nfp-), and functional pancreatic (fp-) NETs ≤ 20 mm. Patients with younger age, larger tumor size, and muscularis invasion were more likely to present with LNM. Additional lymphadenectomy conferred a significant survival advantage in NETs (≤10 mm: HR, 0.47; 95% CI, 0.33–0.66; p < 0.001; 11–20 mm: HR, 0.54; 95% CI, 0.34–0.85; p = 0.008) and fp-NETs ≤ 20 mm (HR, 0.08; 95% CI, 0.02–0.36; p = 0.001), as well as g-NETs (HR, 0.39; 95% CI, 0.16–0.96; p = 0.041) and c-NETs of 11–20 mm (HR, 0.07; 95% CI, 0.01–0.48; p = 0.007). Survival benefits of additional lymphadenectomy were not found in a-NETs, r-NETs, and nfp-NETs with a small size.ConclusionsGiven the increased risk for nodal metastasis, primary tumor resection with regional lymphadenectomy is a potential optimal surgical strategy for si-NETs and fp-NETs ≤ 20 mm, as well as g-NETs and c-NETs of 11–20 mm. Local resection is an appropriate and reliable surgical approach for a-NETs, r-NETs, and nfp-NETs ≤ 20 mm

Segment Anything Model (SAM) for Radiation Oncology

In this study, we evaluate the performance of the Segment Anything Model (SAM) model in clinical radiotherapy. We collected real clinical cases from four regions at the Mayo Clinic: prostate, lung, gastrointestinal, and head \& neck, which are typical treatment sites in radiation oncology. For each case, we selected the OARs of concern in radiotherapy planning and compared the Dice and Jaccard outcomes between clinical manual delineation, automatic segmentation using SAM's "segment anything" mode, and automatic segmentation using SAM with box prompt. Our results indicate that SAM performs better in automatic segmentation for the prostate and lung regions, while its performance in the gastrointestinal and head \& neck regions was relatively inferior. When considering the size of the organ and the clarity of its boundary, SAM displays better performance for larger organs with clear boundaries, such as the lung and liver, and worse for smaller organs with unclear boundaries, like the parotid and cochlea. These findings align with the generally accepted variations in difficulty level associated with manual delineation of different organs at different sites in clinical radiotherapy. Given that SAM, a single trained model, could handle the delineation of OARs in four regions, these results also demonstrate SAM's robust generalization capabilities in automatic segmentation for radiotherapy, i.e., achieving delineation of different radiotherapy OARs using a generic automatic segmentation model. SAM's generalization capabilities across different regions make it technically feasible to develop a generic model for automatic segmentation in radiotherapy

SAMAug: Point Prompt Augmentation for Segment Anything Model

This paper introduces SAMAug, a novel visual point augmentation method for the Segment Anything Model (SAM) that enhances interactive image segmentation performance. SAMAug generates augmented point prompts to provide more information about the user's intention to SAM. Starting with an initial point prompt, SAM produces an initial mask, which is then fed into our proposed SAMAug to generate augmented point prompts. By incorporating these extra points, SAM can generate augmented segmentation masks based on both the augmented point prompts and the initial prompt, resulting in improved segmentation performance. We conducted evaluations using four different point augmentation strategies: random sampling, sampling based on maximum difference entropy, maximum distance, and saliency. Experiment results on the COCO, Fundus, COVID QUEx, and ISIC2018 datasets show that SAMAug can boost SAM's segmentation results, especially using the maximum distance and saliency. SAMAug demonstrates the potential of visual prompt augmentation for computer vision. Codes of SAMAug are available at github.com/yhydhx/SAMAu

When Brain-inspired AI Meets AGI

Artificial General Intelligence (AGI) has been a long-standing goal of humanity, with the aim of creating machines capable of performing any intellectual task that humans can do. To achieve this, AGI researchers draw inspiration from the human brain and seek to replicate its principles in intelligent machines. Brain-inspired artificial intelligence is a field that has emerged from this endeavor, combining insights from neuroscience, psychology, and computer science to develop more efficient and powerful AI systems. In this article, we provide a comprehensive overview of brain-inspired AI from the perspective of AGI. We begin with the current progress in brain-inspired AI and its extensive connection with AGI. We then cover the important characteristics for both human intelligence and AGI (e.g., scaling, multimodality, and reasoning). We discuss important technologies toward achieving AGI in current AI systems, such as in-context learning and prompt tuning. We also investigate the evolution of AGI systems from both algorithmic and infrastructural perspectives. Finally, we explore the limitations and future of AGI

AGI for Agriculture

Artificial General Intelligence (AGI) is poised to revolutionize a variety of sectors, including healthcare, finance, transportation, and education. Within healthcare, AGI is being utilized to analyze clinical medical notes, recognize patterns in patient data, and aid in patient management. Agriculture is another critical sector that impacts the lives of individuals worldwide. It serves as a foundation for providing food, fiber, and fuel, yet faces several challenges, such as climate change, soil degradation, water scarcity, and food security. AGI has the potential to tackle these issues by enhancing crop yields, reducing waste, and promoting sustainable farming practices. It can also help farmers make informed decisions by leveraging real-time data, leading to more efficient and effective farm management. This paper delves into the potential future applications of AGI in agriculture, such as agriculture image processing, natural language processing (NLP), robotics, knowledge graphs, and infrastructure, and their impact on precision livestock and precision crops. By leveraging the power of AGI, these emerging technologies can provide farmers with actionable insights, allowing for optimized decision-making and increased productivity. The transformative potential of AGI in agriculture is vast, and this paper aims to highlight its potential to revolutionize the industry