Rethinking Medical Report Generation: Disease Revealing Enhancement with Knowledge Graph

Knowledge Graph (KG) plays a crucial role in Medical Report Generation (MRG) because it reveals the relations among diseases and thus can be utilized to guide the generation process. However, constructing a comprehensive KG is labor-intensive and its applications on the MRG process are under-explored. In this study, we establish a complete KG on chest X-ray imaging that includes 137 types of diseases and abnormalities. Based on this KG, we find that the current MRG data sets exhibit a long-tailed problem in disease distribution. To mitigate this problem, we introduce a novel augmentation strategy that enhances the representation of disease types in the tail-end of the distribution. We further design a two-stage MRG approach, where a classifier is first trained to detect whether the input images exhibit any abnormalities. The classified images are then independently fed into two transformer-based generators, namely, ``disease-specific generator" and ``disease-free generator" to generate the corresponding reports. To enhance the clinical evaluation of whether the generated reports correctly describe the diseases appearing in the input image, we propose diverse sensitivity (DS), a new metric that checks whether generated diseases match ground truth and measures the diversity of all generated diseases. Results show that the proposed two-stage generation framework and augmentation strategies improve DS by a considerable margin, indicating a notable reduction in the long-tailed problem associated with under-represented diseases

Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision

Joint representation learning of words and entities benefits many NLP tasks, but has not been well explored in cross-lingual settings. In this paper, we propose a novel method for joint representation learning of cross-lingual words and entities. It captures mutually complementary knowledge, and enables cross-lingual inferences among knowledge bases and texts. Our method does not require parallel corpora, and automatically generates comparable data via distant supervision using multi-lingual knowledge bases. We utilize two types of regularizers to align cross-lingual words and entities, and design knowledge attention and cross-lingual attention to further reduce noises. We conducted a series of experiments on three tasks: word translation, entity relatedness, and cross-lingual entity linking. The results, both qualitatively and quantitatively, demonstrate the significance of our method.Comment: 11 pages, EMNLP201

The Faster the Better? Innovation Speed and User Interest in Open Source Software

It is often believed that for open source software (OSS) projects the faster the release, the better for attracting user interest in the software. Whether this is true, however, is still open to question. There is considerable information asymmetry between OSS projects and potential users as project quality is unobservable to users. We suggest that innovation speed of OSS project can signal the unobservable project quality and attract users’ interest in downloading and using the software. We contextualize innovation speed of OSS projects as initial release speed and update speed and examine their impacts on user interest. Drawing on the signaling theory, we propose a signaling effect through which a higher initial release speed or update speed increases user interest, while the effect diminishes as initial release or update speed increases. Using a large-scale panel data set from 7442 OSS projects on SourceForge between 2007 and 2010, our results corroborate the inverted U-shaped relationships between initial release speed and user downloads and between update speed and user downloads

Interpreting mechanism of Synergism of drug combinations using attention based hierarchical graph pooling

The synergistic drug combinations provide huge potentials to enhance therapeutic efficacy and to reduce adverse reactions. However, effective and synergistic drug combination prediction remains an open question because of the unknown causal disease signaling pathways. Though various deep learning (AI) models have been proposed to quantitatively predict the synergism of drug combinations. The major limitation of existing deep learning methods is that they are inherently not interpretable, which makes the conclusion of AI models un-transparent to human experts, henceforth limiting the robustness of the model conclusion and the implementation ability of these models in the real-world human-AI healthcare. In this paper, we develop an interpretable graph neural network (GNN) that reveals the underlying essential therapeutic targets and mechanism of the synergy (MoS) by mining the sub-molecular network of great importance. The key point of the interpretable GNN prediction model is a novel graph pooling layer, Self-Attention based Node and Edge pool (henceforth SANEpool), that can compute the attention score (importance) of nodes and edges based on the node features and the graph topology. As such, the proposed GNN model provides a systematic way to predict and interpret the drug combination synergism based on the detected crucial sub-molecular network. We evaluate SANEpool on molecular networks formulated by genes from 46 core cancer signaling pathways and drug combinations from NCI ALMANAC drug combination screening data. The experimental results indicate that 1) SANEpool can achieve the current state-of-art performance among other popular graph neural networks; and 2) the sub-molecular network detected by SANEpool are self-explainable and salient for identifying synergistic drug combinations

Weyl type f(Q,T)f(Q,T) gravity, and its cosmological implications

We consider an $f(Q,T)$ type gravity model in which the scalar non-metricity $Q_{\alpha \mu \nu}$ of the space-time is expressed in its standard Weyl form, and it is fully determined by a vector field $w_{\mu}$. The field equations of the theory are obtained under the assumption of the vanishing of the total scalar curvature, a condition which is added into the gravitational action via a Lagrange multiplier. The gravitational field equations are obtained from a variational principle, and they explicitly depend on the scalar nonmetricity and on the Lagrange multiplier. The covariant divergence of the matter energy-momentum tensor is also determined, and it follows that the nonmetricity-matter coupling leads to the nonconservation of the energy and momentum. The energy and momentum balance equations are explicitly calculated, and the expressions of the energy source term and of the extra force are found. We investigate the cosmological implications of the theory, and we obtain the cosmological evolution equations for a flat, homogeneous and isotropic geometry, which generalize the Friedmann equations of standard general relativity. We consider several cosmological models by imposing some simple functional forms of the function $f(Q,T)$, and we compare the predictions of the theory with the standard $\Lambda$CDM model.Comment: 22 pages, 14 figures, accepted for publication in Eur. Phys. Journal C. arXiv admin note: text overlap with arXiv:1908.0476

GNNHLS: Evaluating Graph Neural Network Inference via High-Level Synthesis

With the ever-growing popularity of Graph Neural Networks (GNNs), efficient GNN inference is gaining tremendous attention. Field-Programming Gate Arrays (FPGAs) are a promising execution platform due to their fine-grained parallelism, low-power consumption, reconfigurability, and concurrent execution. Even better, High-Level Synthesis (HLS) tools bridge the gap between the non-trivial FPGA development efforts and rapid emergence of new GNN models. In this paper, we propose GNNHLS, an open-source framework to comprehensively evaluate GNN inference acceleration on FPGAs via HLS, containing a software stack for data generation and baseline deployment, and FPGA implementations of 6 well-tuned GNN HLS kernels. We evaluate GNNHLS on 4 graph datasets with distinct topologies and scales. The results show that GNNHLS achieves up to 50.8x speedup and 423x energy reduction relative to the CPU baselines. Compared with the GPU baselines, GNNHLS achieves up to 5.16x speedup and 74.5x energy reduction

Segment Anything Model for Medical Image Analysis: an Experimental Study

Training segmentation models for medical images continues to be challenging due to the limited availability and acquisition expense of data annotations. Segment Anything Model (SAM) is a foundation model trained on over 1 billion annotations, predominantly for natural images, that is intended to be able to segment the user-defined object of interest in an interactive manner. Despite its impressive performance on natural images, it is unclear how the model is affected when shifting to medical image domains. Here, we perform an extensive evaluation of SAM's ability to segment medical images on a collection of 11 medical imaging datasets from various modalities and anatomies. In our experiments, we generated point prompts using a standard method that simulates interactive segmentation. Experimental results show that SAM's performance based on single prompts highly varies depending on the task and the dataset, i.e., from 0.1135 for a spine MRI dataset to 0.8650 for a hip x-ray dataset, evaluated by IoU. Performance appears to be high for tasks including well-circumscribed objects with unambiguous prompts and poorer in many other scenarios such as segmentation of tumors. When multiple prompts are provided, performance improves only slightly overall, but more so for datasets where the object is not contiguous. An additional comparison to RITM showed a much better performance of SAM for one prompt but a similar performance of the two methods for a larger number of prompts. We conclude that SAM shows impressive performance for some datasets given the zero-shot learning setup but poor to moderate performance for multiple other datasets. While SAM as a model and as a learning paradigm might be impactful in the medical imaging domain, extensive research is needed to identify the proper ways of adapting it in this domain.Comment: Link to our code: https://github.com/mazurowski-lab/segment-anything-medica