Side4Video: Spatial-Temporal Side Network for Memory-Efficient Image-to-Video Transfer Learning

Large pre-trained vision models achieve impressive success in computer vision. However, fully fine-tuning large models for downstream tasks, particularly in video understanding, can be prohibitively computationally expensive. Recent studies turn their focus towards efficient image-to-video transfer learning. Nevertheless, existing efficient fine-tuning methods lack attention to training memory usage and exploration of transferring a larger model to the video domain. In this paper, we present a novel Spatial-Temporal Side Network for memory-efficient fine-tuning large image models to video understanding, named Side4Video. Specifically, we introduce a lightweight spatial-temporal side network attached to the frozen vision model, which avoids the backpropagation through the heavy pre-trained model and utilizes multi-level spatial features from the original image model. Extremely memory-efficient architecture enables our method to reduce 75% memory usage than previous adapter-based methods. In this way, we can transfer a huge ViT-E (4.4B) for video understanding tasks which is 14x larger than ViT-L (304M). Our approach achieves remarkable performance on various video datasets across unimodal and cross-modal tasks (i.e., action recognition and text-video retrieval), especially in Something-Something V1&V2 (67.3% & 74.6%), Kinetics-400 (88.6%), MSR-VTT (52.3%), MSVD (56.1%) and VATEX (68.8%). We release our code at https://github.com/HJYao00/Side4Video.Comment: Technical repor

SEA: A Scalable Entity Alignment System

Entity alignment (EA) aims to find equivalent entities in different knowledge graphs (KGs). State-of-the-art EA approaches generally use Graph Neural Networks (GNNs) to encode entities. However, most of them train the models and evaluate the results in a fullbatch fashion, which prohibits EA from being scalable on largescale datasets. To enhance the usability of GNN-based EA models in real-world applications, we present SEA, a scalable entity alignment system that enables to (i) train large-scale GNNs for EA, (ii) speed up the normalization and the evaluation process, and (iii) report clear results for users to estimate different models and parameter settings. SEA can be run on a computer with merely one graphic card. Moreover, SEA encompasses six state-of-the-art EA models and provides access for users to quickly establish and evaluate their own models. Thus, SEA allows users to perform EA without being involved in tedious implementations, such as negative sampling and GPU-accelerated evaluation. With SEA, users can gain a clear view of the model performance. In the demonstration, we show that SEA is user-friendly and is of high scalability even on computers with limited computational resources.Comment: SIGIR'23 Demo Trac

Multiscale microstructures and microstructural effects on the reliability of microbumps in three-dimensional integration

The dimensions of microbumps in three-dimensional integration reach microscopic scales and thus necessitate a study of the multiscale microstructures in microbumps. Here, we present simulated mesoscale and atomic-scale microstructures of microbumps using phase field and phase field crystal models. Coupled microstructure, mechanical stress, and electromigration modeling was performed to highlight the microstructural effects on the reliability of microbumps. The results suggest that the size and geometry of microbumps can influence both the mesoscale and atomic-scale microstructural formation during solidification. An external stress imposed on the microbump can cause ordered phase growth along the boundaries of the microbump. Mesoscale microstructures formed in the microbumps from solidification, solid state phase separation, and coarsening processes suggest that the microstructures in smaller microbumps are more heterogeneous. Due to the differences in microstructures, the von Mises stress distributions in microbumps of different sizes and geometries vary. In addition, a combined effect resulting from the connectivity of the phase morphology and the amount of interface present in the mesoscale microstructure can influence the electromigration reliability of microbumps

Multiscale microstructures and microstructural effects on the reliability of microbumps in three-dimensional integration

The dimensions of microbumps in three-dimensional integration reach microscopic scales and thus necessitate a study of the multiscale microstructures in microbumps. Here, we present simulated mesoscale and atomic-scale microstructures of microbumps using phase field and phase field crystal models. Coupled microstructure, mechanical stress, and electromigration modeling was performed to highlight the microstructural effects on the reliability of microbumps. The results suggest that the size and geometry of microbumps can influence both the mesoscale and atomic-scale microstructural formation during solidification. An external stress imposed on the microbump can cause ordered phase growth along the boundaries of the microbump. Mesoscale microstructures formed in the microbumps from solidification, solid state phase separation, and coarsening processes suggest that the microstructures in smaller microbumps are more heterogeneous. Due to the differences in microstructures, the von Mises stress distributions in microbumps of different sizes and geometries vary. In addition, a combined effect resulting from the connectivity of the phase morphology and the amount of interface present in the mesoscale microstructure can influence the electromigration reliability of microbumps

LiveRetro: Visual Analytics for Strategic Retrospect in Livestream E-Commerce

Livestream e-commerce integrates live streaming and online shopping, allowing viewers to make purchases while watching. However, effective marketing strategies remain a challenge due to limited empirical research and subjective biases from the absence of quantitative data. Current tools fail to capture the interdependence between live performances and feedback. This study identified computational features, formulated design requirements, and developed LiveRetro, an interactive visual analytics system. It enables comprehensive retrospective analysis of livestream e-commerce for streamers, viewers, and merchandise. LiveRetro employs enhanced visualization and time-series forecasting models to align performance features and feedback, identifying influences at channel, merchandise, feature, and segment levels. Through case studies and expert interviews, the system provides deep insights into the relationship between live performance and streaming statistics, enabling efficient strategic analysis from multiple perspectives.Comment: Accepted by IEEE VIS 202

Association of dietary inflammatory index with immune-inflammatory biomarkers in rheumatoid arthritis patients: results from NHANES 1999–2018

BackgroundSynovial inflammation is the main reason for joint damage in patients with rheumatoid arthritis (RA). Diet is recognized as one of the therapeutic strategies to control the inflammatory activity in RA. However, few studies have investigated the association between diet and immune-inflammatory biomarkers in RA patients. Our study aims to examine the correlation between dietary inflammatory potential and systemic immune-inflammation Index (SII), neutrophil-lymphocyte ratio (NLR), platelet-lymphocyte ratio (PLR), and lymphocyte-monocyte ratio (LMR) in the RA population.Materials and methodsThe National Health and Nutrition Examination Survey (NHANES) was the data source utilized in this study, spanning from 1999 to 2018. The study encompassed 2,500 RA participants in total. The dietary inflammatory potential was calculated by the dietary inflammation index (DII) score based on dietary recall interviews. The generalized multiple linear regression analyses were used to evaluate the relationship between DII and immune-inflammatory markers. Furthermore, subgroup analyses and restricted cubic spline models were performed.ResultsAfter full adjustments, there were significant positive correlations between DII levels and SII/NLR in RA patients (SII, β: 14.82, 95% CI: 5.14–24.50, p = 0.003; NLR, β: 0.04, 95% CI: 0.01–0.08, p = 0.005). It was noteworthy that inconsistent results were observed in the association between DII and SII as well as NLR in subgroups of red blood cell levels (Interaction p-value <0.001).ConclusionPro-inflammatory dietary status in the RA population is significantly positively correlated with SII and NLR, influenced by variations in red blood cell levels

Towards Understanding the Capability of Large Language Models on Code Clone Detection: A Survey

Code cloning, the duplication of code fragments, is common in software development. While some reuse aids productivity, excessive cloning hurts maintainability and introduces bugs. Hence, automatic code clone detection is vital. Meanwhile, large language models (LLMs) possess diverse code-related knowledge, making them versatile for various software engineering challenges. However, LLMs' performance in code clone detection is unclear and needs more study for accurate assessment. In this paper, we provide the first comprehensive evaluation of LLMs for clone detection, covering different clone types, languages, and prompts. We find advanced LLMs excel in detecting complex semantic clones, surpassing existing methods. Adding intermediate reasoning steps via chain-of-thought prompts noticeably enhances performance. Additionally, representing code as vector embeddings, especially with text encoders, effectively aids clone detection.Lastly, the ability of LLMs to detect code clones differs among various programming languages. Our study suggests that LLMs have potential for clone detection due to their language capabilities, offering insights for developing robust LLM-based methods to enhance software engineering.Comment: 13 pages, 3 figure

Impact of collimator leaf width and treatment technique on stereotactic radiosurgery and radiotherapy plans for intra- and extracranial lesions

<p>Abstract</p> <p>Background</p> <p>This study evaluated the dosimetric impact of various treatment techniques as well as collimator leaf width (2.5 vs 5 mm) for three groups of tumors – spine tumors, brain tumors abutting the brainstem, and liver tumors. These lesions often present challenges in maximizing dose to target volumes without exceeding critical organ tolerance. Specifically, this study evaluated the dosimetric benefits of various techniques and collimator leaf sizes as a function of lesion size and shape.</p> <p>Methods</p> <p>Fifteen cases (5 for each site) were studied retrospectively. All lesions either abutted or were an integral part of critical structures (brainstem, liver or spinal cord). For brain and liver lesions, treatment plans using a 3D-conformal static technique (3D), dynamic conformal arcs (DARC) or intensity modulation (IMRT) were designed with a conventional linear accelerator with standard 5 mm leaf width multi-leaf collimator, and a linear accelerator dedicated for radiosurgery and hypofractionated therapy with a 2.5 mm leaf width collimator. For the concave spine lesions, intensity modulation was required to provide adequate conformality; hence, only IMRT plans were evaluated using either the standard or small leaf-width collimators.</p> <p>A total of 70 treatment plans were generated and each plan was individually optimized according to the technique employed. The Generalized Estimating Equation (GEE) was used to separate the impact of treatment technique from the MLC system on plan outcome, and t-tests were performed to evaluate statistical differences in target coverage and organ sparing between plans.</p> <p>Results</p> <p>The lesions ranged in size from 2.6 to 12.5 cc, 17.5 to 153 cc, and 20.9 to 87.7 cc for the brain, liver, and spine groups, respectively. As a group, brain lesions were smaller than spine and liver lesions. While brain and liver lesions were primarily ellipsoidal, spine lesions were more complex in shape, as they were all concave. Therefore, the brain and the liver groups were compared for volume effect, and the liver and spine groups were compared for shape. For the brain and liver groups, both the radiosurgery MLC and the IMRT technique contributed to the dose sparing of organs-at-risk(OARs), as dose in the high-dose regions of these OARs was reduced up to 15%, compared to the non-IMRT techniques employing a 5 mm leaf-width collimator. Also, the dose reduction contributed by the fine leaf-width MLC decreased, as dose savings at all levels diminished from 4 – 11% for the brain group to 1 – 5% for the liver group, as the target structures decreased in volume. The fine leaf-width collimator significantly improved spinal cord sparing, with dose reductions of 14 – 19% in high to middle dose regions, compared to the 5 mm leaf width collimator.</p> <p>Conclusion</p> <p>The fine leaf-width MLC in combination with the IMRT technique can yield dosimetric benefits in radiosurgery and hypofractionated radiotherapy. Treatment of small lesions in cases involving complex target/OAR geometry will especially benefit from use of a fine leaf-width MLC and the use of IMRT.</p