RS5M: A Large Scale Vision-Language Dataset for Remote Sensing Vision-Language Foundation Model

Pre-trained Vision-Language Foundation Models utilizing extensive image-text paired data have demonstrated unprecedented image-text association capabilities, achieving remarkable results across various downstream tasks. A critical challenge is how to make use of existing large-scale pre-trained VLMs, which are trained on common objects, to perform the domain-specific transfer for accomplishing domain-related downstream tasks. In this paper, we propose a new framework that includes the Domain Foundation Model (DFM), bridging the gap between the General Foundation Model (GFM) and domain-specific downstream tasks. Moreover, we present an image-text paired dataset in the field of remote sensing (RS), RS5M, which has 5 million RS images with English descriptions. The dataset is obtained from filtering publicly available image-text paired datasets and captioning label-only RS datasets with pre-trained VLM. These constitute the first large-scale RS image-text paired dataset. Additionally, we tried several Parameter-Efficient Fine-Tuning methods on RS5M to implement the DFM. Experimental results show that our proposed dataset are highly effective for various tasks, improving upon the baseline by $8 \% \sim 16 \%$ in zero-shot classification tasks, and obtaining good results in both Vision-Language Retrieval and Semantic Localization tasks. \url{https://github.com/om-ai-lab/RS5M}Comment: RS5M dataset v

GA based adaptive singularity-robust path planning of space robot for on-orbit detection

As a new on-orbit detection platform, the space robot could ensure stable and reliable operation of spacecraft in complex space environments. The tracking accuracy of the space manipulator end-effector is crucial to the detection precision. In this paper, the Cartesian path planning method of velocity level inverse kinematics based on generalized Jacobian matrix (GJM) is proposed. The GJM will come across singularity issue in path planning, which leads to the infinite or incalculable joint velocity. To solve this issue, firstly, the singular value decomposition (SVD) is used for exposition of the singularity avoidance principle of the damped least squares (DLS) method. After that, the DLS method is improved by introducing an adaptive damping factor which changes with the singularity. Finally, in order to improve the tracking accuracy of the singularity-robust algorithm, the objective function is established, and two adaptive parameters are optimized by genetic algorithm (GA). The simulation of a 6-DOF free-floating space robot is carried out, and the results show that, compared with DLS method, the proposed method could improve the tracking accuracy of space manipulator end-effector

Federated Large Language Model: A Position Paper

Large scale language models (LLM) have received significant attention and found diverse applications across various domains, but their development encounters challenges in real-world scenarios. These challenges arise due to the scarcity of public domain data availability and the need to maintain privacy with respect to private domain data. To address these issues, federated learning (FL) has emerged as a promising technology that enables collaborative training of shared models while preserving decentralized data. We propose the concept of federated LLM, which comprises three key components, i.e., federated LLM pre-training, federated LLM fine-tuning, and federated LLM prompt engineering. For each component, we discuss its advantage over traditional LLM training methods and propose specific engineering strategies for implementation. Furthermore, we explore the novel challenges introduced by the integration of FL and LLM. We analyze existing solutions and identify potential obstacles faced by these solutions within the context of federated LLM.Comment: 11 pages, 4 figure