Device-Edge Cooperative Fine-Tuning of Foundation Models as a 6G Service

Foundation models (FoMos), referring to large-scale AI models, possess human-like capabilities and are able to perform competitively in the domain of human intelligence. The breakthrough in FoMos has inspired researchers to deploy such models in the sixth-generation (6G) mobile networks for automating a broad range of tasks in next-generation mobile applications. While the sizes of FoMos are reaching their peaks, their next phase is expected to focus on fine-tuning the models to specific downstream tasks. This inspires us to propose the vision of FoMo fine-tuning as a 6G service. Its key feature is the exploitation of existing parameter-efficient fine-tuning (PEFT) techniques to tweak only a small fraction of model weights for a FoMo to become customized for a specific task. To materialize the said vision, we survey the state-of-the-art PEFT and then present a novel device-edge fine-tuning (DEFT) framework for providing efficient and privacy-preserving fine-tuning services at the 6G network edge. The framework consists of the following comprehensive set of techniques: 1) Control of fine-tuning parameter sizes in different transformer blocks of a FoMo; 2) Over-the-air computation for realizing neural connections in DEFT; 3) Federated DEFT in a multi-device system by downloading a FoMo emulator or gradients; 4) On-the-fly prompt-ensemble tuning; 5) Device-to-device prompt transfer among devices. Experiments are conducted using pre-trained FoMos with up to 11 billion parameters to demonstrate the effectiveness of DEFT techniques. The article is concluded by presenting future research opportunities.Comment: 13 pages, 6 figure

Spatial and temporal variations of Rb/Sr ratios of the bulk surface sediments in Lake Qinghai

The Rb/Sr ratios of lake sediments have been suggested as indicators of weathering intensity by increasing work. However, the geochemistry of Rb/Sr ratios of lake sediments is variable between different lakes. In this study, we investigated the spatial and temporal patterns of Rb/Sr ratios, as well as those of other major elements in surface sediments of Lake Qinghai. We find that the spatial pattern of Rb/Sr ratios of the bulk sediments correlates well with that of the mass accumulation rate, and those of the terrigenous fractions, e.g., SiO2, Ti, and Fe. The temporal variations of Rb/Sr ratios also synchronize with those of SiO2, Ti, and Fe of each individual core. These suggest that Rb/Sr ratios of the surface sediments are closely related to terrigenous input from the catchment. Two out of eight cores show similar trends between Rb/Sr ratios and precipitation indices on decadal scales; however, the other cores do not show such relationship. The result of this study suggests that physical weathering and chemical weathering in Lake Qinghai catchment have opposite influence on Rb/Sr ratios of the bulk sediments, and they compete in dominating the Rb/Sr ratios of lake sediments on different spatial and temporal scales. Therefore, it is necessary to study the geochemistry of Rb/Sr ratio of lake sediments (especially that on short term timescales) particularly before it is used as an indicator of weathering intensity of the catchment

Three-dimensional numerical study of flow characteristic and membrane fouling evolution in an enzymatic membrane reactor

In order to enhance the understanding of membrane fouling mechanism, the hydrodynamics of granular flow in a stirred enzymatic membrane reactor was numerically investigated in the present study. A three-dimensional Euler-Euler model, coupled with k-e mixture turbulence model and drag function for interphase momentum exchange, was applied to simulate the two-phase (fluid-solid) turbulent flow. Numerical simulations of single- or two-phase turbulent flow under various stirring speed were implemented. The numerical results coincide very well with some published experimental data. Results for the distributions of velocity, shear stress and turbulent kinetic energy were provided. Our results show that the increase of stirring speed could not only enlarge the circulation loops in the reactor, but it can also increase the shear stress on the membrane surface and accelerate the mixing process of granular materials. The time evolution of volumetric function of granular materials on the membrane surface has qualitatively explained the evolution of membrane fouling.Comment: 10 panges, 8 figure