HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaverse

Federated Learning (FL), as a rapidly evolving privacy-preserving collaborative machine learning paradigm, is a promising approach to enable edge intelligence in the emerging Industrial Metaverse. Even though many successful use cases have proved the feasibility of FL in theory, in the industrial practice of Metaverse, the problems of non-independent and identically distributed (non-i.i.d.) data, learning forgetting caused by streaming industrial data, and scarce communication bandwidth remain key barriers to realize practical FL. Facing the above three challenges simultaneously, this paper presents a high-performance and efficient system named HFEDMS for incorporating practical FL into Industrial Metaverse. HFEDMS reduces data heterogeneity through dynamic grouping and training mode conversion (Dynamic Sequential-to-Parallel Training, STP). Then, it compensates for the forgotten knowledge by fusing compressed historical data semantics and calibrates classifier parameters (Semantic Compression and Compensation, SCC). Finally, the network parameters of the feature extractor and classifier are synchronized in different frequencies (Layer-wiseAlternative Synchronization Protocol, LASP) to reduce communication costs. These techniques make FL more adaptable to the heterogeneous streaming data continuously generated by industrial equipment, and are also more efficient in communication than traditional methods (e.g., Federated Averaging). Extensive experiments have been conducted on the streamed non-i.i.d. FEMNIST dataset using 368 simulated devices. Numerical results show that HFEDMS improves the classification accuracy by at least 6.4% compared with 8 benchmarks and saves both the overall runtime and transfer bytes by up to 98%, proving its superiority in precision and efficiency.Comment: This paper is submitted to IEEE Transaction on Cloud Computin

Exploring Memorization in Fine-tuned Language Models

LLMs have shown great capabilities in various tasks but also exhibited memorization of training data, thus raising tremendous privacy and copyright concerns. While prior work has studied memorization during pre-training, the exploration of memorization during fine-tuning is rather limited. Compared with pre-training, fine-tuning typically involves sensitive data and diverse objectives, thus may bring unique memorization behaviors and distinct privacy risks. In this work, we conduct the first comprehensive analysis to explore LMs' memorization during fine-tuning across tasks. Our studies with open-sourced and our own fine-tuned LMs across various tasks indicate that fine-tuned memorization presents a strong disparity among tasks. We provide an understanding of this task disparity via sparse coding theory and unveil a strong correlation between memorization and attention score distribution. By investigating its memorization behavior, multi-task fine-tuning paves a potential strategy to mitigate fine-tuned memorization

PBX3 and MEIS1 Cooperate in Hematopoietic Cells to Drive Acute Myeloid Leukemias Characterized by a Core Transcriptome of the MLL-Rearranged Disease

Overexpression of HOXA/MEIS1/PBX3 homeobox genes is the hallmark of mixed lineage leukemia (MLL)-rearranged acute myeloid leukemia (AML). HOXA9 and MEIS1 are considered to be the most critical targets of MLL fusions and their co-expression rapidly induces AML. MEIS1 and PBX3 are not individually able to transform cells and were therefore hypothesized to function as cofactors of HOXA9. However, in this study we demonstrate that co-expression of PBX3 and MEIS1 (PBX3/MEIS1), without ectopic expression of a HOX gene, is sufficient for transformation of normal mouse hematopoietic stem/progenitor cells in vitro. Moreover, PBX3/MEIS1 overexpression also caused AML in vivo, with a leukemic latency similar to that caused by forced expression of MLL-AF9, the most common form of MLL fusions. Furthermore, gene expression profiling of hematopoietic cells demonstrated that PBX3/MEIS1 overexpression, but not HOXA9/MEIS1, HOXA9/PBX3 or HOXA9 overexpression, recapitulated the MLL-fusion-mediated core transcriptome, particularly upregulation of the endogenous Hoxa genes. Disruption of the binding between MEIS1 and PBX3 diminished PBX3/MEIS1-mediated cell transformation and HOX gene upregulation. Collectively, our studies strongly implicate the PBX3/MEIS1 interaction as a driver of cell transformation and leukemogenesis, and suggest that this axis may play a critical role in the regulation of the core transcriptional programs activated in MLL-rearranged and HOX-overexpressing AML. Therefore, targeting the MEIS1/PBX3 interaction may represent a promising therapeutic strategy to treat these AML subtypes

miR-22 has a potent anti-tumour role with therapeutic potential in acute myeloid leukaemia

MicroRNAs are subject to precise regulation and have key roles in tumorigenesis. In contrast to the oncogenic role of miR-22 reported in myelodysplastic syndrome (MDS) and breast cancer, here we show that miR-22 is an essential anti-tumour gatekeeper in de novo acute myeloid leukaemia (AML) where it is significantly downregulated. Forced expression of miR-22 significantly suppresses leukaemic cell viability and growth in vitro, and substantially inhibits leukaemia development and maintenance in vivo. Mechanistically, miR-22 targets multiple oncogenes, including CRTC1, FLT3 and MYCBP, and thus represses the CREB and MYC pathways. The downregulation of miR-22 in AML is caused by TET1/GFI1/EZH2/SIN3A-mediated epigenetic repression and/or DNA copy-number loss. Furthermore, nanoparticles carrying miR-22 oligos significantly inhibit leukaemia progression in vivo. Together, our study uncovers a TET1/GFI1/EZH2/SIN3A/miR-22/CREB-MYC signalling circuit and thereby provides insights into epigenetic/genetic mechanisms underlying the pathogenesis of AML, and also highlights the clinical potential of miR-22-based AML therapy

Data from: Effect of carboxylic group on the compatibility with retarder and the retarding side effect of the fluid loss control additive used in oil well cement

The retarding side effect and the compatibility with other additives are the main problem to limit the field application of the fluid loss control additive (FLCA) synthesized. The effect of the type and content of carboxylic acid group on the retarding side effect of FLCA and the compatibility between FLCA and the retarder AMPS-IA synthesized by 2-acrylamido-2-methyl propane sulfonic acid (AMPS) and itaconic acid (IA) was studied in this paper. The type and content of carboxylic acid group have a great influence on the fluid loss control ability, the compatibility with retarder and the retarding side effect of FLCA. FLCA containing IA or maleic acid (MA) shows better compatibility with retarder than FLCA containing acrylic acid (AA), but the retarding side effect of FLCA containing MA is weaker than that of FLCA containing IA, so MA is the most suitable monomer to synthesizing FLCA having good compatibility with retarder AMPS-IA

One-part alkali activated slag using Ca(OH)2 and Na2CO3 instead of NaOH as activator: more excellent compressive strength and microstructure

In this paper, sodium carbonate and calcium hydroxide instead of sodium hydroxide used as composite activators, and slag are applied to prepare one-part alkali-activated slag. Furthermore, the properties of the slag activated by sodium hydroxide are compared. The compressive strength, XRD, TGA, MIP, and SEM analysis of the two systems are performed. The results show that the one-part alkali-activated slag prepared from sodium carbonate and calcium hydroxide has a superior compressive strength against the slag activated by sodium hydroxide. It is also found that the hydration speed of the sodium hydroxide activated slag is faster, and thus the higher compressive strength of the one-part alkali-activated slag is not caused by the hydration speed. The hydration of one-part alkali-activated slag produces a certain amount of calcium carbonate, resulting in a lower porosity in comparison with the slag activated by sodium hydroxide, which may be the reason for the better compressive strength of one-part alkali-activated slag

data about intrinsic viscosity, API fluid loss and setting time

data about intrinsic viscosity, API fluid loss and setting tim

data.rar

data about API fluid and setting time;data about intrinsic viscosit

Influence of Pore Structure and Solid Bitumen on the Development of Deep Carbonate Gas Reservoirs: A Case Study of the Longwangmiao Reservoir in Gaoshiti–Longnusi Area, Sichuan Basin, SW China

A variable sedimentary environment and accumulation process leads to a complex pore structure in deep carbonate gas reservoirs, and the physical properties are quite different between layers. Moreover, some pores and throats are filled with solid bitumen (SB), which not only interferes with reservoir analysis, but also affects efficient development. However, previous studies on SB mainly focused on the accumulation process and reservoir analysis, and there are few reports about the influence on development. In this paper, through scanning electron microscope analysis, SB extraction, gas flow experiments and depletion experiments, and a similar transformation between experimental results and reservoir production, the production characteristics of carbonate gas reservoirs with different pore structures were studied, and the influence of SB on pore structure, reservoir analysis and development were systematically analyzed. The results show that permeability is one of the key factors affecting gas production rate and recovery, and the production is mainly contributed by high-permeability layers. Although the reserves are abundant, the gas production rate and recovery of layers with a low permeability are relatively low. The SB reduces the pore and throat radius, resulting in porosity and permeability being decreased by 4.73–6.28% and 36.02–3.70%, respectively. With the increase in original permeability, the permeability loss rate decreases. During development, the loss rate of gas production rate is much higher than that of permeability. Increasing the production pressure difference is conducive to reducing the influence. SB also reduces the recovery, which leads to the loss rate of gas production being much higher than that of porosity. For reservoirs with a high permeability, the loss rates of gas production rate and the amount produced are close to those of permeability and porosity. Therefore, in the reservoir analysis and development of carbonate gas reservoirs bearing SB, it is necessary and significant to analyze the influence of reservoir types