Exploring Memorization in Fine-tuned Language Models

LLMs have shown great capabilities in various tasks but also exhibited memorization of training data, thus raising tremendous privacy and copyright concerns. While prior work has studied memorization during pre-training, the exploration of memorization during fine-tuning is rather limited. Compared with pre-training, fine-tuning typically involves sensitive data and diverse objectives, thus may bring unique memorization behaviors and distinct privacy risks. In this work, we conduct the first comprehensive analysis to explore LMs' memorization during fine-tuning across tasks. Our studies with open-sourced and our own fine-tuned LMs across various tasks indicate that fine-tuned memorization presents a strong disparity among tasks. We provide an understanding of this task disparity via sparse coding theory and unveil a strong correlation between memorization and attention score distribution. By investigating its memorization behavior, multi-task fine-tuning paves a potential strategy to mitigate fine-tuned memorization

Association between daily screen time and risk of stroke among middle-aged and elderly people: research based on China health and nutrition survey

BackgroundWe aimed to explore the independent associations between screen time and the risk of stroke among Chinese adults based on the China Health and Nutrition Survey (CHNS).MethodsData on Chinese adults aged older than 40 years from the CHNS in during 2004–2009 were selected. A total of 4,587 individuals were included in 2009, including screen time and the risk of stroke. Simultaneously, we traced the previous screen time to 2004 for those with outcome measures in 2009 (n = 2,100). Basic information, lifestyle, and screen behavior were obtained through face-to-face interviews and self-completed questionnaires. Anthropometric data collected included blood pressure, body weight, height, hip circumference, and waist circumference. Fasting blood was obtained for measurements of lipid and glucose levels. Cross-sectional analysis and cohort analysis were both performed using multivariate logistic regression.ResultsOf all participants, 3,004 (65.49%) participants spent more than 2 h per day on screen time. Taking the men who spent less than 2 h on screen per day as reference, the crude odds ratio (OR) of the high risk of stroke was 1.53 [95% confidence interval (CI), 1.20–1.95] for the men who spent 2–3 h per day on screen and 2.37 (95% CI, 1.78–3.16) for the men who spent more than 3 h per day on screen. This difference remained significant after adjusting for confounding factors. No association was observed among women. However, in the cohort analysis with screen time in 2006 as the independent variable, the association between screen time and stroke risk was found both in men [OR, 1.83 (95% CI, 1.19–2.82)] and women [OR, 1.48 (95% CI, 1.10–1.99)]).ConclusionWe found that the high screen time was associated with an increased stroke risk, which was pronounced in men, warranting a universal need to limit screen time in order to improve health

Non-Invasive Chromosome Screening for Embryo Preimplantation Using Cell-Free DNA

Preimplantation genetic testing (PGT) is widely adopted to select embryos with normal ploidy but requires invasive embryo biopsy procedures. Therefore, non-invasive PGT (niPGT) detection of cell-free DNA (cfDNA) in blastocyst culture medium has gradually become a hot area in the field of assisted reproduction. This chapter will systematically summarize how researchers use embryonic cfDNA to conduct niPGT detection worldwide. It will also thoroughly review the factors that affect the accuracy of the test and its underlying issues, as well as prospective applications. We hope to provide a useful reference for the standardized operation of non-invasive PGT that can be widely applied in clinical practice

Efficient and ultra-stable perovskite light-emitting diodes

Perovskite light-emitting diodes (PeLEDs) have emerged as a strong contender for next-generation display and information technologies. However, similar to perovskite solar cells, the poor operational stability remains the main obstacle toward commercial applications. Here we demonstrate ultra-stable and efficient PeLEDs with extraordinary operational lifetimes (T50) of 1.0x10^4 h, 2.8x10^4 h, 5.4x10^5 h, and 1.9x10^6 h at initial radiance (or current densities) of 3.7 W/sr/m2 (~5 mA/cm2), 2.1 W/sr/m2 (~3.2 mA/cm2), 0.42 W/sr/m2 (~1.1 mA/cm2), and 0.21 W/sr/m2 (~0.7 mA/cm2) respectively, and external quantum efficiencies of up to 22.8%. Key to this breakthrough is the introduction of a dipolar molecular stabilizer, which serves two critical roles simultaneously. First, it prevents the detrimental transformation and decomposition of the alpha-phase FAPbI3 perovskite, by inhibiting the formation of lead and iodide intermediates. Secondly, hysteresis-free device operation and microscopic luminescence imaging experiments reveal substantially suppressed ion migration in the emissive perovskite. The record-long PeLED lifespans are encouraging, as they now satisfy the stability requirement for commercial organic LEDs (OLEDs). These results remove the critical concern that halide perovskite devices may be intrinsically unstable, paving the path toward industrial applications.Comment: This is a preprint of the paper prior to peer review. New and updated results may be available in the final version from the publishe

CRISPR-Cas13a-Based Detection for Bovine Viral Diarrhea Virus

Bovine Viral Diarrhea Virus (BVDV) is the main pathogen of bovine viral diarrhea disease (BVD), which leads to enormous economic losses in the cattle industry. A sensitive and specific detection for BVDV is advantageous to the control of BVDV. Clustered regularly interspaced short palindromic repeats (CRISPR)-Cas systems have been used for detecting virus RNA. In this study, the expression and purification of LwCas13a protein was optimized and the RNase activity of LwCas13a in vitro was verified. CRISPR-LwCas13a system could detect BVDV virus and BVDV RNA with high specificity and simplicity. The detection limit of the LwCas13a system was 103 pM, and there were no cross-reactions with HEK293T and MDBK. In summary, a sensitive, specific, and simple nucleic acid detection method based on CRISPR-Cas13a was developed for BVDV. This method provides a new detection strategy for early diagnosis of BVDV

Farmland boundary extraction based on the AttMobile-DeeplabV3+ network and least squares fitting of straight lines

The rapid extraction of farmland boundaries is key to implementing autonomous operation of agricultural machinery. This study addresses the issue of incomplete farmland boundary segmentation in existing methods, proposing a method for obtaining farmland boundaries based on unmanned aerial vehicle (UAV) remote sensing images. The method is divided into two steps: boundary image acquisition and boundary line fitting. To acquire the boundary image, an improved semantic segmentation network, AttMobile-DeeplabV3+, is designed. Subsequently, a boundary tracing function is used to track the boundaries of the binary image. Lastly, the least squares method is used to obtain the fitted boundary line. The paper validates the method through experiments on both crop-covered and non-crop-covered farmland. Experimental results show that on crop-covered and non-crop-covered farmland, the network’s intersection over union (IoU) is 93.25% and 93.14%, respectively; the pixel accuracy (PA) for crop-covered farmland is 96.62%. The average vertical error and average angular error of the extracted boundary line are 0.039 and 1.473°, respectively. This research provides substantial and accurate data support, offering technical assistance for the positioning and path planning of autonomous agricultural machinery

Mechanisms and modelling of phosphorus solid-liquid transformation during the hydrothermal processing of swine manure

Phosphorus (P) recovery from swine manure by hydrothermal processes has recently attracted considerable interest; however, research has been limited by knowledge gaps and challenges in understanding the mechanisms of soluble and insoluble P transformations and the evaluation of the effects of the reaction conditions. In this study, the transformation mechanisms were investigated and the soluble and insoluble phosphorus distributions in swine manure during the hydrothermal processes were modelled. By increasing the severity of the exogenous conditions, P transformed from insoluble to soluble, and then polymerized with the formation of orthophosphates; meanwhile, the formation of hydrochar was enhanced thereby facilitating further P reclamation. The effects of the endogenous conditions showed there may be a threshold of calcium content, which limited the combination of Ca and P. Calcium ions mainly reacted with P in the form of hydroxyapatite and octacalcium phosphate. The modelling and prediction results showed that a coalification model gives a good fit (RSP2= 0.9205 andRIP2= 0.8559) for changes in the concentrations of solid total P and liquid inorganic P. The prediction level of mean absolute error was good as well (MAESP= 0.74 mg g−1and MAEIP= 0.62 mg g−1). These findings provide a range of scientific opportunities for achieving a comprehensive understanding of the basis of sustainable utilisation of P

A Farm Management Information System for Semi-Supervised Path Planning and Autonomous Vehicle Control

This paper presents a farm management information system targeting improvements in the ease of use and sustainability of robot farming systems. The system integrates the functionalities of field survey, path planning, monitoring, and controlling agricultural vehicles in real time. Firstly, a Grabcut-based semi-supervised field registration method is proposed for arable field detection from the orthoimage taken by the drone with an RGB camera. It partitions a complex field into simple geometric entities with simple user interaction. The average Mean Intersection over Union is about 0.95 when the field size ranges from 2.74 ha to 5.06 ha. In addition, a desktop software and a web application are developed as the entity of an FMIS. Compared to existing FMISs, this system provides more advanced features in robot farming, while providing simpler user interaction and better results. It allows clients to invoke web services and receive responses independent of programming language and platforms. Moreover, the system is compatible with other services, users, and devices following the open-source access protocol. We have evaluated the system by controlling 5 robot tractors with a 2 Hz communication frequency. The communication protocols will be publicly available to protentional users

A Novel Speech/Noise Discrimination Method for Embedded ASR System

<p/> <p>The problem of speech/noise discrimination has become increasingly important as the automatic speech recognition (ASR) system is applied in the real world. Robustness and simplicity are two challenges to the speech/noise discrimination method for an embedded system. The energy-based feature is the most suitable and applicable feature for speech/noise discrimination for embedded ASR system because of effectiveness and simplicity. A new method based on a noise model is proposed to discriminate speech signals from noise signals. The noise model is initialized and then updated according to the signal energy. The experiment shows the effectiveness and robustness of the new method in noisy environments.</p

Rheological properties of concrete with manufactured sand : a multi-level prediction

Conventional rheological predictions of concrete were mostly developed based on spherical aggregate assumptions whereas the particle shape of irregular aggregate significantly influences the rheological properties of concrete. In this study, rheological properties of concrete with manufactured sand (MS) are predicted concerning the particle shape of aggregate based on a multi-level biphase assumption. The relative plastic viscosity and relative yield stress at each level are demonstrated to present power-law relationships with the relative thickness of the corresponding suspending media. The proposed models are proved with high accuracy and robustness for predicting the rheological properties of mixtures with MS and coarse aggregate that have various particle shapes and particle size distributions. Based on the proposed predictions, the influences of particle shapes of MS and coarse aggregate on the rheological properties of mixtures can be represented by their effects on the relative paste film thickness (R_PFT) and relative mortar film thickness (R_MFT), respectively. The proposed multi-level prediction lays the foundation of mix proportioning of concrete with irregular aggregate according to the specified rheological requirements