CycleAlign: Iterative Distillation from Black-box LLM to White-box Models for Better Human Alignment

Language models trained on large-scale corpus often generate content that is harmful, toxic, or contrary to human preferences, making their alignment with human values a critical concern. Reinforcement learning from human feedback (RLHF) with algorithms like PPO is a prevalent approach for alignment but is often complex, unstable, and resource-intensive. Recently, ranking-based alignment methods have emerged, offering stability and effectiveness by replacing the RL framework with supervised fine-tuning, but they are costly due to the need for annotated data. Considering that existing large language models (LLMs) like ChatGPT are already relatively well-aligned and cost-friendly, researchers have begun to align the language model with human preference from AI feedback. The common practices, which unidirectionally distill the instruction-following responses from LLMs, are constrained by their bottleneck. Thus we introduce CycleAlign to distill alignment capabilities from parameter-invisible LLMs (black-box) to a parameter-visible model (white-box) in an iterative manner. With in-context learning (ICL) as the core of the cycle, the black-box models are able to rank the model-generated responses guided by human-craft instruction and demonstrations about their preferences. During iterative interaction, the white-box models also have a judgment about responses generated by them. Consequently, the agreement ranking could be viewed as a pseudo label to dynamically update the in-context demonstrations and improve the preference ranking ability of black-box models. Through multiple interactions, the CycleAlign framework could align the white-box model with the black-box model effectively in a low-resource way. Empirical results illustrate that the model fine-tuned by CycleAlign remarkably exceeds existing methods, and achieves the state-of-the-art performance in alignment with human value

PolyaryleneEther Nitrile and Barium TitanateNanocomposite Plasticized by CarboxylatedZinc Phthalocyanine Buffer

Barium titanate (BT) and polyarylene ether nitrile (PEN) nanocomposites with enhanced dielectric properties were obtained by using carboxylatedzinc phthalocyanine (ZnPc-COOH) buffer as the plasticizer. Carboxylated zinc phthalocyanine, prepared through hydrolyzing ZnPc in NaOH solution, reacted with the hydroxyl groups on the peripheral of hydrogen peroxide treated BT (BT-OH) yielding core-shell structured BT@ZnPc. Thermogravimetric analysis (TGA), transmission electron microscopy (TEM), TEM energy dispersive spectrometer mapping, scanning electron microscopy (SEM), X-ray diffraction (XRD), X-ray photoelectron spectroscopy (XPS), and Fourier transform infrared (FTIR) demonstrated successful preparation of BT@ZnPc. The fabricated BT@ZnPc was incorporated into the PEN matrix through the solution casting method. Rheological measurements demonstrated that the ZnPc-COOH buffer can improve the compatibility between BT and PEN effectively. With the existence of the ZnPc-COOH buffer, the prepared BT@ZnPc/PEN nanocomposites exhibit a high dielectric constant of 5.94 and low dielectric loss (0.016 at 1000 Hz). BT@ZnPc/PEN dielectric composite films can be easily prepared, presenting great application prospects in the field of organic film capacitors

Effects of Applying Different Organic Materials on Grain Yield and Soil Fertility in a Double-Season Rice Cropping System

Double-cropping rice cultivation reduces soil fertility, and the extensive use of chemical fertilizers has harmful effects on both the environment and grain yield. The application of organic materials could be used as a practical strategy to maintain soil fertility and improve grain yield in a double-season rice cropping system. For this purpose, field experiments with six growing seasons over three years, from 2016 to 2018, were conducted to assess the effects of five organic materials (biochar, Chinese milk vetch, rice straw, rapeseed cake fertilizer, and manure) on the grain yield and soil fertility, aiming to save about 25% of the chemical nitrogen (N) fertilizer required for all rice growing stages. The result showed that, compared with CK (the most common dose of fertilizer in this study region; 100% chemical fertilizer without organic fertilizer), the grain yield and soil fertility of double-cropped rice were increased after applying organic fertilizers for three consecutive years. Specifically, the CRC treatment (Chinese milk vetch (10.77 t ha−1 in fresh)/rice straw (26.51 t ha−1 in fresh) + 75% chemical fertilizer) showed significantly higher rates of effective panicles (4.65–10.92%) and annual grain yield (8.00–8.82%). The total N, total phosphorus (P), total potassium (K), alkaline N, and available P content in the CRC soil were significantly increased by 11.85%, 12.22%, 15.08%, 23.32%, and 41.04%, respectively, relative to CK. The decomposition of the applied Chinese milk vetch and rice straw combined with 75% chemical fertilizer resulted in more soil humus (9.50 g kg−1), humic acid (3.19 g kg−1), fulvic acid (3.26 g kg−1), and active organic carbon (5.78 g kg−1) and a significantly higher carbon pool management index (13.5%), as well as significantly higher soil urease activity (18.10%) and acid phosphatase activity (17.64%). Therefore, in this study, Chinese milk vetch (10.77 t ha−1 in fresh) in the early rice season/rice straw (26.51 t ha−1 fresh) in the late rice season + 75% chemical fertilizer treatment was the optimal dose for the double-season rice cropping system. It resulted in higher rice yields and has the potential to be used for more sustainable soil fertility