COPF: Continual Learning Human Preference through Optimal Policy Fitting

The technique of Reinforcement Learning from Human Feedback (RLHF) is a commonly employed method to improve pre-trained Language Models (LM), enhancing their ability to conform to human preferences. Nevertheless, the current RLHF-based LMs necessitate full retraining each time novel queries or feedback are introduced, which becomes a challenging task because human preferences can vary between different domains or tasks. Retraining LMs poses practical difficulties in many real-world situations due to the significant time and computational resources required, along with concerns related to data privacy. To address this limitation, we propose a new method called Continual Optimal Policy Fitting (COPF), in which we estimate a series of optimal policies using the Monte Carlo method, and then continually fit the policy sequence with the function regularization. COPF involves a single learning phase and doesn't necessitate complex reinforcement learning. Importantly, it shares the capability with RLHF to learn from unlabeled data, making it flexible for continual preference learning. Our experimental results show that COPF outperforms strong Continuous learning (CL) baselines when it comes to consistently aligning with human preferences on different tasks and domains

Reliability model and algorithm research on HVDC system and flexible HVDC system

The main difference between flexible high-voltage direct current (HVDC) system and the traditional HVDC system is that there are two control degrees of freedom in the flexible HVDC system that is the amplitude and phase angle of the output voltage. While, there is only one control degrees of freedom in the traditional HVDC system which is the amplitude of the output voltage. Therefore, compared with the traditional HVDC system, there will not be the problem of commutation failure in the flexible HVDC system, and the control systems of the rectifier side and the inverter side being operated independently make the reliability of the system greatly improved. This article analyses and compares the main methods of reliability assessment. Analysing and comparing the reliability parameters of the two systems by using the state enumeration method, and using the economic characterisation to make the comparison more intuitive

The Relative Distance Prediction of Transmembrane Protein Surface Residue Based on Improved Residual Networks

(1) Background: Transmembrane proteins (TMPs) act as gateways connecting the intra- and extra-biomembrane environments, exchanging material and signals crossing the biofilm. Relevant evidence shows that corresponding interactions mostly happen on the TMPs’ surface. Therefore, knowledge of the relative distance among surface residues is critically helpful in discovering the potential local structural characters and setting the foundation for the protein’s interaction with other molecules. However, the prediction of fine-grained distances among residues with sequences remains challenging; (2) Methods: In this study, we proposed a deep-learning method called TMP-SurResD, which capitalized on the combination of the Residual Block (RB) and Squeeze-and-Excitation (SE) for simultaneously predicting the relative distance of functional surface residues based on sequences’ information; (3) Results: The comprehensive evaluation demonstrated that TMP-SurResD could successfully capture the relative distance between residues, with a Pearson Correlation Coefficient (PCC) of 0.7105 and 0.6999 on the validation and independent sets, respectively. In addition, TMP-SurResD outperformed other methods when applied to TMPs surface residue contact prediction, and the maximum Matthews Correlation Coefficient (MCC) reached 0.602 by setting a threshold to the predicted distance of 10; (4) Conclusions: TMP-SurResD can serve as a useful tool in supporting a sequence-based local structural feature construction and exploring the function and biological mechanisms of structure determination in TMPs, which can thus significantly facilitate the research direction of molecular drug action, target design, and disease treatment

The Relative Distance Prediction of Transmembrane Protein Surface Residue Based on Improved Residual Networks

(1) Background: Transmembrane proteins (TMPs) act as gateways connecting the intra- and extra-biomembrane environments, exchanging material and signals crossing the biofilm. Relevant evidence shows that corresponding interactions mostly happen on the TMPs’ surface. Therefore, knowledge of the relative distance among surface residues is critically helpful in discovering the potential local structural characters and setting the foundation for the protein’s interaction with other molecules. However, the prediction of fine-grained distances among residues with sequences remains challenging; (2) Methods: In this study, we proposed a deep-learning method called TMP-SurResD, which capitalized on the combination of the Residual Block (RB) and Squeeze-and-Excitation (SE) for simultaneously predicting the relative distance of functional surface residues based on sequences’ information; (3) Results: The comprehensive evaluation demonstrated that TMP-SurResD could successfully capture the relative distance between residues, with a Pearson Correlation Coefficient (PCC) of 0.7105 and 0.6999 on the validation and independent sets, respectively. In addition, TMP-SurResD outperformed other methods when applied to TMPs surface residue contact prediction, and the maximum Matthews Correlation Coefficient (MCC) reached 0.602 by setting a threshold to the predicted distance of 10; (4) Conclusions: TMP-SurResD can serve as a useful tool in supporting a sequence-based local structural feature construction and exploring the function and biological mechanisms of structure determination in TMPs, which can thus significantly facilitate the research direction of molecular drug action, target design, and disease treatment

The characteristic analysis of phase-controlled array thermo-acoustic emission with multiple emitting surfaces

Abstract Thermo-acoustic (TA) ultrasound, particularly when combined with phased-controlled array technology, has garnered significant interest in the past decade due to its numerous advantages. This paper establishes a theoretical expression for thermo-acoustic phased array (TAPA) emission to investigate different acoustic fields based on input heat flow frequencies, quantities and distances of TA emission surfaces, area of emission surfaces, and phase changes between emission surfaces. The study finds that a TAPA with two emitting surfaces in a line pattern produces a consistent acoustic field compared to a single emitting surface arranged in a semicircle. Additionally, applying different phases on the surfaces narrows the scanning range with an increase in frequency, area of the TA emission surface, and the amount of emission surfaces, while enhancing the directivity of the TA wave. Moreover, increasing the distance between emitting surfaces in a square-shaped TAPA does not affect the ultrasound pressure of the main TA ultrasound but increases the quantity and size of side lobes. Furthermore, enlarging the area of emitting surfaces enhances the directivity of the TA ultrasound, necessitating a reduction in the distance between emitting surfaces or an increase in the area of the emitting surfaces in a square-shaped TAPA to enhance directivity. This paper provides a comprehensive study of TAPA to aid further research in this field

Isolation, Identification, and Characterization of Two Potential Probiotics from Marine Fish

Marine fish are rich in minerals (calcium and iron), vitamins (riboflavin and niacin), nutrients that are beneficial for brain (fatty acids), docosahexaenoic acid that prevent cardiovascular and cerebrovascular diseases, and are an important source of protein for humans. In recent years, the nexus between the increased demand of high-quality protein and the decrease in marine fishery resources resulted in the intensification of marine farming. However, this high-density farming has led to various bacterial diseases frequently due to lack of disease control methods, which has caused economic losses in the farming industry and impeded the healthy development of the industry. Therefore, it is particularly important to reduce the rates of bacterial diseases and improve the survival rate of cultured fish. It is of particular concern on how to maintain intestinal health of cultured fish under this farming method. Probiotics, a kind of living microorganisms, are beneficial to the health of the host. In the 1980s, probiotics have been viewed as an environmentally friendly and effective product in aquaculture; they improve the host health by influencing intestinal microbiota and nonspecific immunity to increase disease resistance ability. Moreover, they can be used as an ideal substitute for antibiotics in aquaculture production. Probiotics from Bacillus, lactic acid bacteria, saccharomyces, and nitrobacteria are diverse, and the function of each varies widely. As the typical representative of probiotics, lactic acid bacteria, may inhibit the growth of pathogens by the production of its metabolites, such as lactic acid, acetic acid, peroxide hydrogen, and bacteriocin. Lactic acid bacteria are also able to balance intestinal microecological imbalance and maintain intestinal microbiota balance when the intestinal microbiota contains pathogenic bacteria or the host is treated with antibiotics. Bacillus is an aerobic or facultative anaerobic gram-positive bacteria, which is stable, possesses strong stress resistance and high resurrection rate, and can produce various macromolecules, such as proteases and amylases and thus can improve the digestive function of the host by promoting the absorption of nutrients. Furthermore, as a non-specific immune antigen, Bacillus can improve the immune resistance of the host by stimulation of the components of cells or cell walls. The application of lactic acid bacteria and Bacillus has demonstrated favorable results, but this has been limited in marine fish culture because the non-fish origin of some strains and the different specificity of strains for different fish species or the same growth stage make their application difficult. Thus, it is essential to develop marine fish-derived probiotics, analyze their characteristics, define growth conditions, verify safety effects, and determine dosage and methods for their administer in marine fish culture. In this study, probiotics from marine fish were screened for the development of microbial ecological agents; 80 strains of culturable bacteria were obtained by separating bacteria from the mucosa samples of the digestive tract of wild Sebastes schlegelii and Hexagrammos otakii. The enzyme-producing ability of the strains was determined using a selective culture medium. The common pathogenic bacteria infecting marine fish were selected as indicator bacteria to determine the antibacterial activity of the isolated strains. Two potential probiotics, strains TS2 and TH8, were screened, and their physiological and biochemical identification, 16S rDNA sequence, growth characteristics, and host safety were determined. The results showed that TS2 had the strongest ability to hydrolyze starch, protein, and fat, and its sterile culture products could significantly inhibit the growth of Vibrio anguillarum, V. parahaemolyticus, V. Harvey, and Pseudoalteromonas nigrifaciens. TH8 has the strongest ability to hydrolyze protein and fat, and its sterile culture products could significantly inhibit the growth of V. anguillarum, V. alginolyticus, V. parahaemolyticus, P. nigrifaciens, Aeromonas hydrophila, Staphyloccocus aureus, and Escherichia coli. According to the analysis of the physiological and biochemical characteristics of the bacteria and 16S rDNA sequence alignment analysis, strain TS2 was identified as Bacillus subtilis and strain TH8 as Vagococcus fluvialis. Strain TS2 showed significant growth at 15–40 ℃, sodium chloride concentration of 0–0.08 g/L, and pH of 5–9; it entered the logarithmic phase after 6 h and the stable phase after 26 h. TH8 grew rapidly at 20–40 ℃, sodium chloride concentration of 0–0.08 g/L, and pH of 5–12; it entered the logarithmic phase after 2 h and the stable phase after 14 h. The safety of strains TS2 and TH8 was analyzed on a homologous host, and it was found that the strains were relatively safe for the homologous host at the concentration of 108 CFU/mL. The screened B. subtilis TS2 and V. fluvialis TH8 strains have a strong enzyme production ability and inhibit the growth of various pathogens by their metabolites; they have the following advantages: Wide temperature and salt tolerance, acid and alkali resistance, and fast growth speed. Thus, they could be considered as potential probiotic candidates for the development of microbial pharmaceuticals and can be used in more applications in marine aquaculture

Research on the Influence of Battery Cell Static Parameters on the Capacity of Different Topology Battery Packs

The parameter inconsistency of the battery cells and the series-parallel connection mode are closely related to the battery pack capacity. Studying the degree of influence of battery pack capacity by battery cell parameters is of great significance to the series-parallel design of battery packs. This paper establishes battery cell models and battery pack models with different topologies. In the MATLAB/Simulink environment, simulation studies were conducted to study the influence of the battery pack capacity by the monomer parameters as the number of cells in series and parallel in the topology changes. Then, from a statistical point of view, the simulation results were analyzed in principle. Finally, a small-scale battery pack experimental platform was built in the laboratory environment to verify the correctness of the simulation conclusions and theoretical analysis

Lisinopril Protects Against the Adriamycin Nephropathy and Reverses the Renalase Reduction: Potential Role of Renalase in Adriamycin Nephropathy

Aims: To investigate the potential role of renalase in adriamycin nephropathy and the effect of lisinopril on the regulation of renalase. Methods: Adriamycin nephropathy was induced in male Wistar rats (n=12) by a single injection of adriamycin at 2 mg/kg body weight. Rats were then randomly assigned to a model group or a treatment group, to which were administered distilled water or the angiotensin converting enzyme inhibitor lisinopril, respectively, for 12 weeks. Six normal rats served as controls. At the end of study, physiological parameters and systolic blood pressure were measured. Glomerulosclerosis and tubulointerstitial injury were assessed by histopathology Renalase protein expression in kidney was quantified by immunohistochemistry and immunoblotting. The serum concentration and urinary excretion of renalase were determined by enzyme-linked immunosorbent assay. Results: In model group rats, proteinuria and systolic blood pressure were elevated. Increased serum renalase concentration was observed; however, renalase protein expression in the kidney was significantly decreased. Compared with the model group, decreased proteinuria, lower systolic blood pressure, and fewer morphologic lesions were detected in the treatment group. Although levels of serum renalase were similar, accumulation of renalase in urine and kidney tissue increased notably in the treatment group compared with the model group. Conclusions: This study suggests that renalase may be involved in the process of adriamycin-induced renal injuries. Lisinopril may attenuate adriamycin-induced kidney injury by controlling blood pressure, which may be partially attributed to the renalase expression and secretion