Rethinking Learning Rate Tuning in the Era of Large Language Models

Large Language Models (LLMs) represent the recent success of deep learning in achieving remarkable human-like predictive performance. It has become a mainstream strategy to leverage fine-tuning to adapt LLMs for various real-world applications due to the prohibitive expenses associated with LLM training. The learning rate is one of the most important hyperparameters in LLM fine-tuning with direct impacts on both fine-tuning efficiency and fine-tuned LLM quality. Existing learning rate policies are primarily designed for training traditional deep neural networks (DNNs), which may not work well for LLM fine-tuning. We reassess the research challenges and opportunities of learning rate tuning in the coming era of Large Language Models. This paper makes three original contributions. First, we revisit existing learning rate policies to analyze the critical challenges of learning rate tuning in the era of LLMs. Second, we present LRBench++ to benchmark learning rate policies and facilitate learning rate tuning for both traditional DNNs and LLMs. Third, our experimental analysis with LRBench++ demonstrates the key differences between LLM fine-tuning and traditional DNN training and validates our analysis

A New Anthracene Derivative from Marine Streptomyces sp. W007 Exhibiting Highly and Selectively Cytotoxic Activities

A new anthracene derivative, 3-hydroxy-1-keto-3-methyl-8-methoxy-1,2,3, 4-tetrahydro-benz[α]anthracene, was isolated from the marine strain Streptomyces sp. W007, and its structure was established by spectroscopic analysis including mass spectra, 1D- and 2D-NMR (1H–1H COSY, HMBC, HSQC and NOESY) experiments. 3-hydroxy-1-keto-3-methyl-8-methoxy-1,2,3,4-tetrahydro-benz[α]anthracene showed cytotoxicity against human lung adenocarcinoma cell line A549

Photometric and Spectroscopic Studies of V582 Lyr and V1016 Oph

We present new CCD photometric light curves about two eclipsing binaries of V582 Lyr and V1016 Oph. Our observations were carried out by the SARA 91.4 cm telescope of America in 2016 and the 60 cm telescope of Chile in 2018. V582 Lyr’s spectra type was classified as K5, and its radial velocity was determined using the LAMOST spectral survey. There are absorptions in the observed Hα line and excess emissions in the subtracted Hα line, which show weak chromospheric activity. We obtained the updated ephemeris information for V582 Lr and V1016 Oph, and found that their orbital periods are both decreasing.We concluded that the decreased rate is −0.474 (±0.011)Å~10−7 days yr−1 for V582 Lyr and 3.460 (±0.014)Å~10−7 days yr−1 for V1016 Oph. For V582 Lyr, the period variation was interpreted as a mass transfer from the secondary component to the primary one, and the corresponding rate is dM2/dt=−1.10 (±0.03)Å~10−7 Me yr−1. For V1016 Oph, we explain it by transferring from the primary component to the secondary one, and the corresponding rate is dM1/dt=−2.69 (±0.04)Å~10−7 Me yr−1. The photometric solution of V1016 Oph was obtained by analyzing the CCD photometry with the Wilson–Devinney program. We also obtained the orbital parameters of V1016 Oph by simultaneously analyzing our BVRI light curves and radial-velocity curve from the LAMOST low-resolution spectral survey. Finally, our orbital solution shows that they are contact eclipsing binaries with contact factors of 3.35 (±0.08)% for V582 Lyr and 41.0 (±0.1)% for V1016 Oph

Lifetime prediction and reliability analysis for aluminum electrolytic capacitors in EV charging module based on mission profiles

The charging modules of Electric vehicles (EVs) always run in a complex and variable state. As the weakness in the reliable operation of charging modules, the accurate lifetime prediction of aluminum electrolytic capacitors (Al-caps) is important for the later maintenance and reliability design. The hotspot temperature calculation method and lifetime model limit the accuracy of aluminum electrolytic capacitors lifetime prediction methods, which cannot meet the increasing requirements for reliability. In order to solve the problems above, this paper has proposed a hotspot temperature calculation method based on the ripple current with frequency characteristics and the cooling conditions on the heat generation and thermal conductivity of the capacitors. Furthermore, the lifetime model under reference voltage has been constructed with 3D surface fitting toolbox, which describes the trends of capacitor lifetime with ambient temperature and hotspot temperature under the constant voltage condition. Considering the variation of voltage, the multiple lifetime model of capacitor is established with a voltage correction coefficient. With the proposed method, it can be realized about the real-time lifetime prediction of capacitors under multiple operating profiles such as ripple current, thermal dissipation conditions, ambient temperature and operating voltage. Finally, the effectiveness of the proposed method is verified with the annual profiles of a 30 kW EV charging module

Systematic review and network meta-analysis of pre-emptive embolization of the aneurysm sac side branches and aneurysm sac coil embolization to improve the outcomes of endovascular aneurysm repair

ObjectivePrevious reports have revealed a high incidence of type II endoleak (T2EL) after endovascular aneurysm repair (EVAR). The incidence of T2EL after EVAR is reduced by pre-emptive embolization of aneurysm sac side branches (ASSB) and aneurysm sac coil embolization (ASCE). This study aimed to investigate whether different preventive interventions for T2EL were correlated with suppression of aneurysm sac expansion and reduction of the re-intervention rate.MethodsThe PubMed, Web of Science, MEDLINE and Embase databases, and conference proceedings were searched to identify articles on EVAR with or without embolization. The study was developed in line with the Participants, Interventions, Comparisons, Outcomes, and Study design principles and was conducted and reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines. We used network meta-analysis based on multivariate random-effects meta-analysis to indirectly compare outcomes of different strategies for embolization during EVAR.ResultsA total of 31 studies met all inclusion criteria and were included in the qualitative and quantitative syntheses. The included studies were published between 2001 and 2022 and analyzed a total of 18,542 patients, including 1,882 patients who received prophylactic embolization treatment during EVAR (experimental group) and 16,660 who did not receive prophylactic embolization during EVAR (control group). The effect of pre-emptive embolization of the inferior mesenteric artery (IMA) (IMA-ASSB) in preventing T2EL was similar (relative risk [RR] 1.01, 95% confidence interval [CI] 0.38–2.63) to the effects of non-selective embolization of ASSB (NS-ASSB) and ASCE (RR 0.88, 95% CI 0.40–1.96). IMA-ASSB showed a better clinical effect in suppressing the aneurysm sac expansion (RR 0.27, 95% CI 0.09–2.25 compared with NS-ASSB; RR 0.93, 95% CI 0.16–5.56 compared with ASCE) and reducing the re-intervention rate (RR 0.34, 95% CI 0.08–1.53 compared with NS-ASSB; RR 0.66, 95% CI 0.19–2.22 compared with ASCE). All prophylactic embolization strategies improved the clinical outcomes of EVAR.ConclusionProphylactic embolization during EVAR effectively prevents T2EL, suppresses the aneurysm sac expansion, and reduces the re-intervention rate. IMA embolization demonstrated benefits in achieving long-term aneurysm sac stability and lowering the risk of secondary surgery. NS-ASSB more effectively reduces the incidence of T2EL, while IMA embolization alone or in combination with ASCE enhances the clinical benefits of EVAR. In addition, as network meta-analysis is still an indirect method based on a refinement of existing data, more studies and evidence are still needed in the future to establish more credible conclusions