Adaptive Optimizers with Sparse Group Lasso for Neural Networks in CTR Prediction

We develop a novel framework that adds the regularizers of the sparse group lasso to a family of adaptive optimizers in deep learning, such as Momentum, Adagrad, Adam, AMSGrad, AdaHessian, and create a new class of optimizers, which are named Group Momentum, Group Adagrad, Group Adam, Group AMSGrad and Group AdaHessian, etc., accordingly. We establish theoretically proven convergence guarantees in the stochastic convex settings, based on primal-dual methods. We evaluate the regularized effect of our new optimizers on three large-scale real-world ad click datasets with state-of-the-art deep learning models. The experimental results reveal that compared with the original optimizers with the post-processing procedure which uses the magnitude pruning method, the performance of the models can be significantly improved on the same sparsity level. Furthermore, in comparison to the cases without magnitude pruning, our methods can achieve extremely high sparsity with significantly better or highly competitive performance. The code is available at https://github.com/intelligent-machine-learning/dlrover/blob/master/tfplus.Comment: 24 pages. Published as a conference paper at ECML PKDD 2021. This version includes Appendix which was not included in the published version because of page limi

Investigation of Low-Frequency Sound Radiation Characteristics and Active Control Mechanism of a Finite Cylindrical Shell

In this paper, the radiation characteristics and active structural acoustic control of a submerged cylindrical shell at low frequencies are investigated. First, the coupled vibro-acoustic equations for a submerged finite cylindrical shell are solved by a modal decomposition method, and the radiation impedance is obtained by the fast Fourier transform. The modal shapes of the first ten acoustic radiation modes and the structure-dependent radiation modes are presented. The relationships between the vibration modes and the radiation modes as well as the contributions of the radiation modes to the radiated sound power are given at low frequencies. Finally, active structural acoustic control of a submerged finite cylindrical shell is investigated by considering the fluid-structure coupled interactions. The physical mechanism of the active control is discussed based on the relationship between the vibration and radiation modes. The results showed that, at low frequencies, only the first several radiation modes contributed to the sound power radiated from a submerged finite cylindrical shell excited by a radial point force. By determining the radiation modes that dominate the contribution to the radiated sound, the physical mechanism of the active control is explained, providing a potential tool to allow active control of the vibro-acoustic responses of submerged structures more effectively

Machine Learning in Process Monitoring and Control for Wire-Arc Additive Manufacturing

Wire-arc additive manufacturing (WAAM) is an arc-based directed energy deposition approach that uses an electrical arc as a source of fusion to melt the wire feedstock and deposit layer by layer. It’s applicable in fabricating large-scale components. At this stage, there are still some issues that need to be researched deeply, such as manufacturing accuracy control, process parameters optimization, path planning, and online monitoring. Machine learning is a new emerging artificial intelligence technology, which is more and more applied in modern industry. In this study, a machine learning based control algorithm was applied in melt pool width control. To monitor the WAAM process, deep learning algorithms were applied in anomalies recognition. At the same time, machine learning methods were employed to predict the deposited surface roughness during the WAAM process

A Robust Weighted Combination Forecasting Method Based on Forecast Model Filtering and Adaptive Variable Weight Determination

Medium-and-long-term load forecasting plays an important role in energy policy implementation and electric department investment decision. Aiming to improve the robustness and accuracy of annual electric load forecasting, a robust weighted combination load forecasting method based on forecast model filtering and adaptive variable weight determination is proposed. Similar years of selection is carried out based on the similarity between the history year and the forecast year. The forecast models are filtered to select the better ones according to their comprehensive validity degrees. To determine the adaptive variable weight of the selected forecast models, the disturbance variable is introduced into Immune Algorithm-Particle Swarm Optimization (IA-PSO) and the adaptive adjustable strategy of particle search speed is established. Based on the forecast model weight determined by improved IA-PSO, the weighted combination forecast of annual electric load is obtained. The given case study illustrates the correctness and feasibility of the proposed method

The efficacy and safety of lenvatinib plus transarterial chemoembolization in combination with PD-1 antibody in treatment of unresectable recurrent hepatocellular carcinoma: a case series report

PurposeTo explore the safety and efficacy of lenvatinib in combination with trans-arterial chemoembolization (TACE) and programmed death receptor 1 (PD-1) antibody in the treatment of unresectable recurrent hepatocellular carcinoma (urHCC).Patients and methodsThe clinical data of 16 patients with unresectable recurrent hepatocellular carcinoma admitted to the Department of Liver Surgery and Liver Transplantation Center, West China Hospital, Sichuan University, and received the conversion therapy of lenvatinib + TACE + PD-1 antibody between January 2019 and January 2022 were retrospectively analyzed.ResultsThere were 25% (4/16) patients suffering from grade 3 adverse events and no patients suffering from grade 4 or higher adverse events. After 4 months of treatment of 16 patients, according to the modified Response Evaluation Criteria in Solid Tumors (mRECIST), two, five, three, and six cases were in complete response (CR), partial response (PR), stable disease (SD), and progressive disease (PD), respectively, and the objective response rate (ORR) was 43.8% (7/16). The 1-year overall survival (OS) rate and 1-year progression-free survival (PFS) rate were 86.2% and 46.9%, respectively. In our subgroup analysis, the ORR of patients with multiple lesions reached up to 60%, which was higher than that of patients with single lesions.ConclusionsLenvatinib in combination with TACE and PD-1 antibody is safe and effective in the treatment of unresectable recurrent hepatocellular carcinoma