SGEM: stochastic gradient with energy and momentum

In this paper, we propose SGEM, Stochastic Gradient with Energy and Momentum, to solve a large class of general non-convex stochastic optimization problems, based on the AEGD method that originated in the work [AEGD: Adaptive Gradient Descent with Energy. arXiv: 2010.05109]. SGEM incorporates both energy and momentum at the same time so as to inherit their dual advantages. We show that SGEM features an unconditional energy stability property, and derive energy-dependent convergence rates in the general nonconvex stochastic setting, as well as a regret bound in the online convex setting. A lower threshold for the energy variable is also provided. Our experimental results show that SGEM converges faster than AEGD and generalizes better or at least as well as SGDM in training some deep neural networks.Comment: 24 pages, 4 figure

Citation advantage of COVID-19 related publications

With the global spread of the COVID-19 pandemic, scientists from various disciplines responded quickly to this historical public health emergency. The sudden boom of COVID-19 related papers in a short period of time may bring unexpected influence to some commonly used bibliometric indicators. By a large-scale investigation using Science Citation Index Expanded and Social Sciences Citation Index, this brief communication confirms the citation advantage of COVID-19 related papers empirically through the lens of Essential Science Indicators' highly cited paper. More than 8% of COVID-19 related papers published during 2020 and 2021 were selected as Essential Science Indicators highly cited papers, which was much higher than the set global benchmark value of 1%. The citation advantage of COVID-19 related papers for different Web of Science categories/countries/journal impact factor quartiles were also demonstrated. The distortions of COVID-19 related papers' citation advantage to some bibliometric indicators such as journal impact factor were discussed at the end of this brief communication.Comment: Journal of Information Science (2023

Adaptive Preconditioned Gradient Descent with Energy

We propose an adaptive time step with energy for a large class of preconditioned gradient descent methods, mainly applied to constrained optimization problems. Our strategy relies on representing the usual descent direction by the product of an energy variable and a transformed gradient, with a preconditioning matrix, for example, to reflect the natural gradient induced by the underlying metric in parameter space or to endow a projection operator when linear equality constraints are present. We present theoretical results on both unconditional stability and convergence rates for three respective classes of objective functions. In addition, our numerical results shed light on the excellent performance of the proposed method on several benchmark optimization problems.Comment: 32 pages, 3 figure

Simulation of the performance of phase-sensitive OTDR based on ultra-weak FBG array using double pulses

A phase-sensitive optical time domain reflectometry sensing system based on Ultra-weak FBG Array using double pulses is proposed. The principle of this sensing system is elaborated. A series of simulations are carried out to optimize the parameters to increase the sensing distance of this system. The parameters contain reflectivity of UWFBG the interval between adjacent UWFBGs and pulse width. The optimal reflectivity and pulse width corresponding to different grating space are found out to achieve a longer sensing distance

Phase sensitive distributed vibration sensing based on ultraweak fiber Bragg grating array using double-pulse

A distributed vibration sensing technique using double-optical-pulse based on phase-sensitive optical time-domain reflectometry (φ-OTDR) and an ultraweak fiber Bragg grating (UWFBG) array is proposed for the first time. The single-mode sensing fiber is integrated with the UWFBG array that has uniform spatial interval and ultraweak reflectivity. The relatively high reflectivity of the UWFBG, compared with the Rayleigh scattering, gains a high signal-to-noise ratio for the signal, which can make the system achieve the maximum detectable frequency limited by the round-trip time of the probe pulse in fiber. A corresponding experimental φ-OTDR system with a 4.5 km sensing fiber integrated with the UWFBG array was setup for the evaluation of the system performance. Distributed vibration sensing is successfully realized with spatial resolution of 50 m. The sensing range of the vibration frequency can cover from 3 Hz to 9 kHz

High performance interrogation by a composite-double-probe-pulse for ultra-weak FBG array

We propose and experimentally demonstrate a technique using a composite-double-probe-pulse (CDPP) to eliminate the effect of polarization fading for phase-sensitive optical time-domain reflectometry (Φ-OTDR) based on ultra-weak FBG (UWFBG) array. The CDPP is composed of two optical pulses whose spatial interval is equal to twice the spatial interval of adjacent UWFBGs in the UWFBG array. One optical pulse is a long optical pulse, and the other optical pulse is composed of two continuous short optical pulses, whose polarization states are orthogonal to each other. The width of the short pulse is equal to half of the width of the normal pulse and their frequencies are different from the long pulse. By using such a method to perform the sensing for the UWFBG array, distributed quantitative measurement can be realized with only direct detection scheme and the influence of polarization fading in the demodulation of signal is thoroughly eliminated

Efficient influenza vaccine manufacturing: Single MDCK suspension cells in chemically defined medium

Facing the constant global high demand for influenza vaccines, improving production capacity is most important. For influenza vaccine production, cell culture-based processes have advantages regarding flexibility, efficiency, and safety in comparison with the traditional egg-based processes. To avoid problems related to microcarrier-based approaches and serum containing media, growth of suspension cells in chemically-defined media is favoured. In addition, such a process has advantages regarding the improvement of virus titers, the scale-up of the production process, and overall productivity in up- and downstream processing. In this study, a previously developed MDCK suspension cell line [1] was cultivated in an in-house chemically defined medium to evaluate cell growth and virus production. For the purpose of process intensification, virus adaptation and infection strategies were investigated to achieve high cell densities and to maximize virus titers. Therefore, an adapted influenza virus strain (A/PR/8/34 H1N1 RK1) was generated by a series of virus passages with low multiplicity of infection (MOI). Virus infections were carried out by supplementing 100% of fresh medium, infecting cells with a MOI of 10-3, and with trypsin addition at 72 h of cell cultivations in shake flasks and bioreactors. For scale-up, MDCK cells were cultivated in a DASGIP bioreactor system, optimizing stirring speed, time of infection, pH and DO levels both for cell growth and virus infection. Cell count, viability, main extracellular metabolites, and virus titers were measured to compare productivity between shake flasks and bioreactors. In batch culture (shake flasks and bioreactors), single MDCK cells were grown to maximum cell densities of 1.2 x107 cells/ml with cell viabilities exceeding 98% at high cell specific growth rates of 0.036 h-1. Virus adaptation to the MDCK suspension cell line led to a fast infection and stable virus titers over time. Regarding process optimization, optimal pH (cell growth: 7.00, infection: 7.20), DO (40%) and agitation speed (80 rpm) were chosen for influenza A virus production in three parallel bioreactors. Cell densities of 1.0 x107 cells/ml were achieved at time of infection (72 h) before performing a dilution step. Post infection, similar virus infection dynamics were observed in shake flasks and bioreactors. For both cultivation systems maximal HA titers of 3.6 log10(HAU/100µl) were achieved without reduction of cell-specific virus titer (12,000 virions/cell). Overall, a highly efficient and scalable upstream process was realized by cultivation of MDCK suspension cells as single cells in chemically defined medium. This is a strong basis for a promising application in large-scale influenza vaccine manufacturing and potential process intensification towards high cell density virus production. [1] Huang D. et al., PloS One 10 (2015): e0141686. doi: 10.1371/journal.pone.014168