Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptions

Stochastic gradient descent (SGD) is a popular and efficient method with wide applications in training deep neural nets and other nonconvex models. While the behavior of SGD is well understood in the convex learning setting, the existing theoretical results for SGD applied to nonconvex objective functions are far from mature. For example, existing results require to impose a nontrivial assumption on the uniform boundedness of gradients for all iterates encountered in the learning process, which is hard to verify in practical implementations. In this paper, we establish a rigorous theoretical foundation for SGD in nonconvex learning by showing that this boundedness assumption can be removed without affecting convergence rates. In particular, we establish sufficient conditions for almost sure convergence as well as optimal convergence rates for SGD applied to both general nonconvex objective functions and gradient-dominated objective functions. A linear convergence is further derived in the case with zero variances.Comment: Accepted by IEEE Transactions on Neural Networks and Learning Systems. DOI: 10.1109/TNNLS.2019.295221

Differentially Private SGDA for Minimax Problems

Stochastic gradient descent ascent (SGDA) and its variants have been the workhorse for solving minimax problems. However, in contrast to the well-studied stochastic gradient descent (SGD) with differential privacy (DP) constraints, there is little work on understanding the generalization (utility) of SGDA with DP constraints. In this paper, we use the algorithmic stability approach to establish the generalization (utility) of DP-SGDA in different settings. In particular, for the convex-concave setting, we prove that the DP-SGDA can achieve an optimal utility rate in terms of the weak primal-dual population risk in both smooth and non-smooth cases. To our best knowledge, this is the first-ever-known result for DP-SGDA in the non-smooth case. We further provide its utility analysis in the nonconvex-strongly-concave setting which is the first-ever-known result in terms of the primal population risk. The convergence and generalization results for this nonconvex setting are new even in the non-private setting. Finally, numerical experiments are conducted to demonstrate the effectiveness of DP-SGDA for both convex and nonconvex cases

A Note on the Bias of Rotational Differential-Linear Distinguishers

This note solves the open problem of finding a closed formula for the bias of a rotational differential-linear distinguisher proposed in IACR ePrint 2021/189 (EUROCRYPT 2021), completely generalizing the results on ordinary differential-linear distinguishers due to Blondeau, Leander, and Nyberg (JoC 2017) to the case of rotational differential-linear distinguishers

An infectious clone of enterovirus 71(EV71) that is capable of infecting neonatal immune competent mice without adaptive mutations

Enterovirus 71 (EV71) is a major pathogen that causes hand, foot and mouth disease (HFMD), which is a life threatening disease in certain children. The pathogenesis of EV71-caused HFMD is poorly defined due to the lack of simple and robust animal models with severe phenotypes that recapitulate symptoms observed in humans. Here, we generated the infectious clone of a clinical isolate from a severe HFMD patient. Virus rescued from the cDNA clone was infectious in cell lines. When administrated intraperitoneally to neonatal ICR, BALB/c and C57 immune competent mice at a dosage of1.4 × 104 pfu per mouse, the virus caused weight loss, paralysis and death in the infected mice after 4-5 days of infection. In the infected mice, detectable viral replication was detected in various tissues such as heart, liver, brain, lung, kidney, small intestine, leg skeletal muscle and medulla oblongata. The histology of the infected mice included massive myolysis, glomerular atrophy, villous blunting in small intestine, widened alveolar septum, diminished alveolar spaces and lymphocytes infiltration into the lung. By using the UV-inactivated virus as a control, we elucidated that the virus first amplified in the leg skeletal muscle tissue and the muscle tissue served as a primary viral replication site. In summary, we generated a stable EV71 infectious clone that is capable of infecting neonatal immune competent mice without adaptive mutations and provide a simple, valuable animal model for the studies of EV71pathogenesis and therapy.</p

Hepatitis B virus spliced variants are associated with an impaired response to interferon therapy

During hepatitis B virus (HBV) replication, spliced HBV genomes and splice-generated proteins have been widely described, however, their biological and clinical significance remains to be defined. Here, an elevation of the proportion of HBV spliced variants in the sera of patients with chronic hepatitis B (CHB) is shown to correlate with an impaired respond to interferon-α (IFN-α) therapy. Transfection of the constructs encoding the three most dominant species of spliced variants into cells or ectopic expression of the two major spliced protein including HBSP and N-terminal-truncated viral polymerase protein result in strong suppression of IFN-α signaling transduction, while mutation of the major splicing-related sites of HBV attenuates the viral anti-IFN activities in both cell and mouse models. These results have associated the productions of HBV spliced variants with the failure response to IFN therapy and illuminate a novel mechanism where spliced viral products are employed to resist IFN-mediated host defense. </p

Infection of inbred BALB/c and C57BL/6 and outbred Institute of Cancer Research mice with the emerging H7N9 avian influenza virus

A new avian-origin influenza virus A (H7N9) recently crossed the species barrier and infected humans; therefore, there is an urgent need to establish mammalian animal models for studying the pathogenic mechanism of this strain and the immunological response. In this study, we attempted to develop mouse models of H7N9 infection because mice are traditionally the most convenient models for studying influenza viruses. We showed that the novel A (H7N9) virus isolated from a patient could infect inbred BALB/c and C57BL/6 mice as well as outbred Institute of Cancer Research (ICR) mice. The amount of bodyweight lost showed differences at 7 days post infection (d.p.i.) (BALB/c mice 30%, C57BL/6 and ICR mice approximately 20%), and the lung indexes were increased both at 3 d.p.i. and at 7 d.p.i.. Immunohistochemistry demonstrated the existence of the H7N9 viruses in the lungs of the infected mice, and these findings were verified by quantitative real-time polymerase chain reaction (RT-PCR) and 50% tissue culture infectious dose (TCID50) detection at 3 d.p.i. and 7 d.p.i.. Histopathological changes occurred in the infected lungs, including pulmonary interstitial inflammatory lesions, pulmonary oedema and haemorrhages. Furthermore, because the most clinically severe cases were in elderly patients, we analysed the H7N9 infections in both young and old ICR mice. The old ICR mice showed more severe infections with more bodyweight lost and a higher lung index than the young ICR mice. Compared with the young ICR mice, the old mice showed a delayed clearance of the H7N9 virus and higher inflammation in the lungs. Thus, old ICR mice could partially mimic the more severe illness in elderly patients. </p

Boosting Heterosubtypic Neutralization Antibodies in Recipients of 2009 Pandemic H1N1 Influenza Vaccine

Our data demonstrated that the inoculation with vaccine derived from the 2009 pandemic influenza raised vigorous neutralization antibodies against both cognate H1N1 and heterotypic influenza viruses. This observation has important implication for vaccine development

Novel, Real-Time Cell Analysis for Measuring Viral Cytopathogenesis and the Efficacy of Neutralizing Antibodies to the 2009 Influenza A (H1N1) Virus

A novel electronic cell sensor array technology, the real-time cell analysis (RTCA) system, was developed to monitor cell events. Unlike the conventional methods labeling the target cells with fluorescence, luminescence, or light absorption, the RTCA system allows for label-free detection of cell processes directly without the incorporation of labels. Here, we used this new format to measure the cytopathic effect (CPE) of the 2009 influenza A (H1N1) virus and the efficacy of neutralizing antibodies in human sera to this virus. The real-time dynamic monitoring of CPE was performed on MDCK cell cultures infected with the H1N1 virus, ranging from 5.50×102 to 5.50×107 copies/mL. The resulting CPE kinetic curves were automatically recorded and were both time and viral load dependent. The CPE kinetics were also distinguishable between different H1N1 stains, as the onset of CPE induced by the A/Shanghai/37T/2009 H1N1 virus was earlier than that of the A/Shanghai/143T/2009 H1N1 virus. Furthermore, inhibition of H1N1 virus-induced CPE in the presence of human specific anti-sera was detected and quantified using the RTCA system. Antibody titers determined using this new neutralization test correlated well with those obtained independently via the standard hemagglutination inhibition test. Taken together, this new CPE assay format provided label-free and high-throughput measurement of viral growth and the effect of neutralizing antibodies, illustrating its potential in influenza vaccine studies

Automatic 2D line extraction from 3D geometric model

A new approach that extracts outlines from 3D geometric models is investigated. A 3D model is created in Maya based on 2D designs provided by artists. Outlines, including occluding contour, suggestive contour, and boundary, that contain shape information of the model are examined mathematically. APIs in RTSC, Trimesh, and OpenGL are applied to extract the outlines from the model and project them onto 2D planes. Hidden lines and redundant points in the projected results are removed to produce more accurate and compact shape information. Finally, similarity comparison is performed between the results and the original 2D drafts by using shape matching algorithms.Bachelor of Engineering (Computer Engineering