Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation

Convolutional neural networks have been widely deployed in various application scenarios. In order to extend the applications' boundaries to some accuracy-crucial domains, researchers have been investigating approaches to boost accuracy through either deeper or wider network structures, which brings with them the exponential increment of the computational and storage cost, delaying the responding time. In this paper, we propose a general training framework named self distillation, which notably enhances the performance (accuracy) of convolutional neural networks through shrinking the size of the network rather than aggrandizing it. Different from traditional knowledge distillation - a knowledge transformation methodology among networks, which forces student neural networks to approximate the softmax layer outputs of pre-trained teacher neural networks, the proposed self distillation framework distills knowledge within network itself. The networks are firstly divided into several sections. Then the knowledge in the deeper portion of the networks is squeezed into the shallow ones. Experiments further prove the generalization of the proposed self distillation framework: enhancement of accuracy at average level is 2.65%, varying from 0.61% in ResNeXt as minimum to 4.07% in VGG19 as maximum. In addition, it can also provide flexibility of depth-wise scalable inference on resource-limited edge devices.Our codes will be released on github soon.Comment: 10page

Stochastic reaction-diffusion problems in modeling biochemical systems

The dynamics of many biological processes rely on an interplay between spatial transport and chemical reactions. In particular, spatial dynamics can play a critical role in the successful functioning of cellular signaling processes, where as basic a prop- erty as cell shape can significantly influence the behavior of signaling pathways. The inside of cells is a complex spatial environment, filled with organelles, filaments and proteins. We investigate the question of how cell signaling pathways function robustly in the presence of such spatial heterogeneity for the most basic of chemical signals. Due to the noisy environment of a cell, particle-based stochastic reaction-diffusion models are a widely used approach for studying such cellular processes, explicitly modeling the diffusion of, and reactions between, individual molecules. However, the computational expense of such methods can greatly limit the size of chemical systems that can be studied. To overcome this challenge, we rigorously derive coarse-grained deterministic partial integro-differential equation models that provide a mean field ap- proximation to the particle-based stochastic reaction-diffusion model. Relationships between the mean field models and standard reaction-diffusion partial differential equation models are further investigated for general biochemical reaction systems. Comparisons between these models are illustrated through mathematical analysis and numerical examples

Coenzyme Q10 attenuates airway inflammation and oxidative stress in neonatal asthmatic rats

Purpose: To determine the therapeutic effect of coenzyme Q10 (CoQ10) on ovalbumin (OVA)-provoked asthma in neonatal rats.Methods: Asthma was induced by exposing neonatal rats to OVA. The levels of SOD, CAT, GPx, GSH, MDA and MPO were estimated using standard biochemical kits, while ELISA was used to measure the concentrations of Ig E and Th2 cytokines. Gene expressions were assayed with qRT-PCR, and protein expressions were determined with western blotting.Results: OVA treatment led to increases in levels of BALF inflammatory cells, lipid peroxidation, serum IgE and BALF Th2 cytokines, but it decreased antioxidant levels. Furthermore, the protein expression of NF-κB and mRNA expression levels of proinflammatory cytokines and inducible nitric oxide synthase (iNOS) were upregulated in the asthmatic rats (p < 0.05). However, coenzyme Q10 supplementation significantly decreased lipid peroxidation, and reduced inflammatory cells and IgE levels, while the antioxidant levels were enhanced (p < 0.05). Moreover, coenzyme Q10 reduced the levels of Th2 cytokines and downregulated the expressions of NF-κB, TNF-α, IL-6, and iNOS in the neonatal asthmatic rats (p < 0.05).Conclusion: Coenzyme Q10 attenuates airway inflammation and oxidative stress in neonatal asthmatic rats. Thus, coenzyme Q10 has promising therapeutic potential in the management of asthma. Keywords: Asthma, Neonatal, Coenzyme Q10, Th2, cytokines, Oxidative stress, Antiinflammatio

Fluctuation analysis for particle-based stochastic reaction-diffusion models

Recent works have derived and proven the large-population mean-field limit for several classes of particle-based stochastic reaction-diffusion (PBSRD) models. These limits correspond to systems of partial integral-differential equations (PIDEs) that generalize standard mass-action reaction-diffusion PDE models. In this work we derive and prove the next order fluctuation corrections to such limits, which we show satisfy systems of stochastic PIDEs with Gaussian noise. Numerical examples are presented to illustrate how including the fluctuation corrections can enable the accurate estimation of higher order statistics of the underlying PBSRD model

Defective DP-colorings of sparse simple graphs

DP-coloring (also known as correspondence coloring) is a generalization of list coloring developed recently by Dvo\v{r}\'ak and Postle. We introduce and study $(i,j)$-defective DP-colorings of simple graphs. Let $g_{DP}(i,j,n)$ be the minimum number of edges in an $n$-vertex DP-$(i,j)$-critical graph. In this paper we determine sharp bound on $g_{DP}(i,j,n)$ for each $i\geq3$ and $j\geq 2i+1$ for infinitely many $n$.Comment: 17 page

Recent development in kinetic theory of granular materials: analysis and numerical methods

33 pagesOver the past decades, kinetic description of granular materials has received a lot of attention in mathematical community and applied fields such as physics and engineering. This article aims to review recent mathematical results in kinetic granular materials, especially for those which arose since the last review by Villani on the same subject. We will discuss both theoretical and numerical developments. We will finally showcase some important open problems and conjectures by means of numerical experiments based on spectral methods