Deep Residual Learning via Large Sample Mean-Field Stochastic Optimization

We study a class of stochastic optimization problems of the mean-field type arising in the optimal training of a deep residual neural network. We consider the sampling problem arising from a continuous layer idealization, and establish the existence of optimal relaxed controls when the training set has finite size. The core of our paper is to prove the Gamma-convergence of the sequence of sampled objective functionals, i.e., show that as the size of the training set grows large, the minimizer of the sampled relaxed problem converges to that of the limiting optimization problem. We connect the limit of the large sampled objective functional to the unique solution, in the trajectory sense, of a nonlinear Fokker-Planck-Kolmogorov (FPK) equation in a random environment. We construct an example to show that, under mild assumptions, the optimal network weights can be numerically computed by solving a second-order differential equation with Neumann boundary conditions in the sense of distributions

Real-time Information, Uncertainty and Quantum Feedback Control

Feedback is the core concept in cybernetics and its effective use has made great success in but not limited to the fields of engineering, biology, and computer science. When feedback is used to quantum systems, two major types of feedback control protocols including coherent feedback control (CFC) and measurement-based feedback control (MFC) have been developed. In this paper, we compare the two types of quantum feedback control protocols by focusing on the real-time information used in the feedback loop and the capability in dealing with parameter uncertainty. An equivalent relationship is established between quantum CFC and non-selective quantum MFC in the form of operator-sum representation. Using several examples of quantum feedback control, we show that quantum MFC can theoretically achieve better performance than quantum CFC in stabilizing a quantum state and dealing with Hamiltonian parameter uncertainty. The results enrich understanding of the relative advantages between quantum MFC and quantum CFC, and can provide useful information in choosing suitable feedback protocols for quantum systems.Comment: 24 page