Orthogonally Decoupled Variational Gaussian Processes

Gaussian processes (GPs) provide a powerful non-parametric framework for reasoning over functions. Despite appealing theory, its superlinear computational and memory complexities have presented a long-standing challenge. State-of-the-art sparse variational inference methods trade modeling accuracy against complexity. However, the complexities of these methods still scale superlinearly in the number of basis functions, implying that that sparse GP methods are able to learn from large datasets only when a small model is used. Recently, a decoupled approach was proposed that removes the unnecessary coupling between the complexities of modeling the mean and the covariance functions of a GP. It achieves a linear complexity in the number of mean parameters, so an expressive posterior mean function can be modeled. While promising, this approach suffers from optimization difficulties due to ill-conditioning and non-convexity. In this work, we propose an alternative decoupled parametrization. It adopts an orthogonal basis in the mean function to model the residues that cannot be learned by the standard coupled approach. Therefore, our method extends, rather than replaces, the coupled approach to achieve strictly better performance. This construction admits a straightforward natural gradient update rule, so the structure of the information manifold that is lost during decoupling can be leveraged to speed up learning. Empirically, our algorithm demonstrates significantly faster convergence in multiple experiments.Comment: Appearing NIPS 201

Orthogonally Decoupled Variational Gaussian Processes

Gaussian processes (GPs) provide a powerful non-parametric framework for reasoning over functions. Despite appealing theory, its superlinear computational and memory complexities have presented a long-standing challenge. State-of-the-art sparse variational inference methods trade modeling accuracy against complexity. However, the complexities of these methods still scale superlinearly in the number of basis functions, implying that that sparse GP methods are able to learn from large datasets only when a small model is used. Recently, a decoupled approach was proposed that removes the unnecessary coupling between the complexities of modeling the mean and the covariance functions of a GP. It achieves a linear complexity in the number of mean parameters, so an expressive posterior mean function can be modeled. While promising, this approach suffers from optimization difficulties due to ill-conditioning and non-convexity. In this work, we propose an alternative decoupled parametrization. It adopts an orthogonal basis in the mean function to model the residues that cannot be learned by the standard coupled approach. Therefore, our method extends, rather than replaces, the coupled approach to achieve strictly better performance. This construction admits a straightforward natural gradient update rule, so the structure of the information manifold that is lost during decoupling can be leveraged to speed up learning. Empirically, our algorithm demonstrates significantly faster convergence in multiple experiments

Multi-component optical solitary waves

We discuss several novel types of multi-component (temporal and spatial) envelope solitary waves that appear in fiber and waveguide nonlinear optics. In particular, we describe multi-channel solitary waves in bit-parallel-wavelength fiber transmission systems for high performance computer networks, multi-colour parametric spatial solitary waves due to cascaded nonlinearities of quadratic materials, and quasiperiodic envelope solitons due to quasi-phase-matching in Fibonacci optical superlattices.Comment: 12 pages, 11 figures; To be published in: Proceedings of the Dynamics Days Asia-Pacific: First International Conference on Nonlinear Science (Hong-Kong, 13-16 July, 1999), Editor: Bambi Hu (Elsevier Publishers, 2000

Implicit Anatomical Rendering for Medical Image Segmentation with Stochastic Experts

Integrating high-level semantically correlated contents and low-level anatomical features is of central importance in medical image segmentation. Towards this end, recent deep learning-based medical segmentation methods have shown great promise in better modeling such information. However, convolution operators for medical segmentation typically operate on regular grids, which inherently blur the high-frequency regions, i.e., boundary regions. In this work, we propose MORSE, a generic implicit neural rendering framework designed at an anatomical level to assist learning in medical image segmentation. Our method is motivated by the fact that implicit neural representation has been shown to be more effective in fitting complex signals and solving computer graphics problems than discrete grid-based representation. The core of our approach is to formulate medical image segmentation as a rendering problem in an end-to-end manner. Specifically, we continuously align the coarse segmentation prediction with the ambiguous coordinate-based point representations and aggregate these features to adaptively refine the boundary region. To parallelly optimize multi-scale pixel-level features, we leverage the idea from Mixture-of-Expert (MoE) to design and train our MORSE with a stochastic gating mechanism. Our experiments demonstrate that MORSE can work well with different medical segmentation backbones, consistently achieving competitive performance improvements in both 2D and 3D supervised medical segmentation methods. We also theoretically analyze the superiority of MORSE.Comment: Accepted at International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2023

Scattering of the vector soliton in coupled nonlinear Schrödinger equation with Gaussian potential

Nonlinear Schrodinger equation (NLSE) is the fundamental equation which describes the wave field envelope dynamics in a nonlinear and dispersive medium. However, if the fields have many components, one should consider the Coupled Nonlinear Schrodinger equation (CNLSE). We considered the interactions of orthogonally polarized and equal-amplitude vector solitons with two polarization directions. In this paper, we focused on the effect of Gaussian potential on the scattering of the vector soliton in CNLSE. The scattering process was investigated by the variational approximation method and direct numerical solution of CNLSE. Analytically, we analyzed the dynamics of the width and center of mass position of a soliton by the variational approximation method. Soliton may be reflected from each other or transmitted through or trapped. Initially, uncoupled solitons may form the coupled state if the kinetic energy of solitons less than the potential of attractive interaction between solitons but when its’ velocity above the critical velocity, the soliton will pass through each other easily. Meanwhile, a direct numerical simulation of CNLSE had been run to check the accuracy of the approximation. The result of the variational model gives a slightly similar pattern with direct numerical simulation of CNLSE by fixing the parameters for both solutions with the same value. The interaction of the vector soliton with Gaussian potential depends on the initial velocity and amplitude of the soliton and the strength of the external potential