Synchronization between Bidirectional Coupled Nonautonomous Delayed Cohen-Grossberg Neural Networks

Based on using suitable Lyapunov function and the properties of M-matrix, sufficient conditions for complete synchronization of bidirectional coupled nonautonomous Cohen-Grossberg neural networks are obtained. The methods for discussing synchronization avoid complicated error system of Cohen-Grossberg neural networks. Two numerical examples are given to show the effectiveness of the proposed synchronization method

VSA: Learning Varied-Size Window Attention in Vision Transformers

Attention within windows has been widely explored in vision transformers to balance the performance, computation complexity, and memory footprint. However, current models adopt a hand-crafted fixed-size window design, which restricts their capacity of modeling long-term dependencies and adapting to objects of different sizes. To address this drawback, we propose \textbf{V}aried-\textbf{S}ize Window \textbf{A}ttention (VSA) to learn adaptive window configurations from data. Specifically, based on the tokens within each default window, VSA employs a window regression module to predict the size and location of the target window, i.e., the attention area where the key and value tokens are sampled. By adopting VSA independently for each attention head, it can model long-term dependencies, capture rich context from diverse windows, and promote information exchange among overlapped windows. VSA is an easy-to-implement module that can replace the window attention in state-of-the-art representative models with minor modifications and negligible extra computational cost while improving their performance by a large margin, e.g., 1.1\% for Swin-T on ImageNet classification. In addition, the performance gain increases when using larger images for training and test. Experimental results on more downstream tasks, including object detection, instance segmentation, and semantic segmentation, further demonstrate the superiority of VSA over the vanilla window attention in dealing with objects of different sizes. The code will be released https://github.com/ViTAE-Transformer/ViTAE-VSA.Comment: 23 pages, 13 tables, and 5 figure

Vision Transformer with Quadrangle Attention

Window-based attention has become a popular choice in vision transformers due to its superior performance, lower computational complexity, and less memory footprint. However, the design of hand-crafted windows, which is data-agnostic, constrains the flexibility of transformers to adapt to objects of varying sizes, shapes, and orientations. To address this issue, we propose a novel quadrangle attention (QA) method that extends the window-based attention to a general quadrangle formulation. Our method employs an end-to-end learnable quadrangle regression module that predicts a transformation matrix to transform default windows into target quadrangles for token sampling and attention calculation, enabling the network to model various targets with different shapes and orientations and capture rich context information. We integrate QA into plain and hierarchical vision transformers to create a new architecture named QFormer, which offers minor code modifications and negligible extra computational cost. Extensive experiments on public benchmarks demonstrate that QFormer outperforms existing representative vision transformers on various vision tasks, including classification, object detection, semantic segmentation, and pose estimation. The code will be made publicly available at \href{https://github.com/ViTAE-Transformer/QFormer}{QFormer}.Comment: 15 pages, the extension of the ECCV 2022 paper (VSA: Learning Varied-Size Window Attention in Vision Transformers

Rethinking Hierarchies in Pre-trained Plain Vision Transformer

Self-supervised pre-training vision transformer (ViT) via masked image modeling (MIM) has been proven very effective. However, customized algorithms should be carefully designed for the hierarchical ViTs, e.g., GreenMIM, instead of using the vanilla and simple MAE for the plain ViT. More importantly, since these hierarchical ViTs cannot reuse the off-the-shelf pre-trained weights of the plain ViTs, the requirement of pre-training them leads to a massive amount of computational cost, thereby incurring both algorithmic and computational complexity. In this paper, we address this problem by proposing a novel idea of disentangling the hierarchical architecture design from the self-supervised pre-training. We transform the plain ViT into a hierarchical one with minimal changes. Technically, we change the stride of linear embedding layer from 16 to 4 and add convolution (or simple average) pooling layers between the transformer blocks, thereby reducing the feature size from 1/4 to 1/32 sequentially. Despite its simplicity, it outperforms the plain ViT baseline in classification, detection, and segmentation tasks on ImageNet, MS COCO, Cityscapes, and ADE20K benchmarks, respectively. We hope this preliminary study could draw more attention from the community on developing effective (hierarchical) ViTs while avoiding the pre-training cost by leveraging the off-the-shelf checkpoints. The code and models will be released at https://github.com/ViTAE-Transformer/HPViT.Comment: Tech report, work in progres

An immersogeometric formulation for free-surface flows with application to marine engineering problems

An immersogeometric formulation is proposed to simulate free-surface flows around structures with complex geometry. The fluid–fluid interface (air–water interface) is handled by the level set method, while the fluid–structure interface is handled through an immersogeometric approach by immersing structures into non-boundary-fitted meshes and enforcing Dirichlet boundary conditions weakly. Residual-based variational multiscale method (RBVMS) is employed to stabilize the coupled Navier–Stokes equations of incompressible flows and level set convection equation. Other level set techniques, including re-distancing and mass balancing, are also incorporated into the immersed formulation. Adaptive quadrature rule is used to better capture the geometry of the immersed structure boundary by accurately integrating the intersected background elements. Generalized-α role= presentation style= box-sizing: border-box; margin: 0px; padding: 0px; display: inline-block; line-height: normal; font-size: 16.2px; word-spacing: normal; overflow-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; position: relative; \u3eα method is adopted for time integration, which results in a two-stage predictor multi-corrector algorithm. GMRES solver preconditioned with block Jacobian matrices of individual fluid and level set subproblems is used for solving the coupled linear systems arising from the multi-corrector stage. The capability and accuracy of the proposed method are assessed by simulating three challenging marine engineering problems, which are a solitary wave impacting a stationary platform, dam break with an obstacle, and planing of a DTMB 5415 ship model. A refinement study is performed. The predictions of key quantities of interest by the proposed formulation are in good agreement with experimental results and boundary-fitted simulation results from others. The proposed formulation has great potential for wide applications in marine engineering problems

Stability and Bifurcation of a Class of Discrete-Time Cohen-Grossberg Neural Networks with Delays

A class of discrete-time Cohen-Grossberg neural networks with delays is investigated in this paper. By analyzing the corresponding characteristic equations, the asymptotical stability of the null solution and the existence of Neimark-Sacker bifurcations are discussed. By applying the normal form theory and the center manifold theorem, the direction of the Neimark-Sacker bifurcation and the stability of bifurcating periodic solutions are obtained. Numerical simulations are given to illustrate the obtained results

Gold substrate-induced single-mode lasing of GaN nanowires

We demonstrate a method for mode-selection by coupling a GaN nanowire laser to an underlying gold substrate. Multimode lasing of GaN nanowires is converted to single-mode behavior following placement onto a gold film. A mode-dependent loss is generated by the absorbing substrate to suppress multiple transverse-mode operation with a concomitant increase in lasing threshold of only ∼13%. This method provides greater flexibility in realizing practical single-mode nanowire lasers and offers insight into the design of metal-contacted nanoscale optoelectronics