TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers

CutMix is a popular augmentation technique commonly used for training modern convolutional and transformer vision networks. It was originally designed to encourage Convolution Neural Networks (CNNs) to focus more on an image's global context instead of local information, which greatly improves the performance of CNNs. However, we found it to have limited benefits for transformer-based architectures that naturally have a global receptive field. In this paper, we propose a novel data augmentation technique TokenMix to improve the performance of vision transformers. TokenMix mixes two images at token level via partitioning the mixing region into multiple separated parts. Besides, we show that the mixed learning target in CutMix, a linear combination of a pair of the ground truth labels, might be inaccurate and sometimes counter-intuitive. To obtain a more suitable target, we propose to assign the target score according to the content-based neural activation maps of the two images from a pre-trained teacher model, which does not need to have high performance. With plenty of experiments on various vision transformer architectures, we show that our proposed TokenMix helps vision transformers focus on the foreground area to infer the classes and enhances their robustness to occlusion, with consistent performance gains. Notably, we improve DeiT-T/S/B with +1% ImageNet top-1 accuracy. Besides, TokenMix enjoys longer training, which achieves 81.2% top-1 accuracy on ImageNet with DeiT-S trained for 400 epochs. Code is available at https://github.com/Sense-X/TokenMix.Comment: ECCV 2022; Code: https://github.com/Sense-X/TokenMi

LEO Satellite-Enabled Grant-Free Random Access with MIMO-OTFS

This paper investigates joint channel estimation and device activity detection in the LEO satellite-enabled grant-free random access systems with large differential delay and Doppler shift. In addition, the multiple-input multiple-output (MIMO) with orthogonal time-frequency space modulation (OTFS) is utilized to combat the dynamics of the terrestrial-satellite link. To simplify the computation process, we estimate the channel tensor in parallel along the delay dimension. Then, the deep learning and expectation-maximization approach are integrated into the generalized approximate message passing with cross-correlation--based Gaussian prior to capture the channel sparsity in the delay-Doppler-angle domain and learn the hyperparameters. Finally, active devices are detected by computing energy of the estimated channel. Simulation results demonstrate that the proposed algorithms outperform conventional methods.Comment: This paper has been accepted for presentation at the IEEE GLOBECOM 2022. arXiv admin note: text overlap with arXiv:2202.1305

Interlayer Electronic Coupling in Arbitrarily Stacked MoS₂ Bilayers Controlled by Interlayer S–S Interaction

The vertically heterostructured MoS₂ bilayers display a wide range of lattice registry relative to bulk MoS₂ through a single or combined in-plane displacement, out-of-plane displacement, and in-plane rotation. Here using density functional theory and numerical structural analysis, we examine both the atomic and electronic structures of the arbitrarily stacked MoS₂ bilayers that form either commensurate or incommensurate superstructures. Our analysis shows that the interlayer electronic coupling between two MoS₂ layers yields an indirect band gap that varies with the lattice registry, thus confirming the previous theoretical findings. The variation of the coupling strength and the indirect band gap with respect to the lattice registry can be attributed to the change in the mean interlayer sulfur–sulfur distance upon displacement. Importantly, our analysis further shows when the twisted MoS₂ bilayers form an incommensurate structure, the interlayer sulfur–sulfur distance is uniformly sampled in-plane and yields a distribution independent of the incommensurate twist angle. When a commensurate structure is formed, while the distribution becomes angular dependent, the mean in-plane sulfur–sulfur distance is found nearly independent of the twist angle. Consequently, the magnitude of the indirect band gap in the bilayers exhibits a weak angular dependence, until the twist angle recovers the high symmetry stacking sequence when the gap changes significantly. Our analysis provides the thorough theoretical explanation to the recently measured photoluminescence spectroscopy in twisted MoS₂ bilayers and can form the basis for understanding the coupling in vertically heterostructured bilayers composed of other transition-metal dichalcogenide monolayers

Interpretable spatial identity neural network-based epidemic prediction

Abstract Epidemic spatial–temporal risk analysis, e.g., infectious number forecasting, is a mainstream task in the multivariate time series research field, which plays a crucial role in the public health management process. With the rise of deep learning methods, many studies have focused on the epidemic prediction problem. However, recent primary prediction techniques face two challenges: the overcomplicated model and unsatisfactory interpretability. Therefore, this paper proposes an Interpretable Spatial IDentity (ISID) neural network to predict infectious numbers at the regional weekly level, which employs a light model structure and provides post-hoc explanations. First, this paper streamlines the classical spatio-temporal identity model (STID) and retains the optional spatial identity matrix for learning the contagion relationship between regions. Second, the well-known SHapley Additive explanations (SHAP) method was adopted to interpret how the ISID model predicts with multivariate sliding-window time series input data. The prediction accuracy of ISID is compared with several models in the experimental study, and the results show that the proposed ISID model achieves satisfactory epidemic prediction performance. Furthermore, the SHAP result demonstrates that the ISID pays particular attention to the most proximate and remote data in the input sequence (typically 20 steps long) while paying little attention to the intermediate steps. This study contributes to reliable and interpretable epidemic prediction through a more coherent approach for public health experts

Stretchable One-Dimensional Conductors for Wearable Applications.

Continuous, one-dimensional (1D) stretchable conductors have attracted significant attention for the development of wearables and soft-matter electronics. Through the use of advanced spinning, printing, and textile technologies, 1D stretchable conductors in the forms of fibers, wires, and yarns can be designed and engineered to meet the demanding requirements for different wearable applications. Several crucial parameters, such as microarchitecture, conductivity, stretchability, and scalability, play essential roles in designing and developing wearable devices and intelligent textiles. Methodologies and fabrication processes have successfully realized 1D conductors that are highly conductive, strong, lightweight, stretchable, and conformable and can be readily integrated with common fabrics and soft matter. This review summarizes the latest advances in continuous, 1D stretchable conductors and emphasizes recent developments in materials, methodologies, fabrication processes, and strategies geared toward applications in electrical interconnects, mechanical sensors, actuators, and heaters. This review classifies 1D conductors into three categories on the basis of their electrical responses: (1) rigid 1D conductors, (2) piezoresistive 1D conductors, and (3) resistance-stable 1D conductors. This review also evaluates the present challenges in these areas and presents perspectives for improving the performance of stretchable 1D conductors for wearable textile and flexible electronic applications

Hypolipidemic and Antithrombotic Effect of 6′-<i>O</i>-Caffeoylarbutin from <i>Vaccinium dunalianum</i> Based on Zebrafish Model, Network Pharmacology, and Molecular Docking

Vaccinium dunalianum leaf buds make one of the most commonly used herbal teas of the Yi people in China, which is used to treat articular rheumatism, relax tendons, and stimulates blood circulation in the body. In addition, 6′-O-caffeoylarbutin (CA) is a standardized extract of V. dunalianum, which has been found in dried leaf buds, reaching levels of up to 31.76%. Because of the uncommon phenomenon, it is suggested that CA may have a potential therapeutic role in hyperlipidemia and thrombosis. This study was designed to study the efficacy of CA on treating hyperlipidemia and thrombosis and the possible mechanisms behind these effects. Hyperlipidemia and thrombosis zebrafish models were treated with CA to observe variations of the integrated optical density within the vessels and the intensity of erythrocyte staining within the hearts. The possible mechanisms were explored using network pharmacology and molecular docking. The results demonstrate that CA exhibits an excellent hypolipidemic effect on zebrafish at concentrations ranging from 3.0 to 30.0 μg/mL and shows thrombosis inhibitory activity in zebrafish at a concentration of 30.0 μg/mL, with an inhibition rate of 44%. Moreover, network pharmacological research shows that MMP9, RELA, MMP2, PRKCA, HSP90AA1, and APP are major targets of CA for therapy of hyperlipidemia and thrombosis, and may relate to pathways in cancer, chemical carcinogenesis-receptor activation, estrogen signaling pathway, and the AGE–RAGE signaling pathway in diabetic complications