Improved delay-dependent stability criteria for discrete-time stochastic neural networks with time-varying delays

AbstractThis letter, investigates the problem of mean square exponential stability for a class of discrete-time stochastic neural network with time-varying delays. By constructing a appropriate Lyapunov-Krasovskii functional, combining the stochastic stability theory, and the convex theory method, a delay-dependent exponential stability criteria is obtained in term of LMIs. Finally, a numerical example is exploited to show the usefulness of the results derived

Revisiting Vision Transformer from the View of Path Ensemble

Vision Transformers (ViTs) are normally regarded as a stack of transformer layers. In this work, we propose a novel view of ViTs showing that they can be seen as ensemble networks containing multiple parallel paths with different lengths. Specifically, we equivalently transform the traditional cascade of multi-head self-attention (MSA) and feed-forward network (FFN) into three parallel paths in each transformer layer. Then, we utilize the identity connection in our new transformer form and further transform the ViT into an explicit multi-path ensemble network. From the new perspective, these paths perform two functions: the first is to provide the feature for the classifier directly, and the second is to provide the lower-level feature representation for subsequent longer paths. We investigate the influence of each path for the final prediction and discover that some paths even pull down the performance. Therefore, we propose the path pruning and EnsembleScale skills for improvement, which cut out the underperforming paths and re-weight the ensemble components, respectively, to optimize the path combination and make the short paths focus on providing high-quality representation for subsequent paths. We also demonstrate that our path combination strategies can help ViTs go deeper and act as high-pass filters to filter out partial low-frequency signals. To further enhance the representation of paths served for subsequent paths, self-distillation is applied to transfer knowledge from the long paths to the short paths. This work calls for more future research to explain and design ViTs from new perspectives.Comment: Accepted by ICCV 2023, oral presentatio

Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigm

Parameter-Efficient Transfer Learning (PETL) aims at efficiently adapting large models pre-trained on massive data to downstream tasks with limited task-specific data. In view of the practicality of PETL, previous works focus on tuning a small set of parameters for each downstream task in an end-to-end manner while rarely considering the task distribution shift issue between the pre-training task and the downstream task. This paper proposes a novel two-stage paradigm, where the pre-trained model is first aligned to the target distribution. Then the task-relevant information is leveraged for effective adaptation. Specifically, the first stage narrows the task distribution shift by tuning the scale and shift in the LayerNorm layers. In the second stage, to efficiently learn the task-relevant information, we propose a Taylor expansion-based importance score to identify task-relevant channels for the downstream task and then only tune such a small portion of channels, making the adaptation to be parameter-efficient. Overall, we present a promising new direction for PETL, and the proposed paradigm achieves state-of-the-art performance on the average accuracy of 19 downstream tasks.Comment: 11 page

Open-World Weakly-Supervised Object Localization

While remarkable success has been achieved in weakly-supervised object localization (WSOL), current frameworks are not capable of locating objects of novel categories in open-world settings. To address this issue, we are the first to introduce a new weakly-supervised object localization task called OWSOL (Open-World Weakly-Supervised Object Localization). During training, all labeled data comes from known categories and, both known and novel categories exist in the unlabeled data. To handle such data, we propose a novel paradigm of contrastive representation co-learning using both labeled and unlabeled data to generate a complete G-CAM (Generalized Class Activation Map) for object localization, without the requirement of bounding box annotation. As no class label is available for the unlabelled data, we conduct clustering over the full training set and design a novel multiple semantic centroids-driven contrastive loss for representation learning. We re-organize two widely used datasets, i.e., ImageNet-1K and iNatLoc500, and propose OpenImages150 to serve as evaluation benchmarks for OWSOL. Extensive experiments demonstrate that the proposed method can surpass all baselines by a large margin. We believe that this work can shift the close-set localization towards the open-world setting and serve as a foundation for subsequent works. Code will be released at https://github.com/ryylcc/OWSOL

Assessing Callous-Unemotional Traits in Chinese Detained Boys: Factor Structure and Construct Validity of the Inventory of Callous-Unemotional Traits

The Inventory of Callous-Unemotional Traits (ICU) was designed to evaluate multiple facets of Callous-Unemotional (CU) traits in youths. However, no study has examined the factor structure and psychometrical properties of the ICU in Chinese detained juveniles. The current study assesses the factor structure, internal consistency and convergent validity of the ICU in 613 Chinese detained boys. Confirmatory factor analysis results indicated that the original three-factor model with 24 items showed an unacceptable fit to the data, however, the 11-item shortened version of the ICU (ICU-11) with callousness and uncaring dimensions showed the best fit. Moreover, the ICU-11 total score and factor scores had good and acceptable internal consistencies. The convergent and criterion validity of the ICU-11 was demonstrated by comparable and significant associations in the expected direction with relevant external criteria (e.g., psychopathy, aggression, and empathy). In conclusion, present findings indicated that the ICU-11 is a reliable and efficient instrument to replace the original ICU when assessing CU traits in the Chinese male detained juvenile sample.This work was supported by the National Natural Science Foundation of China (Grant Nos. 31800945 and 31400904) and Guangzhou University’s 2017 training program for young topnotch personnels (BJ201715)

SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels

Pre-trained vision transformers have strong representation benefits to various downstream tasks. Recently, many parameter-efficient fine-tuning (PEFT) methods have been proposed, and their experiments demonstrate that tuning only 1% of extra parameters could surpass full fine-tuning in low-data resource scenarios. However, these methods overlook the task-specific information when fine-tuning diverse downstream tasks. In this paper, we propose a simple yet effective method called "Salient Channel Tuning" (SCT) to leverage the task-specific information by forwarding the model with the task images to select partial channels in a feature map that enables us to tune only 1/8 channels leading to significantly lower parameter costs. Experiments outperform full fine-tuning on 18 out of 19 tasks in the VTAB-1K benchmark by adding only 0.11M parameters of the ViT-B, which is 780$\times$ fewer than its full fine-tuning counterpart. Furthermore, experiments on domain generalization and few-shot learning surpass other PEFT methods with lower parameter costs, demonstrating our proposed tuning technique's strong capability and effectiveness in the low-data regime.Comment: This work has been accepted by IJCV202

Edge-Termination and Core-Modification Effects of Hexagonal Nanosheet Graphene

[[abstract]]Optimized geometries and electronic structures of two different hexagonal grapheme nanosheets (HGNSs), with armchair (n-A-HGNS, n = 3–11) and zigzag (n-Z-HGNS, n = 1–8) edges have been calculated by using the GGA/PBE method implemented in the SIESTA package, with the DZP basis set, where n represents the number of peripheral rings. The computed HOMO-LUMO energy gap (Eg = ELUMO − EHOMO) decreases for fully H-terminated A- and Z-HGNSs with increasing n, i.e., with increasing nanosheet size and pπ-orbitals being widely delocalized over the sheet surface. The full terminations, calculated with various functional groups, including the electron-withdrawing (F-, Cl-, and CN-) and -donating (OH-, and SH-) substitutions, were addressed. Significant lowering of EHOMO and ELUMO was obtained for CN-terminated HGNS as compared to those for H-terminated ones due to the mesomeric effect. The calculated Eg value decreases with increasing n for all terminations, whereby for the SH-termination in HGNS, the termination effect becomes less significant with increasing n. Further, the calculation results for stabilities of HGNS oxides support the tendency toward the oxidative reactivity at the edge site of the sheet, which shows most pronounced C-C bond length alternation, by chemical modification. Physical properties of HGNSs with various numbers of the core-defects, which can be obtained by strong oxidation, were also investigated. Their structures can change drastically from planar to saddle-like shapes. These conformations could be used as stationary phases with controlled interaction in the separation methods such as HPLC and the other chemical analysis techniques.[[notice]]補正完畢[[incitationindex]]SCI[[booktype]]電子

High prevalence of thalassemia in migrant populations in Guangdong Province, China

BACKGROUND: The objectives of this study were to estimate the prevalence of thalassemia and to analyze the need for public health services for migrant populations in different cities in Guangdong Province, China. METHODS: A cross-sectional survey was conducted in 21 cities of Guangdong Province. Twenty-three types of a- and β-globin gene mutations were detected in a total of 14,230 pregnant women and 14,249 husbands. RESULTS: There was a 16.45% prevalence of thalassemia among the 28,479 individuals, and the prevalences of α-, β-, and combined α-/β- thalassemia were 12.03%, 3.80%, and 0.63%, respectively. Compared with the native city residents in the province, the migrants from within the province and the immigrants from outside the province had lower prevalences of thalassemia, but the prevalence values were >11%. CONCLUSIONS: The prevalence values for thalassemia gene mutations were high in all three population groups studied in Guangdong Province. The results indicate that all segments of the Guangdong population should be screened for thalassemia