Deep Span Representations for Named Entity Recognition

Span-based models are one of the most straightforward methods for named entity recognition (NER). Existing span-based NER systems shallowly aggregate the token representations to span representations. However, this typically results in significant ineffectiveness for long-span entities, a coupling between the representations of overlapping spans, and ultimately a performance degradation. In this study, we propose DSpERT (Deep Span Encoder Representations from Transformers), which comprises a standard Transformer and a span Transformer. The latter uses low-layered span representations as queries, and aggregates the token representations as keys and values, layer by layer from bottom to top. Thus, DSpERT produces span representations of deep semantics. With weight initialization from pretrained language models, DSpERT achieves performance higher than or competitive with recent state-of-the-art systems on eight NER benchmarks. Experimental results verify the importance of the depth for span representations, and show that DSpERT performs particularly well on long-span entities and nested structures. Further, the deep span representations are well structured and easily separable in the feature space

Efficient Mixed Transformer for Single Image Super-Resolution

Recently, Transformer-based methods have achieved impressive results in single image super-resolution (SISR). However, the lack of locality mechanism and high complexity limit their application in the field of super-resolution (SR). To solve these problems, we propose a new method, Efficient Mixed Transformer (EMT) in this study. Specifically, we propose the Mixed Transformer Block (MTB), consisting of multiple consecutive transformer layers, in some of which the Pixel Mixer (PM) is used to replace the Self-Attention (SA). PM can enhance the local knowledge aggregation with pixel shifting operations. At the same time, no additional complexity is introduced as PM has no parameters and floating-point operations. Moreover, we employ striped window for SA (SWSA) to gain an efficient global dependency modelling by utilizing image anisotropy. Experimental results show that EMT outperforms the existing methods on benchmark dataset and achieved state-of-the-art performance. The Code is available at https://github. com/Fried-Rice-Lab/EMT.git.Comment: Super-resolution, Long-range attention, Transformer, Localit

A Localization-to-Segmentation Framework for Automatic Tumor Segmentation in Whole-Body PET/CT Images

Fluorodeoxyglucose (FDG) positron emission tomography (PET) combined with computed tomography (CT) is considered the primary solution for detecting some cancers, such as lung cancer and melanoma. Automatic segmentation of tumors in PET/CT images can help reduce doctors' workload, thereby improving diagnostic quality. However, precise tumor segmentation is challenging due to the small size of many tumors and the similarity of high-uptake normal areas to the tumor regions. To address these issues, this paper proposes a localization-to-segmentation framework (L2SNet) for precise tumor segmentation. L2SNet first localizes the possible lesions in the lesion localization phase and then uses the location cues to shape the segmentation results in the lesion segmentation phase. To further improve the segmentation performance of L2SNet, we design an adaptive threshold scheme that takes the segmentation results of the two phases into consideration. The experiments with the MICCAI 2023 Automated Lesion Segmentation in Whole-Body FDG-PET/CT challenge dataset show that our method achieved a competitive result and was ranked in the top 7 methods on the preliminary test set. Our work is available at: https://github.com/MedCAI/L2SNet.Comment: 7 pages,3 figure

A new result on observer-based sliding mode control design for a class of uncertain Ito^ stochastic delay systems

© 2017 The Franklin Institute This paper develops a new observer-based sliding mode control (SMC) scheme for a general class of Ito^ stochastic delay systems (SDS). The key merit of the presented scheme lies in its simplicity and integrity in design process of the traditional sliding mode observer (SMO) strategy, i.e., the state observer and sliding surface design as well as the associated sliding mode controller synthesis. For guaranteeing to use the scheme, a new LMIs-based criterion is established to ensure the exponential stability of the underlying sliding mode dynamics (SMDs) in mean-square sense with H∞ performance. A bench test example is provided to numerically demonstrate the efficacy of the scheme and illustrate the application procedure for potential readers/users with interest in their ad hoc applications and methodology expansion

SimpleX: A Simple and Strong Baseline for Collaborative Filtering

Collaborative filtering (CF) is a widely studied research topic in recommender systems. The learning of a CF model generally depends on three major components, namely interaction encoder, loss function, and negative sampling. While many existing studies focus on the design of more powerful interaction encoders, the impacts of loss functions and negative sampling ratios have not yet been well explored. In this work, we show that the choice of loss function as well as negative sampling ratio is equivalently important. More specifically, we propose the cosine contrastive loss (CCL) and further incorporate it to a simple unified CF model, dubbed SimpleX. Extensive experiments have been conducted on 11 benchmark datasets and compared with 29 existing CF models in total. Surprisingly, the results show that, under our CCL loss and a large negative sampling ratio, SimpleX can surpass most sophisticated state-of-the-art models by a large margin (e.g., max 48.5% improvement in NDCG@20 over LightGCN). We believe that SimpleX could not only serve as a simple strong baseline to foster future research on CF, but also shed light on the potential research direction towards improving loss function and negative sampling. Our source code will be available at https://reczoo.github.io/SimpleX.Comment: Accepted by CIKM 2021. Code available at https://reczoo.github.io/Simple

Business Process Decomposition based on Service Relevance Mining

Abstrac

Image Super-Resolution using Efficient Striped Window Transformer

Transformers have achieved remarkable results in single-image super-resolution (SR). However, the challenge of balancing model performance and complexity has hindered their application in lightweight SR (LSR). To tackle this challenge, we propose an efficient striped window transformer (ESWT). We revisit the normalization layer in the transformer and design a concise and efficient transformer structure to build the ESWT. Furthermore, we introduce a striped window mechanism to model long-term dependencies more efficiently. To fully exploit the potential of the ESWT, we propose a novel flexible window training strategy that can improve the performance of the ESWT without additional cost. Extensive experiments show that ESWT outperforms state-of-the-art LSR transformers, and achieves a better trade-off between model performance and complexity. The ESWT requires fewer parameters, incurs faster inference, smaller FLOPs, and less memory consumption, making it a promising solution for LSR.Comment: SOTA lightweight super-resolution transformer. 8 pages, 9 figures and 6 tables. The Code is available at https://github.com/Fried-Rice-Lab/FriedRiceLa

Ground clutter mitigation for slow-time MIMO radar using independent component analysis

The detection of low, slow and small (LSS) targets, such as small drones, is a developing area of research in radar, wherein the presence of ground clutter can be quite challenging. LSS targets, because of their unusual flying mode, can be easily shadowed by ground clutter, leading to poor radar detection performance. In this study, we investigated the feasibility and performance of a ground clutter mitigation method combining slow-time multiple-input multiple-output (st-MIMO) waveforms and independent component analysis (ICA) in a ground-based MIMO radar focusing on LSS target detection. The modeling of ground clutter under the framework of st-MIMO was first defined. Combining the spatial and temporal steering vector of st-MIMO, a universal signal model including the target, ground clutter, and noise was established. The compliance of the signal model for conducting ICA to separate the target was analyzed. Based on this, a st-MIMO-ICA processing scheme was proposed to mitigate ground clutter. The effectiveness of the proposed method was verified with simulation and experimental data collected from an S-band st-MIMO radar system with a desirable target output signal-to-clutter-plus-noise ratio (SCNR). This work can shed light on the use of ground clutter mitigation techniques for MIMO radar to tackle LSS targets