DA-TransUNet: integrating spatial and channel dual attention with transformer U-net for medical image segmentation

Accurate medical image segmentation is critical for disease quantification and treatment evaluation. While traditional U-Net architectures and their transformer-integrated variants excel in automated segmentation tasks. Existing models also struggle with parameter efficiency and computational complexity, often due to the extensive use of Transformers. However, they lack the ability to harness the image’s intrinsic position and channel features. Research employing Dual Attention mechanisms of position and channel have not been specifically optimized for the high-detail demands of medical images. To address these issues, this study proposes a novel deep medical image segmentation framework, called DA-TransUNet, aiming to integrate the Transformer and dual attention block (DA-Block) into the traditional U-shaped architecture. Also, DA-TransUNet tailored for the high-detail requirements of medical images, optimizes the intermittent channels of Dual Attention (DA) and employs DA in each skip-connection to effectively filter out irrelevant information. This integration significantly enhances the model’s capability to extract features, thereby improving the performance of medical image segmentation. DA-TransUNet is validated in medical image segmentation tasks, consistently outperforming state-of-the-art techniques across 5 datasets. In summary, DA-TransUNet has made significant strides in medical image segmentation, offering new insights into existing techniques. It strengthens model performance from the perspective of image features, thereby advancing the development of high-precision automated medical image diagnosis. The codes and parameters of our model will be publicly available at https://github.com/SUN-1024/DA-TransUnet

MVP: Multi-task Supervised Pre-training for Natural Language Generation

Pre-trained language models (PLMs) have achieved remarkable success in natural language generation (NLG) tasks. Up to now, most NLG-oriented PLMs are pre-trained in an unsupervised manner using the large-scale general corpus. In the meanwhile, an increasing number of models pre-trained with labeled data (i.e. "supervised pre-training") showcase superior performance compared to unsupervised pre-trained models. Motivated by the success of supervised pre-training, we propose Multi-task superVised Pre-training (MVP) for natural language generation. We collect a large-scale natural language generation corpus, MVPCorpus, from $77$ datasets over $11$ diverse NLG tasks. Then we unify these examples into a general text-to-text format to pre-train the text generation model MVP in a supervised manner. For each task, we further pre-train specific soft prompts to stimulate the model's capacity to perform a specific task. Our MVP model can be seen as a practice that utilizes recent instruction tuning on relatively small PLMs. Extensive experiments have demonstrated the effectiveness and generality of our MVP model in a number of NLG tasks, which achieves state-of-the-art performance on $13$ out of $17$ datasets, outperforming BART by $9.3\%$ and Flan-T5 by $5.8\%$.Comment: Accepted by ACL 202

Co-exposure of C60 fullerene with benzo[a]pyrene results in enhanced biological effects in cells as determined by Fourier-transform infrared spectroscopy

C60 fullerene (C60) is a promising manufactured carbon-based nanoparticles (NPs). With an increasing number of applications, it is being found in the environment. In addition, C60 is likely to associate with other environmental toxic contaminants. How such interactions with C60 can impact on the environmental fate, transport and bioavailability of toxicants remains unknown. Benzo[a]pyrene (B[a]P) is a polycyclic aromatic hydrocarbon (PAH). Herein, two cell lines (fish gill or MCF-7 cells) were employed to explore the biological impacts of co-exposure to C60 and B[a]P. Post-exposure cells were interrogated using Fourier-transformation infrared (FTIR) microspectroscopy. By inputting spectral data into principal component analysis and linear discriminant analysis, data reduction allowed for visualisation of cell categorization and identification of wavenumber-related biomarkers corresponding to cellular alterations. Our results indicate that low-dose C60 increases B[a]P-induced alterations, while C60 at high concentrations reduces these effects. We also found that although C60 co-exposure increases B[a]P-induced CYP1A1 induction, co-exposure seemingly attenuates the levels of oxidative damage induced by either agent singly. This suggests that interactions between environmental NPs and contaminants are complex and unpredictable

Ensemble learning based defect detection of laser sintering

In rapid development, Selective Laser Sintering (SLS) creates prototypes by processing industrial materials, for example, polymers. Such materials are usually in powder form and fused by a laser beam. The manufacturing quality depends on the interaction between a high-energy laser beam and the powdered material. However, in-homogeneous temperature distribution, unstable laser powder, and inconsistent powder densities can cause defects in the final product, for example, Powder Bed Defects. Such factors can lead to irregularities, for example, warping, distortion, and inadequate powder bed fusion. These irregularities may affect the profitable SLS production. Consequently, detecting powder bed defects requires automation. An ensemble learning-based approach is proposed for detecting defects in SLS powder bed images from this perceptive. The proposed approach first pre-processes the images to reduce the computational complexity. Then, the Convolutional Neural Network (CNN) based ensembled models (off-the-shelf CNN, bagged CNN, and boosted CNN) are implemented and compared. The ensemble learning CNN (bagged and boosted CNN) is good for powder bed detection. The evaluation results indicate that the performance of bagged CNN is significant. It also indicates that preprocessing of the images, mainly cropping to the region of interest, improves the performance of the proposed approach. The training and testing accuracy of the bagged CNN is 96.1% and 95.1%, respectively.© 2023The Authors. IET Optoelectronics published by John Wiley& Sons Ltd on behalf of The Institution of Engineering and Technology. This is an open access article under the terms of the Creative Commons Attribution License,which permits use, distribution and reproduction in any medium, provided the original work is properly cited.fi=vertaisarvioitu|en=peerReviewed

Constrained Reinforcement Learning for Dynamic Material Handling

As one of the core parts of flexible manufacturing systems, material handling involves storage and transportation of materials between workstations with automated vehicles. The improvement in material handling can impulse the overall efficiency of the manufacturing system. However, the occurrence of dynamic events during the optimisation of task arrangements poses a challenge that requires adaptability and effectiveness. In this paper, we aim at the scheduling of automated guided vehicles for dynamic material handling. Motivated by some real-world scenarios, unknown new tasks and unexpected vehicle breakdowns are regarded as dynamic events in our problem. We formulate the problem as a constrained Markov decision process which takes into account tardiness and available vehicles as cumulative and instantaneous constraints, respectively. An adaptive constrained reinforcement learning algorithm that combines Lagrangian relaxation and invalid action masking, named RCPOM, is proposed to address the problem with two hybrid constraints. Moreover, a gym-like dynamic material handling simulator, named DMH-GYM, is developed and equipped with diverse problem instances, which can be used as benchmarks for dynamic material handling. Experimental results on the problem instances demonstrate the outstanding performance of our proposed approach compared with eight state-of-the-art constrained and non-constrained reinforcement learning algorithms, and widely used dispatching rules for material handling.Comment: accepted by the 2023 International Joint Conference on Neural Networks (IJCNN

BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models

Large language models (LLMs) have achieved dramatic proficiency over NLP tasks with normal length. Recently, multiple studies have committed to extending the context length and enhancing the long text modeling capabilities of LLMs. To comprehensively evaluate the long context ability of LLMs, we propose BAMBOO, a multi-task long context benchmark. BAMBOO has been designed with four principles: comprehensive capacity evaluation, avoidance of data contamination, accurate automatic evaluation, and different length levels. It consists of 10 datasets from 5 different long text understanding tasks, i.e. question answering, hallucination detection, text sorting, language modeling, and code completion, to cover core capacities and various domains of LLMs. We conduct experiments with five long context models on BAMBOO and further discuss four key research questions of long text. We also qualitatively analyze current long context models and point out future directions for enhancing long text modeling capacities. We release our data, prompts, and code at https://github.com/RUCAIBox/BAMBOO

HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models

Large language models (LLMs), such as ChatGPT, are prone to generate hallucinations, i.e., content that conflicts with the source or cannot be verified by the factual knowledge. To understand what types of content and to which extent LLMs are apt to hallucinate, we introduce the Hallucination Evaluation benchmark for Large Language Models (HaluEval), a large collection of generated and human-annotated hallucinated samples for evaluating the performance of LLMs in recognizing hallucination. To generate these samples, we propose a ChatGPT-based two-step framework, i.e., sampling-then-filtering. Besides, we also hire some human labelers to annotate the hallucinations in ChatGPT responses. The empirical results suggest that ChatGPT is likely to generate hallucinated content in specific topics by fabricating unverifiable information (i.e., about $19.5\%$ responses). Moreover, existing LLMs face great challenges in recognizing the hallucinations in texts. However, our experiments also prove that providing external knowledge or adding reasoning steps can help LLMs recognize hallucinations. Our benchmark can be accessed at https://github.com/RUCAIBox/HaluEval.Comment: Accepted to EMNLP 2023 Main Conference (Long Paper