Story Ending Generation with Incremental Encoding and Commonsense Knowledge

Generating a reasonable ending for a given story context, i.e., story ending generation, is a strong indication of story comprehension. This task requires not only to understand the context clues which play an important role in planning the plot but also to handle implicit knowledge to make a reasonable, coherent story. In this paper, we devise a novel model for story ending generation. The model adopts an incremental encoding scheme to represent context clues which are spanning in the story context. In addition, commonsense knowledge is applied through multi-source attention to facilitate story comprehension, and thus to help generate coherent and reasonable endings. Through building context clues and using implicit knowledge, the model is able to produce reasonable story endings. context clues implied in the post and make the inference based on it. Automatic and manual evaluation shows that our model can generate more reasonable story endings than state-of-the-art baselines.Comment: Accepted in AAAI201

LSPT: Long-term Spatial Prompt Tuning for Visual Representation Learning

Visual Prompt Tuning (VPT) techniques have gained prominence for their capacity to adapt pre-trained Vision Transformers (ViTs) to downstream visual tasks using specialized learnable tokens termed as prompts. Contemporary VPT methodologies, especially when employed with self-supervised vision transformers, often default to the introduction of new learnable prompts or gated prompt tokens predominantly sourced from the model's previous block. A pivotal oversight in such approaches is their failure to harness the potential of long-range previous blocks as sources of prompts within each self-supervised ViT. To bridge this crucial gap, we introduce Long-term Spatial Prompt Tuning (LSPT) - a revolutionary approach to visual representation learning. Drawing inspiration from the intricacies of the human brain, LSPT ingeniously incorporates long-term gated prompts. This feature serves as temporal coding, curbing the risk of forgetting parameters acquired from earlier blocks. Further enhancing its prowess, LSPT brings into play patch tokens, serving as spatial coding. This is strategically designed to perpetually amass class-conscious features, thereby fortifying the model's prowess in distinguishing and identifying visual categories. To validate the efficacy of our proposed method, we engaged in rigorous experimentation across 5 FGVC and 19 VTAB-1K benchmarks. Our empirical findings underscore the superiority of LSPT, showcasing its ability to set new benchmarks in visual prompt tuning performance

A Large-scale Medical Visual Task Adaptation Benchmark

Visual task adaptation has been demonstrated to be effective in adapting pre-trained Vision Transformers (ViTs) to general downstream visual tasks using specialized learnable layers or tokens. However, there is yet a large-scale benchmark to fully explore the effect of visual task adaptation on the realistic and important medical domain, particularly across diverse medical visual modalities, such as color images, X-ray, and CT. To close this gap, we present Med-VTAB, a large-scale Medical Visual Task Adaptation Benchmark consisting of 1.68 million medical images for diverse organs, modalities, and adaptation approaches. Based on Med-VTAB, we explore the scaling law of medical prompt tuning concerning tunable parameters and the generalizability of medical visual adaptation using non-medical/medical pre-train weights. Besides, we study the impact of patient ID out-of-distribution on medical visual adaptation, which is a real and challenging scenario. Furthermore, results from Med-VTAB indicate that a single pre-trained model falls short in medical task adaptation. Therefore, we introduce GMoE-Adapter, a novel method that combines medical and general pre-training weights through a gated mixture-of-experts adapter, achieving state-of-the-art results in medical visual task adaptation

Exploring Socially Shared Regulation with an AI Deep Learning Approach Using Multimodal Data

AbstractSocially shared regulation of learning (SSRL) is essential for the success of collaborative learning, yet learners often neglect needed regulation while facing challenges. In order to provide targeted support when needed, it is critical to identify the precise events that trigger regulation. Multimodal collaborative learning data may offer opportunities for this. This study aims to lay such a foundation by exploring the potential for using machine-learned models trained on multimodal data, including electrodermal activities (EDA), speech, and video, to detect the presence of SSRL-relevant process-level indicators in successful and less successful groups. The study involves thirty groups of secondary students (N=94) working collaboratively in five physics lessons. Considering the demonstrated positive results of machine-learned models, the advantages and limitations of the technical approach are discussed, and further development directions are suggested.Abstract Socially shared regulation of learning (SSRL) is essential for the success of collaborative learning, yet learners often neglect needed regulation while facing challenges. In order to provide targeted support when needed, it is critical to identify the precise events that trigger regulation. Multimodal collaborative learning data may offer opportunities for this. This study aims to lay such a foundation by exploring the potential for using machine-learned models trained on multimodal data, including electrodermal activities (EDA), speech, and video, to detect the presence of SSRL-relevant process-level indicators in successful and less successful groups. The study involves thirty groups of secondary students (N=94) working collaboratively in five physics lessons. Considering the demonstrated positive results of machine-learned models, the advantages and limitations of the technical approach are discussed, and further development directions are suggested

Effect of cyclic compression on the micromechanical properties of a Zr-based metallic glass

In this study, the effect of cyclic compression on the micromechanical properties of a Zr-based metallic glass (MG) was investigated via nanoindentation. Cyclic compression significantly softened the surface of the sample, with a maximum hardness loss of 19.93%. The number of cyclic compression passes had a greater effect on the hardness of the sample than the cyclic compression load. The elastic modulus exhibited a nonlinear variation upon increasing the cyclic loading or number of passes at a lower loading rate due to the coupling effect of loading rate and cyclic compression treatment. Then, the serration behavior and strain rate sensitivity analysis were applied. The calculated m-values obtained for MGs were all negative and gradually tended to zero upon further cyclic compression treatment. This demonstrated the weakening effect of cyclic compression on the strain rate sensitivity of MG, and the underlying mechanism was discussed. This study provides a process reference for studying the fatigue failure behaviors of MGs from the perspective of mechanical properties, which is useful for understanding their fatigue generation

EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Model

Self-supervised learning has emerged as a highly effective approach in the fields of natural language processing and computer vision. It is also applicable to brain signals such as electroencephalography (EEG) data, given the abundance of available unlabeled data that exist in a wide spectrum of real-world medical applications ranging from seizure detection to wave analysis. The existing works leveraging self-supervised learning on EEG modeling mainly focus on pretraining upon each individual dataset corresponding to a single downstream task, which cannot leverage the power of abundant data, and they may derive sub-optimal solutions with a lack of generalization. Moreover, these methods rely on end-to-end model learning which is not easy for humans to understand. In this paper, we present a novel EEG foundation model, namely EEGFormer, pretrained on large-scale compound EEG data. The pretrained model cannot only learn universal representations on EEG signals with adaptable performance on various downstream tasks but also provide interpretable outcomes of the useful patterns within the data. To validate the effectiveness of our model, we extensively evaluate it on various downstream tasks and assess the performance under different transfer settings. Furthermore, we demonstrate how the learned model exhibits transferable anomaly detection performance and provides valuable interpretability of the acquired patterns via self-supervised learning.Comment: A preprint version of an ongoing wor

ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling

Modeling continuous-time dynamics on irregular time series is critical to account for data evolution and correlations that occur continuously. Traditional methods including recurrent neural networks or Transformer models leverage inductive bias via powerful neural architectures to capture complex patterns. However, due to their discrete characteristic, they have limitations in generalizing to continuous-time data paradigms. Though neural ordinary differential equations (Neural ODEs) and their variants have shown promising results in dealing with irregular time series, they often fail to capture the intricate correlations within these sequences. It is challenging yet demanding to concurrently model the relationship between input data points and capture the dynamic changes of the continuous-time system. To tackle this problem, we propose ContiFormer that extends the relation modeling of vanilla Transformer to the continuous-time domain, which explicitly incorporates the modeling abilities of continuous dynamics of Neural ODEs with the attention mechanism of Transformers. We mathematically characterize the expressive power of ContiFormer and illustrate that, by curated designs of function hypothesis, many Transformer variants specialized in irregular time series modeling can be covered as a special case of ContiFormer. A wide range of experiments on both synthetic and real-world datasets have illustrated the superior modeling capacities and prediction performance of ContiFormer on irregular time series data. The project link is https://seqml.github.io/contiformer/.Comment: Neurips 2023 Poste

Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals

Seeing is believing, however, the underlying mechanism of how human visual perceptions are intertwined with our cognitions is still a mystery. Thanks to the recent advances in both neuroscience and artificial intelligence, we have been able to record the visually evoked brain activities and mimic the visual perception ability through computational approaches. In this paper, we pay attention to visual stimuli reconstruction by reconstructing the observed images based on portably accessible brain signals, i.e., electroencephalography (EEG) data. Since EEG signals are dynamic in the time-series format and are notorious to be noisy, processing and extracting useful information requires more dedicated efforts; In this paper, we propose a comprehensive pipeline, named NeuroImagen, for reconstructing visual stimuli images from EEG signals. Specifically, we incorporate a novel multi-level perceptual information decoding to draw multi-grained outputs from the given EEG data. A latent diffusion model will then leverage the extracted information to reconstruct the high-resolution visual stimuli images. The experimental results have illustrated the effectiveness of image reconstruction and superior quantitative performance of our proposed method.Comment: A preprint version of an ongoing wor

Tumor-associated macrophages regulate gastric cancer cell invasion and metastasis through TGF beta 2/NF-kappa B/Kindlin-2 axis

Objective: Recent studies have shown that tumor-associated macrophages (TAMs) play an important role in cancer invasion and metastasis. Our previous studies have reported that TAMs promote the invasion and metastasis of gastric cancer (GC) cells through the Kindlin-2 pathway. However, the mechanism needs to be clarified. Methods: THP-1 monocytes were induced by PMA/interleukin (IL)-4/IL-13 to establish an efficient TAM model in vitro and M2 macrophages were isolated via flow cytometry. A dual luciferase reporter system and chromatin immunoprecipitation (ChIP) assay were used to investigate the mechanism of transforming growth factor beta 2 (TGF beta 2) regulating Kindlin-2 expression. Immunohistochemistry was used to study the relationships among TAM infiltration in human GC tissues, Kindlin-2 protein expression, clinicopathological parameters and prognosis in human GC tissues. A nude mouse oncogenesis model was used to verify the invasion and metastasis mechanisms in vivo. Results: We found that Kindlin-2 expression was upregulated at both mRNA and protein levels in GC cells cocultured with TAMs, associated with higher invasion rate. Kindlin-2 knockdown reduced the invasion rate of GC cells under coculture condition. TGF beta 2 secreted by TAMs regulated the expression of Kindlin-2 through the transcription factor NF-kappa B. TAMs thus participated in the progression of GC through the TGF beta 2/NF-kappa B/Kindlin-2 axis. Kindlin-2 expression and TAM infiltration were significantly positively correlated with TNM stage, and patients with high Kindlin-2 expression had significantly poorer overall survival than patients with low Kindlin-2 expression. Furthermore, Kindlin-2 promoted the invasion of GC cells in vivo. Conclusions: This study elucidates the mechanism of TAMs participating in GC cell invasion and metastasis through the TGF beta 2/NF-kappa B/Kindlin-2 axis, providing a possibility for new treatment options and approaches.Peer reviewe