MolCPT: Molecule Continuous Prompt Tuning to Generalize Molecular Representation Learning

Molecular representation learning is crucial for the problem of molecular property prediction, where graph neural networks (GNNs) serve as an effective solution due to their structure modeling capabilities. Since labeled data is often scarce and expensive to obtain, it is a great challenge for GNNs to generalize in the extensive molecular space. Recently, the training paradigm of "pre-train, fine-tune" has been leveraged to improve the generalization capabilities of GNNs. It uses self-supervised information to pre-train the GNN, and then performs fine-tuning to optimize the downstream task with just a few labels. However, pre-training does not always yield statistically significant improvement, especially for self-supervised learning with random structural masking. In fact, the molecular structure is characterized by motif subgraphs, which are frequently occurring and influence molecular properties. To leverage the task-related motifs, we propose a novel paradigm of "pre-train, prompt, fine-tune" for molecular representation learning, named molecule continuous prompt tuning (MolCPT). MolCPT defines a motif prompting function that uses the pre-trained model to project the standalone input into an expressive prompt. The prompt effectively augments the molecular graph with meaningful motifs in the continuous representation space; this provides more structural patterns to aid the downstream classifier in identifying molecular properties. Extensive experiments on several benchmark datasets show that MolCPT efficiently generalizes pre-trained GNNs for molecular property prediction, with or without a few fine-tuning steps

Heterophily-Based Graph Neural Network for Imbalanced Classification

Graph neural networks (GNNs) have shown promise in addressing graph-related problems, including node classification. However, conventional GNNs assume an even distribution of data across classes, which is often not the case in real-world scenarios, where certain classes are severely underrepresented. This leads to suboptimal performance of standard GNNs on imbalanced graphs. In this paper, we introduce a unique approach that tackles imbalanced classification on graphs by considering graph heterophily. We investigate the intricate relationship between class imbalance and graph heterophily, revealing that minority classes not only exhibit a scarcity of samples but also manifest lower levels of homophily, facilitating the propagation of erroneous information among neighboring nodes. Drawing upon this insight, we propose an efficient method, called Fast Im-GBK, which integrates an imbalance classification strategy with heterophily-aware GNNs to effectively address the class imbalance problem while significantly reducing training time. Our experiments on real-world graphs demonstrate our model's superiority in classification performance and efficiency for node classification tasks compared to existing baselines.Comment: Accepted by Twelfth International Conference on Complex Networks & Their Application

From Data Inferring to Physics Representing: A Novel Mobile MIMO Channel Prediction Scheme Based on Neural ODE

In this paper, we propose an innovative learning-based channel prediction scheme so as to achieve higher prediction accuracy and reduce the requirements of huge amount and strict sequential format of channel data. Inspired by the idea of the neural ordinary differential equation (Neural ODE), we first prove that the channel prediction problem can be modeled as an ODE problem with a known initial value through analyzing the physical process of electromagnetic wave propagation within a varying space. Then, we design a novel physics-inspired spatial channel gradient network (SCGNet), which represents the derivative process of channel varying as a special neural network and can obtain the gradients at any relative displacement needed for the ODE solving. With the SCGNet, the static channel at any location served by the base station is accurately inferred through consecutive propagation and integration. Finally, we design an efficient recurrent positioning algorithm based on some prior knowledge of user mobility to obtain the velocity vector, and propose an approximate Doppler compensation method to make up the instantaneous angular-delay domain channel. Only discrete historical channel data is needed for the training, whereas only a few fresh channel measurements is needed for the prediction, which ensures the scheme's practicability

Improving Fairness for Data Valuation in Horizontal Federated Learning

Federated learning is an emerging decentralized machine learning scheme that allows multiple data owners to work collaboratively while ensuring data privacy. The success of federated learning depends largely on the participation of data owners. To sustain and encourage data owners' participation, it is crucial to fairly evaluate the quality of the data provided by the data owners and reward them correspondingly. Federated Shapley value, recently proposed by Wang et al. [Federated Learning, 2020], is a measure for data value under the framework of federated learning that satisfies many desired properties for data valuation. However, there are still factors of potential unfairness in the design of federated Shapley value because two data owners with the same local data may not receive the same evaluation. We propose a new measure called completed federated Shapley value to improve the fairness of federated Shapley value. The design depends on completing a matrix consisting of all the possible contributions by different subsets of the data owners. It is shown under mild conditions that this matrix is approximately low-rank by leveraging concepts and tools from optimization. Both theoretical analysis and empirical evaluation verify that the proposed measure does improve fairness in many circumstances

Tetris: A compilation Framework for VQE Applications

Quantum computing has shown promise in solving complex problems by leveraging the principles of superposition and entanglement. The Variational Quantum Eigensolver (VQE) algorithm stands as a pivotal approach in the realm of quantum algorithms, enabling the simulation of quantum systems on quantum hardware. In this paper, we introduce two innovative techniques, namely "Tetris" and "Fast Bridging," designed to enhance the efficiency and effectiveness of VQE tasks. The "Tetris" technique addresses a crucial aspect of VQE optimization by unveiling cancellation opportunities within the logical circuit phase of UCCSD ansatz. Tetris demonstrates a remarkable reduction up to 20% in CNOT gate counts, about 119048 CNOT gates, and 30% depth reduction compared to the state-of-the-art compiler 'Paulihedral'. In addition to Tetris, we present the "Fast Bridging" technique as an alternative to the conventional qubit routing methods that heavily rely on swap operations. The fast bridging offers a novel approach to qubit routing, mitigating the limitations associated with swap-heavy routing. By integrating the fast bridging into the VQE framework, we observe further reductions in CNOT gate counts and circuit depth. The bridging technique can achieve up to 27% CNOT gate reduction in the QAOA application. Through a combination of Tetris and the fast bridging, we present a comprehensive strategy for enhancing VQE performance. Our experimental results showcase the effectiveness of Tetris in uncovering cancellation opportunities and demonstrate the symbiotic relationship between Tetris and the fast bridging in minimizing gate counts and circuit depth. This paper contributes not only to the advancement of VQE techniques but also to the broader field of quantum algorithm optimization

Lawyer LLaMA Technical Report

Large Language Models (LLMs), like LLaMA, have exhibited remarkable performances across various tasks. Nevertheless, when deployed to specific domains such as law or medicine, the models still confront the challenge of a deficiency in domain-specific knowledge and an inadequate capability to leverage that knowledge to resolve domain-related problems. In this paper, we focus on the legal domain and explore how to inject domain knowledge during the continual training stage and how to design proper supervised finetune tasks to help the model tackle practical issues. Moreover, to alleviate the hallucination problem during model's generation, we add a retrieval module and extract relevant articles before the model answers any queries. Augmenting with the extracted evidence, our model could generate more reliable responses. We release our data and model at https://github.com/AndrewZhe/lawyer-llama.Comment: Work in progres

The temporal signature of self: Temporal measures of restingâ state EEG predict selfâ consciousness

The self is the core of our mental life. Previous investigations have demonstrated a strong neural overlap between selfâ related activity and resting state activity. This suggests that information about selfâ relatedness is encoded in our brain’s spontaneous activity. The exact neuronal mechanisms of such â restâ self containment,â however, remain unclear. The present EEG study investigated temporal measures of resting state EEG to relate them to selfâ consciousness. This was obtained with the selfâ consciousness scale (SCS) which measures Private, Public, and Social dimensions of self. We demonstrate positive correlations between Private selfâ consciousness and three temporal measures of resting state activity: scaleâ free activity as indexed by the powerâ law exponent (PLE), the autoâ correlation window (ACW), and modulation index (MI). Specifically, higher PLE, longer ACW, and stronger MI were related to higher degrees of Private selfâ consciousness. Finally, conducting eLORETA for spatial tomography, we found significant correlation of Private selfâ consciousness with activity in cortical midline structures such as the perigenual anterior cingulate cortex and posterior cingulate cortex. These results were reinforced with a dataâ driven analysis; a machine learning algorithm accurately predicted an individual as having a â highâ or â lowâ Private selfâ consciousness score based on these measures of the brain’s spatiotemporal structure. In conclusion, our results demonstrate that Private selfâ consciousness is related to the temporal structure of resting state activity as featured by temporal nestedness (PLE), temporal continuity (ACW), and temporal integration (MI). Our results support the hypothesis that selfâ related information is temporally contained in the brain’s resting state. â Restâ self containmentâ can thus be featured by a temporal signature.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/147871/1/hbm24412.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/147871/2/hbm24412_am.pd

Disrupted neural variability during propofol‐induced sedation and unconsciousness

Variability quenching is a widespread neural phenomenon in which trial‐to‐trial variability (TTV) of neural activity is reduced by repeated presentations of a sensory stimulus. However, its neural mechanism and functional significance remain poorly understood. Recurrent network dynamics are suggested as a candidate mechanism of TTV, and they play a key role in consciousness. We thus asked whether the variability‐quenching phenomenon is related to the level of consciousness. We hypothesized that TTV reduction would be compromised during reduced level of consciousness by propofol anesthetics. We recorded functional magnetic resonance imaging signals of resting‐state and stimulus‐induced activities in three conditions: wakefulness, sedation, and unconsciousness (i.e., deep anesthesia). We measured the average (trial‐to‐trial mean, TTM) and variability (TTV) of auditory stimulus‐induced activity under the three conditions. We also examined another form of neural variability (temporal variability, TV), which quantifies the overall dynamic range of ongoing neural activity across time, during both the resting‐state and the task. We found that (a) TTM deceased gradually from wakefulness through sedation to anesthesia, (b) stimulus‐induced TTV reduction normally seen during wakefulness was abolished during both sedation and anesthesia, and (c) TV increased in the task state as compared to resting‐state during both wakefulness and sedation, but not anesthesia. Together, our results reveal distinct effects of propofol on the two forms of neural variability (TTV and TV). They imply that the anesthetic disrupts recurrent network dynamics, thus prevents the stabilization of cortical activity states. These findings shed new light on the temporal dynamics of neuronal variability and its alteration during anesthetic‐induced unconsciousness.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/146388/1/hbm24304_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/146388/2/hbm24304.pd

A practical low-cost approach to build membrane electrode assemblies using decal transfer technique

Membrane electrode assembly (MEA) is the heart of a proton exchange membrane (PEM) fuel cell. In this paper, a practical and low-cost approach to build membrane electrode assemblies in lab using decal transfer technique is described and verified. The procedure is based on current literatures but using different materials for convenience and cost-saving. All devices and tools used are easy to find in normal labs and all materials used are affordable and easy to obtain from market. Two Membrane electrode assemblies are built and tested using a PEM fuel cell test and control system. Two MEAs showed voltages of about 0.82V and 0.70V respectively, measured with a digital multimeter, which are lower than that of the commercial MEA. It is found that the unscratched MEA with more catalyst has a lower voltage, but can generate higher current density and much water. Some water oozed through the bipolar plates which usually is not seen in a single cell tes

TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise

Large Language Models (LLMs) exhibit impressive reasoning and data augmentation capabilities in various NLP tasks. However, what about small models? In this work, we propose TeacherLM-7.1B, capable of annotating relevant fundamentals, chain of thought, and common mistakes for most NLP samples, which makes annotation more than just an answer, thus allowing other models to learn "why" instead of just "what". The TeacherLM-7.1B model achieved a zero-shot score of 52.3 on MMLU, surpassing most models with over 100B parameters. Even more remarkable is its data augmentation ability. Based on TeacherLM-7.1B, we augmented 58 NLP datasets and taught various student models with different parameters from OPT and BLOOM series in a multi-task setting. The experimental results indicate that the data augmentation provided by TeacherLM has brought significant benefits. We will release the TeacherLM series of models and augmented datasets as open-source.Comment: 5 figures, 15 page