660 research outputs found
MolCPT: Molecule Continuous Prompt Tuning to Generalize Molecular Representation Learning
Molecular representation learning is crucial for the problem of molecular
property prediction, where graph neural networks (GNNs) serve as an effective
solution due to their structure modeling capabilities. Since labeled data is
often scarce and expensive to obtain, it is a great challenge for GNNs to
generalize in the extensive molecular space. Recently, the training paradigm of
"pre-train, fine-tune" has been leveraged to improve the generalization
capabilities of GNNs. It uses self-supervised information to pre-train the GNN,
and then performs fine-tuning to optimize the downstream task with just a few
labels. However, pre-training does not always yield statistically significant
improvement, especially for self-supervised learning with random structural
masking. In fact, the molecular structure is characterized by motif subgraphs,
which are frequently occurring and influence molecular properties. To leverage
the task-related motifs, we propose a novel paradigm of "pre-train, prompt,
fine-tune" for molecular representation learning, named molecule continuous
prompt tuning (MolCPT). MolCPT defines a motif prompting function that uses the
pre-trained model to project the standalone input into an expressive prompt.
The prompt effectively augments the molecular graph with meaningful motifs in
the continuous representation space; this provides more structural patterns to
aid the downstream classifier in identifying molecular properties. Extensive
experiments on several benchmark datasets show that MolCPT efficiently
generalizes pre-trained GNNs for molecular property prediction, with or without
a few fine-tuning steps
From Data Inferring to Physics Representing: A Novel Mobile MIMO Channel Prediction Scheme Based on Neural ODE
In this paper, we propose an innovative learning-based channel prediction
scheme so as to achieve higher prediction accuracy and reduce the requirements
of huge amount and strict sequential format of channel data. Inspired by the
idea of the neural ordinary differential equation (Neural ODE), we first prove
that the channel prediction problem can be modeled as an ODE problem with a
known initial value through analyzing the physical process of electromagnetic
wave propagation within a varying space. Then, we design a novel
physics-inspired spatial channel gradient network (SCGNet), which represents
the derivative process of channel varying as a special neural network and can
obtain the gradients at any relative displacement needed for the ODE solving.
With the SCGNet, the static channel at any location served by the base station
is accurately inferred through consecutive propagation and integration.
Finally, we design an efficient recurrent positioning algorithm based on some
prior knowledge of user mobility to obtain the velocity vector, and propose an
approximate Doppler compensation method to make up the instantaneous
angular-delay domain channel. Only discrete historical channel data is needed
for the training, whereas only a few fresh channel measurements is needed for
the prediction, which ensures the scheme's practicability
Heterophily-Based Graph Neural Network for Imbalanced Classification
Graph neural networks (GNNs) have shown promise in addressing graph-related
problems, including node classification. However, conventional GNNs assume an
even distribution of data across classes, which is often not the case in
real-world scenarios, where certain classes are severely underrepresented. This
leads to suboptimal performance of standard GNNs on imbalanced graphs. In this
paper, we introduce a unique approach that tackles imbalanced classification on
graphs by considering graph heterophily. We investigate the intricate
relationship between class imbalance and graph heterophily, revealing that
minority classes not only exhibit a scarcity of samples but also manifest lower
levels of homophily, facilitating the propagation of erroneous information
among neighboring nodes. Drawing upon this insight, we propose an efficient
method, called Fast Im-GBK, which integrates an imbalance classification
strategy with heterophily-aware GNNs to effectively address the class imbalance
problem while significantly reducing training time. Our experiments on
real-world graphs demonstrate our model's superiority in classification
performance and efficiency for node classification tasks compared to existing
baselines.Comment: Accepted by Twelfth International Conference on Complex Networks &
Their Application
Survey of Computerized Adaptive Testing: A Machine Learning Perspective
Computerized Adaptive Testing (CAT) provides an efficient and tailored method
for assessing the proficiency of examinees, by dynamically adjusting test
questions based on their performance. Widely adopted across diverse fields like
education, healthcare, sports, and sociology, CAT has revolutionized testing
practices. While traditional methods rely on psychometrics and statistics, the
increasing complexity of large-scale testing has spurred the integration of
machine learning techniques. This paper aims to provide a machine
learning-focused survey on CAT, presenting a fresh perspective on this adaptive
testing method. By examining the test question selection algorithm at the heart
of CAT's adaptivity, we shed light on its functionality. Furthermore, we delve
into cognitive diagnosis models, question bank construction, and test control
within CAT, exploring how machine learning can optimize these components.
Through an analysis of current methods, strengths, limitations, and challenges,
we strive to develop robust, fair, and efficient CAT systems. By bridging
psychometric-driven CAT research with machine learning, this survey advocates
for a more inclusive and interdisciplinary approach to the future of adaptive
testing
Improving Fairness for Data Valuation in Horizontal Federated Learning
Federated learning is an emerging decentralized machine learning scheme that
allows multiple data owners to work collaboratively while ensuring data
privacy. The success of federated learning depends largely on the participation
of data owners. To sustain and encourage data owners' participation, it is
crucial to fairly evaluate the quality of the data provided by the data owners
and reward them correspondingly. Federated Shapley value, recently proposed by
Wang et al. [Federated Learning, 2020], is a measure for data value under the
framework of federated learning that satisfies many desired properties for data
valuation. However, there are still factors of potential unfairness in the
design of federated Shapley value because two data owners with the same local
data may not receive the same evaluation. We propose a new measure called
completed federated Shapley value to improve the fairness of federated Shapley
value. The design depends on completing a matrix consisting of all the possible
contributions by different subsets of the data owners. It is shown under mild
conditions that this matrix is approximately low-rank by leveraging concepts
and tools from optimization. Both theoretical analysis and empirical evaluation
verify that the proposed measure does improve fairness in many circumstances
Tetris: A compilation Framework for VQE Applications
Quantum computing has shown promise in solving complex problems by leveraging
the principles of superposition and entanglement. The Variational Quantum
Eigensolver (VQE) algorithm stands as a pivotal approach in the realm of
quantum algorithms, enabling the simulation of quantum systems on quantum
hardware. In this paper, we introduce two innovative techniques, namely
"Tetris" and "Fast Bridging," designed to enhance the efficiency and
effectiveness of VQE tasks. The "Tetris" technique addresses a crucial aspect
of VQE optimization by unveiling cancellation opportunities within the logical
circuit phase of UCCSD ansatz. Tetris demonstrates a remarkable reduction up to
20% in CNOT gate counts, about 119048 CNOT gates, and 30% depth reduction
compared to the state-of-the-art compiler 'Paulihedral'. In addition to Tetris,
we present the "Fast Bridging" technique as an alternative to the conventional
qubit routing methods that heavily rely on swap operations. The fast bridging
offers a novel approach to qubit routing, mitigating the limitations associated
with swap-heavy routing. By integrating the fast bridging into the VQE
framework, we observe further reductions in CNOT gate counts and circuit depth.
The bridging technique can achieve up to 27% CNOT gate reduction in the QAOA
application. Through a combination of Tetris and the fast bridging, we present
a comprehensive strategy for enhancing VQE performance. Our experimental
results showcase the effectiveness of Tetris in uncovering cancellation
opportunities and demonstrate the symbiotic relationship between Tetris and the
fast bridging in minimizing gate counts and circuit depth. This paper
contributes not only to the advancement of VQE techniques but also to the
broader field of quantum algorithm optimization
Lawyer LLaMA Technical Report
Large Language Models (LLMs), like LLaMA, have exhibited remarkable
performances across various tasks. Nevertheless, when deployed to specific
domains such as law or medicine, the models still confront the challenge of a
deficiency in domain-specific knowledge and an inadequate capability to
leverage that knowledge to resolve domain-related problems. In this paper, we
focus on the legal domain and explore how to inject domain knowledge during the
continual training stage and how to design proper supervised finetune tasks to
help the model tackle practical issues. Moreover, to alleviate the
hallucination problem during model's generation, we add a retrieval module and
extract relevant articles before the model answers any queries. Augmenting with
the extracted evidence, our model could generate more reliable responses. We
release our data and model at https://github.com/AndrewZhe/lawyer-llama.Comment: Work in progres
The temporal signature of self: Temporal measures of restingâ state EEG predict selfâ consciousness
The self is the core of our mental life. Previous investigations have demonstrated a strong neural overlap between selfâ related activity and resting state activity. This suggests that information about selfâ relatedness is encoded in our brainâs spontaneous activity. The exact neuronal mechanisms of such â restâ self containment,â however, remain unclear. The present EEG study investigated temporal measures of resting state EEG to relate them to selfâ consciousness. This was obtained with the selfâ consciousness scale (SCS) which measures Private, Public, and Social dimensions of self. We demonstrate positive correlations between Private selfâ consciousness and three temporal measures of resting state activity: scaleâ free activity as indexed by the powerâ law exponent (PLE), the autoâ correlation window (ACW), and modulation index (MI). Specifically, higher PLE, longer ACW, and stronger MI were related to higher degrees of Private selfâ consciousness. Finally, conducting eLORETA for spatial tomography, we found significant correlation of Private selfâ consciousness with activity in cortical midline structures such as the perigenual anterior cingulate cortex and posterior cingulate cortex. These results were reinforced with a dataâ driven analysis; a machine learning algorithm accurately predicted an individual as having a â highâ or â lowâ Private selfâ consciousness score based on these measures of the brainâs spatiotemporal structure. In conclusion, our results demonstrate that Private selfâ consciousness is related to the temporal structure of resting state activity as featured by temporal nestedness (PLE), temporal continuity (ACW), and temporal integration (MI). Our results support the hypothesis that selfâ related information is temporally contained in the brainâs resting state. â Restâ self containmentâ can thus be featured by a temporal signature.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/147871/1/hbm24412.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/147871/2/hbm24412_am.pd
Disrupted neural variability during propofolâinduced sedation and unconsciousness
Variability quenching is a widespread neural phenomenon in which trialâtoâtrial variability (TTV) of neural activity is reduced by repeated presentations of a sensory stimulus. However, its neural mechanism and functional significance remain poorly understood. Recurrent network dynamics are suggested as a candidate mechanism of TTV, and they play a key role in consciousness. We thus asked whether the variabilityâquenching phenomenon is related to the level of consciousness. We hypothesized that TTV reduction would be compromised during reduced level of consciousness by propofol anesthetics. We recorded functional magnetic resonance imaging signals of restingâstate and stimulusâinduced activities in three conditions: wakefulness, sedation, and unconsciousness (i.e., deep anesthesia). We measured the average (trialâtoâtrial mean, TTM) and variability (TTV) of auditory stimulusâinduced activity under the three conditions. We also examined another form of neural variability (temporal variability, TV), which quantifies the overall dynamic range of ongoing neural activity across time, during both the restingâstate and the task. We found that (a) TTM deceased gradually from wakefulness through sedation to anesthesia, (b) stimulusâinduced TTV reduction normally seen during wakefulness was abolished during both sedation and anesthesia, and (c) TV increased in the task state as compared to restingâstate during both wakefulness and sedation, but not anesthesia. Together, our results reveal distinct effects of propofol on the two forms of neural variability (TTV and TV). They imply that the anesthetic disrupts recurrent network dynamics, thus prevents the stabilization of cortical activity states. These findings shed new light on the temporal dynamics of neuronal variability and its alteration during anestheticâinduced unconsciousness.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/146388/1/hbm24304_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/146388/2/hbm24304.pd
Research on the relationship between carbon performance and financial performance of electric power enterprises under the background of âdual carbonâ
Under the background of âdual carbon,â the power industry, as a pillar industry of the national economy, is ushering in changes. Based on the data of listed companies in the electric power production and supply industry from 2010 to 2020, this paper takes the operating income corresponding to each unit of carbon emission as the substitute variable of carbon performance (CP). After dimensionality reduction of 12 financial indicators through factor analysis, this paper establishes a comprehensive indicator of financial performance (FP), and establishes panel data to explore the relationship between CP and FP of electric power enterprises. To mitigate the endogeneity problem, 2SLS regression was performed using instrumental variables. The results show that CP has a positive and sustainable impact on the FP, which indicates that power enterprises need to pay attention to the long-term management of carbon emission reduction, so that the improvement of FP of enterprises can achieve sustainable development, which is in line with the expectations of Porterâs hypothesis and stakeholder theory. In addition, firm size plays a negative moderating role in the relationship between CP and FP. The research results provide a path and basis for encouraging power enterprises to improve CP and help China achieve the goal of âdual carbonâ as soon as possible
- âŚ