132 research outputs found
Numerical Methods for Distributed Stochastic Compositional Optimization Problems with Aggregative Structure
The paper studies the distributed stochastic compositional optimization
problems over networks, where all the agents' inner-level function is the sum
of each agent's private expectation function. Focusing on the aggregative
structure of the inner-level function, we employ the hybrid variance reduction
method to obtain the information on each agent's private expectation function,
and apply the dynamic consensus mechanism to track the information on each
agent's inner-level function. Then by combining with the standard distributed
stochastic gradient descent method, we propose a distributed aggregative
stochastic compositional gradient descent method. When the objective function
is smooth, the proposed method achieves the optimal convergence rate
. We further combine the proposed method with
the communication compression and propose the communication compressed variant
distributed aggregative stochastic compositional gradient descent method. The
compressed variant of the proposed method maintains the optimal convergence
rate . Simulated experiments on decentralized
reinforcement learning verify the effectiveness of the proposed methods
Concurrent On-the-fly SCC Detection for Automata-based Model Checking with Fairness Assumption
Evaluating Self-Supervised Learning for Molecular Graph Embeddings
Graph Self-Supervised Learning (GSSL) provides a robust pathway for acquiring
embeddings without expert labelling, a capability that carries profound
implications for molecular graphs due to the staggering number of potential
molecules and the high cost of obtaining labels. However, GSSL methods are
designed not for optimisation within a specific domain but rather for
transferability across a variety of downstream tasks. This broad applicability
complicates their evaluation. Addressing this challenge, we present "Molecular
Graph Representation Evaluation" (MOLGRAPHEVAL), generating detailed profiles
of molecular graph embeddings with interpretable and diversified attributes.
MOLGRAPHEVAL offers a suite of probing tasks grouped into three categories: (i)
generic graph, (ii) molecular substructure, and (iii) embedding space
properties. By leveraging MOLGRAPHEVAL to benchmark existing GSSL methods
against both current downstream datasets and our suite of tasks, we uncover
significant inconsistencies between inferences drawn solely from existing
datasets and those derived from more nuanced probing. These findings suggest
that current evaluation methodologies fail to capture the entirety of the
landscape.Comment: update result
A Group Symmetric Stochastic Differential Equation Model for Molecule Multi-modal Pretraining
Molecule pretraining has quickly become the go-to schema to boost the
performance of AI-based drug discovery. Naturally, molecules can be represented
as 2D topological graphs or 3D geometric point clouds. Although most existing
pertaining methods focus on merely the single modality, recent research has
shown that maximizing the mutual information (MI) between such two modalities
enhances the molecule representation ability. Meanwhile, existing molecule
multi-modal pretraining approaches approximate MI based on the representation
space encoded from the topology and geometry, thus resulting in the loss of
critical structural information of molecules. To address this issue, we propose
MoleculeSDE. MoleculeSDE leverages group symmetric (e.g., SE(3)-equivariant and
reflection-antisymmetric) stochastic differential equation models to generate
the 3D geometries from 2D topologies, and vice versa, directly in the input
space. It not only obtains tighter MI bound but also enables prosperous
downstream tasks than the previous work. By comparing with 17 pretraining
baselines, we empirically verify that MoleculeSDE can learn an expressive
representation with state-of-the-art performance on 26 out of 32 downstream
tasks
UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye View
In the field of 3D object detection for autonomous driving, the sensor
portfolio including multi-modality and single-modality is diverse and complex.
Since the multi-modal methods have system complexity while the accuracy of
single-modal ones is relatively low, how to make a tradeoff between them is
difficult. In this work, we propose a universal cross-modality knowledge
distillation framework (UniDistill) to improve the performance of
single-modality detectors. Specifically, during training, UniDistill projects
the features of both the teacher and the student detector into Bird's-Eye-View
(BEV), which is a friendly representation for different modalities. Then, three
distillation losses are calculated to sparsely align the foreground features,
helping the student learn from the teacher without introducing additional cost
during inference. Taking advantage of the similar detection paradigm of
different detectors in BEV, UniDistill easily supports LiDAR-to-camera,
camera-to-LiDAR, fusion-to-LiDAR and fusion-to-camera distillation paths.
Furthermore, the three distillation losses can filter the effect of misaligned
background information and balance between objects of different sizes,
improving the distillation effectiveness. Extensive experiments on nuScenes
demonstrate that UniDistill effectively improves the mAP and NDS of student
detectors by 2.0%~3.2%
ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedback
Recent advancements in conversational large language models (LLMs), such as
ChatGPT, have demonstrated remarkable promise in various domains, including
drug discovery. However, existing works mainly focus on investigating the
capabilities of conversational LLMs on chemical reaction and retrosynthesis.
While drug editing, a critical task in the drug discovery pipeline, remains
largely unexplored. To bridge this gap, we propose ChatDrug, a framework to
facilitate the systematic investigation of drug editing using LLMs. ChatDrug
jointly leverages a prompt module, a retrieval and domain feedback (ReDF)
module, and a conversation module to streamline effective drug editing. We
empirically show that ChatDrug reaches the best performance on 33 out of 39
drug editing tasks, encompassing small molecules, peptides, and proteins. We
further demonstrate, through 10 case studies, that ChatDrug can successfully
identify the key substructures (e.g., the molecule functional groups, peptide
motifs, and protein structures) for manipulation, generating diverse and valid
suggestions for drug editing. Promisingly, we also show that ChatDrug can offer
insightful explanations from a domain-specific perspective, enhancing
interpretability and enabling informed decision-making. This research sheds
light on the potential of ChatGPT and conversational LLMs for drug editing. It
paves the way for a more efficient and collaborative drug discovery pipeline,
contributing to the advancement of pharmaceutical research and development
- …