145 research outputs found
An Element-wise RSAV Algorithm for Unconstrained Optimization Problems
We present a novel optimization algorithm, element-wise relaxed scalar
auxiliary variable (E-RSAV), that satisfies an unconditional energy dissipation
law and exhibits improved alignment between the modified and the original
energy. Our algorithm features rigorous proofs of linear convergence in the
convex setting. Furthermore, we present a simple accelerated algorithm that
improves the linear convergence rate to super-linear in the univariate case. We
also propose an adaptive version of E-RSAV with Steffensen step size. We
validate the robustness and fast convergence of our algorithm through ample
numerical experiments.Comment: 25 pages, 7 figure
Beam Retrieval: General End-to-End Retrieval for Multi-Hop Question Answering
Multi-hop QA involves finding multiple relevant passages and step-by-step
reasoning to answer complex questions. While previous approaches have developed
retrieval modules for selecting relevant passages, they face challenges in
scenarios beyond two hops, owing to the limited performance of one-step methods
and the failure of two-step methods when selecting irrelevant passages in
earlier stages. In this work, we introduce Beam Retrieval, a general end-to-end
retrieval framework for multi-hop QA. This approach maintains multiple partial
hypotheses of relevant passages at each step, expanding the search space and
reducing the risk of missing relevant passages. Moreover, Beam Retrieval
jointly optimizes an encoder and two classification heads by minimizing the
combined loss across all hops. To establish a complete QA system, we
incorporate a supervised reader or a zero-shot GPT-3.5. Experimental results
demonstrate that Beam Retrieval achieves a nearly 50% improvement compared with
baselines on challenging MuSiQue-Ans, and it also surpasses all previous
retrievers on HotpotQA and 2WikiMultiHopQA. Providing high-quality context,
Beam Retrieval helps our supervised reader achieve new state-of-the-art
performance and substantially improves (up to 28.8 points) the QA performance
of zero-shot GPT-3.5.Comment: Code is available at https://github.com/canghongjian/beam_retrieve
Efficient Cross-Device Federated Learning Algorithms for Minimax Problems
In many machine learning applications where massive and privacy-sensitive
data are generated on numerous mobile or IoT devices, collecting data in a
centralized location may be prohibitive. Thus, it is increasingly attractive to
estimate parameters over mobile or IoT devices while keeping data localized.
Such learning setting is known as cross-device federated learning. In this
paper, we propose the first theoretically guaranteed algorithms for general
minimax problems in the cross-device federated learning setting. Our algorithms
require only a fraction of devices in each round of training, which overcomes
the difficulty introduced by the low availability of devices. The communication
overhead is further reduced by performing multiple local update steps on
clients before communication with the server, and global gradient estimates are
leveraged to correct the bias in local update directions introduced by data
heterogeneity. By developing analyses based on novel potential functions, we
establish theoretical convergence guarantees for our algorithms. Experimental
results on AUC maximization, robust adversarial network training, and GAN
training tasks demonstrate the efficiency of our algorithms
Efficient Projection-Free Online Methods with Stochastic Recursive Gradient
This paper focuses on projection-free methods for solving smooth Online
Convex Optimization (OCO) problems. Existing projection-free methods either
achieve suboptimal regret bounds or have high per-iteration computational
costs. To fill this gap, two efficient projection-free online methods called
ORGFW and MORGFW are proposed for solving stochastic and adversarial OCO
problems, respectively. By employing a recursive gradient estimator, our
methods achieve optimal regret bounds (up to a logarithmic factor) while
possessing low per-iteration computational costs. Experimental results
demonstrate the efficiency of the proposed methods compared to
state-of-the-arts.Comment: 15 pages, 3 figure
An optimized quantum minimum searching algorithm with sure-success probability and its experiment simulation with Cirq
Finding a minimum is an essential part of mathematical models, and it plays
an important role in some optimization problems. Durr and Hoyer proposed a
quantum searching algorithm (DHA), with a certain probability of success, to
achieve quadratic speed than classical ones. In this paper, we propose an
optimized quantum minimum searching algorithm with sure-success probability,
which utilizes Grover-Long searching to implement the optimal exact searching,
and the dynamic strategy to reduce the iterations of our algorithm. Besides, we
optimize the oracle circuit to reduce the number of gates by the simplified
rules. The performance evaluation including the theoretical success rate and
computational complexity shows that our algorithm has higher accuracy and
efficiency than DHA algorithm. Finally, a simulation experiment based on Cirq
is performed to verify its feasibility.Comment: 15 pages, 8 figures. arXiv admin note: text overlap with
arXiv:1908.07943 by other author
EMID: An Emotional Aligned Dataset in Audio-Visual Modality
In this paper, we propose Emotionally paired Music and Image Dataset (EMID),
a novel dataset designed for the emotional matching of music and images, to
facilitate auditory-visual cross-modal tasks such as generation and retrieval.
Unlike existing approaches that primarily focus on semantic correlations or
roughly divided emotional relations, EMID emphasizes the significance of
emotional consistency between music and images using an advanced 13-dimension
emotional model. By incorporating emotional alignment into the dataset, it aims
to establish pairs that closely align with human perceptual understanding,
thereby raising the performance of auditory-visual cross-modal tasks. We also
design a supplemental module named EMI-Adapter to optimize existing cross-modal
alignment methods. To validate the effectiveness of the EMID, we conduct a
psychological experiment, which has demonstrated that considering the emotional
relationship between the two modalities effectively improves the accuracy of
matching in abstract perspective. This research lays the foundation for future
cross-modal research in domains such as psychotherapy and contributes to
advancing the understanding and utilization of emotions in cross-modal
alignment. The EMID dataset is available at https://github.com/ecnu-aigc/EMID
Three-step Formation of Diamonds in Shock-compressed Hydrocarbons: Decomposition, Species Separation, and Nucleation
The accumulation and circulation of carbon-hydrogen dictate the chemical
evolution of ice giant planets. Species separation and diamond precipitation
have been reported in carbon-hydrogen systems, verified by static and
shock-compression experiments. Nevertheless, the dynamic formation processes
for the above-mentioned phenomena are still insufficiently understood. Here,
combing deep learning model, we demonstrate that diamonds form through a
three-step process involving decomposition, species separation and nucleation
procedures. Under shock condition of 125 GPa and 4590 K, hydrocarbons are
decomposed to give hydrogen and low-molecular-weight alkanes (CH4 and C2H6),
which escape from the carbon chains resulting in C/H species separation. The
remaining carbon atoms without C-H bonds accumulate and nucleate to form
diamond crystals. The process of diamond growth is found to associated with a
critical nucleus size where dynamic energy barrier plays a key role. These
dynamic processes for diamonds formation are insightful in establishing the
model for ice giant planet evolution.Comment: 5 figure
Lactobacillus rhamnosus CY12 Enhances Intestinal Barrier Function by Regulating Tight Junction Protein Expression, Oxidative Stress, and Inflammation Response in Lipopolysaccharide-Induced Caco-2 Cells
he intestinal barrier is vital for preventing inflammatory bowel disease (IBD). The objectives of this study were to assess whether the Lactobacillus rhamnosus CY12 could alleviate oxidative stress, inflammation, and the disruption of tight junction (TJ) barrier functions induced by lipopolysaccharide (LPS), and therefore to explore the potential underlying molecular mechanisms. Our results showed that LPS-induced Cancer coli-2 (Caco-2) cells significantly increased the levels of reactive oxygen species (ROS), lactate dehydrogenase, inflammatory cytokines interleukin-1β, interleukin-6, interleukin-8, and tumor necrosis factor-α (IL-1β, IL-6, IL-8, and TNF-α), and the cell apoptosis rate while decreasing the levels of TJ proteins occludin, zonula occludens-1 (ZO-1), and claudin and antioxidant enzymes, such as catalase, superoxide dismutase, and glutathione peroxidase(CAT, SOD, and GSH-Px) (p < 0.05). However, Lactobacillus rhamnosus CY12 could relieve cytotoxicity, apoptosis, oxidative stress, and pro-inflammatory cytokine expressions, and also inhibit the Toll-like receptor 4/nuclear factor kappa-B(TLR4/NF-κB) signaling pathway. Furthermore, the gene expression of antioxidant enzymes, as well as the mRNA and protein expressions of TJ proteins, was improved. Particularly, the concentration of 108 cfu/mL significantly prevented the inflammatory injury induced by LPS in Caco-2 cells (p < 0.05). These findings support a potential application of Lactobacillus rhamnosus CY12 as a probiotic to prevent LPS-induced intestinal injury and treat intestinal barrier dysfunction
- …