52 research outputs found
Arabic-SOS: Segmentation, stemming, and orthography standardization for classical and pre-modern standard Arabic
This is an accepted manuscript of an article published by ACM in DATeCH2019: Proceedings of the 3rd International Conference on Digital Access to Textual Cultural Heritage in May 2019, available online: https://doi.org/10.1145/3322905.3322927
The accepted version of the publication may differ from the final published version.While morphological segmentation has always been a hot topic in Arabic, due to the morphological complexity of the language
and the orthography, most effort has focused on Modern Standard Arabic. In this paper, we focus on pre-MSA texts. We use the Gradient Boosting algorithm to train a morphological segmenter with a corpus derived from Al-Manar, a late 19th/early 20th century magazine that focused on the Arabic and Islamic heritage. Since most of the cultural heritage Arabic available suffers from substandard orthography, we have trained a machine learner to standardize the text. Our segmentation accuracy reaches 98.47%, and the orthography standardization an F-macro of 0.98 and an F-micro of 0.99. We also produce stemming as a by-product of segmentation
Do Deep Learning Methods Really Perform Better in Molecular Conformation Generation?
Molecular conformation generation (MCG) is a fundamental and important
problem in drug discovery. Many traditional methods have been developed to
solve the MCG problem, such as systematic searching, model-building, random
searching, distance geometry, molecular dynamics, Monte Carlo methods, etc.
However, they have some limitations depending on the molecular structures.
Recently, there are plenty of deep learning based MCG methods, which claim they
largely outperform the traditional methods. However, to our surprise, we design
a simple and cheap algorithm (parameter-free) based on the traditional methods
and find it is comparable to or even outperforms deep learning based MCG
methods in the widely used GEOM-QM9 and GEOM-Drugs benchmarks. In particular,
our design algorithm is simply the clustering of the RDKIT-generated
conformations. We hope our findings can help the community to revise the deep
learning methods for MCG. The code of the proposed algorithm could be found at
https://gist.github.com/ZhouGengmo/5b565f51adafcd911c0bc115b2ef027c
Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+
Recent developments in deep learning have made remarkable progress in
speeding up the prediction of quantum chemical (QC) properties by removing the
need for expensive electronic structure calculations like density functional
theory. However, previous methods learned from 1D SMILES sequences or 2D
molecular graphs failed to achieve high accuracy as QC properties primarily
depend on the 3D equilibrium conformations optimized by electronic structure
methods, far different from the sequence-type and graph-type data. In this
paper, we propose a novel approach called Uni-Mol+ to tackle this challenge.
Uni-Mol+ first generates a raw 3D molecule conformation from inexpensive
methods such as RDKit. Then, the raw conformation is iteratively updated to its
target DFT equilibrium conformation using neural networks, and the learned
conformation will be used to predict the QC properties. To effectively learn
this update process towards the equilibrium conformation, we introduce a
two-track Transformer model backbone and train it with the QC property
prediction task. We also design a novel approach to guide the model's training
process. Our extensive benchmarking results demonstrate that the proposed
Uni-Mol+ significantly improves the accuracy of QC property prediction in
various datasets. We have made the code and model publicly available at
\url{https://github.com/dptech-corp/Uni-Mol}
Boosted ab initio Cryo-EM 3D Reconstruction with ACE-EM
The central problem in cryo-electron microscopy (cryo-EM) is to recover the
3D structure from noisy 2D projection images which requires estimating the
missing projection angles (poses). Recent methods attempted to solve the 3D
reconstruction problem with the autoencoder architecture, which suffers from
the latent vector space sampling problem and frequently produces suboptimal
pose inferences and inferior 3D reconstructions. Here we present an improved
autoencoder architecture called ACE (Asymmetric Complementary autoEncoder),
based on which we designed the ACE-EM method for cryo-EM 3D reconstructions.
Compared to previous methods, ACE-EM reached higher pose space coverage within
the same training time and boosted the reconstruction performance regardless of
the choice of decoders. With this method, the Nyquist resolution (highest
possible resolution) was reached for 3D reconstructions of both simulated and
experimental cryo-EM datasets. Furthermore, ACE-EM is the only amortized
inference method that reached the Nyquist resolution
Light Multi-segment Activation for Model Compression
Model compression has become necessary when applying neural networks (NN)
into many real application tasks that can accept slightly-reduced model
accuracy with strict tolerance to model complexity. Recently, Knowledge
Distillation, which distills the knowledge from well-trained and highly complex
teacher model into a compact student model, has been widely used for model
compression. However, under the strict requirement on the resource cost, it is
quite challenging to achieve comparable performance with the teacher model,
essentially due to the drastically-reduced expressiveness ability of the
compact student model. Inspired by the nature of the expressiveness ability in
Neural Networks, we propose to use multi-segment activation, which can
significantly improve the expressiveness ability with very little cost, in the
compact student model. Specifically, we propose a highly efficient
multi-segment activation, called Light Multi-segment Activation (LMA), which
can rapidly produce multiple linear regions with very few parameters by
leveraging the statistical information. With using LMA, the compact student
model is capable of achieving much better performance effectively and
efficiently, than the ReLU-equipped one with same model scale. Furthermore, the
proposed method is compatible with other model compression techniques, such as
quantization, which means they can be used jointly for better compression
performance. Experiments on state-of-the-art NN architectures over the
real-world tasks demonstrate the effectiveness and extensibility of the LMA
3D Molecular Generation via Virtual Dynamics
Structure-based drug design, i.e., finding molecules with high affinities to
the target protein pocket, is one of the most critical tasks in drug discovery.
Traditional solutions, like virtual screening, require exhaustively searching
on a large molecular database, which are inefficient and cannot return novel
molecules beyond the database. The pocket-based 3D molecular generation model,
i.e., directly generating a molecule with a 3D structure and binding position
in the pocket, is a new promising way to address this issue. Herein, we propose
VD-Gen, a novel pocket-based 3D molecular generation pipeline. VD-Gen consists
of several carefully designed stages to generate fine-grained 3D molecules with
binding positions in the pocket cavity end-to-end. Rather than directly
generating or sampling atoms with 3D positions in the pocket like in early
attempts, in VD-Gen, we first randomly initialize many virtual particles in the
pocket; then iteratively move these virtual particles, making the distribution
of virtual particles approximate the distribution of molecular atoms. After
virtual particles are stabilized in 3D space, we extract a 3D molecule from
them. Finally, we further refine atoms in the extracted molecule by iterative
movement again, to get a high-quality 3D molecule, and predict a confidence
score for it. Extensive experiment results on pocket-based molecular generation
demonstrate that VD-Gen can generate novel 3D molecules to fill the target
pocket cavity with high binding affinities, significantly outperforming
previous baselines
Adversarial Meta Sampling for Multilingual Low-Resource Speech Recognition
Low-resource automatic speech recognition (ASR) is challenging, as the
low-resource target language data cannot well train an ASR model. To solve this
issue, meta-learning formulates ASR for each source language into many small
ASR tasks and meta-learns a model initialization on all tasks from different
source languages to access fast adaptation on unseen target languages. However,
for different source languages, the quantity and difficulty vary greatly
because of their different data scales and diverse phonological systems, which
leads to task-quantity and task-difficulty imbalance issues and thus a failure
of multilingual meta-learning ASR (MML-ASR). In this work, we solve this
problem by developing a novel adversarial meta sampling (AMS) approach to
improve MML-ASR. When sampling tasks in MML-ASR, AMS adaptively determines the
task sampling probability for each source language. Specifically, for each
source language, if the query loss is large, it means that its tasks are not
well sampled to train ASR model in terms of its quantity and difficulty and
thus should be sampled more frequently for extra learning. Inspired by this
fact, we feed the historical task query loss of all source language domain into
a network to learn a task sampling policy for adversarially increasing the
current query loss of MML-ASR. Thus, the learnt task sampling policy can master
the learning situation of each language and thus predicts good task sampling
probability for each language for more effective learning. Finally, experiment
results on two multilingual datasets show significant performance improvement
when applying our AMS on MML-ASR, and also demonstrate the applicability of AMS
to other low-resource speech tasks and transfer learning ASR approaches.Comment: accepted in AAAI202
- …