4,235 research outputs found
Learning Pruned Structure and Weights Simultaneously from Scratch: an Attention based Approach
As a deep learning model typically contains millions of trainable weights,
there has been a growing demand for a more efficient network structure with
reduced storage space and improved run-time efficiency. Pruning is one of the
most popular network compression techniques. In this paper, we propose a novel
unstructured pruning pipeline, Attention-based Simultaneous sparse structure
and Weight Learning (ASWL). Unlike traditional channel-wise or weight-wise
attention mechanism, ASWL proposed an efficient algorithm to calculate the
pruning ratio through layer-wise attention for each layer, and both weights for
the dense network and the sparse network are tracked so that the pruned
structure is simultaneously learned from randomly initialized weights. Our
experiments on MNIST, Cifar10, and ImageNet show that ASWL achieves superior
pruning results in terms of accuracy, pruning ratio and operating efficiency
when compared with state-of-the-art network pruning methods
Subnatural-Linewidth Polarization-Entangled Photon Pairs with Controllable Temporal Length
We demonstrate an efficient experimental scheme for producing
polarization-entangled photon pairs from spontaneous four-wave mixing (SFWM) in
a laser-cooled Rb atomic ensemble, with a bandwidth (as low as 0.8 MHz)
much narrower than the rubidium atomic natural linewidth. By stabilizing the
relative phase between the two SFWM paths in a Mach-Zehnder interferometer
configuration, we are able to produce all four Bell states. These
subnatural-linewidth photon pairs with polarization entanglement are ideal
quantum information carriers for connecting remote atomic quantum nodes via
efficient light-matter interaction in a photon-atom quantum network.Comment: Title changed, published version, 5 pages + 3 pages Supplemental
Materia
Synthesis of Epoxidatied Castor Oil and Its Effect on the Properties of Waterborne Polyurethane
AbstractIn this study, a new method for synthesis poxidatied castor oil (ECO) is engaged. A series of waterborne polyurethane dispersions (WPUs) were synthesized using polytetramethylene ether glycol (PTMEG), toluene diisocyanate (TDI-80), and ECO. These WPUs can be crosslinked spontaneously upon drying, without extra additives or processing steps. Moreover, the particle size, and morphology of WPUs were examined with light scattering ultrafine particle analyzer, and transmission electron microscopy. The anti-water, thermal and mechanical properties were also studied. Results reveal that the particle size of WPUs mainly depends on the concentrations of ECO. The particle size decreases when the ECO is used. Furthermore, increased amount of ECO results in an improvement of the anti-water, thermal and mechanical properties of WPU films
Segatron: Segment-Aware Transformer for Language Modeling and Understanding
Transformers are powerful for sequence modeling. Nearly all state-of-the-art
language models and pre-trained language models are based on the Transformer
architecture. However, it distinguishes sequential tokens only with the token
position index. We hypothesize that better contextual representations can be
generated from the Transformer with richer positional information. To verify
this, we propose a segment-aware Transformer (Segatron), by replacing the
original token position encoding with a combined position encoding of
paragraph, sentence, and token. We first introduce the segment-aware mechanism
to Transformer-XL, which is a popular Transformer-based language model with
memory extension and relative position encoding. We find that our method can
further improve the Transformer-XL base model and large model, achieving 17.1
perplexity on the WikiText-103 dataset. We further investigate the pre-training
masked language modeling task with Segatron. Experimental results show that
BERT pre-trained with Segatron (SegaBERT) can outperform BERT with vanilla
Transformer on various NLP tasks, and outperforms RoBERTa on zero-shot sentence
representation learning.Comment: Accepted by AAAI 202
A refined numerical investigation of a large equivalent shallow-depth underwater explosion
The large equivalent shallow-depth explosion problem is very significant in
the field of naval architecture and ocean engineering, as such explosions can
be used to attack and demolish ships and anti-ship missiles. In the current
work, a refined numerical study of the flow-field characteristics of a large
equivalent shallow-depth explosion is carried out using a self-developed
Eulerian finite element solver. Firstly, the numerical model is validated
against theoretical results and a small equivalent explosion test in a tank.
The numerical results are found to agree well with the theoretical and
experimental results. In the next step, the cavitation cut-off effect is added
to the underwater explosion model, and the cavitation phenomenon is
quantitatively analyzed through the flow-field pressure. In addition, the
dynamic characteristics of the bubble and water hump under various initial
conditions for different stand-off parameters are analyzed. The effect of
gravity on these physical processes is also discussed. The bubble pulsation
period, taking into account the free surface effect, is then quantitatively
studied and compared with Cole's experimental formula for an underwater
explosion. Overall, when the stand-off parameter > 2, the influence of the free
surface on the empirical period of the bubble is not significant. Our
investigation provides broad insights into shallow-depth underwater explosions
from theoretical, experimental, and numerical perspectives
Inverse Geometry Design of Radiative Enclosures Using Particle Swarm Optimization Algorithms
Three different Particle Swarm Optimization (PSO) algorithms—standard PSO, stochastic PSO (SPSO) and differential evolution PSO (DEPSO)—are applied to solve the inverse geometry design problems of radiative enclosures. The design purpose is to satisfy a uniform distribution of radiative heat flux on the designed surface. The design surface is discretized into a series of control points, the PSO algorithms are used to optimize the locations of these points and the Akima cubic interpolation is utilized to approximate the changing boundary shape. The retrieval results show that PSO algorithms can be successfully applied to solve inverse geometry design problems and SPSO achieves the best performance on computational time. The influences of the number of control points and the radiative properties of the media on the retrieval geometry design results are also investigated
- …