Pretraining Deep Learning Models for Natural Language Understanding

Since the first bidirectional deep learn- ing model for natural language understanding, BERT, emerged in 2018, researchers have started to study and use pretrained bidirectional autoencoding or autoregressive models to solve language problems. In this project, I conducted research to fully understand BERT and XLNet and applied their pretrained models to two language tasks: reading comprehension (RACE) and part-of-speech tagging (The Penn Treebank). After experimenting with those released models, I implemented my own version of ELECTRA, a pretrained text encoder as a discriminator instead of a generator to improve compute-efficiency, with BERT as its underlying architecture. To reduce the number of parameters, I replaced BERT with ALBERT in ELEC- TRA and named the new model, ALE (A Lite ELECTRA). I compared the performance of BERT, ELECTRA, and ALE on GLUE benchmark dev set after pretraining them with the same datasets for the same amount of training FLOPs

A Typha Angustifolia-like MoS2/carbon nanofiber composite for high performance Li-S batteries

A Typha Angustifolia-like MoS2/carbon nanofiber composite as both a chemically trapping agent and redox conversion catalyst for lithium polysulfides has been successfully synthesized via a simple hydrothermal method. Cycling performance and coulombic efficiency have been improved significantly by applying the Typha Angustifolia-like MoS2/carbon nanofiber as the interlayer of a pure sulfur cathode, resulting in a capacity degradation of only 0.09% per cycle and a coulombic efficiency which can reach as high as 99%

Pixelated Semantic Colorization

While many image colorization algorithms have recently shown the capability of producing plausible color versions from gray-scale photographs, they still suffer from limited semantic understanding. To address this shortcoming, we propose to exploit pixelated object semantics to guide image colorization. The rationale is that human beings perceive and distinguish colors based on the semantic categories of objects. Starting from an autoregressive model, we generate image color distributions, from which diverse colored results are sampled. We propose two ways to incorporate object semantics into the colorization model: through a pixelated semantic embedding and a pixelated semantic generator. Specifically, the proposed convolutional neural network includes two branches. One branch learns what the object is, while the other branch learns the object colors. The network jointly optimizes a color embedding loss, a semantic segmentation loss and a color generation loss, in an end-to-end fashion. Experiments on PASCAL VOC2012 and COCO-stuff reveal that our network, when trained with semantic segmentation labels, produces more realistic and finer results compared to the colorization state-of-the-art

Three-dimensional super-resolution correlation-differential confocal microscopy with nanometer axial focusing accuracy

We present a correlation-differential confocal microscopy (CDCM), a novel method that can simultaneously improve the three-dimensional spatial resolution and axial focusing accuracy of confocal microscopy (CM). CDCM divides the CM imaging light path into two paths, where the detectors are before and after the focus with an equal axial offset in opposite directions. Then, the light intensity signals received from the two paths are processed by the correlation product and differential subtraction to improve the CM spatial resolution and axial focusing accuracy, respectively. Theoretical analyses and preliminary experiments indicate that, for the excitation wavelength of λ = 405 nm, numerical aperture of NA = 0.95, and the normalized axial offset of uM = 5.21, the CDCM resolution is improved by more than 20% and more than 30% in the lateral and axial directions, respectively, compared with that of the CM. Also, the axial focusing resolution important for the imaging of sample surface profiles is improved to 1 nm