Inter-KD: Intermediate Knowledge Distillation for CTC-Based Automatic Speech Recognition

Recently, the advance in deep learning has brought a considerable improvement in the end-to-end speech recognition field, simplifying the traditional pipeline while producing promising results. Among the end-to-end models, the connectionist temporal classification (CTC)-based model has attracted research interest due to its non-autoregressive nature. However, such CTC models require a heavy computational cost to achieve outstanding performance. To mitigate the computational burden, we propose a simple yet effective knowledge distillation (KD) for the CTC framework, namely Inter-KD, that additionally transfers the teacher's knowledge to the intermediate CTC layers of the student network. From the experimental results on the LibriSpeech, we verify that the Inter-KD shows better achievements compared to the conventional KD methods. Without using any language model (LM) and data augmentation, Inter-KD improves the word error rate (WER) performance from 8.85 % to 6.30 % on the test-clean.Comment: Accepted by 2022 SLT Worksho

EM-Network: Oracle Guided Self-distillation for Sequence Learning

We introduce EM-Network, a novel self-distillation approach that effectively leverages target information for supervised sequence-to-sequence (seq2seq) learning. In contrast to conventional methods, it is trained with oracle guidance, which is derived from the target sequence. Since the oracle guidance compactly represents the target-side context that can assist the sequence model in solving the task, the EM-Network achieves a better prediction compared to using only the source input. To allow the sequence model to inherit the promising capability of the EM-Network, we propose a new self-distillation strategy, where the original sequence model can benefit from the knowledge of the EM-Network in a one-stage manner. We conduct comprehensive experiments on two types of seq2seq models: connectionist temporal classification (CTC) for speech recognition and attention-based encoder-decoder (AED) for machine translation. Experimental results demonstrate that the EM-Network significantly advances the current state-of-the-art approaches, improving over the best prior work on speech recognition and establishing state-of-the-art performance on WMT'14 and IWSLT'14.Comment: ICML 202

Comparative analysis of pepper and tomato reveals euchromatin expansion of pepper genome caused by differential accumulation of Ty3/Gypsy-like elements

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Abstract Background Among the Solanaceae plants, the pepper genome is three times larger than that of tomato. Although the gene repertoire and gene order of both species are well conserved, the cause of the genome-size difference is not known. To determine the causes for the expansion of pepper euchromatic regions, we compared the pepper genome to that of tomato. Results For sequence-level analysis, we generated 35.6 Mb of pepper genomic sequences from euchromatin enriched 1,245 pepper BAC clones. The comparative analysis of orthologous gene-rich regions between both species revealed insertion of transposons exclusively in the pepper sequences, maintaining the gene order and content. The most common type of the transposon found was the LTR retrotransposon. Phylogenetic comparison of the LTR retrotransposons revealed that two groups of Ty3/Gypsy-like elements (Tat and Athila) were overly accumulated in the pepper genome. The FISH analysis of the pepper Tat elements showed a random distribution in heterochromatic and euchromatic regions, whereas the tomato Tat elements showed heterochromatin-preferential accumulation. Conclusions Compared to tomato pepper euchromatin doubled its size by differential accumulation of a specific group of Ty3/Gypsy-like elements. Our results could provide an insight on the mechanism of genome evolution in the Solanaceae family

Weighted Mask R-CNN for Improving Adjacent Boundary Segmentation

In the recent era of AI, instance segmentation has significantly advanced boundary and object detection especially in diverse fields (e.g., biological and environmental research). Despite its progress, edge detection amid adjacent objects (e.g., organism cells) still remains intractable. This is because homogeneous and heterogeneous objects are prone to being mingled in a single image. To cope with this challenge, we propose the weighted Mask R-CNN designed to effectively separate overlapped objects in virtue of extra weights to adjacent boundaries. For numerical study, a range of experiments are performed with applications to simulated data and real data (e.g., Microcystis, one of the most common algae genera and cell membrane images). It is noticeable that the weighted Mask R-CNN outperforms the standard Mask R-CNN, given that the analytic experiments show on average 92.5% of precision and 96.4% of recall in algae data and 94.5% of precision and 98.6% of recall in cell membrane data. Consequently, we found that a majority of sample boundaries in real and simulated data are precisely segmented in the midst of object mixtures

CoFeS2@CoS2 Nanocubes Entangled with CNT for Efficient Bifunctional Performance for Oxygen Evolution and Oxygen Reduction Reactions

Exploring bifunctional electrocatalysts to lower the activation energy barriers for sluggish electrochemical reactions for both the oxygen evolution reaction (OER) and oxygen reduction reaction (ORR) are of great importance in achieving lower energy consumption and higher conversion efficiency for future energy conversion and storage system. Despite the excellent performance of precious metal-based electrocatalysts for OER and ORR, their high cost and scarcity hamper their large-scale industrial application. As alternatives to precious metal-based electrocatalysts, the development of earth-abundant and efficient catalysts with excellent electrocatalytic performance in both the OER and the ORR is urgently required. Herein, we report a core–shell CoFeS2@CoS2 heterostructure entangled with carbon nanotubes as an efficient bifunctional electrocatalyst for both the OER and the ORR. The CoFeS2@CoS2 nanocubes entangled with carbon nanotubes show superior electrochemical performance for both the OER and the ORR: a potential of 1.5 V (vs. RHE) at a current density of 10 mA cm−2 for the OER in alkaline medium and an onset potential of 0.976 V for the ORR. This work suggests a processing methodology for the development of the core–shell heterostructures with enhanced bifunctional performance for both the OER and the ORR

Highly sensitive pregnancy test kit via oriented antibody conjugation on brush-type ligand-coated quantum beads

Lateral flow assays (LFA) enable development of portable and rapid diagnostic kits; however, their capacity to detect low levels of disease markers remains poor. Here, we report a highly sensitive pregnancy test kit as a proof of concept, by combining brush-type ligand-coated quantum beads (B-type QBs) and nanobody, which can control the antibody orientation and enhance sensitivity. The brush-type ligand provided excellent dispersion stability and high-binding capacity toward antibody. Fc-binding nanobody increased the antigen-binding capacity of conjugated antibodies on the B-type QBs. To facilitate convenient acquisition of the LFA results, we developed a smartphone-based reader with a 3D-printed optical imaging module, and validated the diagnostic performance of the sensing platform. The pregnancy test kit achieved a 5.1 pg mL???1 limit of detection, corresponding to the levels for early-stage detection of heart disease and malaria. Our LFA application can potentially be expanded to diagnosis other diseases by simply changing the antibody pair in the kit

Association of Tim-3/Gal-9 Axis with NLRC4 Inflammasome in Glioma Malignancy: Tim-3/Gal-9 Induce the NLRC4 Inflammasome

Tim-3/Gal-9 and the NLRC4 inflammasome contribute to glioma progression. However, the underlying mechanisms involved are unclear. Here, we observed that Tim-3/Gal-9 expression increased with glioma malignancy and found that Tim-3/Gal-9 regulate NLRC4 inflammasome formation and activation. Tim-3/Gal-9 and NLRC4 inflammasome-related molecule expression levels increased with WHO glioma grade, and this association was correlated with low survival. We investigated NLRC4 inflammasome formation by genetically regulating Tim-3 and its ligand Gal-9. Tim-3/Gal-9 regulation was positively correlated with the NLRC4 inflammasome, NLRC4, and caspase-1 expression. Tim-3/Gal-9 did not trigger IL-1β secretion but were strongly positively correlated with caspase-1 activity as they induced programmed cell death in glioma cells. A protein–protein interaction analysis revealed that the FYN-JAK1-ZNF384 pathways are bridges in NLRC4 inflammasome regulation by Tim-3/Gal-9. The present study showed that Tim-3/Gal-9 are associated with poor prognosis in glioma patients and induce NLRC4 inflammasome formation and activation. We proposed that a Tim-3/Gal-9 blockade could be beneficial in glioma therapy as it would reduce the inflammatory microenvironment by downregulating the NLRC4 inflammasome