Hearing Lips: Improving Lip Reading by Distilling Speech Recognizers

Lip reading has witnessed unparalleled development in recent years thanks to deep learning and the availability of large-scale datasets. Despite the encouraging results achieved, the performance of lip reading, unfortunately, remains inferior to the one of its counterpart speech recognition, due to the ambiguous nature of its actuations that makes it challenging to extract discriminant features from the lip movement videos. In this paper, we propose a new method, termed as Lip by Speech (LIBS), of which the goal is to strengthen lip reading by learning from speech recognizers. The rationale behind our approach is that the features extracted from speech recognizers may provide complementary and discriminant clues, which are formidable to be obtained from the subtle movements of the lips, and consequently facilitate the training of lip readers. This is achieved, specifically, by distilling multi-granularity knowledge from speech recognizers to lip readers. To conduct this cross-modal knowledge distillation, we utilize an efficacious alignment scheme to handle the inconsistent lengths of the audios and videos, as well as an innovative filtering strategy to refine the speech recognizer's prediction. The proposed method achieves the new state-of-the-art performance on the CMLR and LRS2 datasets, outperforming the baseline by a margin of 7.66% and 2.75% in character error rate, respectively.Comment: AAAI 202

CAFE Learning to Condense Dataset by Aligning Features

Dataset condensation aims at reducing the network training effort through condensing a cumbersome training set into a compact synthetic one. State-of-the-art approaches largely rely on learning the synthetic data by matching the gradients between the real and synthetic data batches. Despite the intuitive motivation and promising results, such gradient-based methods, by nature, easily overfit to a biased set of samples that produce dominant gradients, and thus lack global supervision of data distribution. In this paper, we propose a novel scheme to Condense dataset by Aligning FEatures (CAFE), which explicitly attempts to preserve the real-feature distribution as well as the discriminant power of the resulting synthetic set, lending itself to strong generalization capability to various architectures. At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales, while accounting for the classification of real samples. Our scheme is further backed up by a novel dynamic bi-level optimization, which adaptively adjusts parameter updates to prevent over-/under-fitting. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art: on the SVHN dataset, for example, the performance gain is up to 11%. Extensive experiments and analyses verify the effectiveness and necessity of proposed designs.Comment: The manuscript has been accepted by CVPR-2022

Discovery of Plant Viruses From Tea Plant (Camellia sinensis (L.) O. Kuntze) by Metagenomic Sequencing

The tea plant (Camellia sinensis (L.) O. Kuntze) is an economically important woody species. In this study, we collected 26 tea plant samples with typical discoloration symptoms from different tea gardens and performed metagenomic analysis based on next-generation sequencing. Homology annotation and PCR sequencing validation finally identified seven kinds of plant viruses from tea plant. Based on abundance distribution analysis, the two most abundant plant viruses were highlighted. Genetic characterization suggested that they are two novel virus species with relatively high homology to Blueberry necrotic ring blotch virus and American plum line pattern virus. We named the newly discovered viruses tea plant necrotic ring blotch virus (TPNRBV) and tea plant line pattern virus (TPLPV). Evolutionary relationship analysis indicated that TPNRBV and TPLPV should be grouped into the Blunervirus and the Ilarvirus genera, respectively. TPLPV might have same genome activation process with known ilarviruses based on sequence analysis. Moreover, specific primers for both viruses detection were designed and validated. The symptoms and ultrastructure of TPNRBV infected leaves were first recorded. Virus detections in the symptomatic and asymptomatic tissues from field plants showing tea plant necrotic ring blotch disease suggest that TPNRBV has a systemic movement feature. In summary, we first identified seven kinds of putative plant viruses by metagenomic analysis and report two novel viruses being latent pathogens to tea plant. The results will advance our understanding of tea plant virology and have significance for the genetic breeding of tea plants in the future

Effects of obesity with reduced 25(OH)D levels on bone health in elderly Chinese people: a nationwide cross-sectional study

BackgroundObesity is often accompanied by lower 25(OH)D levels, whereas these two parameters exhibit opposite effects on bone health. It is uncertain what are the effects of lower 25(OH)D levels in obesity on bone health in elderly Chinese people.MethodsA nationally representative cross-sectional analysis of China Community-based Cohort of Osteoporosis (CCCO) was performed from 2016 to 2021, which consisted of 22,081 participants. Demographic data, disease history, Body mass index (BMI), bone mineral density (BMD), the levels of the biomarkers of vitamin D status and those of bone metabolism markers were measured for all participants (N = 22,081). The genes (rs12785878, rs10741657, rs4588, rs7041, rs2282679 and rs6013897) related to 25(OH)D transportation and metabolism were performed in a selected subgroup (N = 6008).ResultsObese subjects exhibited lower 25(OH)D levels (p < 0.05) and higher BMD (p < 0.001) compared with those of normal subjects following adjustment. The genotypes and allele frequency of rs12785878, rs10741657, rs6013897, rs2282679, rs4588 and rs7041 indicated no significant differences among three BMI groups following correction by the Bonferroni’s method (p > 0.05). The levels of total 25(OH)D (ToVD) were significantly different among the GC1F, GC1S and GC2 haplotype groups (p < 0.05). Correlation analysis indicated that ToVD levels were significantly correlated with parathyroid hormone levels, BMD, risk of osteoporosis (OP) and the concentration levels of other bone metabolism markers (p < 0.05). Generalized varying coefficient models demonstrated that the increasing BMI, ToVD levels and their interactions were positively associated with BMD outcomes (p < 0.001), whereas the reduced levels of ToVD and BMI increased the risk of OP, which was noted notably for the subjects with reduced ToVD levels (less than 20.69 ng/ml) combined with decreased BMI (less than 24.05 kg/m2).ConclusionThere was a non-linear interaction of BMI and 25(OH)D. And higher BMI accompanied by decreased 25(OH)D levels is associated with increased BMD and decreased incidence of OP, optimal ranges exist for BMI and 25(OH)D levels. The cutoff value of BMI at approximately 24.05 kg/m2 combined with an approximate value of 25(OH)D at 20.69 ng/ml are beneficial for Chinese elderly subjects

Enhanced particle swarm optimization based on principal component analysis and line search

Particle swarm optimization (PSO) guides its search direction by a linear learning strategy, in which each particle updates its velocity through a linear combination among its present status, historical best experience and the swarm best experience. The current position of each particle can be seen as a velocity accumulator. Such a storage strategy is easy to achieve, however, it is inefficient when searching in a complex space and has a great restriction on the achieved heuristic information for the promising solutions. Therefore, a new PSO searching mechanism (PCA-PSO) is proposed based on principal component analysis (PCA) and Line Search (LS), in which PCA is mainly used to efficiently mine population information for the promising principal component directions and then LS strategy is utilized on them. PCA-PSO can inherit most of the velocity information of all the particles to guide them to the most promising directions, which have great difference in learning mechanism with usual PSOs. Experimental results and extensive comparisons with hybrid PSOs, pPSA, PCPSO, CLPSO, GL-25, and CoDE show that PCA-PSO consistently and significantly outperforms some PSO variants and is competitive for other state-of-the-art algorithms. © 2013 Elsevier Inc. All rights reserved

complex event routing in pub/sub systems using traffic analysis model

Bringing in complex events in distributed Pub/Sub systems enables users to observe complicated event correlations over time and space. However, the routing of complex events in Pub/Sub systems is still largely unexplored, especially lacking q