Causal-Based Supervision of Attention in Graph Neural Network: A Better and Simpler Choice towards Powerful Attention

Recent years have witnessed the great potential of attention mechanism in graph representation learning. However, while variants of attention-based GNNs are setting new benchmarks for numerous real-world datasets, recent works have pointed out that their induced attentions are less robust and generalizable against noisy graphs due to lack of direct supervision. In this paper, we present a new framework which utilizes the tool of causality to provide a powerful supervision signal for the learning process of attention functions. Specifically, we estimate the direct causal effect of attention to the final prediction, and then maximize such effect to guide attention attending to more meaningful neighbors. Our method can serve as a plug-and-play module for any canonical attention-based GNNs in an end-to-end fashion. Extensive experiments on a wide range of benchmark datasets illustrated that, by directly supervising attention functions, the model is able to converge faster with a clearer decision boundary, and thus yields better performances

ASP: Automatic Selection of Proxy dataset for efficient AutoML

Deep neural networks have gained great success due to the increasing amounts of data, and diverse effective neural network designs. However, it also brings a heavy computing burden as the amount of training data is proportional to the training time. In addition, a well-behaved model requires repeated trials of different structure designs and hyper-parameters, which may take a large amount of time even with state-of-the-art (SOTA) hyper-parameter optimization (HPO) algorithms and neural architecture search (NAS) algorithms. In this paper, we propose an Automatic Selection of Proxy dataset framework (ASP) aimed to dynamically find the informative proxy subsets of training data at each epoch, reducing the training data size as well as saving the AutoML processing time. We verify the effectiveness and generalization of ASP on CIFAR10, CIFAR100, ImageNet16-120, and ImageNet-1k, across various public model benchmarks. The experiment results show that ASP can obtain better results than other data selection methods at all selection ratios. ASP can also enable much more efficient AutoML processing with a speedup of 2x-20x while obtaining better architectures and better hyper-parameters compared to utilizing the entire dataset.Comment: This paper was actually finished in 202

Familial multinodular goiter syndrome with papillary thyroid carcinomas: mutational analysis of the associated genes in 5 cases from 1 Chinese family

Background: Familial papillary thyroid cancer (fPTC) is recognized as a distinct entity only recently and no fPTC predisposing genes have been identified. Several potential regions and susceptibility loci for sporadic PTC have been reported. We aimed to evaluate the role of the reported susceptibility loci and potential risk genomic region in a Chinese familial multinodular goiter (fMNG) with PTC family. Methods: We sequenced the related risk genomic regions and analyzed the known PTC susceptibility loci in the Chinese family members who consented to join the study. These loci included (1) the point mutations of the BRAF and RET; (2) the possible susceptibility loci to sporadic PTC; and (3) the suggested potential fMNG syndrome with PTC risk region. Results: The members showed no mutations in the common susceptible BRAF and RET genomic region, although contained several different heterozygous alleles in the RET introns. All the members were homozygous for PTC risk alleles of rs966423 (C) at chromosome 2q35, rs2910164 (C) at chromosome 5q24 and rs2439302 (G) at chromosome 8p12; while carried no risk allele of rs4733616 (T) at chromosome 8q24, rs965513 (A) or rs1867277 (A) at chromosome 9q22 which were associated with radiation-related PTC. The frequency of the risk allele of rs944289 (T) but not that of rs116909374 (T) at chromosome 14q13 was increased in the MNG or PTC family members. Conclusions: Our work provided additional evidence to the genetic predisposition to a Chinese familial form of MNG with PTC. The family members carried quite a few risk alleles found in sporadic PTC; particularly, homozygous rs944289 (T) at chromosome 14q13 which was previously shown to be linked to a form of fMNG with PTC. Moreover, the genetic determinants of radiation-related PTC were not presented in this family

Easy Begun is Half Done: Spatial-Temporal Graph Modeling with ST-Curriculum Dropout

Spatial-temporal (ST) graph modeling, such as traffic speed forecasting and taxi demand prediction, is an important task in deep learning area. However, for the nodes in graph, their ST patterns can vary greatly in difficulties for modeling, owning to the heterogeneous nature of ST data. We argue that unveiling the nodes to the model in a meaningful order, from easy to complex, can provide performance improvements over traditional training procedure. The idea has its root in Curriculum Learning which suggests in the early stage of training models can be sensitive to noise and difficult samples. In this paper, we propose ST-Curriculum Dropout, a novel and easy-to-implement strategy for spatial-temporal graph modeling. Specifically, we evaluate the learning difficulty of each node in high-level feature space and drop those difficult ones out to ensure the model only needs to handle fundamental ST relations at the beginning, before gradually moving to hard ones. Our strategy can be applied to any canonical deep learning architecture without extra trainable parameters, and extensive experiments on a wide range of datasets are conducted to illustrate that, by controlling the difficulty level of ST relations as the training progresses, the model is able to capture better representation of the data and thus yields better generalization

Identification and characterization of circular RNAs in mammary gland tissue from sheep at peak lactation and during the nonlactating period

Circular RNAs are a class of noncoding RNA with a widespread occurrence in eukaryote tissues, and with some having been demonstrated to have clear biological function. In sheep, little is known about the role of circular RNAs in mammary gland tissue, and therefore an RNA sequencing approach was used to compare mammary gland tissue expression of circular RNAs in 9 Small Tail Han sheep at peak lactation, and subsequently when they were not lactating. These 9 sheep had their RNA pooled for analysis into 3 libraries from peak lactation and 3 from the nonlactating period. A total of 3,278 and 1,756 circular RNAs were identified in the peak lactation and nonlactating mammary gland tissues, respectively, and the expression and identity of 9 of them was confirmed using reverse transcriptase-polymerase chain reaction analysis and DNA sequencing. The type, chromosomal location and length of the circular RNAs identified were ascertained. Forty upregulated and one downregulated circular RNAs were characterized in the mammary gland tissue at peak lactation compared with the nonlactating mammary gland tissue. Gene ontology enrichment analysis revealed that the parental genes of these differentially expressed circular RNAs were related to molecular function, binding, protein binding, ATP binding, and ion binding. Five differentially expression circular RNAs were selected for further analysis to predict their target microRNAs, and some microRNAs reportedly associated with the development of the mammary gland were found in the constructed circular RNA–microRNA network. This study reveals the expression profiles and characterization of circular RNAs at 2 key stages of mammary gland activity, thereby providing an improved understanding of the roles of circular RNAs in the mammary gland of sheep

Net exchanges of CO2, CH4, and N2O between China's terrestrial ecosystems and the atmosphere and their contributions to global climate warming

Author Posting. © American Geophysical Union, 2011. This article is posted here by permission of American Geophysical Union for personal use, not for redistribution. The definitive version was published in Journal of Geophysical Research 116 (2011): G02011, doi:10.1029/2010JG001393.China's terrestrial ecosystems have been recognized as an atmospheric CO2 sink; however, it is uncertain whether this sink can alleviate global warming given the fluxes of CH4 and N2O. In this study, we used a process-based ecosystem model driven by multiple environmental factors to examine the net warming potential resulting from net exchanges of CO2, CH4, and N2O between China's terrestrial ecosystems and the atmosphere during 1961–2005. In the past 45 years, China's terrestrial ecosystems were found to sequestrate CO2 at a rate of 179.3 Tg C yr−1 with a 95% confidence range of (62.0 Tg C yr−1, 264.9 Tg C yr−1) while emitting CH4 and N2O at rates of 8.3 Tg C yr−1 with a 95% confidence range of (3.3 Tg C yr−1, 12.4 Tg C yr−1) and 0.6 Tg N yr−1 with a 95% confidence range of (0.2 Tg N yr−1, 1.1 Tg N yr−1), respectively. When translated into global warming potential, it is highly possible that China's terrestrial ecosystems mitigated global climate warming at a rate of 96.9 Tg CO2eq yr−1 (1 Tg = 1012 g), substantially varying from a source of 766.8 Tg CO2eq yr−1 in 1997 to a sink of 705.2 Tg CO2eq yr−1 in 2002. The southeast and northeast of China slightly contributed to global climate warming; while the northwest, north, and southwest of China imposed cooling effects on the climate system. Paddy land, followed by natural wetland and dry cropland, was the largest contributor to national warming potential; forest, followed by woodland and grassland, played the most significant role in alleviating climate warming. Our simulated results indicate that CH4 and N2O emissions offset approximately 84.8% of terrestrial CO2 sink in China during 1961–2005. This study suggests that the relieving effects of China's terrestrial ecosystems on climate warming through sequestering CO2 might be gradually offset by increasing N2O emission, in combination with CH4 emission.This study has been supported by NASA LCLUC Program (NNX08AL73G_S01) , NASA IDS Program (NNG04GM39C), and China’s Ministry of Science and Technology (MOST) 973 Program (2002CB412500)