Rethinking Attention Mechanism in Time Series Classification

Attention-based models have been widely used in many areas, such as computer vision and natural language processing. However, relevant applications in time series classification (TSC) have not been explored deeply yet, causing a significant number of TSC algorithms still suffer from general problems of attention mechanism, like quadratic complexity. In this paper, we promote the efficiency and performance of the attention mechanism by proposing our flexible multi-head linear attention (FMLA), which enhances locality awareness by layer-wise interactions with deformable convolutional blocks and online knowledge distillation. What's more, we propose a simple but effective mask mechanism that helps reduce the noise influence in time series and decrease the redundancy of the proposed FMLA by masking some positions of each given series proportionally. To stabilize this mechanism, samples are forwarded through the model with random mask layers several times and their outputs are aggregated to teach the same model with regular mask layers. We conduct extensive experiments on 85 UCR2018 datasets to compare our algorithm with 11 well-known ones and the results show that our algorithm has comparable performance in terms of top-1 accuracy. We also compare our model with three Transformer-based models with respect to the floating-point operations per second and number of parameters and find that our algorithm achieves significantly better efficiency with lower complexity

A Multiobjective Computation Offloading Algorithm for Mobile Edge Computing

In mobile edge computing (MEC), smart mobile devices (SMDs) with limited computation resources and battery lifetime can offload their computing-intensive tasks to MEC servers, thus to enhance the computing capability and reduce the energy consumption of SMDs. Nevertheless, offloading tasks to the edge incurs additional transmission time and thus higher execution delay. This paper studies the trade-off between the completion time of applications and the energy consumption of SMDs in MEC networks. The problem is formulated as a multiobjective computation offloading problem (MCOP), where the task precedence, i.e. ordering of tasks in SMD applications, is introduced as a new constraint in the MCOP. An improved multiobjective evolutionary algorithm based on decomposition (MOEA/D) with two performance enhancing schemes is proposed.1) The problem-specific population initialization scheme uses a latency-based execution location initialization method to initialize the execution location (i.e. either local SMD or MEC server) for each task. 2) The dynamic voltage and frequency scaling based energy conservation scheme helps to decrease the energy consumption without increasing the completion time of applications. The simulation results clearly demonstrate that the proposed algorithm outperforms a number of state-of-the-art heuristics and meta-heuristics in terms of the convergence and diversity of the obtained nondominated solutions

CapMatch: Semi-Supervised Contrastive Transformer Capsule With Feature-Based Knowledge Distillation for Human Activity Recognition

This article proposes a semi-supervised contrastive capsule transformer method with feature-based knowledge distillation (KD) that simplifies the existing semisupervised learning (SSL) techniques for wearable human activity recognition (HAR), called CapMatch. CapMatch gracefully hybridizes supervised learning and unsupervised learning to extract rich representations from input data. In unsupervised learning, CapMatch leverages the pseudolabeling, contrastive learning (CL), and feature-based KD techniques to construct similarity learning on lower and higher level semantic information extracted from two augmentation versions of the data“, weak” and “timecut”, to recognize the relationships among the obtained features of classes in the unlabeled data. CapMatch combines the outputs of the weak-and timecut-augmented models to form pseudolabeling and thus CL. Meanwhile, CapMatch uses the feature-based KD to transfer knowledge from the intermediate layers of the weak-augmented model to those of the timecut-augmented model. To effectively capture both local and global patterns of HAR data, we design a capsule transformer network consisting of four capsule-based transformer blocks and one routing layer. Experimental results show that compared with a number of state-of-the-art semi-supervised and supervised algorithms, the proposed CapMatch achieves decent performance on three commonly used HAR datasets, namely, HAPT, WISDM, and UCI_HAR. With only 10% of data labeled, CapMatch achieves F1 values of higher than 85.00% on these datasets, outperforming 14 semi-supervised algorithms. When the proportion of labeled data reaches 30%, CapMatch obtains F1 values of no lower than 88.00% on the datasets above, which is better than several classical supervised algorithms, e.g., decision tree and k -nearest neighbor (KNN)

DTCM: Deep Transformer Capsule Mutual Distillation for Multivariate Time Series Classification

This paper proposes a dual-network-based feature extractor, perceptive capsule network (PCapN), for multivariate time series classification (MTSC), including a local feature network (LFN) and a global relation network (GRN). The LFN has two heads (i.e., Head_A and Head_B), each containing two squash CNN blocks and one dynamic routing block to extract the local features from the data and mine the connections among them. The GRN consists of two capsule-based transformer blocks and one dynamic routing block to capture the global patterns of each variable and correlate the useful information of multiple variables. Unfortunately, it is difficult to directly deploy PCapN on mobile devices due to its strict requirement for computing resources. So, this paper designs a lightweight capsule network (LCapN) to mimic the cumbersome PCapN. To promote knowledge transfer from PCapN to LCapN, this paper proposes a deep transformer capsule mutual (DTCM) distillation method. It is targeted and offline, using one- and two-way operations to supervise the knowledge distillation process for the dual-network-based student and teacher models. Experimental results show that the proposed PCapN and DTCM achieve excellent performance on UEA2018 datasets regarding top-1 accuracy

Health Risk Assessment of Vegetables Grown on the Contaminated Soils in Daye City of Hubei Province, China

China is an agriculturally-producing country and the safety of its vegetables will have an extensive attention at home and abroad. Recently, contamination of soils and vegetables caused by mining activities is of great social concern because of the potential risk to human health, especially to the residents whom live near metal or metalloid mines. In this study, 18 topsoil and 141 vegetable samples were collected from the contaminated areas in Daye City Hubei Province, China and the concentrations of copper (Cu), zinc (Zn), arsenic (As), cadmium (Cd) and lead (Pb) were analyzed. A self-designed questionnaire was assigned to obtain the exposure scenario and the USEPA health risk assessment model was adopted to assess two type of risks (non-carcinogenic risks and carcinogenic risks) of vegetables to humans. The results showed that the average contents of metal(loid)s in soils exceeded the background value of Daye City. The average contents of metal(loid)s, especially As, Cd, Pb, in three kinds of vegetables were significantly higher than the permissible values based on Chinese national standard. Leafy vegetables had relatively higher concentrations and the transfer factors of As (0.015), Cd (0.080) and Pb (0.003) were comparable to leguminous and fruit vegetables. Leguminous vegetables had relatively higher concentrations and transfer factors of Cu (0.032) and Zn (0.094) than leafy and fruit vegetables. The transfer factors from soil to plants follows a decreasing order as Cd (0.068), Zn (0.047) > Cu (0.023) > As (0.006), Pb (0.002). Furthermore, health risk assessment revealed the following results: the non-carcinogenic risk decreased in the order of children, adult, adolescent, while the carcinogenic risk followed a decreasing order of adult, adolescent, children; the calculated carcinogenic and non-carcinogenic risk of the metal(loid)s by vegetable consumption decreased in the order of leafy vegetables > fruit vegetables > leguminous vegetables. The relatively lower transfer factors and lower risks may suggest that leguminous and fruit vegetables are more suitable for planting in Daye City. Based on the contributions of five kinds of metal(loid)s from three types of vegetables, Cd and As are found to be the dominant sources of health risk

Bulk phosphorous deposition at four typical land use sites in Southwest China

Whilst ongoing increases in the deposition of atmospheric nitrogen (N) in China have attracted a lot of attention, to date there has been little research on phosphorus (P) deposition. In this study, we quantified inorganic P (PO43−), dissolved organic P (DOP) and total P (TP) in bulk deposition at four sites in the Sichuan Basin, Southwest China. Chengdu (CD), Shifang (SF), Yanting (YT), and Gongga (GG). They represent the land use categories urban, suburban, agricultural and forest, respectively, during 2008–2018 at CD and YT, 2015–2018 at SF, and 2007–2014 at GG. Annual average TP concentrations (deposition rates) were 0.07 (0.61), 0.49 (3.22), 0.17 (1.07) and 0.01 (0.20) mg L−1 (kg ha−1 yr−1), at CD, SF, YT and GG, respectively. The TP concentrations at YT and GG showed significant increasing trends over the years, with very little change at CD and a decline at SF because of the implementation of environmental control measures. Average PO43− to DOP ratios were 0.65, 0.95, 0.82 and 0.81 at CD, SF, YT and GG, respectively, indicating that DOP accounts for a higher proportion at the urban site, and dominated by combustion sources. Bulk P deposition showed higher deposition rates in summer and lower in winter. These results highlight the importance of long term monitoring in detecting spatial and temporal changing trends of the chemical composition, so as to implement effective policies to eliminate air pollution, especially for Southwest China, where there is limited research on atmospheric P deposition

How many human genes can be defined as housekeeping with current expression data?

BACKGROUND: Housekeeping (HK) genes are ubiquitously expressed in all tissue/cell types and constitute a basal transcriptome for the maintenance of basic cellular functions. Partitioning transcriptomes into HK and tissue-specific (TS) genes relatively is fundamental for studying gene expression and cellular differentiation. Although many studies have aimed at large-scale and thorough categorization of human HK genes, a meaningful consensus has yet to be reached. RESULTS: We collected two latest gene expression datasets (both EST and microarray data) from public databases and analyzed the gene expression profiles in 18 human tissues that have been well-documented by both two data types. Benchmarked by a manually-curated HK gene collection (HK408), we demonstrated that present data from EST sampling was far from saturated, and the inadequacy has limited the gene detectability and our understanding of TS expressions. Due to a likely over-stringent threshold, microarray data showed higher false negative rate compared with EST data, leading to a significant underestimation of HK genes. Based on EST data, we found that 40.0% of the currently annotated human genes were universally expressed in at least 16 of 18 tissues, as compared to only 5.1% specifically expressed in a single tissue. Our current EST-based estimate on human HK genes ranged from 3,140 to 6,909 in number, a ten-fold increase in comparison with previous microarray-based estimates. CONCLUSION: We concluded that a significant fraction of human genes, at least in the currently annotated data depositories, was broadly expressed. Our understanding of tissue-specific expression was still preliminary and required much more large-scale and high-quality transcriptomic data in future studies. The new HK gene list categorized in this study will be useful for genome-wide analyses on structural and functional features of HK genes

How Do Variable Substitution Rates Influence Ka and Ks Calculations?

The ratio of nonsynonymous substitution rate (Ka) to synonymous substitution rate (Ks) is widely used as an indicator of selective pressure at sequence level among different species, and diverse mutation models have been incorporated into several computing methods. We have previously developed a new γ-MYN method by capturing a key dynamic evolution trait of DNA nucleotide sequences, in consideration of varying mutation rates across sites. We now report a further improvement of NG, LWL, MLWL, LPB, MLPB, and YN methods based on an introduction of gamma distribution to illustrate the variation of raw mutation rate over sites. The novelty comes in two ways: (1) we incorporate an optimal gamma distribution shape parameter a into γ-NG, γ-LWL, γ-MLWL, γ-LPB, γ-MLPB, and γ-YN methods; (2) we investigate how variable substitution rates affect the methods that adopt different models as well as the interplay among four evolutional features with respect to Ka/Ks computations. Our results suggest that variable substitution rates over sites under negative selection exhibit an opposite effect on ω estimates compared with those under positive selection. We believe that the sensitivity of our new methods has been improved than that of their original methods under diverse conditions and it is advantageous to introduce novel parameters for Ka/Ks computation

Structure-based virtual screening and molecular dynamics of potential inhibitors targeting sodium-bile acid co-transporter of carcinogenic liver fluke Clonorchis sinensis.

BackgroundClonorchis sinensis requires bile acid transporters as this fluke inhabits bile juice-filled biliary ducts, which provide an extreme environment. Clonorchis sinensis sodium-bile acid co-transporter (CsSBAT) is indispensable for the fluke's survival in the final host, as it circulates taurocholate and prevents bile toxicity in the fluke; hence, it is recognized as a useful drug target.Methodology and principal findingsIn the present study, using structure-based virtual screening approach, we presented inhibitor candidates targeting a bile acid-binding pocket of CsSBAT. CsSBAT models were built using tertiary structure modeling based on a bile acid transporter template (PDB ID: 3zuy and 4n7x) and were applied into AutoDock Vina for competitive docking simulation. First, potential compounds were identified from PubChem (holding more than 100,000 compounds) by applying three criteria: i) interacting more favorably with CsSBAT than with a human homolog, ii) intimate interaction to the inward- and outward-facing conformational states, iii) binding with CsSBAT preferably to natural bile acids. Second, two compounds were identified following the Lipinski's rule of five. Third, other two compounds of molecular weight higher than 500 Da (Mr > 500 Da) were presumed to efficiently block the transporter via a feasible rational screening strategy. Of these candidates, compound 9806452 exhibited the least hepatotoxicity that may enhance drug-likeness properties.ConclusionsIt is proposed that compound 9806452 act as a potential inhibitor toward CsSBAT and further studies are warranted for drug development process against clonorchiasis