Efficient N:M Sparse DNN Training Using Algorithm, Architecture, and Dataflow Co-Design

Sparse training is one of the promising techniques to reduce the computational cost of DNNs while retaining high accuracy. In particular, N:M fine-grained structured sparsity, where only N out of consecutive M elements can be nonzero, has attracted attention due to its hardware-friendly pattern and capability of achieving a high sparse ratio. However, the potential to accelerate N:M sparse DNN training has not been fully exploited, and there is a lack of efficient hardware supporting N:M sparse training. To tackle these challenges, this paper presents a computation-efficient training scheme for N:M sparse DNNs using algorithm, architecture, and dataflow co-design. At the algorithm level, a bidirectional weight pruning method, dubbed BDWP, is proposed to leverage the N:M sparsity of weights during both forward and backward passes of DNN training, which can significantly reduce the computational cost while maintaining model accuracy. At the architecture level, a sparse accelerator for DNN training, namely SAT, is developed to neatly support both the regular dense operations and the computation-efficient N:M sparse operations. At the dataflow level, multiple optimization methods ranging from interleave mapping, pre-generation of N:M sparse weights, and offline scheduling, are proposed to boost the computational efficiency of SAT. Finally, the effectiveness of our training scheme is evaluated on a Xilinx VCU1525 FPGA card using various DNN models and datasets. Experimental results show the SAT accelerator with the BDWP sparse training method under 2:8 sparse ratio achieves an average speedup of 1.75x over that with the dense training, accompanied by a negligible accuracy loss of 0.56% on average. Furthermore, our proposed training scheme significantly improves the training throughput by 2.97~25.22x and the energy efficiency by 1.36~3.58x over prior FPGA-based accelerators.Comment: To appear in the IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD

Collaborative Rebate Strategy of Business-to-Customer Platforms Considering Recycling and Trade-Ins Simultaneously

B2C (business to customer) platforms like JD.com and Suning.com often cooperate with professional recycling companies, and implement recycling programs and trade-in programs simultaneously, especially for electronic products. The former means that platforms recycle old products from customers with cash, whereas the latter means that platforms allow customers to trade in old products for new ones. Under this background, we discuss how to develop the optimal rebate strategy for B2C platforms based on the market recovery price of old products, and give the optimal rebate prices and feasible conditions of single-rebate, dual-rebate, and none-rebate strategies. The results show that the single-recycling rebate strategy is dominant when the residual value of old products is low, and when the residual value of old products is high, platforms should choose in turn the single-trade-in rebate strategy, dual-rebate strategy, single-recycling rebate strategy, and non-rebate strategy with the increase in the cost of new products. In order to obtain higher profits, B2C platforms should provide appropriate rebates to better coordinate the recycling program and the trade-in program on the basis of the market recovery price, the residual value, and the durability of old products as well as the cost, the selling price, and the upgrade range of new products

Advance selling and service cancelation when consumers are overconfident

Purpose: This paper aims to study the joint decision making of advance selling and service cancelation for service provides with limited capacity when consumers are overconfident. Design/methodology/approach: For the case in which consumers encounter uncertainties about product valuation and consumption states in the advance period and are overconfident about the probability of a good state, we study how the service provider chooses the optimal sales strategy among the non-advance selling strategy, the advance selling and disallowing cancelation strategy, and the advance selling and allowing cancelation strategy. We also discuss how overconfidence influences the service provider’s decision making. Findings: The results show that when service capacity is sufficient, the service provider should adopt advance selling and disallow cancelation; when service capacity is insufficient, the service provider should still implement advance selling but allow cancelation; and when service capacity is extremely insufficient, the service provider should offer spot sales. Moreover, overconfidence weakens the necessity to allow cancelation under sufficient service capacity and enhances it under insufficient service capacity but is always advantageous to advance selling. Practical implications: The obtained results provide managerial insights for service providers to make advance selling decisions. Originality/value: This paper is among the first to explore the effect of consumers’ overconfidence on the joint decision of advance selling and service cancelation under capacity constraints

Compressive hard thresholding pursuit algorithm for sparse signal recovery

Hard Thresholding Pursuit (HTP) is one of the important and efficient algorithms for reconstructing sparse signals. Unfortunately, the hard thresholding operator is independent of the objective function and hence leads to numerical oscillation in the course of iterations. To alleviate this drawback, the hard thresholding operator should be applied to a compressible vector. Motivated by this idea, we propose a new algorithm called Compressive Hard Thresholding Pursuit (CHTP) by introducing a compressive step first to the standard HTP. Convergence analysis and stability of CHTP are established in terms of the restricted isometry property of a sensing matrix. Numerical experiments show that CHTP is competitive with other mainstream algorithms such as the HTP, Orthogonal Matching Pursuit (OMP) and Subspace Pursuit (SP) algorithms both in the sparse signal reconstruction ability and average recovery runtime

Rotational Differential-Linear Distinguishers of ARX Ciphers with Arbitrary Output Linear Masks

The rotational differential-linear attacks, proposed at EUROCRYPT 2021, is a generalization of differential-linear attacks by replacing the differential part of the attacks with rotational differentials. At EUROCRYPT 2021, Liu et al. presented a method based on Morawiecki et al.’s technique (FSE 2013) for evaluating the rotational differential-linear correlations for the special cases where the output linear masks are unit vectors. With this method, some powerful (rotational) differential-linear distinguishers with output linear masks being unit vectors against Friet, Xoodoo, and Alzette were discovered. However, how to compute the rotational differential-linear correlations for arbitrary output masks was left open. In this work, we partially solve this open problem by presenting an efficient algorithm for computing the (rotational) differential-linear correlation of modulo additions for arbitrary output linear masks, based on which a technique for evaluating the (rotational) differential-linear correlation of ARX ciphers is derived. We apply the technique to Alzette, Siphash, Chacha, and Speck. As a result, significantly improved (rotational) differential-linear distinguishers including deterministic ones are identified. All results of this work are practical and experimentally verified to confirm the validity of our methods. In addition, we try to explain the experimental distinguishers employed in FSE 2008, FSE 2016, and CRYPTO 2020 against Chacha. The predicted correlations are close to the experimental ones

A Note on the Bias of Rotational Differential-Linear Distinguishers

This note solves the open problem of finding a closed formula for the bias of a rotational differential-linear distinguisher proposed in IACR ePrint 2021/189 (EUROCRYPT 2021), completely generalizing the results on ordinary differential-linear distinguishers due to Blondeau, Leander, and Nyberg (JoC 2017) to the case of rotational differential-linear distinguishers

Characterization and identification of the integrin family in silkworm, Bombyx mori

YesAs an important economic insect, Bombyx mori is also a useful model organism for lepidopteran insect. Integrins are evolutionarily conserved fromsponges to humans, and play vital roles inmany physiological and pathological processes. To explore their diverse functions of integrins in insect, eleven integrins including sixα and five β subunitswere cloned and characterized fromsilkworm. Our results showed that integrins fromsilkwormown more family members compared to other invertebrates. Among those α subunits, integrins α1, α2, and the other four subunits belong to PS1, PS2, and PS3 groups, respectively. The β subunits mainly gather in the insect βν group except the β1 subunit which belongs to the insect β group. Expression profiles demonstrated that the integrins exhibited distinct patterns, but were mainly expressed in hemocytes. α1 and β2 subunits are the predominant ones either in the embryogenesis or larva stages. Interestingly, integrins were significantly up-regulated after stimulated by 20-hydroxyecdysone (20-E) in vivo. These results indicate that integrins performdiverse functions in hemocytes of silkworm. Overall, our results provide a newinsight into the functional and evolutionary features of integrins.National Basic Research Programof China (No. 2012cb114603), the Research Fund for the Doctoral Program of Higher Education of China (20130182110003), the Natural Science Foundation of Chongqing (cstc2013jcyjys0007), and the Fundamental Research Funds for the Central Universities (SWU111014)

A Reconstruction Method for Hyperspectral Remote Sensing Reflectance in the Visible Domain and Applications

A reconstruction method was developed for hyperspectral remote sensing reflectance (Rrs)data in the visible domain (400–700 nm) based on in situ observations. A total of 2,647 Rrs spectra were collected over a wide variety of water environments including open ocean, coastal and inland waters. Ten schemes with different band numbers (6 to 15) were tested based on a nonlinear model. It was found that the accuracy of the reconstruction increased with the increase of input band numbers. Eight of these schemes met the accuracy criterion with the mean absolute error (MAE) and mean relative error (MRE)values between reconstructed and in situ Rrs less than 0.00025 sr-1 and 5%, respectively. We chose the eight-band scheme for further evaluation because of its decent performance. The results revealed that the parameterization derived by the eight-band scheme was efficient for restoring Rrs spectra from different water bodies. In contrast to the previous studies that used a linear model with 15 spectral bands, the nonlinear model with the eight-band scheme yielded a comparable reconstruction performance. The MAE andMRE values were generally less than 0.00016 sr-1 and 3% respectively; much lower than the uncertainties in satellite-derived Rrs products. Furthermore, a preliminary experiment of this method on the data from the Hyperspectral Imager for the Coastal Ocean (HICO) showed high potential in the future applications for reconstructing Rrs spectra from space-borne optical sensors. Overall, the eight-band scheme with our non-linear model was proven to be optimal for hyperspectral Rrs reconstruction in the visible domain

Unfaithful Maintenance of Methylation Imprints Due to Loss of Maternal Nuclear Dnmt1 during Somatic Cell Nuclear Transfer

The low success rate of somatic cell nuclear transfer (SCNT) in mammalian cloning is largely due to imprinting problems. However, little is known about the mechanisms of reprogramming imprinted genes during SCNT. Parental origin-specific DNA methylation regulates the monoallelic expression of imprinted genes. In natural fertilization, methylation imprints are established in the parental germline and maintained throughout embryonic development. However, it is unclear whether methylation imprints are protected from global changes of DNA methylation in cloned preimplantation embryos. Here, we demonstrate that cloned porcine preimplantation embryos exhibit demethylation at differentially methylated regions (DMRs) of imprinted genes; in particular, demethylation occurs during the first two cell cycles. By RNAi-mediated knockdown, we found that Dnmt1 is required for the maintenance of methylation imprints in porcine preimplantation embryos. However, no clear signals were detected in the nuclei of oocytes and preimplantation embryos by immunofluorescence. Thus, Dnmt1 is present at very low levels in the nuclei of porcine oocytes and preimplantation embryos and maintains methylation imprints. We further showed that methylation imprints were rescued in nonenucleated metaphase II (MII) oocytes. Our results indicate that loss of Dnmt1 in the maternal nucleus during SCNT significantly contributes to the unfaithful maintenance of methylation imprints in cloned embryos