Dual Discriminator Adversarial Distillation for Data-free Model Compression

Knowledge distillation has been widely used to produce portable and efficient neural networks which can be well applied on edge devices for computer vision tasks. However, almost all top-performing knowledge distillation methods need to access the original training data, which usually has a huge size and is often unavailable. To tackle this problem, we propose a novel data-free approach in this paper, named Dual Discriminator Adversarial Distillation (DDAD) to distill a neural network without any training data or meta-data. To be specific, we use a generator to create samples through dual discriminator adversarial distillation, which mimics the original training data. The generator not only uses the pre-trained teacher's intrinsic statistics in existing batch normalization layers but also obtains the maximum discrepancy from the student model. Then the generated samples are used to train the compact student network under the supervision of the teacher. The proposed method obtains an efficient student network which closely approximates its teacher network, despite using no original training data. Extensive experiments are conducted to to demonstrate the effectiveness of the proposed approach on CIFAR-10, CIFAR-100 and Caltech101 datasets for classification tasks. Moreover, we extend our method to semantic segmentation tasks on several public datasets such as CamVid and NYUv2. All experiments show that our method outperforms all baselines for data-free knowledge distillation

Preparation of Green Biosorbent using Rice Hull to Preconcentrate, Remove and Recover Heavy Metal and Other Metal Elements from Water

Sodium hydroxide treated rice hulls were investigated to preconcentrate, remove, and recover metal ions including Be2+, Al3+, Cr3+, CO2+, Ni2+, Cu2+, Zn2+, Sr2+, Ag+, Cd2+, Ba2+, and Pb2+ in both batch mode and column mode. Sodium hydroxide treatment significantly improved the removal efficiency for all metal ions of interest compared to the untreated rice hull. The removal kinetics were extremely fast for Co, Ni, Cu, Zn, Sr, Cd, and Ba, which made the treated rice hull a promising economic green adsorbent to preconcentrate, remove, and recover low-level metal ions in column mode at relatively high throughput. The principal removal mechanism is believed to be the electrostatic attraction between the negatively charged rice hulls and the positively charged metal ions. pH had a drastic impact on the removal for different metal ions and a pH of 5 worked best for most of the metal ions of interest. Processed rice hulls provide an economic alternative to costly resins that are currently commercially available products designed for metal ion preconcentration for trace metal analysis, and more importantly, for toxic heavy metal removal and recovery from the environment

Deep Learning Methods for Calibrated Photometric Stereo and Beyond

Photometric stereo recovers the surface normals of an object from multiple images with varying shading cues, i.e., modeling the relationship between surface orientation and intensity at each pixel. Photometric stereo prevails in superior per-pixel resolution and fine reconstruction details. However, it is a complicated problem because of the non-linear relationship caused by non-Lambertian surface reflectance. Recently, various deep learning methods have shown a powerful ability in the context of photometric stereo against non-Lambertian surfaces. This paper provides a comprehensive review of existing deep learning-based calibrated photometric stereo methods. We first analyze these methods from different perspectives, including input processing, supervision, and network architecture. We summarize the performance of deep learning photometric stereo models on the most widely-used benchmark data set. This demonstrates the advanced performance of deep learning-based photometric stereo methods. Finally, we give suggestions and propose future research trends based on the limitations of existing models.Comment: 19 pages, 11 figures, 4 table

Explaining the differences of gait patterns between high and low-mileage runners with machine learning

Running gait patterns have implications for revealing the causes of injuries between higher-mileage runners and low-mileage runners. However, there is limited research on the possible relationships between running gait patterns and weekly running mileages. In recent years, machine learning algorithms have been used for pattern recognition and classification of gait features to emphasize the uniqueness of gait patterns. However, they all have a representative problem of being a black box that often lacks the interpretability of the predicted results of the classifier. Therefore, this study was conducted using a Deep Neural Network (DNN) model and Layer-wise Relevance Propagation (LRP) technology to investigate the differences in running gait patterns between higher-mileage runners and low-mileage runners. It was found that the ankle and knee provide considerable information to recognize gait features, especially in the sagittal and transverse planes. This may be the reason why high-mileage and low-mileage runners have different injury patterns due to their different gait patterns. The early stages of stance are very important in gait pattern recognition because the pattern contains effective information related to gait. The findings of the study noted that LRP completes a feasible interpretation of the predicted results of the model, thus providing more interesting insights and more effective information for analyzing gait patterns

Anchor Retouching via Model Interaction for Robust Object Detection in Aerial Images

Object detection has made tremendous strides in computer vision. Small object detection with appearance degradation is a prominent challenge, especially for aerial observations. To collect sufficient positive/negative samples for heuristic training, most object detectors preset region anchors in order to calculate Intersection-over-Union (IoU) against the ground-truthed data. In this case, small objects are frequently abandoned or mislabeled. In this paper, we present an effective Dynamic Enhancement Anchor (DEA) network to construct a novel training sample generator. Different from the other state-of-the-art techniques, the proposed network leverages a sample discriminator to realize interactive sample screening between an anchor-based unit and an anchor-free unit to generate eligible samples. Besides, multi-task joint training with a conservative anchor-based inference scheme enhances the performance of the proposed model while reducing computational complexity. The proposed scheme supports both oriented and horizontal object detection tasks. Extensive experiments on two challenging aerial benchmarks (i.e., DOTA and HRSC2016) indicate that our method achieves state-of-the-art performance in accuracy with moderate inference speed and computational overhead for training. On DOTA, our DEA-Net which integrated with the baseline of RoI-Transformer surpasses the advanced method by 0.40% mean-Average-Precision (mAP) for oriented object detection with a weaker backbone network (ResNet-101 vs ResNet-152) and 3.08% mean-Average-Precision (mAP) for horizontal object detection with the same backbone. Besides, our DEA-Net which integrated with the baseline of ReDet achieves the state-of-the-art performance by 80.37%. On HRSC2016, it surpasses the previous best model by 1.1% using only 3 horizontal anchors

Genome-wide association study reveals genetic loci and candidate genes for meat quality traits in a four-way crossbred pig population

Meat quality traits (MQTs) have gained more attention from breeders due to their increasing economic value in the commercial pig industry. In this genome-wide association study (GWAS), 223 four-way intercross pigs were genotyped using the specific-locus amplified fragment sequencing (SLAF-seq) and phenotyped for PH at 45 min post mortem (PH45), meat color score (MC), marbling score (MA), water loss rate (WL), drip loss (DL) in the longissimus muscle, and cooking loss (CL) in the psoas major muscle. A total of 227, 921 filtered single nucleotide polymorphisms (SNPs) evenly distributed across the entire genome were detected to perform GWAS. A total of 64 SNPs were identified for six meat quality traits using the mixed linear model (MLM), of which 24 SNPs were located in previously reported QTL regions. The phenotypic variation explained (PVE) by the significant SNPs was from 2.43% to 16.32%. The genomic heritability estimates based on SNP for six meat-quality traits were low to moderate (0.07–0.47) being the lowest for CL and the highest for DL. A total of 30 genes located within 10 kb upstream or downstream of these significant SNPs were found. Furthermore, several candidate genes for MQTs were detected, including pH45 (GRM8), MC (ANKRD6), MA (MACROD2 and ABCG1), WL (TMEM50A), CL (PIP4K2A) and DL (CDYL2, CHL1, ABCA4, ZAG and SLC1A2). This study provided substantial new evidence for several candidate genes to participate in different pork quality traits. The identification of these SNPs and candidate genes provided a basis for molecular marker-assisted breeding and improvement of pork quality traits

Imaging-guided synergistic targeting-promoted photo-chemotherapy against cancers by methotrexate-conjugated hyaluronic acid nanoparticles

Abstract(#br)A combination of chemotherapy and photothermal therapy (PTT) against cancer, overcoming the intrinsic limitations of single-modal chemotherapy or PTT, has emerged as a promising strategy to achieve synergistic therapeutic effect. However, the lack of precise drug delivery and intelligent drug release based on photo-chemotherapy at specific tumor sites remained a challenge. Hence, the both tumor-specific targeting molecule (methotrexate) and ligand (hyaluronic acid)-introduced, glutathione-responsive amphiphiles (deoxycholic acid-hyaluronic acid-methotrexate, DA-SS-HA-MTX) were developed for synchronous delivery of indocyanine green (ICG) and doxorubicin (DOX). The as-synthesized DOX/ICG@DSHM remarkably improved the intracellular drug uptake and accumulation owing to both the CD44/folate receptors-mediated synergistic targeting and the glutathione-triggered rapid drug release. Moreover, DOX/ICG@DSHM efficiently accumulated at the tumor sites, realizing the notable tumor ablation under the guidance of dual-modal optical imaging. Taken together, this study provided a promising nanotheranostic agent for imaging-guided chemo-photothermal combination therapy

Solvothermal Synthesis and Characterization of Chalcopyrite CuInSe2 Nanoparticles

The ternary I-III-VI2 semiconductor of CuInSe2 nanoparticles with controllable size was synthesized via a simple solvothermal method by the reaction of elemental selenium powder and CuCl as well as InCl3 directly in the presence of anhydrous ethylenediamine as solvent. X-ray diffraction patterns and scanning electron microscopy characterization confirmed that CuInSe2 nanoparticles with high purity were obtained at different temperatures by varying solvothermal time, and the optimal temperature for preparing CuInSe2 nanoparticles was found to be between 180 and 220 °C. Indium selenide was detected as the intermediate state at the initial stage during the formation of pure ternary compound, and the formation of copper-related binary phase was completely deterred in that the more stable complex [Cu(C2H8N2)2]+ was produced by the strong N-chelation of ethylenediamine with Cu+. These CuInSe2 nanoparticles possess a band gap of 1.05 eV calculated from UV–vis spectrum, and maybe can be applicable to the solar cell devices

AXL targeting restores PD-1 blockade sensitivity of STK11/LKB1 mutant NSCLC through expansion of TCF1+ CD8 T cells

Mutations in STK11/LKB1 in non-small cell lung cancer (NSCLC) are associated with poor patient responses to immune checkpoint blockade (ICB), and introduction of a Stk11/Lkb1 (L) mutation into murine lung adenocarcinomas driven by mutant Kras and Trp53 loss (KP) resulted in an ICB refractory syngeneic KPL tumor. Mechanistically this occurred because KPL mutant NSCLCs lacked TCF1-expressing CD8 T cells, a phenotype recapitulated in human STK11/LKB1 mutant NSCLCs. Systemic inhibition of Axl results in increased type I interferon secretion from dendritic cells that expanded tumor-associated TCF1+PD-1+CD8 T cells, restoring therapeutic response to PD-1 ICB in KPL tumors. This was observed in syngeneic immunocompetent mouse models and in humanized mice bearing STK11/LKB1 mutant NSCLC human tumor xenografts. NSCLC-affected individuals with identified STK11/LKB1 mutations receiving bemcentinib and pembrolizumab demonstrated objective clinical response to combination therapy. We conclude that AXL is a critical targetable driver of immune suppression in STK11/LKB1 mutant NSCLC.publishedVersio