Holistic CNN Compression via Low-rank Decomposition with Knowledge Transfer

近日，国际顶级学术刊物《IEEE Transactions on Pattern Analysis and Machine Intelligence》（PAMI）接收了厦门大学信息科学与技术学院纪荣嵘团队的最新研究成果“Holistic CNN Compression via Low-rank Decomposition with Knowledge Transfer”。PAMI是计算机科学领域最顶级的国际期刊，其影响因子为 9.45。 该论文提出了一种统一的全局卷积神经网络压缩框架，简称为LRDKT，其目标在于统一加速与压缩卷积神经网络。该工作是厦门大学博士生林绍辉和导师纪荣嵘教授团队的阶段性研究成果，目前论文相关代码已开源。团队该方向的前期成果已经发表在AAAI/IJCAI等CCF-A类国际会议上。该论文由我校博士生林绍辉与其导师纪荣嵘教授（通讯作者）、硕士研究生陈超、悉尼大学陶大成教授、美国罗彻斯特大学罗杰波教授等合作完成，这也是我校研究生第二次在计算机领域的最顶级刊物上以第一作者身份发表论文，标志着我校信息学科研究生培养质量的突破。【Abstract】Convolutional neural networks (CNNs) have achieved remarkable success in various computer vision tasks, which are extremely powerful to deal with massive training data by using tens of millions of parameters. However, CNNs often cost significant memory and computation consumption, which prohibits their usage in resource-limited environments such as mobile or embedded devices. To address the above issues, the existing approaches typically focus on either accelerating the convolutional layers or compressing the fully-connected layers separatedly, without pursuing a joint optimum. In this paper, we overcome such a limitation by introducing a holistic CNN compression framework, termed LRDKT, which works throughout both convolutional and fully-connected layers. First, a low-rank decomposition (LRD) scheme is proposed to remove redundancies across both convolutional kernels and fully-connected matrices, which has a novel closed-form solver to significantly improve the efficiency of the existing iterative optimization solvers. Second, a novel knowledge transfer (KT) based training scheme is introduced. To recover the accumulated accuracy loss and overcome the vanishing gradient, KT explicitly aligns outputs and intermediate responses from a teacher (original) network to its student (compressed) network. We have comprehensively analyzed and evaluated the compression and speedup ratios of the proposed model on MNIST and ILSVRC 2012 benchmarks. In both benchmarks, the proposed scheme has demonstrated superior performance gains over the state-of-the-art methods. We also demonstrate the proposed compression scheme for the task of transfer learning,including domain adaptation and object detection, which show exciting performance gains over the state-of-the-arts. Our source code and compressed models are available at https://github.com/ShaohuiLin/LRDKT.This work is supported by the National Key R&D Program (No.2017YFC0113000, No.2016YFB1001503), Natural Science Foundation of China (No.U1705262, No.61705262,No.61772443, No.61572410). 该项研究得到了国家重点研发专项（No.2017YFC0113000, and No.2016YFB1001503）、国家自然科学基金联合重点项目（No.U1705262）的资助

Presumptive Differentiation of Phytopathogenic and Non-pathogenic Bacteria by Improved Rapid-Extraction TLC Method

TLC profiles of aminolipids extracted from phytopathogenic and non-pathogenic bacteria with chloroform methanol-0.3%NaCl (2:1:0.2, v/v/v) or 2-propanol are useful for discrimination of bacteria. For many bacteria, each TLC profile is genus or species specific and highly reproducible. For most gram-negative bacteria, the uppermost spot (Up) appeared at ca. Rf 0.7 on the chromatograms developed with chloroform- methanol-0.2% CaCl2・2H2O (55:35:8, v/v/v). This spot was absent on the chromatograms of gram-positive bacteria, Clavibacter michiganensis. The profiles of Agrobacterium spp. and Rhizobium spp. were different from other gram-negative bacteria with the uppermost spots at ca. Rf 0.75. For the case of Agrobacterium spp., the chromatograms of the strains belonging to the same biovar were identical. Distinct differences were found among the profiles of Agrobacterium biovar 1, A. biovar 2, A. biovar 3 and A.rubi. The profiles of Rhizobium spp., except for R. tropici, and their relatives such as Bradyrhizobium, Mesorhizobium and Sinorhizobium spp. were quite simple and different from those of Agrobacterium spp. For the case of Burkholderia species, except for B. andropogonis, three spots (designated as S1, S2, S3) appeared under the uppermost spot (Up) and their profiles were species specific for several species such as B. plantarii and B. caryophylli. On the chromatogram of B. andropogonis, the S1 spot (non-phosphorous) was absent and the S3 spot was faint. The profiles of 96 Ralstonia solanacearum strains from various sources were identical. For the case of Erwinia carotovora an intensive benchmark spot appeared at Rf 0.64 but this spot was absent on the chromatograms of pathovars of E. chrysanthemi and E. herbicola. Clear diversity in profiles was observed between Xanthomonas campestris and X. oryzae. The profile of pathovars of Pseudomonas syringae were identical and simple. Substitution of chloroform solvent systems with less hazardous organic solvents was tested. 2-propanol for the lipid extraction and 1-butanol-acetic acid-water (3:1:1 and 5:3:1, v/v/v) for the developing solvents were usable, though development with the butanol systems was highly time-consuming

中国-巴基斯坦喀喇昆仑公路沿线野生动物区系研究/Wild Fauna Along the China - Pakistan Kara - Koram Highway[J]

中国-巴基斯坦喀喇昆仑公路(以下简称"中巴公路")是中国政府于20世纪60年代中期援建的一条连接中国西部城市喀什和巴基斯坦北部城市塔科特的国际公路.2009年4月、7月、9月和2010年6～7月对中巴公路沿线分布的动物进行了考察,经全面系统分类、整理和统计,该地区共有陆栖野生动物22目51科105属147种,其中爬行类1目3科5属8种,鸟类14目34科72属103种,哺乳类/兽类7目14科28属36种.从野生动物分布的种类数量上看,分布最少的是爬行类(8种),而较多的是鸟类(103种)和哺乳类/兽类(36种),分别占该地区分布野生脊椎动物种数的5.44%、70.07%和24.49%.野生动物区系以北方广布种、高地型、古北型和中亚型为主,全北型、东洋型和喜马拉雅-横断山型渗透混杂,表明古北界温带陆栖动物地理属性是该地区动物区系的基本特征之所在

巴基斯坦红其拉甫国家公园种子植物区系分析/Seed-plant flora in Khunjerab National Park of Pakistan[J]

红其拉甫国家公园(国家自然保护区)位于巴基斯坦北部地区,喀喇昆仑山南坡,介于36°04′～37°05′N和74°38′～75°45′E之间,海拔3 200～6 000 m之间,平均海拔4 000 m,总面积约2 270 km2,建于1975年.通过实地考察和查阅相关文献资料,对分布于该地区的植物从区系组成、区系性质的角度进行了分析.结果表明,该地区共有野生种子植物38科149属369种,其中裸子植物2科2属5种,被子植物36科147属364种.按照吴征镒先生对中国种子植物所划分的15个分布区类型,对所整理和统计的种子植物进行地理成分分析表明,该地区分布的植物可归入10个分布区类型和13个变型.其中北温带分布的属、种最多,有72属199种,占该地区植物总属、种数的56.68%和71.56%.由此表明温带属性是该地区植物区系的基本特点.与此同时,该地区植物区系与旧世界温带及地中海-西亚-中亚交流最多,其次与温带亚洲和东亚有一定的交流,而与热带交流很少