Forward and Backward Information Retention for Accurate Binary Neural Networks

Weight and activation binarization is an effective approach to deep neural network compression and can accelerate the inference by leveraging bitwise operations. Although many binarization methods have improved the accuracy of the model by minimizing the quantization error in forward propagation, there remains a noticeable performance gap between the binarized model and the full-precision one. Our empirical study indicates that the quantization brings information loss in both forward and backward propagation, which is the bottleneck of training accurate binary neural networks. To address these issues, we propose an Information Retention Network (IR-Net) to retain the information that consists in the forward activations and backward gradients. IR-Net mainly relies on two technical contributions: (1) Libra Parameter Binarization (Libra-PB): simultaneously minimizing both quantization error and information loss of parameters by balanced and standardized weights in forward propagation; (2) Error Decay Estimator (EDE): minimizing the information loss of gradients by gradually approximating the sign function in backward propagation, jointly considering the updating ability and accurate gradients. We are the first to investigate both forward and backward processes of binary networks from the unified information perspective, which provides new insight into the mechanism of network binarization. Comprehensive experiments with various network structures on CIFAR-10 and ImageNet datasets manifest that the proposed IR-Net can consistently outperform state-of-the-art quantization methods

A specific and rapid method for detecting Bacillus and Acinetobacter species in Daqu

Daqu is a spontaneous, solid-state cereal fermentation product used for saccharification and as a starter culture for Chinese Baijiu production. Bacillus and Acinetobacter, two dominant microbial genera in Daqu, produce enzymes and organic acids that influence the Daqu quality. However, there are no rapid analytical methods for detecting Bacillus and Acinetobacter. We designed primers specific to the genera Bacillus and Acinetobacter to perform genetic comparisons using the 16 S rRNA. After amplification of polymerase chain reaction using specific primers, high-throughput sequencing was performed to detect strains of Bacillus and Acinetobacter. The results showed that the effective amplification rates for Bacillus and Acinetobacter in Daqu were 86.92% and 79.75%, respectively. Thus, we have devised and assessed a method to accurately identify the species associated with Bacillus and Acinetobacter in Daqu, which can also hold significance for bacterial typing and identification

Oxygen Isotope Geochemistry of Phosphate from Igneous Rock Weathering Profiles

Biological phosphorous (P) -cycling in marine, terrestrial and atmospheric realm is key to evolutionary and climatic changes in Earth history. Oxygen isotope composition of phosphate reveals mechanisms of bond breaking and reforming during P-cycling as well as ambient water oxygen isotope compositions. I first review the state-of-art knowledge of phosphate P-O bond breaking and reforming induced by enzymatic processes (Chapter 1). Literatures in both the geochemistry and biochemistry have shown that on the cellular level, equilibrium exchange between phosphate and water happens within the cell and a non-equilibrium kinetic isotope effect occurs in the bond breaking and forming catalyzed by extracellular enzymes. Triple oxygen isotope could potentially separate the mixed signals between the two dominant processed which are otherwise hard to interpret. After we have concluded that P-O bond cannot be readily altered without extensive biological activities, we move to explore the history of land colonization by biota. I obtained a set of drill core samples from a paleoweathering profile of Middle Cambrian age (~500 Ma). The extracted phosphate oxygen isotope signature shows typical igneous ä18O and ∆17O values with no change between pristine and weathered igneous rocks. This is in contrary to the 13.2 ‰ change in the modern profile as demonstrated by Dustin Boyd’s thesis, suggesting a lack of any significant P-cycling in Middle Cambrian land surface (Chapter 2). The same triple oxygen isotope approach was applied to explore the weathering nature of weathering rinds, which shows a 0.8 ‰ excursion from pristine to weathered rinds, implying that biological activities are playing a role in the formation of the weathering rinds (Chapter 3). Finally, a set of phosphate samples extracted from a paleoweathering profile from the Late Permian Emeishan Large igneous provinces (ca. 260 Ma) reveals not only a modern-like land biological P-cycling but also local meteoric water’s triple oxygen isotope composition at that time, opening up a promising venue for studying paleo-precipitation and paleoaltimitry (Chapter 4). Much of this study is explorative and has revealed many new applications of triple oxygen isotopes of phosphate. It paves the way for a more systematic study of geological materials with extensive sampling in space and time

Gradient Projection with Approximate L0 Norm Minimization for Sparse Reconstruction in Compressed Sensing

In the reconstruction of sparse signals in compressed sensing, the reconstruction algorithm is required to reconstruct the sparsest form of signal. In order to minimize the objective function, minimal norm algorithm and greedy pursuit algorithm are most commonly used. The minimum L1 norm algorithm has very high reconstruction accuracy, but this convex optimization algorithm cannot get the sparsest signal like the minimum L0 norm algorithm. However, because the L0 norm method is a non-convex problem, it is difficult to get the global optimal solution and the amount of calculation required is huge. In this paper, a new algorithm is proposed to approximate the smooth L0 norm from the approximate L2 norm. First we set up an approximation function model of the sparse term, then the minimum value of the objective function is solved by the gradient projection, and the weight of the function model of the sparse term in the objective function is adjusted adaptively by the reconstruction error value to reconstruct the sparse signal more accurately. Compared with the pseudo inverse of L2 norm and the L1 norm algorithm, this new algorithm has a lower reconstruction error in one-dimensional sparse signal reconstruction. In simulation experiments of two-dimensional image signal reconstruction, the new algorithm has shorter image reconstruction time and higher image reconstruction accuracy compared with the usually used greedy algorithm and the minimum norm algorithm

Measurement Matrix Optimization via Mutual Coherence Minimization for Compressively Sensed Signals Reconstruction

For signals reconstruction based on compressive sensing, to reconstruct signals of higher accuracy with lower compression rates, it is required that there is a smaller mutual coherence between the measurement matrix and the sparsifying matrix. Mutual coherence between the measurement matrix and sparsifying matrix can be expressed indirectly by the property of the Gram matrix. On the basis of the Gram matrix, a new optimization algorithm of acquiring a measurement matrix has been proposed in this paper. Firstly, a new mathematical model is designed and a new method of initializing measurement matrix is adopted to optimize the measurement matrix. Then, the loss function of the new algorithm model is solved by the gradient projection-based method of Gram matrix approximating an identity matrix. Finally, the optimized measurement matrix is generated by minimizing mutual coherence between measurement matrix and sparsifying matrix. Compared with the conventional measurement matrices and the traditional optimization methods, the proposed new algorithm effectively improves the performance of optimized measurement matrices in reconstructing one-dimensional sparse signals and two-dimensional image signals that are not sparse. The superior performance of the proposed method in this paper has been fully tested and verified by a large number of experiments

Endoscopic Video-assisted Excision Combined with Mammotome for Major Benign Breast Lesions

BackgroundIt was difficult to excise major benign breast lesions more than 3 cm employing either Mammotome or Endoscopic respectively.MethodsFrom October,2005 and October,2008,10 patients with major benign breast lesions underwent endoscopic,video-assisted breast excision combined with Mammotome in our institution.Cosmetic results were evaluated using the ABNSW scoring system.ResultsThe incision size was 1 to 1.8 cm.The operation time ranged from 30 to 55 minutes with a median of 41 minutes of all patients,from 38 to 55 minutes with a median of 45 minutes of 6 patients with rigid mass and from 30 to 36 minutes with a median of 30 minutes of 4 patients with flexible mass.The median total ABNSW score was 14.1 points,median intraoperative bleeding was 17 ml,and median cost was 7825 ￥.No clinically important complication was encountered,and all patients were extremely satisfied with the cosmetic results of the procedure.ConclusionVideo-assisted endoscopic resection combined with Mammotome is a safe,effective technique to treat major benign breast tumors,and provides esthetic advantages

Application of Low-Altitude UAV Remote Sensing Image Object Detection Based on Improved YOLOv5

With the development of science and technology, the traditional industrial structures are constantly being upgraded. As far as drones are concerned, an increasing number of researchers are using reinforcement learning or deep learning to make drones more intelligent. At present, there are many algorithms for object detection. Although many models have a high accuracy of detection, these models have many parameters and high complexity, making them unable to perform real-time detection. Therefore, it is particularly important to design a lightweight object detection algorithm that is able to meet the needs of real-time detection using UAVs. In response to the above problems, this paper establishes a dataset of six animals in grassland from different angles and during different time periods on the basis of the remote sensing images of drones. In addition, on the basis of the Yolov5s network model, a lightweight object detector is designed. First, Squeeze-and-Excitation Networks are introduced to improve the expressiveness of the network model. Secondly, the convolutional layer of branch 2 in the BottleNeckCSP structure is deleted, and 3/4 of its input channels are directly merged with the results of branch 1 processing, which reduces the number of model parameters. Next, in the SPP module of the network model, a 3 × 3 maximum pooling layer is added to improve the receptive field of the model. Finally, the trained model is applied to NVIDIA-TX2 processor for real-time object detection. After testing, the optimized YOLOv5 grassland animal detection model was able to effectively identify six different forms of grassland animal. Compared with the YOLOv3, EfficientDet-D0, YOLOv4 and YOLOv5s network models, the mAP_0.5 value was improved by 0.186, 0.03, 0.007 and 0.011, respectively, and the mAP_0.5:0.95 value was improved by 0.216, 0.066, 0.034 and 0.051, respectively, with an average detection speed of 26 fps. The experimental results show that the grassland animal detection model based on the YOLOv5 network has high detection accuracy, good robustness, and faster calculation speed in different time periods and at different viewing angles

Design and Application of a UAV Autonomous Inspection System for High-Voltage Power Transmission Lines

As the scale of the power grid continues to expand, the human-based inspection method struggles to meet the needs of efficient grid operation and maintenance. Currently, the existing UAV inspection system in the market generally has short endurance power time, high flight operation requirements, low degree of autonomous flight, low accuracy of intelligent identification, slow generation of inspection reports, and other problems. In view of these shortcomings, this paper designs an intelligent inspection system based on self-developed UAVs, including autonomous planning of inspection paths, sliding film control algorithms, mobile inspection schemes and intelligent fault diagnosis. In the first stage, basic data such as latitude, longitude, altitude, and the length of the cross-arms are obtained from the cloud database of the power grid, while the lateral displacement and vertical displacement during the inspection drone operation are calculated, and the inspection flight path is generated independently according to the inspection type. In the second stage, in order to make the UAV’s flight more stable, the reference-model-based sliding mode control algorithm is introduced to improve the control performance. Meanwhile, during flight, the intelligent UAV uploads the captured photos to the cloud in real time. In the third stage, a mobile inspection program is designed in order to improve the inspection efficiency. The transfer of equipment is realized in the process of UAV inspection. Finally, to improve the detection accuracy, a high-precision object detector is designed based on the YOLOX network model, and the improved model increased the mAP0.5:0.95 metric by 2.22 percentage points compared to the original YOLOX_m for bird’s nest detection. After a large number of flight verifications, the inspection system designed in this paper greatly improves the efficiency of power inspection, shortens the inspection cycle, reduces the investment cost of inspection manpower and material resources, and successfully fuses the object detection algorithm in the field of high-voltage power transmission lines inspection

Estimation of Soil Arsenic Content with Hyperspectral Remote Sensing

With the continuous application of arsenic-containing chemicals, arsenic pollution in soil has become a serious problem worldwide. The detection of arsenic pollution in soil is of great significance to the protection and restoration of soil. Hyperspectral remote sensing is able to effectively monitor heavy metal pollution in soil. However, due to the possible complex nonlinear relationship between soil arsenic (As) content and the spectrum and data redundancy, an estimation model with high efficiency and accuracy is urgently needed. In response to this situation, 62 samples and 27 samples were collected in Daye and Honghu, Hubei Province, respectively. Spectral measurement and physical and chemical analysis were performed in the laboratory to obtain the As content and spectral reflectance. After the continuum removal (CR) was performed, the stable competitive adaptive reweighting sampling algorithm coupled the successive projections algorithm (sCARS-SPA) was used for characteristic band selection, which effectively solves the problem of data redundancy and collinearity. Partial least squares regression (PLSR), radial basis function neural network (RBFNN), and shuffled frog leaping algorithm optimization of the RBFNN (SFLA-RBFNN) were established in the characteristic wavelengths to predict soil As content. These results show that the sCARS-SPA-SFLA-RBFNN model has the best universality and high prediction accuracy in different land-use types, which is a scientific and effective method for estimating the soil As content