Knowledge Distillation with Adversarial Samples Supporting Decision Boundary

Many recent works on knowledge distillation have provided ways to transfer the knowledge of a trained network for improving the learning process of a new one, but finding a good technique for knowledge distillation is still an open problem. In this paper, we provide a new perspective based on a decision boundary, which is one of the most important component of a classifier. The generalization performance of a classifier is closely related to the adequacy of its decision boundary, so a good classifier bears a good decision boundary. Therefore, transferring information closely related to the decision boundary can be a good attempt for knowledge distillation. To realize this goal, we utilize an adversarial attack to discover samples supporting a decision boundary. Based on this idea, to transfer more accurate information about the decision boundary, the proposed algorithm trains a student classifier based on the adversarial samples supporting the decision boundary. Experiments show that the proposed method indeed improves knowledge distillation and achieves the state-of-the-arts performance.Comment: Accepted to AAAI 201

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons

An activation boundary for a neuron refers to a separating hyperplane that determines whether the neuron is activated or deactivated. It has been long considered in neural networks that the activations of neurons, rather than their exact output values, play the most important role in forming classification friendly partitions of the hidden feature space. However, as far as we know, this aspect of neural networks has not been considered in the literature of knowledge transfer. In this paper, we propose a knowledge transfer method via distillation of activation boundaries formed by hidden neurons. For the distillation, we propose an activation transfer loss that has the minimum value when the boundaries generated by the student coincide with those by the teacher. Since the activation transfer loss is not differentiable, we design a piecewise differentiable loss approximating the activation transfer loss. By the proposed method, the student learns a separating boundary between activation region and deactivation region formed by each neuron in the teacher. Through the experiments in various aspects of knowledge transfer, it is verified that the proposed method outperforms the current state-of-the-art.Comment: Accepted to AAAI 201

Backbone Can Not be Trained at Once: Rolling Back to Pre-trained Network for Person Re-Identification

In person re-identification (ReID) task, because of its shortage of trainable dataset, it is common to utilize fine-tuning method using a classification network pre-trained on a large dataset. However, it is relatively difficult to sufficiently fine-tune the low-level layers of the network due to the gradient vanishing problem. In this work, we propose a novel fine-tuning strategy that allows low-level layers to be sufficiently trained by rolling back the weights of high-level layers to their initial pre-trained weights. Our strategy alleviates the problem of gradient vanishing in low-level layers and robustly trains the low-level layers to fit the ReID dataset, thereby increasing the performance of ReID tasks. The improved performance of the proposed strategy is validated via several experiments. Furthermore, without any add-ons such as pose estimation or segmentation, our strategy exhibits state-of-the-art performance using only vanilla deep convolutional neural network architecture.Comment: Accepted to AAAI 201

A case report of successfully treated metachronous gastrointestinal stromal tumor and colon cancer

The diagnosis of gastrointestinal stromal tumor (GIST) has become relatively common in recent years, but little is known about its association with other malignancies. We present a rare case of successfully treated metachronous GIST and colon cancer with concurrent FOLFOX (5-fluorouracil, leucovorin, and oxaliplatin) chemotherapy and imatinib. A 63-year-old man presented with abdominal pain that had started 2 weeks ago, and endoscopic ultrasonography showed masses that were compatible with GIST on the duodenum. He underwent Whipple surgery. One year after the GIST diagnosis, two liver masses were found on abdominal computed tomography images taken for surveillance. A liver biopsy showed metastatic adenocarcinoma, not GIST. Colonoscopy was then performed to identify the primary site of the metastatic adenocarcinoma in the liver, and sigmoid colon cancer was found. He received 12 cycles of adjuvant FOLFOX concurrently with adjuvant imatinib. There were no serious adverse events of grade 3 or higher from either imatinib or chemotherapy. He has completed adjuvant imatinib and FOLFOX chemotherapy and there is no evidence of disease recurrence. When a synchronous or metachronous tumor is found in a GIST patient, the clinician should keep in mind the possibility of another primary tumor of different histopathology, as well as GIST recurrence

Production of Transgenic Cloned Miniature Pigs with Membrane-bound Human Fas Ligand (FasL) by Somatic Cell Nuclear Transfer

Cell-mediated xenograft rejection, including NK cells and CD8+ CTL, is a major obstacle in successful pig-to-human xenotransplantation. Human CD8+ CTL and NK cells display high cytotoxicity for pig cells, mediated at least in part by the Fas/FasL pathway. To prevent cell-mediated xenocytotoxicity, a membrane-bound form of human FasL (mFasL) was generated as an inhibitor for CTL and NK cell cytotoxicity that could not be cleaved by metalloproteinase to produce putative soluble FasL. We produced two healthy transgenic pigs harboring the mFasL gene via somatic cell nuclear transfer (SCNT). In a cytotoxicity assay using transgenic clonal cell lines and transgenic pig ear cells, the rate of CD8+ CTL-mediated cytotoxicity was significantly reduced in transgenic pig's ear cells compared with that in normal minipig fetal fibroblasts. Our data indicate that grafts of transgenic pigs expressing membrane-bound human FasL control the cellular immune response to xenografts, creating a window of opportunity to facilitate xenograft survival

Real-time Image Processing for Microscopy-based Label-free Imaging Flow Cytometry in a Microfluidic Chip

Imaging flow cytometry (IFC) is an emerging technology that acquires single-cell images at high-throughput for analysis of a cell population. Rich information that comes from high sensitivity and spatial resolution of a single-cell microscopic image is beneficial for single-cell analysis in various biological applications. In this paper, we present a fast image-processing pipeline (R-MOD: Real-time Moving Object Detector) based on deep learning for high-throughput microscopy-based label-free IFC in a microfluidic chip. The R-MOD pipeline acquires all single-cell images of cells in flow, and identifies the acquired images as a real-time process with minimum hardware that consists of a microscope and a high-speed camera. Experiments show that R-MOD has the fast and reliable accuracy (500 fps and 93.3% mAP), and is expected to be used as a powerful tool for biomedical and clinical applications.113Ysciescopu

Magnetic resonance imaging analysis of rotator cuff tear after shoulder dislocation in a patient older than 40 years

Background This study was designed to evaluate characters of the rotator cuff tear (RCT) recognized after primary shoulder dislocation in patients older than 40. Methods From 2008 to 2019, patients who visited two hospitals after dislocation were retrospectively reviewed. Inclusion criteria were patients over 40 who had dislocation, with magnetic resonance imaging (MRI) undergone. Exclusion criteria were patients who lost to follow-up, combined with any proximal humerus fracture, brachial plexus injury, and previous operation or dislocation history in the ipsilateral shoulder. Also patients who had only bankart or bony bakart lesion in MRI were excluded. We evaluated RCTs that were recognized by MRI after the primary shoulder dislocation with regard to tear size, degree, involved tendons, fatty degeneration, the age when the first dislocation occurred, and the duration until the MRI was evaluated after the dislocation. Results Fifty-five RCTs were included. According to age groups, the tear size was increased in coronal and sagittal direction, the number of involved tendons was increased, and the degree of fatty degeneration was advanced in infraspinatus muscle. Thirty-two cases (58.2%) conducted MRI after 3 weeks from the first shoulder dislocation event. This group showed that the retraction size of the coronal plane was increased significantly and the fatty accumulation of the supraspinatus muscle had progressed significantly. Conclusions Age is also a strong factor to affect the feature of RCT after the shoulder dislocation in patients over 40. And the delay of the MRI may deteriorate the degree of tear size and fatty degeneration