Regularizing Neural Networks with Meta-Learning Generative Models

This paper investigates methods for improving generative data augmentation for deep learning. Generative data augmentation leverages the synthetic samples produced by generative models as an additional dataset for classification with small dataset settings. A key challenge of generative data augmentation is that the synthetic data contain uninformative samples that degrade accuracy. This is because the synthetic samples do not perfectly represent class categories in real data and uniform sampling does not necessarily provide useful samples for tasks. In this paper, we present a novel strategy for generative data augmentation called meta generative regularization (MGR). To avoid the degradation of generative data augmentation, MGR utilizes synthetic samples in the regularization term for feature extractors instead of in the loss function, e.g., cross-entropy. These synthetic samples are dynamically determined to minimize the validation losses through meta-learning. We observed that MGR can avoid the performance degradation of na\"ive generative data augmentation and boost the baselines. Experiments on six datasets showed that MGR is effective particularly when datasets are smaller and stably outperforms baselines.Comment: Accepted to NeurIPS 202

Transfer Learning with Pre-trained Conditional Generative Models

Transfer learning is crucial in training deep neural networks on new target tasks. Current transfer learning methods always assume at least one of (i) source and target task label spaces overlap, (ii) source datasets are available, and (iii) target network architectures are consistent with source ones. However, holding these assumptions is difficult in practical settings because the target task rarely has the same labels as the source task, the source dataset access is restricted due to storage costs and privacy, and the target architecture is often specialized to each task. To transfer source knowledge without these assumptions, we propose a transfer learning method that uses deep generative models and is composed of the following two stages: pseudo pre-training (PP) and pseudo semi-supervised learning (P-SSL). PP trains a target architecture with an artificial dataset synthesized by using conditional source generative models. P-SSL applies SSL algorithms to labeled target data and unlabeled pseudo samples, which are generated by cascading the source classifier and generative models to condition them with target samples. Our experimental results indicate that our method can outperform the baselines of scratch training and knowledge distillation.Comment: 24 pages, 6 figure

901–85 Lp(a) is a Predictor of Coronary Artery Disease in Pre-menopausal but not in Post-menopausal Women

Coronary artery disease (CAD) risk increases in women after menopause. Although numerous reports suggest that lipid profile worsening after menopause may be associated with an increase in CAD among women, there have been few studies that discussed the contribution of Lp(a). To examine the association between CAD and Lp(a) in pre-menopausal (PR, <55 yo) and post-menopausal (PO, ≥55 yo) women, we evaluated Lp(a) levels and other risk factor prevalence in 180 female patients (20 to 77 yo) with angiographically defined CAD. Six risk factors were assessed: hyper-Lp(a)emia (Hi-Lp(a), Lp(a)≥30mg/dl). hyper-LDLemia (Hi-LDL, LDL≥160mg/dl). hypo-HDLemia (Lo-HDL, HDL<35mg/dl), hypertension, diabetes, and smoking. Cases were defined as those who had >1 coronary artery with >50% stenosis. There were more patients with Hi-Lp(a) (61%*vs 27%), Hi-LDL (61%**vs 5%) and smoking (61%**vs 9%) in PR cases (n=18) as compaved with those in PR controls (n=22). PO cases (n=93) had more Lo-HDL (15%*vs 2%), diabetes (33%**vs 13%) and smokers (61%**vs 9%) than PO controls (n=47) did. The median Lp(a) of PR cases was higher than that of PR controls (38.8*, 22.7mg/dl), and they increased with number of diseased arteries. In contrast, there was no difference in the Lp(a) levels between PO cases and PO controls (21.7, 25.2mg/dl). Logistic regression model also revealed that Hi-Lp(a) was an independent predictor of CAD after controlling for Hi-LDL, Lo-HDL, hypertension, diabetes and smoking among PR (B=2.44, SE=1.20, p<0.05), but not among PO. Our data suggests that Lp(a) may be a strong risk factor for CAD in pre-menopausal women, and in post-menopausal women other risk factors, such as an estrogen deficiency, may play an important role. (*p<0.05, **p<0.01)

Flow-induced Alignment of Amyloid Protofilaments Revealed by Linear Dichroism

This research was originally published in the Journal of Biological Chemistry. Rumi Adachi, Kei-ichi Yamaguchi, Hisashi Yagi, Kazumasa Sakurai, Hironobu Naiki and Yuji Goto. Flow-induced Alignment of Amyloid Protofilaments Revealed by Linear Dichroism. J. Biol. Chem. 2007; 282, 8978-8983. © the American Society for Biochemistry and Molecular Biolog

Bovine Myoblast Differentiation

Satellite cells are involved in postnatal myogenesis and in muscle hypertrophy. A better understanding of the mechanisms of myogenesis is useful to improve the muscle production in farm animals. Herein, we show the cytokine effects on the myogenesis in bovine myoblast cultures. Acidic fibroblast growth factors (aFGF) and interleukin 1 (IL-1) stimulated the cell proliferation, and insulin-1ike growth factor-I (IGF-I) stimulated to form multinucleated myotubes. Thus, it was possible to regulate the bovine myoblast differentiation by aFGF, IL-1 and/or IGF-I. Using the culture system, the expression of myosin heavy chain (MyHC) isoforms was detailed in bovine myoblasts during the differentiation. It was immunohistochemically confirmed that bovine myoblasts expressed β/slow-type MyHC (MyHC-slow), fast-type MyHC (MyHC-fast) and developmental-type MyHC (MyHC-dev) isoforms. Furthermore, the expression of fast 2a and β/slow MyHC mRNA was recognized in the cultures of bovine myoblasts. The results support the existence of bovine myoblast phenotypes that express differentially MyHC isoforms

Millimeter-Thick Single-Walled Carbon Nanotube Forests: Hidden Role of Catalyst Support

A parametric study of so-called "super growth" of single-walled carbon nanotubes(SWNTs) was done by using combinatorial libraries of iron/aluminum oxide catalysts. Millimeter-thick forests of nanotubes grew within 10 min, and those grown by using catalysts with a thin Fe layer (about 0.5 nm) were SWNTs. Although nanotube forests grew under a wide range of reaction conditions such as gas composition and temperature, the window for SWNT was narrow. Fe catalysts rapidly grew nanotubes only when supported on aluminum oxide. Aluminum oxide, which is a well-known catalyst in hydrocarbon reforming, plays an essential role in enhancing the nanotube growth rates.Comment: 11 pages, 3 figures. Jpn. J. Appl. Phys. (Express Letters) in pres