Domain-adaptive deep network compression

Deep Neural Networks trained on large datasets can be easily transferred to new domains with far fewer labeled examples by a process called fine-tuning. This has the advantage that representations learned in the large source domain can be exploited on smaller target domains. However, networks designed to be optimal for the source task are often prohibitively large for the target task. In this work we address the compression of networks after domain transfer. We focus on compression algorithms based on low-rank matrix decomposition. Existing methods base compression solely on learned network weights and ignore the statistics of network activations. We show that domain transfer leads to large shifts in network activations and that it is desirable to take this into account when compressing. We demonstrate that considering activation statistics when compressing weights leads to a rank-constrained regression problem with a closed-form solution. Because our method takes into account the target domain, it can more optimally remove the redundancy in the weights. Experiments show that our Domain Adaptive Low Rank (DALR) method significantly outperforms existing low-rank compression techniques. With our approach, the fc6 layer of VGG19 can be compressed more than 4x more than using truncated SVD alone -- with only a minor or no loss in accuracy. When applied to domain-transferred networks it allows for compression down to only 5-20% of the original number of parameters with only a minor drop in performance.Comment: Accepted at ICCV 201

Recognizing New Classes with Synthetic Data in the Loop : Application to Traffic Sign Recognition

On-board vision systems may need to increase the number of classes that can be recognized in a relatively short period. For instance, a traffic sign recognition system may suddenly be required to recognize new signs. Since collecting and annotating samples of such new classes may need more time than we wish, especially for uncommon signs, we propose a method to generate these samples by combining synthetic images and Generative Adversarial Network (GAN) technology. In particular, the GAN is trained on synthetic and real-world samples from known classes to perform synthetic-to-real domain adaptation, but applied to synthetic samples of the new classes. Using the Tsinghua dataset with a synthetic counterpart, SYNTHIA-TS, we have run an extensive set of experiments. The results show that the proposed method is indeed effective, provided that we use a proper Convolutional Neural Network (CNN) to perform the traffic sign recognition (classification) task as well as a proper GAN to transform the synthetic images. Here, a ResNet101-based classifier and domain adaptation based on CycleGAN performed extremely well for a ratio for new/known classes; even for more challenging ratios such as , the results are also very positive

OLED Encapsulation by Room Temperature Plasma-Assisted ALD Al2O3 Films

Organic light emitting diodes (OLEDs, both small molecule and polymer LEDs) require excellent gas and moisture permeation barrier layers to increase their lifetime. The quality of the barrier layer is ultimately controlled by the presence of defects in the layer. Although a barrier layer may be intrinsically excellent (water vapor transmission rate, WVTR = 10-6 g·m-2·day-1) the protected device may fail in the presence of defects that lead to preferential diffusion pathways for H2O (e.g., defects caused by particles from the environment and/or production process). The state-of-the-art barrier coatings are micrometer-thick multi-layer structure, in which organic interlayers are alternated with inorganic barrier layers with the purpose of decoupling the above-mentioned defects. Recently, atomic layer deposition (ALD) has been successfully tested for the deposition of very thin (<50 nm) single layer permeation barriers on pristine polymer substrates [1,2], showing the potential of this highly uniform and conformal deposition technique in the field of moisture permeation barriers. In this contribution the encapsulation of OLEDs by plasma-assisted ALD of thin (20-40 nm) Al2O3 layers is addressed. The layers are synthesized at room temperature by sequentially exposing the substrate to Al(CH3)3 vapor and a remote inductively coupled O2 plasma in Oxford Instruments FlexALTM and OpALTM reactors. The intrinsic quality of the deposited ALD layers was determined by monitoring the oxidation of a Ca film encapsulated by the Al2O3 film: WVTR values as low as 2·10-6 g·m-2·day-1 have been measured. The potential of ALD layers in encapsulating OLEDs, and therefore in successfully covering the defects present on the device, has been investigated by means of electroluminescence measurements of polymer-LEDs (effective emitting area of 5.8 cm2). The black spot density and area growth were followed as a function of the time under standard conditions of 20°C and 50% relative humidity. Within a 500 h test ALD-encapsulated OLEDs show approximately half the black spot density compared to devices encapsulated by plasma deposited a-SiNx:H (300 nm thick). The black spot density is further reduced by combining the a-SiNx:H and ALD Al2O3 layers. These results point towards a very promising application of ALD Al2O3 layers in the field of OLED encapsulation and will be interpreted in terms of possible mechanisms related to film growth in multi-layer structures

Active Learning for Deep Detection Neural Networks

The cost of drawing object bounding boxes (i.e. labeling) for millions of images is prohibitively high. For instance, labeling pedestrians in a regular urban image could take 35 seconds on average. Active learning aims to reduce the cost of labeling by selecting only those images that are informative to improve the detection network accuracy. In this paper, we propose a method to perform active learning of object detectors based on convolutional neural networks. We propose a new image-level scoring process to rank unlabeled images for their automatic selection, which clearly outperforms classical scores. The proposed method can be applied to videos and sets of still images. In the former case, temporal selection rules can complement our scoring process. As a relevant use case, we extensively study the performance of our method on the task of pedestrian detection. Overall, the experiments show that the proposed method performs better than random selection. Our codes are publicly available at www.gitlab.com/haghdam/deep_active_learning.Comment: Accepted at ICCV 201

The Effectiveness of Mindfulness Training on Behavioral Problems and Attentional Functioning in Adolescents with ADHD

The effectiveness of an 8-week mindfulness training for adolescents aged 11–15 years with ADHD and parallel Mindful Parenting training for their parents was evaluated, using questionnaires as well as computerized attention tests. Adolescents (N = 10), their parents (N = 19) and tutors (N = 7) completed measurements before, immediately after, 8 weeks after and 16 weeks after training. Adolescents reported on their attention and behavioral problems and mindful awareness, and were administered two computerized sustained attention tasks. Parents as well as tutors reported on adolescents’ attention and behavioral problems and executive functioning. Parents further reported on their own parenting, parenting stress and mindful awareness. Both the mindfulness training for the adolescents and their parents was delivered in group format. First, after mindfulness training, adolescents’ attention and behavior problems reduced, while their executive functioning improved, as indicated by self-report measures as well as by father and teacher report. Second, improvements in adolescent’ actual performance on attention tests were found after mindfulness training. Moreover, fathers, but not mothers, reported reduced parenting stress. Mothers reported reduced overreactive parenting, whereas fathers reported an increase. No effect on mindful awareness of adolescents or parents was found. Effects of mindfulness training became stronger at 8-week follow-up, but waned at 16-week follow-up. Our study adds to the emerging body of evidence indicating that mindfulness training for adolescents with ADHD (and their parents) is an effective approach, but maintenance strategies need to be developed in order for this approach to be effective in the longer term

The Mindful Attention Awareness Scale for Adolescents (MAAS-A): Psychometric Properties in a Dutch Sample

The factor structure and psychometric properties of the Dutch version of the Mindful Attention Awareness Scale for Adolescents (MAAS-A) was studied in a sample of adolescents (n = 717; age range, 11–17 years) of the general population. The MAAS-A and other questionnaires measuring other constructs were administered in high schools across the Netherlands. A one-factor structure was demonstrated using principal component analysis and was further confirmed using confirmatory factor analysis. The MAAS-A was shown to have high internal consistency. Expected negative correlations between mindfulness and self-reported stress and emotion regulation strategies such as rumination and catastrophizing were found. Further, mindfulness was positively correlated with happiness, healthy self-regulation, and with another recently developed measure of mindfulness in children and adolescents, the Child and Adolescent Mindfulness Measure. Mindfulness as measured by the MAAS-A correlated positively with quality of life, but an expected positive relationship with acceptance was not found. Interestingly, adolescents without meditation experience scored higher on the MAAS-A than adolescents without this experience. Further, adolescents with chronic disorders scored lower on the MAAS-A than adolescents without these disorders. Overall, this study has shown evidence of the first valid and reliable Dutch measure of mindfulness for adolescents. The factor structure, internal consistency, and convergent and divergent validity as well as their relationship to quality of life are comparable to the original MAAS-A

Continuity of Genetic Risk for Aggressive Behavior Across the Life-Course

We test whether genetic influences that explain individual differences in aggression in early life also explain individual differences across the life-course. In two cohorts from The Netherlands (N = 13,471) and Australia (N = 5628), polygenic scores (PGSs) were computed based on a genome-wide meta-analysis of childhood/adolescence aggression. In a novel analytic approach, we ran a mixed effects model for each age (Netherlands: 12–70 years, Australia: 16–73 years), with observations at the focus age weighted as 1, and decaying weights for ages further away. We call this approach a ‘rolling weights’ model. In The Netherlands, the estimated effect of the PGS was relatively similar from age 12 to age 41, and decreased from age 41–70. In Australia, there was a peak in the effect of the PGS around age 40 years. These results are a first indication from a molecular genetics perspective that genetic influences on aggressive behavior that are expressed in childhood continue to play a role later in life

Fast anisotropic Gauss filtering

Abstract. We derive the decomposition of the anisotropic Gaussian in a one dimensional Gauss filter in the x-direction followed by a one dimensional filter in a non-orthogonal direction ϕ. So also the anisotropic Gaussian can be decomposed by dimension. This appears to be extremely efficient from a computing perspective. An implementation scheme for normal convolution and for recursive filtering is proposed. Also directed derivative filters are demonstrated. For the recursive implementation, filtering an 512 × 512 image is performed within 65 msec, independent of the standard deviations and orientation of the filter. Accuracy of the filters is still reasonable when compared to truncation error or recursive approximation error. The anisotropic Gaussian filtering method allows fast calculation of edge and ridge maps, with high spatial and angular accuracy. For tracking applications, the normal anisotropic convolution scheme is more advantageous, with applications in the detection of dashed lines in engineering drawings. The recursive implementation is more attractive in feature detection applications, for instance in affine invariant edge and ridge detection in computer vision. The proposed computational filtering method enables the practical applicability of orientation scale-space analysis