Orthogonal SVD Covariance Conditioning and Latent Disentanglement

Inserting an SVD meta-layer into neural networks is prone to make the covariance ill-conditioned, which could harm the model in the training stability and generalization abilities. In this paper, we systematically study how to improve the covariance conditioning by enforcing orthogonality to the Pre-SVD layer. Existing orthogonal treatments on the weights are first investigated. However, these techniques can improve the conditioning but would hurt the performance. To avoid such a side effect, we propose the Nearest Orthogonal Gradient (NOG) and Optimal Learning Rate (OLR). The effectiveness of our methods is validated in two applications: decorrelated Batch Normalization (BN) and Global Covariance Pooling (GCP). Extensive experiments on visual recognition demonstrate that our methods can simultaneously improve covariance conditioning and generalization. The combinations with orthogonal weight can further boost the performance. Moreover, we show that our orthogonality techniques can benefit generative models for better latent disentanglement through a series of experiments on various benchmarks. Code is available at: \href{https://github.com/KingJamesSong/OrthoImproveCond}{https://github.com/KingJamesSong/OrthoImproveCond}.Comment: Accepted by IEEE T-PAMI. arXiv admin note: substantial text overlap with arXiv:2207.0211

Image style transfer with neural networks

Abstract. The main goal of this thesis is to introduce the reader to different style transfer methods that are based on artificial neural networks. The thesis begins with an introduction to some key concepts regarding artificial neural networks. These concepts include a single artificial neuron, a feedforward neural network, convolutional neural network and generative adversarial networks. Next, the main ideas behind five different style transfer methods are explained. Finally, style transferred images created by these methods are presented and some characteristics of different methods are demonstrated.Kuvien tyylimuunnos neuroverkoilla. Tiivistelmä. Tämän kandidaatintyön tarkoitus on esitellä lukijalle erilaisia keinotekoisiin neuroverkkoihin pohjautuvia kuvien tyylimuunnosmenetelmiä. Aiheeseen johdatellaan käymällä ensin läpi pääpiirteittäin muutamia keinotekoisiin neuroverkkoihin liittyviä konsepteja, kuten yksittäinen keinotekoinen neuroni ja näistä neuroneista muodostettava neuroverkko, konvoluutioneuroverkko sekä generatiivinen kilpaileva verkosto. Seuraavaksi viiden erilaisen tyylimuunnosmenetelmän toimintaperiaate käydään läpi. Lopuksi eri menetelmien tuottamia tyylimuunnettuja kuvia esitetään ja lisäksi joitain menetelmien erityispiirteitä demonstroidaan

A Style-Based Generator Architecture for Generative Adversarial Networks

We propose an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature. The new architecture leads to an automatically learned, unsupervised separation of high-level attributes (e.g., pose and identity when trained on human faces) and stochastic variation in the generated images (e.g., freckles, hair), and it enables intuitive, scale-specific control of the synthesis. The new generator improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation. To quantify interpolation quality and disentanglement, we propose two new, automated methods that are applicable to any generator architecture. Finally, we introduce a new, highly varied and high-quality dataset of human faces.Comment: CVPR 2019 final versio