Cheong Took, Clive

Mandic, Danilo

PALADE, VASILE

RUIZ-GARCIA, ARIEL

Schmidhuber, Jürgen

English

Generative Adversarial Networks (GANs) have proven to be efficient systems for data generation and other machine learning tasks. They owe their success to a minimax learning concept initially proposed by Schmidhuber (1990) to implement Artificial Curiosity. Two learning networks, a generator and an evaluator or discriminator, compete with each other in a zero-sum game. Despite their obvious advantages and their application to a wide range of domains, GANs have yet to overcome several challenges such as non-convergence, overfitting, mode collapse, amongst others. New advancements in deep representation learning (RL) can help improve the learning process in GenerativeAdversarial Learning (GAL). For instance, RL can help address issues such as dataset bias and identify a set of features that are well suited for a given task

Royal Holloway - Pure

EditorialDeep Neural Network Representation and Generative AdversarialLearningGenerative Adversarial Networks (GANs) have provento be efficient systems for data generation and other machinelearning tasks. They owe their success to aminimax learningconcept initially proposed by Schmidhuber (1990) to imple-ment Artificial Curiosity. Two learning networks, a gener-ator and an evaluator or discriminator, compete with eachother in a zero-sum game. Despite their obvious advantagesand their application to a wide range of domains, GANs haveyet to overcome several challenges such as non-convergence,overfitting, mode collapse, amongst others. New advance-ments in deep representation learning (RL) can help improvethe learning process inGenerativeAdversarial Learning (GAL).For instance, RL can help address issues such as dataset biasand identify a set of features that are well suited for a giventask.This special issue on Deep RL and GAL provides a col-lection of high-quality articles addressing theoretical and prac-tical aspects of deep RL and GAL. After a rigorous reviewprocess of 70 articles, 21 articles were selected for inclusionin this special issue. A brief summary of these selected arti-cles is provided hereafter:The popularity and outstanding performance of GANsis attributed to a minimax learning paradigm in which twonetworks compete witch each other in a zero-sum game. In"Generative Adversarial Networks are special cases of Arti-ficial Curiosity (1990) and also closely related to Predictabil-ity Minimization (1991)", Schmidhuber looks into the ori-gins of adversarial learning through a minimax learning ap-proach and establishes that GANs are a special case of Ar-tificial Curiosity. The author also points out that anothermethod of 1991 called Predictability Minimization is basedon a very similar minimax game.The joint training of two models in GANs makes modelinterpretability a challenging task. In Latent Dirichlet allo-cation based generative adversarial networks, Pan et al. ex-plore how to integrate GANs with data structure priors. Theproposed methodology aims to shed some light into modelinterpretability by looking into how multiple generators cor-relate to data structure.The difference in the dimensionality space between thesource and target domains in GANs can lead to an unsta-ble training process. In Multi-projection of unequal dimen-sion optimal transport theory for Generative Adversary Net-works, Lin et al. propose a novel approach in which thesource and target probabilitymeasures are projected into low-dimensional subspaces. The authors treat this optimizationproblem as amulti-marginal optimal transport (OT) problemand provide explicit properties of the solution.ORCID(s):In addition to unstable training, the generator model inGANs is prone to model collapse, which results in failureto generate data with several variations. In Analysis of thetransferability and robustness of GANs evolved for Paretoset approximations, Garciarena et al. propose a novel neuro-evolutionary approach to address mode collapse and demon-strate the scalability of their approach through a series ofexperiments. The authors propose a set of guidelines to findthe best performing GAN architecture for a given task.Image synthesis is one of the most popular applicationsof GANs. However, generating realistic images using GANsremains a challenge, particularly when specific features arerequired. In Investigating object compositionality in Gener-ative Adversarial Networks, van Steenkiste et al. look at us-ing object compositionality as an inductive bias for GANs.The authors propose using a number of generator models,with shared weights and separate latent vectors as input, togenerate images as a composition of individual objects.Generating multi-domain images is also a challengingtask. In Unsupervised multi-domain multimodal image-to-image translation with explicit domain-constrained disen-tanglement, Xia et al. propose a framework formulti-domainimage translation through a single network. Xia et al. showthat disentanglement of content and style can lead to unde-sirable results. The proposed frameworks allows for gener-ation of diverse outputs with unpaired training data. Simi-larly, the authors ofMetalGAN: Multi-domain label-less im-age synthesis using cGANs and meta-learning, propose amulti-domain image synthesis approach also using a singlenetwork. In this paper, Fontanini et al. propose combing aconditional GAN along with Meta-Learning for image gen-eration and domain switch, demonstrating promising results.In CariGAN: Caricature generation through weakly pairedadversarial learning, Li et al. employ facial landmarks toconstrain the generated caricature facial images and generatecaricature images with reasonable facial deformation. Theauthors introduce a loss function to encourage diversity inthe generator.In Lower dimensional kernels for video discriminators,Kahembwe et al. look into the discriminator models used inGANs for video generation. The authors show that uncon-strained discriminator models are difficult to optimize. Theauthors continue by proposing a methodology for the designofGANdiscriminators for video generation, which improvesthe accuracy of contemporary state-of-the-art approaches.Class imbalance is another common problem that leadsto poor generalization performance in deep NNs. The au-thors ofCEGAN:Classification EnhancementGenerative Ad-versarial Networks for Unraveling Data Imbalance Prob-Ruiz-Garcia, A., Schmidhuber, J., Palade, V., Cheong Took, C., Mandic, D.: Preprint submitted to Elsevier Page 1 of 2Deep Representation and Generative Adversarial Learninglems propose generating newminority class samples througha so-called classification enhancement generative adversar-ial network (CEGAN). Similarly, in Deep learning for sym-bols detection and classification in engineering drawings,Elyan et al. use GANs to generate new training samples forminority classes of engineering drawings data and show im-proved classification accuracy. Additionally, Sáez Trigueroset al. generate new training samples to boost face recogni-tion accuracy performance in their articleGenerating photo-realistic training data to improve face recognition accuracy.In Modality independent adversarial network for gener-alized zero shot image classification, Zhang et al. proposeusing GANs within a zero-shot learning context to classifyimages of unseen target classes. The GANmodel learns dis-criminative and high level cross-modal data representations.The authors perform a series of ablation studies to show theoutstanding performance of the proposed approach.Aside from image synthesis, image manipulation is acommon application of GANs. In Image manipulation withnatural language using Two-sided Attentive Conditional Gen-erative Adversarial Network, Zhu et al. introduce a novelapproach to attributed editing through natural language de-scriptions. This method is shown to be robust to scale invari-ance and outperforms contemporary methods. In ClsGAN:Selective Attribute Editing Model based on ClassificationAdversarial Network, Liu et al. propose a classification ad-versarial network that produces high resolution images andhigh attribute transformation accuracy. The authors showstate-of-the-art performance in image quality and transferaccuracy through a series of ablation studies.It is well-established in the literature that model conver-gence can be a challenge for GANs. The authors of FPGAN:Face de-identification method with generative adversarialnetworks for social robots propose an end-to-end frameworkthat is guaranteed to convergence to a given criterion. Lin etal. apply the proposedmethodology to face de-identification,and outline a metric for this task.GANs are increasingly becoming popular in applicationdomains where data is commonly complex. The authors ofHigh tissue contrast image synthesis viamultistage attention-GAN: Application to segmenting brain MR scans investigatethe use of GANS to improve image contrast. Hamghalam etal. argue that their multi-stage approach reduces the gap be-tween the source and target domains. The authors show out-standing performance of the proposed method when appliedto multimodal image segmentation. In a similar application,the authors of in GP-GAN: Brain tumor growth predictionusing stacked 3D generative adversarial networks from lon-gitudinal MR Images propose a data-driven 3D GAN forgrowth prediction of Glioma tumors using magnetic reso-nance images. The proposed method by Elazab et al. canalso be used for image segmentation. Furthermore, in AMD-GAN: Attention encoder and multi-branch structure basedgenerative adversarial networks for fundus disease detectionfrom scanning laser ophthalmoscopy images, Xie et al. pro-pose a GANmethodology for the detection of fundus diseasefrom laser ophthalmocopy images (SLO). In thismethod, thegenerator model is guided by an autoencoder and the dis-criminator model is exploited to classify SLO images.The minimax learning paradigm employed by GANs, in-troduced in early works on Artificial Curiosity (1990) andPredictability Minimization (1991), continues to gain mo-mentum in other domain applications outside of image syn-thesis. In Energy-efficient and damage-recovery slitheringgait design for a snake-like robot based on reinforcementlearning and inverse reinforcement learning, Bing et al. pro-pose using the generator model as the agent that learns togenerate actions that produce optimal rewards. The discrim-inator model is used to rate whether an action was producedby the generator model, or from expert data. The authorsshow promising results in terms of energy efficiency anddamage recovery.In a different application, Javanmardi et al. propose aGAN based approach to tracking in Appearance variationadaptation tracker using adversarial network. The authorspropose using the discriminator model to differentiate be-tween recent and earlier target regions, and the generatormodel to produce an adaptation mask. With this formula-tion, the model is able to align feature vectors of recent andearlier target regions, as well as distinguish between back-ground and foreground.The articles presented here propose a variety of methodsaddressing some of the most complex challenges in RL andGAL. They also highlight that using GANs often requiresdesigning specificmodel architectures and loss functions tai-lored to the task being solved. This special issue aims to pro-mote and encourage the development of theoretical methodsin deep RL and GAL in an attempt to advance this popularfield.The guest editors extend their gratitude to the authors,Editors in Chief and reviewers for their contributions andsupport in the realization of this special issue.ARIEL RUIZ-GARCIA, Guest EditorSeeChange.aiManchester, United Kingdomariel.9arcia@gmail.comJÜRGEN SCHMIDHUBER, Guest EditorNNAISENSESwiss AI Lab IDSIAUSI and SUPSILugano, Switzerlandjuergen@idsia.chVASILE PALADE, Guest EditorCentre for Data ScienceCoventry UniversityCoventry, United Kingdomvasile.palade@coventry.ac.ukCLIVE CHEONG TOOK, Guest EditorDepartment of Electronic EngineeringRoyal Holloway (University of London)Surrey, United KingdomClive.CheongTook@rhul.ac.ukDANILO MANDIC, Guest EditorDepartment of Electrical and Electronic EngineeringImperial College LondonLondon, United Kingdomd.mandic@imperial.ac.ukRuiz-Garcia, A., Schmidhuber, J., Palade, V., Cheong Took, C., Mandic, D.: Preprint submitted to Elsevier Page 2 of 2

Deep neural network representation and Generative Adversarial Learning

https://pure.royalholloway.ac.uk/portal/files/41580705/NN_VSI_RLandGANs_Editorial_CameraReady_final_submission.pdf

Deep neural network representation and Generative Adversarial Learning

Abstract

Similar works

Full text

Available Versions

Royal Holloway - Pure