CoLFI: Cosmological Likelihood-free Inference with Neural Density Estimators

In previous works, we proposed to estimate cosmological parameters with the artificial neural network (ANN) and the mixture density network (MDN). In this work, we propose an improved method called the mixture neural network (MNN) to achieve parameter estimation by combining ANN and MDN, which can overcome shortcomings of the ANN and MDN methods. Besides, we propose sampling parameters in a hyper-ellipsoid for the generation of the training set, which makes the parameter estimation more efficient. A high-fidelity posterior distribution can be obtained using $\mathcal{O}(10^2)$ forward simulation samples. In addition, we develop a code-named CoLFI for parameter estimation, which incorporates the advantages of MNN, ANN, and MDN, and is suitable for any parameter estimation of complicated models in a wide range of scientific fields. CoLFI provides a more efficient way for parameter estimation, especially for cases where the likelihood function is intractable or cosmological models are complex and resource-consuming. It can learn the conditional probability density $p(\boldsymbol\theta|\boldsymbol{d})$ using samples generated by models, and the posterior distribution $p(\boldsymbol\theta|\boldsymbol{d}_0)$ can be obtained for a given observational data $\boldsymbol{d}_0$. We tested the MNN using power spectra of the cosmic microwave background and Type Ia supernovae and obtained almost the same result as the Markov Chain Monte Carlo method. The numerical difference only exists at the level of $\mathcal{O}(10^{-2}\sigma)$. The method can be extended to higher-dimensional data.Comment: 24 pages, 8 tables, 17 figures, ApJS in press, corrected the ELU plot in Table 5. The code repository is available at https://github.com/Guo-Jian-Wang/colf

Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions

Generative Adversarial Networks (GANs) is a novel class of deep generative models which has recently gained significant attention. GANs learns complex and high-dimensional distributions implicitly over images, audio, and data. However, there exists major challenges in training of GANs, i.e., mode collapse, non-convergence and instability, due to inappropriate design of network architecture, use of objective function and selection of optimization algorithm. Recently, to address these challenges, several solutions for better design and optimization of GANs have been investigated based on techniques of re-engineered network architectures, new objective functions and alternative optimization algorithms. To the best of our knowledge, there is no existing survey that has particularly focused on broad and systematic developments of these solutions. In this study, we perform a comprehensive survey of the advancements in GANs design and optimization solutions proposed to handle GANs challenges. We first identify key research issues within each design and optimization technique and then propose a new taxonomy to structure solutions by key research issues. In accordance with the taxonomy, we provide a detailed discussion on different GANs variants proposed within each solution and their relationships. Finally, based on the insights gained, we present the promising research directions in this rapidly growing field.Comment: 42 pages, Figure 13, Table

PassGAN: A Deep Learning Approach for Password Guessing

State-of-the-art password guessing tools, such as HashCat and John the Ripper, enable users to check billions of passwords per second against password hashes. In addition to performing straightforward dictionary attacks, these tools can expand password dictionaries using password generation rules, such as concatenation of words (e.g., "password123456") and leet speak (e.g., "password" becomes "p4s5w0rd"). Although these rules work well in practice, expanding them to model further passwords is a laborious task that requires specialized expertise. To address this issue, in this paper we introduce PassGAN, a novel approach that replaces human-generated password rules with theory-grounded machine learning algorithms. Instead of relying on manual password analysis, PassGAN uses a Generative Adversarial Network (GAN) to autonomously learn the distribution of real passwords from actual password leaks, and to generate high-quality password guesses. Our experiments show that this approach is very promising. When we evaluated PassGAN on two large password datasets, we were able to surpass rule-based and state-of-the-art machine learning password guessing tools. However, in contrast with the other tools, PassGAN achieved this result without any a-priori knowledge on passwords or common password structures. Additionally, when we combined the output of PassGAN with the output of HashCat, we were able to match 51%-73% more passwords than with HashCat alone. This is remarkable, because it shows that PassGAN can autonomously extract a considerable number of password properties that current state-of-the art rules do not encode.Comment: This is an extended version of the paper which appeared in NeurIPS 2018 Workshop on Security in Machine Learning (SecML'18), see https://github.com/secml2018/secml2018.github.io/raw/master/PASSGAN_SECML2018.pd