Genomic studies of sex differences : On mutations, recombination, and sexual antagonism in songbirds

Many organisms have separate sexes, i.e., males and females. The presence of separate sexes causes sex-specific selection regimes and sexual antagonism, which can lead to sex differences in morphology, physiology, and behaviours. Sex and sex differences can be genetically governed and regulated by a pair of sex chromosomes (e.g., X and Y, or Z and W), on which there often are regions without recombination. In this thesis, I used genomic approaches to study sex differences in a songbird, the great reed warbler (Acrocephalus arundinaceus), in which male and females are monochromatic and genetically determined by a pair of sex chromosomes. The thesis starts with a study presenting and evaluating two alternative phylogenetic approaches (the expected likelihood weight (ELW) and the BEAST approach) to determine when different parts of the sex chromosomes stop recombining. My findings highlight the benefits of using fixed topologies to estimate the timing of recombination cessation as done by these approaches. Thereafter, I focus on molecular sex differences using genomic and bioinformatic methods to specifically investigate sex biases in de novo mutations and in recombination patterns, and search for sexually antagonistic loci in the genome.By using whole genome sequencing data from a three-generation pedigree of the great reed warbler, I found a strong sex bias in the numbers of de novo mutations, with males having three times as many mutations as females. Regarding recombination, I found no statistical support for sex-specific recombination rates, but the recombination landscape differed between sexes, with males having more crossovers towards the chromosome ends compared to females. Besides, I developed an interactive R application, RecView ShinyApp, to implement the methodology of locating recombination for future studies within similar topic.Finally, I used statistical approaches based on allele frequency differences and associations with sex per se to search for sexually antagonistic loci with whole genome sequencing data from 100 old great reed warblers that aged between 3 and 5 years. By comparing the top 100 SNPs with the strongest allelic differentiation between the sexes, and the most significant associations with sex, I discovered 50 overlapping SNPs that constitute candidates for future studies of sexual antagonistic selection. To conclude, this thesis has improved the methodology for studying the timing of recombination cessation on sex chromosomes as well as to study recombination per se, identified sex-specific de novo mutation rates and sexually dimorphic recombination landscapes, and obtained candidate loci for sexual antagonism

Variational Hamiltonian Monte Carlo via Score Matching

Traditionally, the field of computational Bayesian statistics has been divided into two main subfields: variational methods and Markov chain Monte Carlo (MCMC). In recent years, however, several methods have been proposed based on combining variational Bayesian inference and MCMC simulation in order to improve their overall accuracy and computational efficiency. This marriage of fast evaluation and flexible approximation provides a promising means of designing scalable Bayesian inference methods. In this paper, we explore the possibility of incorporating variational approximation into a state-of-the-art MCMC method, Hamiltonian Monte Carlo (HMC), to reduce the required gradient computation in the simulation of Hamiltonian flow, which is the bottleneck for many applications of HMC in big data problems. To this end, we use a {\it free-form} approximation induced by a fast and flexible surrogate function based on single-hidden layer feedforward neural networks. The surrogate provides sufficiently accurate approximation while allowing for fast exploration of parameter space, resulting in an efficient approximate inference algorithm. We demonstrate the advantages of our method on both synthetic and real data problems

Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases

For big data analysis, high computational cost for Bayesian methods often limits their applications in practice. In recent years, there have been many attempts to improve computational efficiency of Bayesian inference. Here we propose an efficient and scalable computational technique for a state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian Monte Carlo (HMC). The key idea is to explore and exploit the structure and regularity in parameter space for the underlying probabilistic model to construct an effective approximation of its geometric properties. To this end, we build a surrogate function to approximate the target distribution using properly chosen random bases and an efficient optimization process. The resulting method provides a flexible, scalable, and efficient sampling algorithm, which converges to the correct target distribution. We show that by choosing the basis functions and optimization process differently, our method can be related to other approaches for the construction of surrogate functions such as generalized additive models or Gaussian process models. Experiments based on simulated and real data show that our approach leads to substantially more efficient sampling algorithms compared to existing state-of-the art methods

Optimum advertising pulsation strategies: A dynamic programming approach

This study, using the dynamic programming approach, has addressed the problem of optimally allocating a fixed advertising budget of a monopolistic firm over a planning horizon comprised of n equal periods to maximize two popular measures of advertising performance: (1) profits related to the advertising effort (discount factor r = 0), and (2) present value of profits related to the advertising effort (discount factor r \u3e 0). Two dynamic programming models that use the modified Vidale-Wolfe model to represent sales response to advertising are formulated with respect to whether the time value of money is considered. For a planning horizon comprised of four equal time periods, computing routines are developed to solve two sample problems with respect to the dynamic programming models. Sensitivity analyses are performed to assess the impacts of a change in some key model parameters upon the behavior patterns of the optimum dynamic programming advertising policy and the associated total return. Four alternative types of traditional advertising pulsation policies are modeled for the purpose of comparing their performance with the optimum advertising policy determined by dynamic programming. For a planning horizon comprised of four equal time periods, computing routines are also developed to generate total returns under these traditional advertising pulsation policies. Computational results show that the performance under the optimal advertising policy determined by dynamic programming, as expected, is at least as good as the maximum performance among the four traditional advertising pulsation policies. The plausibility of the modified Vidale-Wolfe model is empirically examined using the well-known Lydia Pinkham vegetable compound annual data covering the period from 1907 to 1960. Model parameters have been estimated using the Gauss-Newton algorithm related to nonlinear regression. The model selected is one corrected for first-order autoregressive residuals. The empirical results indicate that the model parameters are statistically significant and of the expected signs. More important, it is found that the advertising response function is concave

Deep Network Approximation: Beyond ReLU to Diverse Activation Functions

This paper explores the expressive power of deep neural networks for a diverse range of activation functions. An activation function set $\mathscr{A}$ is defined to encompass the majority of commonly used activation functions, such as $\mathtt{ReLU}$, $\mathtt{LeakyReLU}$, $\mathtt{ReLU}^2$, $\mathtt{ELU}$, $\mathtt{SELU}$, $\mathtt{Softplus}$, $\mathtt{GELU}$, $\mathtt{SiLU}$, $\mathtt{Swish}$, $\mathtt{Mish}$, $\mathtt{Sigmoid}$, $\mathtt{Tanh}$, $\mathtt{Arctan}$, $\mathtt{Softsign}$, $\mathtt{dSiLU}$, and $\mathtt{SRS}$. We demonstrate that for any activation function $\varrho\in \mathscr{A}$, a $\mathtt{ReLU}$ network of width $N$ and depth $L$ can be approximated to arbitrary precision by a $\varrho$-activated network of width $6N$ and depth $2L$ on any bounded set. This finding enables the extension of most approximation results achieved with $\mathtt{ReLU}$ networks to a wide variety of other activation functions, at the cost of slightly larger constants

On Enhancing Expressive Power via Compositions of Single Fixed-Size ReLU Network

This paper explores the expressive power of deep neural networks through the framework of function compositions. We demonstrate that the repeated compositions of a single fixed-size ReLU network exhibit surprising expressive power, despite the limited expressive capabilities of the individual network itself. Specifically, we prove by construction that $\mathcal{L}_2\circ \boldsymbol{g}^{\circ r}\circ \boldsymbol{\mathcal{L}}_1$ can approximate $1$-Lipschitz continuous functions on $[0,1]^d$ with an error $\mathcal{O}(r^{-1/d})$, where $\boldsymbol{g}$ is realized by a fixed-size ReLU network, $\boldsymbol{\mathcal{L}}_1$ and $\mathcal{L}_2$ are two affine linear maps matching the dimensions, and $\boldsymbol{g}^{\circ r}$ denotes the $r$-times composition of $\boldsymbol{g}$. Furthermore, we extend such a result to generic continuous functions on $[0,1]^d$ with the approximation error characterized by the modulus of continuity. Our results reveal that a continuous-depth network generated via a dynamical system has immense approximation power even if its dynamics function is time-independent and realized by a fixed-size ReLU network