Serotonin, a hormone produced in the brain, has long been implicated in the regulation of critical behaviors, such as those related to aggression or impulse control. However, most research on serotonin has focused on the proximate connection to behavior, and little is known about its evolution. This is unfortunate, since the serotonin system has great potential to inform our understanding of behavioral evolution. I seek to address this gap in knowledge by investigating the molecular evolution of the serotonin system in macaques (genus Macaca).
The macaque genus represents a useful model for understanding behavioral evolution. Comprised of approximately 19 species, macaques display a wide range of behaviors. It is likely that behavioral differences are caused by differences in neuroendocrinology. Therefore, the serotonin system provides one potential mechanism through which evolution may act to shape macaque behavior. In this dissertation, I sequence four genes that are known to influence serotonin functioning and behavior: HTR1A, HTR1B, TPH2, and SLC6A4. I examine the pattern of genetic variation within and between several species of macaque, and, using an approach based on molecular evolutionary theory, discern which evolutionary force ??? positive selection, balancing selection, purifying selection, or random genetic drift ??? is most likely to have acted on these genes.
Three out of the four genes (HTR1B, TPH2, and SLC6A4), show a low level of overall genetic variation within the coding regions, suggesting that purifying selection is the predominate force acting on these genes. Within non-coding regions, the patterns of genetic variation found are consistent with genetic drift. Thus, positive selection does not seem to be affecting these genes. The genetic variation for these genes may contribute to the behavioral variation found in macaques; however, any effect that these genes have on behavior is likely due to non-adaptive evolutionary forces.
In contrast to the other genes, HTR1A shows a pattern that is clearly distinct. HTR1A displays an unusually high level of interspecific variation, which is consistent with positive selection. Moreover, a subset of macaque species share a codon loss, an extremely rare event in gene evolution, and analyses of the coding region indicate a significant elevation of protein evolution among certain sites of the gene. These results suggest that positive selection has played a significant role in the evolution of the serotonin system and it is likely that the effects of positive selection on HTR1A contributed to macaque behavioral evolution. This research provides an important first step towards gaining a more thorough understanding of the mechanisms underlying the evolution of behavior
