Reproductive Isolation of Hybrid Populations Driven by Genetic Incompatibilities

Despite its role in homogenizing populations, hybridization has also been proposed as a means to generate new species. The conceptual basis for this idea is that hybridization can result in novel phenotypes through recombination between the parental genomes, allowing a hybrid population to occupy ecological niches unavailable to parental species. Here we present an alternative model of the evolution of reproductive isolation in hybrid populations that occurs as a simple consequence of selection against genetic incompatibilities. Unlike previous models of hybrid speciation, our model does not incorporate inbreeding, or assume that hybrids have an ecological or reproductive fitness advantage relative to parental populations. We show that reproductive isolation between hybrids and parental species can evolve frequently and rapidly under this model, even in the presence of substantial ongoing immigration from parental species and strong selection against hybrids. An interesting prediction of our model is that replicate hybrid populations formed from the same pair of parental species can evolve reproductive isolation from each other. This non-adaptive process can therefore generate patterns of species diversity and relatedness that resemble an adaptive radiation. Intriguingly, several known hybrid species exhibit patterns of reproductive isolation consistent with the predictions of our model.Author Summary: Understanding the origin of species is one of the central challenges in evolutionary biology. It has been suggested that hybridization could generate new species because hybrids can display novel combinations of traits that induce reproductive isolation from their parental species (called “hybrid speciation”). Existing models predict that this should only occur in special cases, and indeed there have been only few well-supported examples. We describe a new model of hybrid reproductive isolation that results from selection against genetic incompatibilities in hybrids, which are predicted to be common. Simulations reveal that hybrid populations rapidly and frequently become isolated from parental species by fixing combinations of genes that hinder successful reproduction with parental species. We propose that this process could be an important mechanism for the formation of new species.