Improving heritability estimation by a variable selection approach in sparse high dimensional linear mixed models

Motivated by applications in neuroanatomy, we propose a novel methodology to estimate heritability, which corresponds to the proportion of phenotypic variance that can be explained by genetic factors. Since the phenotypic variations may be due to only a small fraction of the available genetic information, we propose an estimator of heritability that can be used in sparse linear mixed models. Since the real genetic architecture is in general unknown in practice, our method enables the user to determine whether the genetic effects are very sparse: in that case, we propose a variable selection approach to recover the support of these genetic effects before estimating heritability. Otherwise, we use a classical maximum likelihood approach. We apply our method, implemented in the R package EstHer that is available on the Comprehensive R Archive Network, on neuroanatomical data from the project IMAGEN.