Funding lotteries for research grant allocation: An extended taxonomy and evaluation of their fairness

Some research funding organizations (funders) are experimenting with random allocation of funding (funding lotteries), whereby funding is awarded to a random subset of eligible applicants evaluated positively by review panels. There is no consensus on which allocation rule is fairer—traditional peer review or funding lotteries—partly because there exist different ways of implementing funding lotteries, and partly because different selection procedures satisfy different ideas of fairness (desiderata). Here we focus on two desiderata: that funding be allocated by ‘merit’ (epistemic correctness) versus following ethical considerations, for example without perpetuating biases (unbiased fairness) and without concentrating resources in the hands of a few (distributive fairness). We contribute to the debate first by differentiating among different existing lottery types in an extended taxonomy of selection procedures; and second, by evaluating (via Monte Carlo simulations) how these different selection procedures meet the different desiderata under different conditions. The extended taxonomy distinguishes “Types” of selection procedures by the role of randomness in guiding funding decisions, from null (traditional peer review), to minimal and extensive (various types of funding lotteries). Simulations show that low-randomness Types (e.g. ‘tie-breaking’ lotteries) do not differ meaningfully from traditional peer review in the way they prioritize epistemic correctness at the cost of lower unbiased and distributive fairness. Probably unbeknownst to funders, another common lottery Type (lotteries where some favorably-evaluated proposals bypass the lottery) displays marked variation in epistemic correctness and fairness depending on the specific bypass implementation. We discuss implications for funders who run funding lotteries or are considering doing so.