HERC researchers were involved in the first-ever randomized experiment to look at the impact of medical research funding on academic productivity by taking advantage of NZ Health Research Council Explorer Grant Scheme, which uses a stage process including a lottery: https://gateway.hrc.govt.nz/funding/downloads/2024_Explorer_Grant_Application_Guidelines.pdf
Professor Philip Clarke indicated, "While the study is relatively small, involving only 88 individuals, they were follow-up for an average of 3.8 years on a range of outcomes including publication, citation and media impact. While our results are inconclusive on whether research funding has an effect on subsequent research output, it was an important proof-of-concept study.
Our current collaboration with the British Academy, which employs a lottery process as part of its small grants scheme, will enable us to further explore this issue: ‘Promising’ results from first year of innovative grant awarding trial show greater diversity of awardees and institutions given funding | The British Academy
Details of the Paper
Title: The impact of winning funding on researcher productivity, results from a randomized trial
Abstract
The return on investment of funding science has rarely been accurately measured. Previous estimates of the benefits of funding have used observational studies, including regression discontinuity designs. In 2013, the Health Research Council of New Zealand began awarding funding using a modified lottery, with an initial peer review stage followed by funding at random for short-listed applicants. This allowed us to compare research outputs between those awarded funding or not using a randomized experimental study design. The analysis included eighty-eight researchers who were followed for an average of 3.8 years of follow-up. The rate ratios (and 95 per cent credible intervals (CI)) for funding were 0.95 (95 per cent CI 0.67 to 1.39) for publications and 1.06 (95 per cent CI 0.79 to 1.43) for citations, showing no clear impact of funding on research outputs. The wider use of funding lotteries could provide robust estimates of the benefits of research funding to better inform science policy.