The world of research funding is about to get a futuristic twist! UK Research and Innovation (UKRI) is exploring the use of AI to tackle the overwhelming surge in grant applications.
With an annual research funding allocation of over £8 billion, UKRI has seen a staggering 80% increase in applications over the past seven years, while the number of funded grants has halved. This has led the organization to seek innovative solutions to streamline its peer review process.
Enter Mike Thelwall, a data scientist from the University of Sheffield, who, along with his team, is delving into the potential of generative AI. Funded by the UK Metascience Unit, the first government body dedicated to enhancing research processes, their mission is to determine if AI can ease the burden on peer reviewers.
Thelwall's team will have access to a confidential dataset of 1000-2000 full-text grant proposals, either funded or rejected by UKRI. They plan to feed these proposals into large language models (LLMs) to see if AI can accurately predict the scores and decisions made by human reviewers.
But here's where it gets controversial: Thelwall believes that if LLMs can predict grant proposal scores with reasonable accuracy, they could potentially speed up the review process or support human reviewers. However, Mohammad Hosseini, an AI ethics researcher at Northwestern University, raises doubts about LLMs' ability to generate novel ideas, which could impact their effectiveness in detecting creative grant proposals.
And this is the part most people miss: the potential backlash from researchers if funding organizations don't disclose the criteria fed into AI systems. Transparency is key, but it could also lead to applicants gaming the system, writing to please AI rather than human reviewers.
So, how might UKRI utilize generative AI? Thelwall suggests it could be a tiebreaker or an additional reviewer, even assisting in a fast-track desk-reject option to reduce the workload on human experts. He cites the example of the la Caixa Foundation in Barcelona, where AI-assisted peer review still requires 90% of applications to go through full review by human experts.
While the potential benefits are exciting, the ethical considerations and the risk of gaming the system cannot be overlooked. As UKRI explores this innovative path, the research community eagerly awaits the outcomes and the potential impact on the future of grant funding.
What are your thoughts on using AI in grant peer review? Do you think it could revolutionize the process, or are there too many potential pitfalls? We'd love to hear your opinions in the comments!
