What impact do we want our research to have beyond academia – and how do we measure that impact? These were among the questions that were discussed at a seminar in Oxford last week organised by the NIHR Oxford BRC.
The seminar, organised by Prof Trish Greenhalgh, the lead for the BRC’s Partnerships for Health, Wealth and Innovation theme, featured Prof Wilfred Mijnhardt, Policy Director of the Rotterdam School of Management at Erasmus University. Both are international experts in the field of research impact.
The measurement of research impact – and specifically the influence research has on areas such as patient outcomes, health service productivity, public understanding of science, economic growth or environmental sustainability – is subject to much debate.
Performance
Impact metrics are used to compare the performance of higher education institutions against each other, for example via impact case studies, which will make up 25% of the total scores in the next Research Excellence Framework. They are also employed by the Treasury to assess the overall performance of the higher education sector.
Impact metrics are increasingly used to monitor the progress of multi-stakeholder research collaborations such as Biomedical Research Centres, established to strengthening the translation of scientific research into clinical practice that benefits patients, and to build partnerships with industry, clinical services, policymakers and patients.
Prof Greenhalgh said: “This seminar was the first in what we hope will be a series running through the life of this BRC.
“The NIHR [National Institute for Health Research] already has a set of robust metrics for measuring success in conventional terms: publications, income, start-ups, doctoral students and so on; and Oxford performs well against these.
“But we should be alert to the work that is going on in the Netherlands and Scandinavia in particular on ‘socially responsible impact metrics and productive interactions’ that can supplement conventional approaches by encouraging us to also attend to such issues as engagement and sustainability. Whilst such metrics appear ‘soft’ to a superficial analysis, there is actually a strong evidence base for including them, and this is what we will be exploring with Professor Mijnhardt over the coming months,” she concluded.
Social contract
Prof Mijnhardt, who has 20 years of experience in research policies and institutional development, was joined in conversation at the seminar by Dr Pavel Ovseiko, Senior Research Fellow in Health Policy and Management at the University of Oxford’s Radcliffe Department of Medicine, and Dr Alexander Rushforth, post-doctoral researcher with the university’s Nuffield Department of Primary Care Health Sciences.
Dr Ovseiko outlined the current approaches to the measurement of research impact in the UK, including Biomedical Research Centres and their implications for the national and regional innovation systems in the context of the changing social contract between science and society.
He said: “Professor Mijnhardt has challenged us to take a long-term strategic view on impact assessment in the UK’s biomedical research.
“Given that we are at the beginning of the BRC3 funding round, it is an opportune time to think creatively about where we would like to be in five years’ time and start laying the foundations for a successful BRC4 bid in 2022.
“By exploring and measuring different types and mechanisms of research impact in the current and previous BRC funding rounds, we can shape organisational norms, rules, values and behaviours to maximise impact from the NIHR’s investment in biomedical research.”
Collaborate
He welcomed the fact that representatives from the NIHR Central Commissioning Facility attended the seminar, highlighting the opportunity to collaborate across the entire NIHR infrastructure.
Dr Rushforth reflected on the best UK and international practices in research impact assessment and explored the strategic opportunities to maximise the value of biomedical research for society as a whole.
Among the issues that the participants at the meeting considered were: what are the different types of impact; how are researchers framing the relationship between science and society; how is their knowledge being disseminated beyond academically-focused outlets such as journals and conferences; and what metrics can be used for measuring impact success – and what do these metrics say about what is valued and incentivised?
Dr Lorna Henderson, Clinical Research Manager of the Oxford BRC, said: “Oxford is leading in the field of research impact. This seminar was an important and timely discussion about how we can continue to build on this legacy in the future.”
Read more about the seminar here.