Understanding the impact of research

Academic research impacts on everything from healthcare, to economic policy. But how do we understand and evaluate that impact? Martin Szomszor of Digital Science tells us more about their latest work to visualize research impact in the UK.

Academic research has many benefits. Technological advances, richer culture, improved social and economic policies, and better medical care are just a few of the many outputs forming a complex and interconnected network of research impact.

Understanding this behaviour is crucial to governments, funders, charities, and universities to ensure resources are directed appropriately and the proper measures are in place to evaluate its success.

Last year saw the completion of the UK Research Excellence Framework (REF2014), the latest in a series of periodic research assessments by the Higher Education Funding Council for England (HEFCE) to measure the diversity and success of UK research.

Traditionally, this type of assessment is based on bibliometric measures by requiring academics to submit their best work for review – usually articles in prestigious journals, notable books, etc.

REF2014 introduced a novel aspect to the assessment through the inclusion of impact case studies – documents that describe how underpinning research has led to impact, with sources to corroborate claims. Submissions are made to one of 36 disciplinary units of assessment, further categorised into four main panels. A total of 6,975 case studies were submitted, capturing the work of 50,000 researchers working in 154 institutions in the UK.

Digital Science, in conjunction with the Policy Institute at King’s College London, were commissioned by HEFCE to publish these case studies on a searchable website and provide a synthetic analyses of their content.

For the first time, a large collection of evidence is now available describing the impact of university research in the UK, providing a unique insight into the types of impact reported (e.g. economic, political, environmental, etc.), and the various beneficiaries of the research (e.g. schools, hospitals, local government, etc.). It also shows how the primary research ultimately reaches the beneficiaries.

In our recent Digital Research Report, The Diversity of UK Research and Knowledge, Digital Science exemplifies the breadth and depth of research impact through BrainScans – networks where nodes represent the volume of research in disciplines, and edges the interdisciplinary research described in case studies.

Both global (see Figure 1 below), and institutional diagrams can be created, allowing us to inspect and report on the relative strengths and weaknesses of research competencies at national and institutional levels.

Digital Brain Scan
Fig.1. Click to see larger image

Accompanying this report is an online visualisation for exploration of the case study database (Figure 2 below). By performing text analyses, we are able to cluster together related case studies according to the underpinning research. Nodes in the visualisation are colored according to the panel they were submitted to, revealing interesting clusters around topical subjects such as climate change, immigration, biodiversity, and science outreach.

online visualisation
Fig. 2. Click to go through to the online visualization

These kinds of insight are paving the way to an increased understanding of how to collect, model, and report on research impact. As well as demonstrating the diversity and complexity of the problem, the case study database also provides new evidence on how different disciplines come together. For example, how technological outputs are combined with medical research to combat global health issues.

We are sure to see much research on this topic in the future. Academics will have to learn more about how the research leads to impact, institutions will have to do more with academics in reporting this, and analysts will look to more data sources (such as altmetrics) to increase the scope of their understanding.

View the latest posts on the Research in progress blog homepage