Quick start guide to research impact and our services
This quick guide is to help you orientate yourself with:
- how we define research impact;
- how to begin thinking about your own impact journey;
- how we can build on existing impact journeys; and,
- support and guidance available.
The MVLS Research Impact team are working on training sessions for each of these elements, drawing from existing examples of College research impact. These resources will be made available and advertised in due course.
This is an audio-narrated overview of understanding research. If you don't see a slide deck, follow the prompt to sign in to your UofG Microsoft360. Each slide has a voiceover—if it doesn't autoplay, click the three dots in the bottom right of the slide. The presentation can be enlarged (bottom right icon) to open into a new tab/window in the PowerPoint online app—you can then view as a presentation (55 mins). If you struggle with the embedded copy, you can access the presentation directly.
Understanding impact (read time: 5 mins)
Research impact has a range of definitions, but the UK Research & Innovation's (UKRI) Research Excellence Framework (REF) defines it as the demonstrable contribution that excellent research makes to society, the economy (and the natural environment). It must represent an evidenced, measurable effect, benefit or change to:
- activities, attitudes, awareness, behaviour, capacity, opportunity, performance, policy practice, process or understanding among
- a wide range of stakeholders: an audience, beneficiary, community, constituency, organisation or individuals outwith the institution
- in any geographic location whether locally, regionally, nationally or internationally.
Although REF is a useful framework to understand impact, often we can identify impact narratives that don't fit within the strict windows of that exercise, either because the impacts happened over a longer timeframe than the REF allows, or they cannot be unequivocally evidenced to the high bar that REF requires. The MVLS Research Impact Support team are keen to record a broad range of impacts arising from research across the College, and can work with research staff and Impact Champions to help identify and recognise such impacts.
Understanding change at different levels: a logic model
Inputs, activities, outputs, outcomes and impact each describe changes at different levels, but all contribute to the impact narrative. They collectively form your 'logic model', which is a means that many programmes and organisations use to conceptualize the progression of activities through to goals—how each component contributes towards the intended change, and what the assumptions and risks are at each stage. Some of you may also be familiar with a 'theory of change', which perhaps goes a little further to identify the causal mechanisms for why each component is expected to achieve the intended outcomes. However, the primary topic we want to introduce at this stage is the use of language (semantic standards) that we use to describe impact and the processes that lead to it.
Inputs are the people, money, time, materials and infrastructural resources needed to make your activities possible.
Activities include your research itself as well as associated knowledge exchange and engagement activities (and research communications): stakeholder conferences, co-design workshops, training events, sitting on advisory boards, webinars. Often these are elements that you may in the past have written into 'Pathways to Impact'. While extremely valuable when targeted and planned, they are not impact. Most will be familiar that activities are the things we do and impacts are the changes that result, but it's useful to understand what comes inbetween.
Outputs are the deliverables from your research and engagement activities, and include new knowledge, insights, models, datasets, technical innovations, products and services produced directly within both the programme of research. Outputs are within the researchers sphere of control. In some cases, a research publication might be described as an output; however, it is the content of the publication (and the engagements to share that content) that underpins the impact. So, outputs can include:
- practice manuals;
- media engagements;
- a new antibody;
- a new mouse model;
- a new algorithm;
- new understanding about a species or ecosystem;
- evidence of a factor mediating a disease state;
- characterising a patient population; or
- findings from a new health linkage study.
Outputs can also be closer to translation/implementation if co-developed with a non-academic partner; for example:
- new software;
- a new validated drug target or lead molecule; products
- a new assay or diagnostic test;
- a new professional training framework;
- a new ecosystem management plan;
- defining the type or design of a new health intervention;
Outcomes almost always reflect a change made by the outputs. This may be in changes in the attitude, knowledge and skills—namely, the agency or behaviour of other actors, be they an individual, a group, or an organisation—when they make use of, or are influenced by research outputs. Using this definition, outcomes can apply equally within and beyond academia. By contrast, impact is defined by UKRI as outcomes solely beyond academia. Outcomes in this context might include:
- adoption of a new technology or professional practice;
- implementation of a new health intervention;
- adoption of a new species or conservation management approach;
- license of patents; and
- new policy, clinical guidelines or drug approval.
Outcomes are typically finite and measurable changes. So, one may be able to quantitate the scale of uptake of a new technology or how many professionals have changed their practice. For example, if practice change is linked to a weight-loss intervention, then we might determine how many users lost weight and the extent of that weight loss over a set period of time. Likewise, we might quantify how many individuals received an intervention as a result of a new screening tool or diagnostic device, which they wouldn't otherwise have received. We might find out what a company earned from sales, or what geographical area a policy covers. Outcomes are typically beyond the sphere of a researcher's control, but they're not always beyond the sphere of influence and interest. As outcomes represent medium-term effects they are sometimes considered synonymous with impact, which they are to some extent. However, as we see below, impact is subtly broader.
The lines between output and outcome can blur, depending on whose perspective it is viewed from—this is particularly true in applied/co-developed research. In some circumstances, outputs can include the changes resulting from the use of other outputs (insights, products, services) that are relevant to the achievement of outcomes. These might include enhanced understanding or knowledge following a training workshop, or from a trial of an intervention with professional users. In these cases we can record changes in understanding and other key statstics as outputs, because these are deliverables from such research, even if they look like outcomes or impact from the perspective of someone engaged in basic research. However, these outputs are still distinct from how and whether that knowledge is then adopted and used to affect change or realise benefits—for this, see outcomes and impact below.
Impact (better thought of as 'realised benefit') encompasses the wider, long-term secondary effects of outcomes, often a result of changes in behaviour of many actors within wider socioeconomic and environmental contexts. These effects are well beyond a researcher's sphere of control or influence. Impacts include:
- the realised health benefits of a clinical guideline or practice change being applied, for example, reduced hospitalisation or morbidity, and improved quality of life;
- the societal or environmental benefit of a new policy being implemented successfully;
- recovery, improvement or effective management of an ecosystem or species;
- economic or other benefits accrued from a licensee exploting your research;
- the reimbursement and real-world effectiveness of a new drug; and
- the benefits accrued by users of a product that was sold by a company that used your research to create, validate or otherwise improve processes to realise that product.
Understanding whether and how outcomes have led to an impact, and the extent of the impact, is also not always immediately apparent. Often it requires additional reporting from third-party organisations, or discrete evaluation and audit. For practical purposes, impact case studies primarily articulate well qualified and quantified outcomes.
TIP: As outcomes happen in settings beyond the control of academia, researchers should consider how to extend their sphere of influence through engagement with communities of research users—from specialist groups to companies or the public. Engagement can be at the earliest stages of research in the development of your research programme and questions (co-design); or by bringing partners into an existing programme, having brought the potential benefits to their attention (co-delivery). Such partnerships can also help us recognise, qualify and quantify measurable outcomes and eventual impacts when they arise.
Planning for impact (read time: 3 mins; activity time: 30 mins+)
Developing your impact plan at the outset of your research proposal has been a component of research funding applications for some time. This activity can add considerable value to the design of your research, helping you to develop research questions, and identify collaborators (academic and non-academic) with whom to share the challenge and deliver outputs that lead to impact-generating outcomes. Furthermore, by anticipating the change or benefit you are hoping to see, planning helps clarify what evidence you might collect to substantiate these, and from whom.
A note on Pathways to Impact: From 1st March 2020 Pathways to Impact and the Impact summary were removed from all UKRI funding applications. However, impact is still a crucial aspect of research proposals. Here is some guidance on how to embed impact in your proposal: 2020 Impact Festival Slides (see also: EPSRC Embedding Impact 2020 video). While these materials are based on EPSRC applicant and reviewer guidelines, they are broadly applicable to all UKRI funders, including those to whom College members more commonly apply.
Key questions to ask of your research to impact plan
Work through a structured series of questions in our step-by-step guide to help develop your impact plan. Access a downloadable template, which takes you through the questions outlined below.
Planning for impact template: interrogate your research and help to formulate what your research impact vision might look like (this is just a .docx version of the questions below).
1. Identify what is happening (or not happening) ‘out there’ that you think your research findings may change.
Is there a gap:
- in knowledge and understanding (among the public, specific user groups, practitioners)
- in knowledge and understanding (about an action, reaction, process, technology)
- in diagnosis, response to, treatment of problem
- or a demand in the market that your findings could help address?
For example, is a policy non-existent or not fit for purpose to deal with the issue you are examining? Remember that it could be the methods, tools and collaborative networks you establish, rather than your research findings, that others are interested in. If you've developed a partcularly novel tool, make sure you consider wider applications for it. Always ask: "Who are my competitors?"—if you don't know who they are, or you feel there aren't any, then you're probably not in the competition.
2. List anyone you think will be interested in, affected by, or involved in delivering or experiencing the change that may happen as a result of your findings.
- What groups of people will be most directly affected by a change?
- Are there parents or carers for these groups?
- What professional sector deals with these groups (teachers, GPs, police)?
- Is there third sector support or advocacy for any of these groups?
- Also consider who might be adversely affected or against your research.
3. List what these user communities will get – why might they be interested?
- Will they be getting new or improved policies, understanding, practice, or new/improved product, process or system?
- Might there be additional, follow-on impacts? For example, if findings influence policy, is there potential for support/guidance to practitioners delivering new policy?
- If findings change practice, is this a model which might be of interest to other populations/jurisdictions?
4. How will you engage / communicate with these user communities?
- Think about the most effective ways to reach each user group. Whichever channel you choose, always build in mechanisms for people to engage at a deeper level, whether to provide views, ask questions or supply additional information.
- FIND OUT MORE in our section on 'Stakeholder engagement' below.
5. How might you be able to demonstrate / evidence any changes and the link back to your research?
- Consider how you might demonstrate that your research has led to impactful outcomes—how will you know what you have achieved? For some types of research, such as interventional research, it is worth building evaluation in from the beginning, perhaps with an academic or non-academic research partner. Alternatively, you might work with a stakeholder/partner on an audit (structured survey, or testimony based) to follow up instances of research uptake.
- FIND OUT MORE in our section on 'Evidencing impact' below.
Top 40 practical tips researchers said helped them achieve impact, from Fast Track Impact.
Stakeholder engagement (read time: 3 mins)
For any kind of impact to occur, there needs to be engagement with the stakeholder communities. In a broad sense, these are groups of people or organisations that have a vested interest in your research or the anticipated outcomes (alternatively, they may have something to lose from it). Stakeholders include individuals, communities, populations; professionals or professional councils; other academics; policy makers (national or international); groups or agencies (e.g., non-governmental organisations); and industry. There are also end beneficiaries—individuals, communities or populations, who benefit from your research, but who may not be a part of your stakeholder group(s).
Ready to begin? Read a guide to stakeholder analysis from Fast Track Impact.
There are many reasons why, as researchers, you may choose to engage with stakeholders early on in your research:
- To better define the direction of your research
- To understand the overall context of the area
- To evaluate existing knowledge and how this might change as a result of the research (evaluation)
- To understand existing practices (e.g., towards disease prevention)—both those that are problematic and those that work well and should be built upon
- To determine areas where understanding is lacking and where research/interventions have the greatest scope for change
- To establish what could prompt the intended beneficiaries to engage with an issue, what barriers they face (e.g., personal, economic, social, political, moral) and what could motivate them to overcome these barriers
- To co-create research/interventions that are socially and culturally acceptable
- To build sustainability into the research
Stakeholders can facilitate impact from your research by bringing expertise and influence, as well as contributing new ideas and perspectives. They may also provide key resources. To identify the stakeholders that might benefit from, or contribute to, your research, consider who may be interested, affected by, or otherwise care about your findings. You may not need to engage with all possible stakeholders, but you can usually identify a key stakeholder with influence (see Fig.).
- Who is interested in my research?
• Who has a stated interest in your research?
• Who would you like to be interested?
• Consider that it may not be your findings that interest them—it could be a tool or methodology, or your connections.
- Who has the power to help or hinder the impacts arising from your research?
• Which groups/organisations have influence or power over the change you are hoping to see? Note that this could include groups beneficial to you who may not be intrinsically interested or aware of your research.
• Who might be adversely affected or against your intended impacts, and why?
- Who is most impacted by your research?
• If not addressed by the above questions, who stands to benefit most from the outcomes of your research?
• Who is harmed by it?
• Note that research beneficiaries may not have much power or influence over the early development of outcomes from your research, but they may influence those who do have power over your work.
TIP: You don't necessarily need to have a relationship with the end beneficiaries, but it's useful to know a stakeholder that does. Some researchers working in highly applied fields may have strong influence on their research users through to the end beneficiaries/communities; however, this is not always a requirement for strong impact. Other researchers, often in basic research fields, may identify research-user stakeholders just one step removed from their work. This might be through a Material Transfer Agreement or licensing of a tool—sometimes to another research lab, or to a company. The user may then go on to have great impact with your material, so your main responsibility is to retain some sort of contact to find out how it has been used (usually a mandate of MTAs and licensing). Further, not all knowledge is encapsulated in a product or tool, it could be 'know-how'. It is equally important to retain good connections with research users to determine how such knowledge is used.
Convening a stakeholder workshop together with an existing partner can help to identify wider stakeholder groups, communications channels and feedback about research—these workshops could be one-on-one, small group, online, or in-person. Consider involving key stakeholders in shaping your ideas at the outset of research planning (co-design).
- who will be responsible for each key activity or programme of engagement. Will it be the PI for the research programme, a postdoc or collaborator?
- when and where will you hold such a meeting? What are the existing timelines for stakeholder decision-making, guideline development, policy engagement?
- can you collaborate with an existing academic partner, industry workshop or patient/user group to create a larger event.
The MVLS Impact Support team can help you think though such engagements, or put you in touch with academic peers who have considerable practical experience in this area.
TIP: Engaging with stakeholders can be extremely rewarding and partnerships can form both informally and formally. It is worth bearing in mind, at all stages, any structural inequities that might exist between you and stakeholder groups. This is particularly important when you collaborators are volunteers, small charities, patients (or other vulnerable people), or stakeholders in the international development sector. Formal ethical frameworks might exist for working with participants in research, and there are legal teams involved with more commercial partnerships; however, some relationships may fall outwith formal ethical approval processes.
The Glasgow Centre for International Development has produced some valuable guidance on ethical challenges in international research, which has broader considerations for any relationships where there may be potential for inequities or power imbalance between you and your stakeholders.
Academic collaborators. Your pathway to impact may arise via academic collaborators. This means you need to keep in mind instances where you have worked with a collaborator on published research that may come to underpin an impact based on their subsequent engagements and translation. If you are in touch with collaborators regularly, then check in with these developments. If you have lost touch with such a collaborator, consider reaching out to ask the question.
Further help: research communications and public engagement
When initiating engagement with new groups, it is worth assessing how you might reach them and what image you present to them. Your first interaction could be online, in which case what does your online presence look like? Is it discoverable, professional, understandable (at least to those you're trying to engage with) and consistent with the messages you've sent out? This is equally important when communicating your research to mass audiences through the media.
Develop your academic presence online: this academic digital footprint resource provides a nuts and bolts guide to improving your academic digital presence, accessing your staff profile and working with the media.
Public & community engagement: If you need help to plan a public or community engagement event, or to develop an effective evaluation plan for that event, you can reach out to the MVLS Public Engagement Team or to the Research & Innovation Services Public & Community Engagement Advisor.
- Training opportunities are available via CoreHR, see the public engagement training and resources page for details.
Engagement with patients and healthcare practitioners: If you're planning highly targeted engagement with patients, or with practitioners in primary care, you may seek advice from colleagues within one of the Patient Involvement and Engagement (PPIE) groups in the College. See also this College guide to public and patient involvement and engagement.
Evidencing impact (read time: 5 mins)
Evidencing and evaluating impact is the process used to determine whether or not your research has led to change or benefit. At one level it is the process you undertake to identify, capture and document evidence across all stages of your research impact journey. This process can involve recording evidence of your own key activities and engagements that contributed to the change or benefit. It may include the collection of documentary evidence published by others (e.g., clinical guidelines, drug approvals, policy documents that reference your work), or requesting testimonials from stakeholders which describe the research contribution.
However, evidencing may also involve more formal evaluation or assessment of your outcomes through additional research, evaluation or audit process to help understand the impacts, substantiate the causal links, and quantitate the precise 'significance' and 'reach' of the impacts. Briefly, 'significance' describes the magnitude or intensity of the effect of impacts arising from your research; whereas 'reach' can describe the numbers, extent or scale of the impact. Reach might be achieved through social or geographical spread, penetration into a particular community of interest, or scaling beyond the original stakeholder groups (e.g., from grassroots to policymakers or vice versa). Knowing more about the outcomes and impact of your research can add value to future research development—useful collateral for funding applications, and to enhance the impactfulness of your research. For example, knowing the health economic benefits of an intervention you've trialled can improve its uptake. Evaluations may reveal ways to optimise core components of your research, or make it more generally applicable.
The main question we often faced with is: Did the research make a material and distinct contribution to the impact, without which the impact would not have occurred (or would be much diminished)? While also recognising that impact may result from a body of research, rather than a single study, to which other institutions may have similarly contributed.
Knowing what you need to evidence comes partly from your planning for impact exercise. Knowing the change or benefit you hope to see, and who might deliver or benefit from it, will clarify what you need your evidence to show. From there, you can try to identify the types of evidence and indicators that corroborate it. The guide below provides an non-exhaustive range of examples.
TIP: Impact doesn't have to be global. Remember that impact isn't always global and wide-ranging. Impact can be achieved even at comparatively small scales, if that is the aim and intent. For example, many policies or interventions are only relevant for specific populations, at specific times and in specific places. Understanding this should factor into your plan for impact—consider your end goal, but plan for the first steps.
There are many different evidence sources, a useful guide is: ‘Types of impact and examples of evidence required to corroborate the impact’. Another useful resource includes 'Guidance on the types of evidence that could be collected', which lists different types of evidence types and their respective advantages and disadvantages.
How do we use evidence?
There are two principle uses of evidence to substantiate impact: (1) providing a LINK between a piece of UofG research and the impact/research users; and (2) quantifying the significance of the impact. The former is required to establish the relationship between your research and what you might claim as impact. It is essential that you collect linking evidence as early as possible. You may have a direct relationship through a contract or something less formal. Alternatively, more indirect evidence might be secured where research users cite a published piece of work; however, clarifying in-person how they have used it is always beneficial. Evidence on the scale and extent of an impact is also important to consider from the outset of your research. Impacts tend to happen beyond the academic sphere, so sometimes evidence of scale and extent may be created by third-party stakeholders.
- Documentary evidence—materials describing exactly what research was considered and how it led to change—is extremely useful, but often rare. Early stage documentary evidence might include published synopses from workshops held by stakeholders. More mature examples include policy documents, clinical practice guidelines or drug approval documentation from health regulatory bodies that cite your work. However, not all citations are equally valid. Consider the context of the citation: is it just background research or rationale that sets the scene? This is a harder case to make unless your research clearly identified the gap that mandated the policy or guideline recommendation. The strongest evidence is where a policy proposal, clinical or practice recommendation, or drug approval review specifically cites your research study (either by trial name or by research paper) to support that recommendation.
- Testimony evidence—in the absence of published documentation, you can seek testimony from a person or organisation. This should describe what research was received and how it has been used. Testimony evidence is particularly useful at the outset of a research engagement, to help establish the nature of the engagement, the research communicated, and how they plan to use it. This can always be followed up at a later date, to identify some of the outcomes, but the initial record provides the LINK. Similarly, for more mature potential impacts (e.g., if an impact on a clinical guideline isn’t explicit), you may need to ask the guideline chair or another clinical body to identify how your research has contributed—sometimes this is very clear to professionals within a field, but not apparent to those outside that field.
London School of Economics blog post: 'Guidance on testimonials and statements to corroborate impact'.
NOTE about 'activities': Many researchers misconstrue activities (the things they do) with evidence of impact (substantiating the changes that happen as a result). Evidence of activity is not the same as evidence of impact. Recording your activities, be it speaking at a policy event/workshop or academic conference attended by industry stakeholders, has useful narrative value only if you have a plan for how you will follow up on those activities, or know the change that resulted from them. Who was it you were hoping to reach? Did you speak to them or share a document with them? Do you have their contact details? Do they have yours? What did they think of your research? How might they use it? Without such evaluation plans or opportunity to identify potential outcomes, a list of activities is good for esteem, but it is not evidence of impact.
Altmetric Institutional Explorer: It is useful to know which stakeholders might be using your research, so that you can search for (or ask them for) documentary evidence they have published that cites your work. However, you may not always know who is using your research. Altmetric Explorer is a useful tool to investigate this: UofG staff have access—just follow the login link. Most of what Altmetrics provide is 'buzz' in the form of media interest, which can be interesting to note, but is not impact in and of itself (the impact would be if you can link the media piece to a change or benefit as a result of someone reading it—no easy task). However, Altmetrics also includes policy documents, such as some national and international guidelines, policy documents and other grey literature. The Research Impact Officers also have some limited access to Overton.io, a newer platform that indexes policy and business documents.
Audit and evaluation evidence
Audit and evaluation evidence is particularly important in interventional research, where you are developing products, services or other interventions that require more evidence than just functionality or efficacy. Building evaluation into the design of your project, ideally involving a relevant non-academic research partner, will add considerable value to the outcomes of the research. It can also identify gaps to be addressed with further research, both opportunities for additional research funding and the scope for increased impact.
The Research Impact Support Team can prospectively support research programmes in identifying opportunities to evaluate their impacts, especially where such evidence would enhance the potential impact. Examples might include:
- health economics analysis of your intervention;
- surveys of professional practitioners to identify uptake and use of recommendations; and
- analysis of prescription rate data studies to show change in medical treatment.
We hope to be able to connect you to College facilities that may be able to support this, or researchers who have practice in these types of investigations. We anticipate that some of these audit and evaluation activities may be funded through internal funds, such as the Glasgow Knowledge Exchange Fund. However, each opportunity is reviewed on a case-by-case basis.
The MVLS Research Impact Support team can help you to review your plans for evidencing impact. In addition, we can offer our own expertise on where to find sources of information, and what such investigations entail. For impact opportunities that we have identified, we can help you to secure testimonies from key stakeholders if you feel that we, as a third party College service, are better placed to make such a request. For more substantial investigations, including evaluation and audits, we are happy to participate in these discussions, and to either bring together or signpost to College professionals able to help with design of such activities.
Where to store your evidence? All researchers who are engaged with external stakeholders should make efforts to store securely any evidence to substantiate the enaggement and any outcomes from it. The Research Impact Team are happy to advise, and will also help store materials related to those impact opportunities that have been added to the MVLS Impact Opportunity Pipeline. However, UofG also hosts a Knowledge Exchange and Impact Repository within the Englighten platform. This allows you to record and document non-academic collaborations, consultations, public or community engagement, knowledge exchange or impact-generating activity and any other external engagement related to or stemming from your research.
What about innovation? (read time: 1 min)
How does innovation fit with impact?
Innovation has a wide range of definitions, depending on where you look. Sometimes the term is used synonymously with translational research—a process of transforming discoveries from curiosity-driven research into real-world applications with measurable outcomes. The UKRI defines innovation more broadly as, "the application of knowledge or ideas for the development of products, services or processes—whether in business, public services, or non-profit sectors.”
However, when it comes to funding innovation, UKRI often want to see one or more of the following—that the work: is disruptive to the target market; improves on the existing state-of-the-art; produces a commercially viable product, service or process; and has wider impact.
Innovation describes a coordinated process through which impact may be generated through a direct application of research to address a known (or even unknown) gap, but innovation is not the sole means to apply research, nor is it the only pathway to impact. Research can have substantial impacts without ticking any of these boxes. However, the common threads are: engaging and asking questions of stakeholders—identifying interest and need, and establishing routes to enable your research to be used. For research and innovation alike, a key question remains: what happened as a result?
If you've read through this quick start guide and feel you still have ananswered questions, or have an opportunity you'd like to discuss further with us, please get in touch. You can email us directly (email@example.com), however, we'd recommend and prefer that you try using our web-form — this collects and structures the type of information that is most useful for us to assist you.