Research Evaluation: Talking about your outputs
At the University of Glasgow we want to create an environment that produces research of the highest quality. Good-quality research happens within a community that supports, incentivises and rewards a positive research culture.
The fair and inclusive assessment of research activities is central to this ambition. For this reason, in 2020 the University of Glasgow formally declared its commitment to the following international initiatives, to demonstrate our commitment to good practice in research assessment:
These commitments sit alongside the University’s previous adoption, in 2017, of the Leiden Manifesto for Research Metrics.
We have developed the following resources to help researchers in thinking about how they talk about research outputs (e.g. in a job, promotion or grant application) and the sorts of questions to ask about publication strategy during mentoring or career development conversations.
Below we provide:
- Conversation starters for both self-reflection and mentorship discussions relating to DORA-inspired publication strategies and tactics. This aims to ensure that research outcomes reach the desired audiences promptly and openly, thereby maximising both the impact of the research and recognition of the researchers.
- The policies and initiatives that we have introduced since 2015 to assess the range of contributions to research in a fair and transparent manner, and that are sensitive to disciplinary context.
The principles of DORA/Leiden/Hong Kong Manifesto support the University’s own principles for research, as outlined in the 2020-25 Research Strategy i.e. that:
- We value the quality of our research over its quantity.
- The University succeeds when our individual researchers succeed.
- How research is done is as important as what is done.
Conversation starters about individual research and publication strategy
Questions researchers might want to ask or explore:
The Declaration on Research Assessment (DORA) has a fundamental aim to avoid the use of the journal impact factor as a proxy for article quality. However, discussion around the quality of our published outputs and publication strategies remain important, and we encourage supervisors, line managers, mentors, and indeed peers, to support others to reflect on the quality of their work and on how to enable the work to reach the widest relevant audience.
Below, we list possible questions to aid conversations centred around a researcher’s own situation and career needs. The guidance relates not only to the outputs themselves but the research and plans for research that underpin these outputs.
- What are your plans for developing research outputs (e.g. publications, software, performances) over the next 12 months?
- What strategy will give your work the highest international profile: a) with your immediate peers? and b) within the wider discipline?
- What is your approach to sharing your research (before, during, and after formal publication) and getting critical input to it? How has advice from critical friends been incorporated into your research and resulting outputs?
- How would you succinctly describe your strongest research outputs, in terms of the originality of the research, its significance to the field, your contribution to the work?
- What is your approach to open research, e.g. the sharing of data, code or software, and how might you enhance it?
- In terms of your international peers, what has been the most significant output(s) published in the last 12 months, and why? What can you learn about the research direction or how it was shared or presented?
- In the next couple of years, what is the right balance between your outputs targeted at a specialist audience and those aimed at a wider readership?
- (where appropriate) Is this the right time in your career to consider writing a monograph or review paper; if so, what would be the best route to publication?
- In considering your recent publications, who is citing or commenting on your work? What are they saying about it? Of your influential or highly cited outputs, do you know why they have attracted attention?
- (where appropriate) How are you supporting colleagues at an earlier career stage to develop an outputs strategy that is complementary to yours?
The support of the University of Glasgow for DORA, Hong Kong Principles and the Leiden Manifesto is consistent with and embedded within our institutional strategies, policies, and action plans.
- We value the quality of our research over its quantity
- The University succeeds when our individual researchers succeed
- How research is done is as important as what is done
- Institutional Strategic Priorities for Research Culture 2020–2025
- Concordat for the Career Development of Researchers 2020–2023
Externally the University of Glasgow is a member of the UK Reproducibility Network, the European Guild of Research Intensive Universities and has established a Lab for Academic Culture to enhance academic research and teaching culture both by implementing local initiatives and shaping sector policies.
Our policies and practices
Our policies and practices include:
|2020||Updated our institutional KPIs for research to include transparent and normalised bibliographic metrics, including open access, open data, author contribution statements, and uptake of ORCID.||Research KPIs
|Institutional success requires targets and goals. These apply to large groupings, not individuals, and are aligned to the environment we wish to encourage.|
|2012||Introduced academic promotion criteria that follow a preponderance approach. Promotion is determined through meeting transparent criteria in at least 4 of 7 performance dimensions: outputs, income, supervision, teaching, engagement, and esteem. Promotion criteria are detailed and openly available. Targets are benchmarked and subject-normalised.||Academic Promotion Criteria||Recognition that staff contribute to the success of a team/organisation in a variety of equally important ways. No individual is required to meet the criteria for all seven dimensions, but as a team we do.|
|2012||Promotions consider the previous six- year period when assessing the contributions made across the 7 performance dimensions.||Academic Promotion Policies||Research success requires planning and execution over many years and cannot be judged on a single review cycle.|
|2019||Revised professorial promotion criteria to require evidence of collegiality (i.e., support for the careers of others).||Making Collegiality a Core Element of Research Culture at UofG||Government, funders, and the sector more widely are rightly insisting on an improved research culture.|
|2019||Updated professorial promotions to bring parity of reward for impact and outputs, a commitment to open research and minimum standard applied to learning and teaching.||MyGlasgow News: Academic Promotion Criteria||Funders and government require academia to make a more rounded contribution to society.|
|2017||Launched policy on “responsible use of metrics”, committing to fair and transparent mechanisms for evaluating and reporting on research performance.||Responsible Metrics||Quantitative indicators have a vital function in unbiased assessment of research, but must be normalised (by subject and career stage), contexualised, and used in tandem with peer review.|
|2016||Open research (including open access, wide and early sharing of research). Library supports open data, data availability statements, open research; all tracked as part of Research Culture Action Plan.||How do I make my Publication Open Access
|Research is accelerated and integrity is maintained by early, wide and complete reporting.|
|2016||Introduced new recruitment practices within our Fellowship programme, e.g. requiring short narrative on small number of best outputs, and explicitly referring to different output types, and requiring a commitment to open research.||LKAS Fellowships||Output quality cannot be judged on metrics alone and requires context to assess significance, contribution, and contribution to strategy.|
|2019||Ensure that impact factor, or wording that could be perceived as referring to journal-based metrics, is not used as an indicator in research assessment processes.||Statement on Use of Quantitative Indicators in Assessment of Research Quality||The Journal Impact Factor was introduced to compare journals and has no role in the assessment of individual researchers or outputs.|
Development and support
In implementing the policies above we seek to both Develop and Support our staff.
|Development and Support||Links||Why|
|2015-2018||Panel-based external reviews of research (2015, 2018) were aimed at identifying and sharing what makes a good quality article in each discipline.||Interim Research Reviews (VPN required)
Interim Impact Reviews (VPN required)
|Establishing what research excellence means and how it might be recognised at the discipline level.|
|2015||Established a network of research integrity advisors, to record informal queries or concerns received from staff and students on good research practice.||Research Integrity Advisers
Research Integrity Areas of Conduct
|Research integrity is not simply about compliance, but about supporting and advising on best practice.|
|2019||Research culture awards to recognises the contribution of all grades and job families of our staff.||Research Culture Awards||Supporting and celebrating the work that our colleagues undertake, often previously unrecognised, in support of the careers of others|
|2017||Embedded the CRediT taxonomy into our Code of Good Research Practice to encourage researchers to put their author contribution on the public record; we have also enabled researchers to record their author contribution(s) in our institutional outputs repository.||CRediT taxonomy
How do I make my Publications Open Access
Before Submitting Your Manuscript
|Research is often a team activity and traditional author listings fail to capture individual roles and contributions that are central to researchers’ assessment and their careers.|
Advice to individuals on managing publication profiles:
|Managing Your Publications||The communication of our research to the right audiences is as important as the research itself, both in support of the wider community and the progression of individual careers.|
|2021||Conversation starters in support of the careers of others and publication strategies.||Conversation Starters||Career development depends upon local peer group support. This support requires formal and informal conversations, the initiation of which is critical.|
|2020||Developed resources / campaign for PI career conversations (summer 2020) with follow-on survey of researchers in October 2020; articulated what ’10 days of CPD looks like’ for researchers.||PI career conversation with researchers||Career advice depends upon local support and advice. Dialogue, not form filling, is central to the support we seek to encourage.|
|2021||Produced a toolkit for hiring managers to ensure we recruit researchers and academic staff who will embody the values of our new UofG research strategy — to support creativity, collaboration and career development — and ensure a positive research culture.||SCOPE
UofG SCOPE case study
|This resource emerged from a SCOPE workshop we hosted to identify what is important in supporting the careers of others and how we can evaluate this (e.g. in grants, recruitment, appraisals or future REF exercises.|
Related strategies, initiatives and action plans
- University Research Strategy 2020-2025
- Institutional Action Plan for Research Culture 2020-2025
- Action Plan for the Concordat on the Career Development of Researchers 2020-2023
- Research Culture at the University of Glasgow
- Research Integrity at the University of Glasgow
- Research Culture Timeline
- Lab for Academic Culture
- For guidance and support on sharing your research, and on tracking engagement with that research, visit our managing your publications webpages.
- The University encourages the early and wide sharing of research findings wherever this is possible, e.g. through pre-print publication.
- PI career conversations with researchers (Moodle resource)
- Publication support (Moodle resource)