Programme Evaluation and Assessment Blueprinting to address assessment workload
MPhil Textile Conservation Programme, 2 year PGT, based in History of Art, School of Culture and Creative Arts, College of Arts and Humanities.
What was the rationale behind your activity?
One driver for the implementation of an assessment blueprinting exercise was in response to concerns about high assessment load (both formative and summative) on a two-year master’s programme combining academic and professional skills (including practical work). As part of the programme, the practical work requires students to spend time in the studio/labs (involving workshops and supervised studio time) to practice and develop their skills which requires high contact hours and consequently this left less time for independent study to complete assessments.
The programme design had been previously reviewed in 2017 following curriculum mapping and assessment blueprinting exercise (CMAB) which was invaluable in ensuring effective alignment of intended learning outcomes (ILOs) and development of skills, knowledge and understanding both within courses and across the programme. However, over recent years it has become clear that this needed to be looked at again as students and staff were finding the assessment workload was overly high. In addition, students were keen for the level of practical skills work also to be more fully recognised in the assessment weighting as this is a key component of the programme.
Implementation (what was done)
In the academic year 2023/24, another CMAB was undertaken in consultation with Kim Davis (Academic and Digital Development Adviser, Academic Services, and co-lead for the Assessment and Feedback University Strategy Workstream) in conjunction with student/staff/external examiner discussions and consultations. The course aims, ILOs and assessments (both formative and summative) were tabulated and reviewed through the curriculum mapping process. This was invaluable in enabling a comparison of the learning development opportunities, ILOs and it also provided a straightforward means to review the assessments and associated workload. This process highlighted in what ways we were creating a high assessment workload.
Through this process we identified that there was potential to combine the learning and teaching across two of the courses in each semester creating greater alignment of learning, more efficient timetabling opportunities and spread of assessments. As a result, the programme structure was redesigned. Two courses in semester 1 were combined to make one 40-credit changing and this was replicated in semester 2. The 40-credit courses combined two of the professional skills courses. This not only provided a more cohesive approach to the learning and teaching, but it also provides greater breadth and flexibility in the course design as well as reducing the assessment workload.
The teaching team revised ILOs to provide greater clarity. Changes in the assessment workload is reflected in the tables below. Part of the process was also to think more about the workload ensuring the impact of the formative assignments was taken into account in a more transparent manner. This led to an assessment workload reduction of a roughly a third.
Alongside this we have introduced the use of analytic rubrics so we are spending less time on grade justification but working to improve feedback literacy as well as consistency and efficiency of marking focusing on feeding forward to.support learning development. In addition, students are expected to engage more with self-evaluation of the developing work with the aim to promote greater self-regulation.
|
Principles and Practice: Advanced Skills course (20-credits) |
Type of Assessment |
Word Count/Duration or Equivalent |
Weighting |
|
Formative |
Examination of Objects 1 |
1000 words |
0% |
|
Formative |
Examination of objects 2 |
In class discussion |
0% |
|
Formative |
Critical Review of an article part of group work |
2 hours preparation plus 15mins group presentation |
0% |
|
Formative |
Substantiated Conservation report (professional requirement) linked to practical work |
3500 words |
0% |
|
Summative |
Practical: Practical textile conservation |
Continual assessment over course |
30% |
|
Summative |
Report: Substantiated Treatment Proposal – 2.5 days |
3500 words |
50% |
|
Summative |
Viva |
30 mins |
20% |
|
Formative and summative combined |
|
8000 words plus practical work, 45 mins presentation and class discussion. |
|
Plus
|
Conservation in Practice course (20-credit) |
Type of Assessment - |
Word Count/Duration or Equivalent |
Weighting |
|
Formative |
Literature Search |
2 hours and class discussion |
0% |
|
Formative |
Annotated bibliography (peer review) linked to summative assignment |
250 words plus peer review |
0% |
|
Summative |
Annotated bibliography |
1000 words |
20% |
|
Summative |
Literature Review |
4000 words |
80% |
|
Formative and summative combined |
|
5250 words and class discussion |
|
Total word count: 13,520 words, practical work and 45 mins presentation and class discussion across the 20 credit courses.
|
40 credits - new Principles and Practice Advanced Skills |
Type of Assessment |
Word Count/Duration or Equivalent |
Weighting |
|
Formative |
Examination of objects 1 |
1000 words |
0% |
|
Formative |
Examination of objects 2 |
In class discussion |
0% |
|
Formative |
Literature Search |
2 hours preparation and class discussion |
0% |
|
Formative |
Critical Review of an article part of group work linked to workshops |
2 hours preparation plus 15mins group presentation |
0% |
|
Formative |
Annotated bibliography (peer review) linked to summative assignment |
250 words plus peer review |
0% |
|
Summative |
Practical textile conservation |
Continual assessment over 10 weeks |
30% (weighting increased) |
|
Summative |
Report portfolio linked to practical work (professional requirement). Includes formative feedback opportunities across the semester provided to support report development |
3750 words plus reflective self-evaluation |
30% (now summative) |
|
Summative |
Written Assignment: Annotated Bibliography (student choice – they can link it to an aspect of practical work, object study in another course or research interest) |
1000 words |
10% |
|
Summative |
Critical report with VIVA |
2000 words & 30mins |
30% (reduced word count) |
|
Formative and summative combined |
|
7750 words, practical work, 45 mins presentation and class discussion in total. |
|
Effectiveness (and what worked well)
More formative opportunities are now embedded in the summative assessments, where appropriate, rather than as separate activities (e.g., a large report that was previously only formative (but is essential as it has professional practice requirements) became a summative assessment with embedded formative opportunities.
We also identified that some assessments could be developed across two core courses spreading the assessment load. One large weighted summative assessment was moved and replaced a less meaningful assessment in another course (e.g., annotated bibliography in semester 1 followed by a literature review in semester 2 previously they have both been in semester 1).
Links across assessment are more visible enabling students to more effectively manage workload and be more cognisant of the programmatic design of their learning.
This is an evolving process, and we now more consciously consider the workload of formative assignments and build this into our workload planning. We are also considering time implications beyond word count alone. Students and staff have access to workload planners, so we all have greater awareness of workload requirements across courses and well as within courses.
We will continue to review this with the students at the end of the semester and see where we need to make adjustments.
Scalability + Transferability
The value of the use of CMAB of the design process has been invaluable while it take time to do it provides a clear and transparent means to evaluate a programme and courses. It is potentially easier for programmes with a larger proportion of core courses and PGT as becomes more complex to manage when working with UG programmes where students have combining different pathways leading to different combinations of courses.
What student benefits are there?
In addition to the reduction in workload through changes to the assessments, the workload is spread more effectively across the semester. In this case study, students are building their assessments incrementally alongside their practical work with opportunities for greater synergy.
What student challenges are there?
What questions remain? Are there any new questions you have as a result of your reflection?
The timing for the assessments does require students to be working at a high level across the programme. We will continue to monitor the workload and any unforeseen implications for progression.
What staff benefits are there?
Evaluation and re-evaluation of the course and programme design can help to ensure that they are developing learning in the most appropriate way and help to manage workload for students and staff.
It has been useful to re-evaluate our learning and teaching practice and recalibrate and reflect on where and whether our assignments are appropriate. As can be seen from the changes, shifting the structure of our courses has given us more flexibility so we should be able to more easily adapt what we do in response to changes in the profession and changing student needs. We hope it will create a more manageable workload for staff too.
What staff challenges are there?
What questions remain? Are there any new questions you have as a result of your reflection?
It was a substantial amount of work across several months to think about the changes and how best to implement adaptions we wanted to make. Support of Academic and Digital Development Advisers was invaluable in providing an objective colleague to discuss and explore different ways to progress this.
We are now focusing on refining our marking processes to ensure that we are more effective in providing meaningful feedback for students, but this is manageable for staff.
References
i.e. did you draw from Assessment & Feedback Literature or best practice resources, or consult with an Assessment and Feedback expert?
Kim Davis (Academic and Digital Development Adviser (Academic Services) and co-lead for the Assessment and Feedback University Strategy Workstream)
Learning and Teaching Assessment and Resources Hub.