2022 Evaluation
As in 2021, we had a pre and post evaluation form for the pupils and teachers to fill in. 43 pupils completed the pre-evaluation and 40 the post-evaluation. Here are blank version of our Evaluation forms - Pre-activity Evaluation and Post-activity Evaluation.
The pupils were asked to report their self-knowledge at the start of the visit. We were impressed to find that 97% (41/43) agreed that knew what a vaccine is – this was backed up by conversations we had on the first afternoon where many pupils were happy to raise their hand to tell us what a vaccine is. But in the pre-evaluation, only 9% (4/43) said they knew a lot about the contents of vaccines. In contrast, many pupils correctly noted that we can predict whether people are protected against infections (70%, 30/43) and that vaccines can protect more people that just those who are vaccinated (79%, 34/43). This high level of knowledge might explain why only 28% (12/43) thought they would learn a lot from the activities and discussions.
It didn’t surprise us that the pupils were familiar with the ideas of viruses and vaccines. The COVID19 pandemic has led to intense discussions in families and the media on this topic and many of the pupils had been offered the COVID19 vaccine.
Despite this high level of self-reported knowledge in the pre-evaluation forms, 65% (26/40) of pupils reported that they had learnt a lot about vaccines across the engagement. Our discussions and activities led to a rise in knowledge in a number of areas. Now 60% (24/40), an increase from 9%, knew a lot about the contents of a vaccine. We also saw an increase in pupils who agreed that we can predict whether people are protected from infections by looking at their blood, up to 88% (35/40) from 70%. And also an increase in the pupils who thought vaccines can protect more people than just those who received the vaccine – up to 90% (36/40) from 79%.
These increases in self-reported knowledge convince us that the pupils had learned from our interactions with them, particularly on the contents of vaccines. We think this is an important topic to learn about – understanding what is in a vaccine should increase people’s confidence in receiving a vaccine and help target dangerous mis-information about vaccine content in the wider media.
We also examined the pupils’ understanding of the topics we covered by looking at the worksheets they completed. There were three parts to the worksheet. First the pupils drew a picture of an imaginary virus and included some information about the virus, for example, how it spreads and what symptoms is causes. Part 2 asked the pupils to draw a picture of someone receiving the vaccine and finally the pupils had to draw a poster to encourage people to take their vaccine.
We were impressed by the imaginative responses of the pupils! We used three of the viruses the pupils drew in the 2022 Glasgow Science Festival activity because they were so colourful and interesting looking!
Many pupils did not explicitly note how their virus is spread or the symptoms it causes. A learning point for us was to be more direct on the worksheets to encourage them to write this information down. However, many did indicate that the viruses had negative consequences with about a fifth including classic viral symptoms including nausea and body pains.
32% of the pupils drew the person administrating the vaccine wearing protective clothing, e.g. gloves, masks or scrubs. And many pupils drew someone receiving the vaccine in the arm which is how most real-world vaccines are administered. Interestingly, just under half the pupils suggested their vaccine could be given in multiple ways – for example orally in pills or with food, as a nasal spray or given within a BandAid. Likely the students picked up on discussion points in our presentation about the vaccine field’s aim to target vaccines to the site of infection – we’d discussed the nasal spray that many of them had been given as a ‘flu vaccine and also the Polio vaccine that is given orally.
Drawing a poster had been a task the pupils had done previously in other projects and we suspect many pupils were influenced by the COVID19 messaging they are likely to have been exposed to. Many of the posters stressed the personal responsibility of vaccination with messaging focussing on what ‘you’ should do. About 30% of the students used the threat of negative consequences, often death, to encourage people to be vaccinated. Two thirds used more positive messaging, e.g. that getting vaccinated will help others. This was a promising response and perhaps reflected conversations we’d had about the importance of herd immunity in the Virus Tag and Herd Immunity activities. Importantly, the vast majority of posters presented an improvement following vaccination – reflecting the main message of our engagement.
We also, with the help of the interns Sophia and Euan, recorded the engagement of the pupils throughout our visit. Here is a version of the form the interns used - In Person Engagement. This more formal assessment of engagement reflected our feelings that the vast majority of pupils were focussed and interested in the activities and discussions with many hands raised to ask and answer questions. The pupils were mainly keen to start work on their worksheets each day, but we did note that their focus waned towards the end of this time as it came close to the end of the day.
We were especially impressed with the ability of the pupils to recall key learning points from the previous day including remembering the contents of vaccines, what antibodies are, and the role Lady Mary Montague played in promoting variolation.
We could also detect the pupils’ learning via their designs for their own activities. Of the 12 groups, 11 were directly related to content that we had covered in our presentation. Interestingly, 5 groups focussed on explaining the contents of vaccines, perhaps reflecting that this was the area that they had learnt most about and considered important. We think the majority of these activities were inspired by the ‘Predicting infection risk’ activity using colour change and mixing liquids to demonstrate their messages. Other groups focussed their activity on herd immunity, how antibodies work, and the history of vaccines, covering the range of the topics we’d talked about over the week.
Improvements
The least successful part of the engagement activity was the ‘heard about herd immunity’ activity. We think there were two reasons for this. First, we were able to extend the engagement to two classes of P6/7 pupils. This was great as we were able to interact with more pupils, but it did mean we had a greater number of pupils per wooden tower than we had planned. The activity would have worked better with 3-4 pupils working together rather than 5-6.
Second, some of the pupils struggled with the different rules for this activity versus their ‘home’ rules that they had for this well-known wooden block game. To get round this problem, we could have made the rules we were using clearer on the powerpoint slide and made sure this was visible while the students played the game.
Despite these challenges, this activity led to some interesting discussions. For example, some groups built their towers with all the infected people at the top, making it easy to remove them and keep the tower stable. This led to discussions about quarantine, including the very extreme version of this used to wall off whole communities during the Black Death in Edinburgh where tenements were blocked up to stop the spread of the plague. We contrasted this response with the ‘home’ quarantine that many of them will have experienced during the COVID19 pandemic.