Research units A-Z

Local and national governments often have to decide how to allocate budgets between different desirable policy options and potential outcomes. For example, the same pot of money could be used to improve social housing, or support job seeking, but not both.

This blog explores using online mixed methods to do this research.

What is this research about?

This research aims to find out how members of the public think about different aspects of wellbeing and how they value the aspects relative to each other. We are focusing on seven aspects of wellbeing, identified through a consulation process - see SIPHER-7 Report.

These are:

• Loneliness
• Neighbourhood safety
• Housing
• Effect of physical health on daily activity
• Effect of mental health on daily activity
• Household disposable income (i.e. how much money households have available after they have paid all taxes and housing costs)
• Employment situation

We will elicit social preferences of the general public across these different aspects of wellbeing. Social preferences are what people think as citizens (as opposed to private individuals) and relate to ideas about what makes a ‘good’ society. So in other words, what would we advise policy makers to consider when they need to prioritise between various possible wellbeing outcomes for the populations they serve?

How are we going to do this research?

Our sample will be broadly representative of the general population. We also want to ensure that there is diversity of views. We do this by asking potential participants about, for example, voting intentions or to agree/disagree with statements relevant to our research topic, such as ‘having a steady job is really important to a person’s wellbeing’. Then, participants who responded differently to these questions will be included in the sample.

The research will be conducted entirely online, using quantitative survey and qualitative discussion group components. Participants will be asked to do two tasks:

Task 1: complete an online survey before an online group discussion.

This online survey will be used to elicit quantitative social preferences for wellbeing outcomes.  Remember that social preferences are not about private preferences that people hold, but how they would like decisions to be made for the good of the whole society. To do this quantitatively, we ask respondents in the online survey to place different wellbeing outcomes on a numerical scale between 0 for the worst outcome and 100 for the best outcome. 

Task 2: join an online group discussion and complete the survey again, during the group.

Online groups will involve five to seven people, with a facilitator to guide the discussion. The quantitative survey data collected from participants within these groups will be analysed to produce a measure of wellbeing. These will then be used by the other workstreams in SIPHER to support making policy decisions. The qualitative data will provide a narrative to explain these quantitative findings and help us understand people’s views.

Moving online – advantages, challenges and solutions

Given the pandemic, we had to develop online methods for both tasks. We have identified several advantages and challenges so far and want to outline these here.

Advantages of moving online

In terms of advantages, evidence suggests online discussion groups can generate meaningful discussion of complex or sensitive issues. Perhaps, at least some people are more comfortable and willing to engage in online discussion with strangers, rather than in person – preferring to interact in the online space for such research, not having to socialise over lunch or finding themselves on the same bus as other participants afterwards. It is also easier to recruit people unwilling or unable to travel or participate face-to-face. Some research costs are reduced, given there are no travel, venue or refreshments expenses. We may be able to get more accurate transcriptions of the sessions, as it is easier to identify different speakers from online audio-visual recordings than audio recordings of in person discussion groups. And, of course, in a pandemic, there is no risk of covid transmission. Even with restrictions lifted, clinically vulnerable people or those concerned about virus transmission may be more likely to participate online than face-to-face.

Challenges of moving online

Yet, there are also challenges. We developed an online survey tool aiming to be as user friendly as possible but the questions can be quite complex and difficult to get your head around. This is necessary to elicit preferences that can be quantitatively analysed. We have created additional audio instructions to support participants to self-complete the survey before the group session. People completing the online survey before joining the online discussion groups will mean that they are already somewhat familiar with key concepts and the tasks. We hope this will maximise time available for meaningful group discussion. We will also share statistics to illustrate that most people when asked about the different aspects of wellbeing we are looking at, only very few report optimum levels on all aspects of wellbeing, whilst the vast majority faces challenges in one or more areas. Providing this information aims to give context and engage participants in this complex topic. We will then use a summary of the pre-group online survey results as a starting point for discussion.

Conducting discussion groups online limits the time available; two hours online was suggested as the maximum time people could concentrate and actively participate by those with experience we consulted. Providing incentives similar to those offered by market research companies conducting online focus groups may be necessary to aid recruitment – these amounts may be more than academic research usually budgets for face-to-face qualitative research. We acknowledge that online communication can be mentally taxing and participating in an online discussion of an unusual and challenging topic may well be very taxing.

Our study requires participants to navigate multiple internet links provided for a consent form, the online survey, an audio guide for the survey and for the online group. During the group, they need to access two screens (one for the survey and one for Zoom). Digital confidence and skill of participants will vary, and some will need extra support e.g. pre-group tester session/s, instructions for connecting and navigating online. There may be technical problems such as broadband outage or hardware malfunction. Controlling the environment from which people participate is not possible, there may be noise, distractions, people may come and go (or drop out), despite providing participant information and making expectations clear.

Obviously, a huge disadvantage of online methods is the exclusion of those not online, often those already marginalised, such as older people or those on low incomes. We will carry out individual telephone interviews with people considered digitally excluded, to compare their responses to those who participated online.

We wanted to utilise existing expertise in online methods to recruit participants and facilitate the online groups, with our support. The Centre for Deliberative Research (CDR) has been appointed to collaborate with us to develop this emerging research method of using online discussion groups to understand social preferences.  

It seems increased online research is inevitable, more and more people have access to the internet and the pandemic has seriously restricted in-person field work. Careful planning and considering how to engage participants in online research tasks is needed more than ever.

The views and opinions expressed in this blog are those of the author/authors.

 

---

Further reading on online discussion groups:

• Future of Britain: facilitating dialogue on post-Brexit policy during the pandemic

• The view from the table: facilitating public dialogue about the environment

References:

• Daniels, N., Gillen, P., Casson, K., & Wilson, I. (2019). STEER: Factors to Consider When Designing Online Focus Groups Using Audiovisual Technology in Health Research. International Journal of Qualitative Methods, 18. https://doi.org/10.1177/1609406919885786

• Woodyatt CR, Finneran CA, Stephenson R. In-Person Versus Online Focus Group Discussions: A Comparative Analysis of Data Quality. Qualitative Health Research. 2016;26(6):741-749. doi:10.1177/1049732316631510

Links:

• SIPHER-7 [PDF] A seven -indicator outcome measure to capture wellbeing for economic evaluational. Tsuchiya, A. and Wu, G. (C.) (2021) (doi: 10.36399/gla.pubs.310977). (Enlighten ID 310977)

• Making Deliberative Dialogue Work Online

• The Centre for Deliberative Research (CDR)

 

First published: 22 October 2021