Im/Material Event #2: Playing in Im/material Worlds
Virtual Encounters with Material Worlds
This event was covered by an article in the Courier, which can be accessed here.
A Summary of Event #2
The second event in the im/material network series, Playing in Im/material Worlds, was held at Abertay University on 13 May 2019. The discussion tapped into experiences of the members of the panel, who are storytellers and game designers, specifically focusing on how they engage with and represent im/material things and places in different media. Robin Sloan, Senior Lecturer in Game Design at Abertay University, moderated the discussion, and the panel was formed by Lynn Love (Lecturer in Animation and Interactive Media at Abertay University), Mona Bozdog (Experience Designer and Lecturer in Immersive Experience Design at Abertay University), Sandy Louchart (Head of Undergraduate Programmes at the School of Simulation and Visualisation at Glasgow School of Art), and Simon Meek (V&A’s Dundee’s first Designer in Residence, and founder/creative director of The Secret Experiment).
What are the current trends around immersive experience and design?
While Virtual Reality (VR) represents a tiny proportion of immersive experiences, it dominates much of the conversation around the topic, and VR is currently driven by commercialism and the ‘gimmick of the immersive.’ While there is new, good work appearing, the panel identified that many industry players are still focused on the visual aspects of immersion and improving the hardware, to the detriment of the narrative, which they see as a critical area on which to focus. Currently, VR game narratives often focus on the director’s vision. This is somewhat ironic and counterintuitive, as you cannot control everything that happens in a VR environment as precisely as in traditional digital media. While not yet a trend, perhaps, the panel emphasized that developers need to allow the player to take ownership within the narrative by ‘creating a playground’, a change that requires taking a significantly different approach to design.
While much work to date has focused on the visual, as noted above, it was pointed out that VR needs to incorporate senses beyond the visual to enhance immersion. The way that we operate our senses in VR, and how this differs from our natural operation of them in the physical world was likewise noted as an area ready for further research and development. In this vein, it was noted that in VR there is currently too much focus on and use of the ‘neck’ as the controller, operating the scope of the visual field, as opposed to full body movement as is more common in naturalistic physical world interactions. Simon Meek pointed out that he uses his neck far more in VR experiences than he does in real life, leading to a subtle breaking of immersivity. Similarly, Sandy Louchart argued that each time hand-held controllers are used in a VR experience, it breaks the immersion because the controller-based interaction significantly diverges from the majority of our experiences of interacting with our surroundings. This lack of interaction fluency, he argues, is detrimental to our ability to seamlessly engage with the narrative and experiential aspects of immersive media that should be at their core.
What do you think about the current state of immersives – is the technology limiting our ability to link the material and immaterial worlds?
Games and play already cut across the everyday material world and our imagined worlds, and operate under their own sets of rules, agreed by individuals or groups. When we are designing play spaces in immersive media, ideally we are designing forms of play that cross and mix the material and immaterial worlds in new and interesting ways. The mixing of elements is important because entirely abstracted play without material supports or hooks is difficult to sustain, and play that is based entirely within the material realm misses imaginative and creative opportunities afforded by the immersive digital. The essential design challenge is in mixing these elements and crossing these boundaries in a satisfying way
The panel argued that, while VR transports the player to new immaterial world, it currently eliminates the material world around the user entirely, rather than providing connections between the two. The next generation of VR headsets are taking the desire to link these worlds into consideration, in that they are designed to be less isolating, either by providing a view through to the material world or through supporting multi-user interaction. Within the currently available technologies, Augmented Reality (AR) offers a more social immersive experience. In AR, multiple players can be grouped around the same thing and be immersed in the same experience. However, design challenges persist. For example, the immaterial element needs to be translucent enough to actually augment the space and yet realized enough to convincingly link the digital components firmly with the material world.
How do you view the focus on the linking of the physical and digital, and what are the opportunities?
While different choices will need to be made in the context of each project, the panel argued that, broadly speaking, replication of the material isn’t necessary to create experiential links; the material and immaterial should work together through the creation of points of connection, but they don’t need to overlap. The participants will fill in the gaps with both material and immaterial elements from their own experiences. There is an inherent value in the physical; this can be seen in museums as visitors experience the past through physical artefacts. In the commercial world, this is seen as an opportunity for companies: they often use nostalgia to create products that link people with past experiences.
From a games or playful media perspective, how would you define the audience?
While the term ‘audience’ may imply a passive spectator, games and playful media shift this definition. The panel agreed that of all the terms that could be used to define an audience (player, user, spectator, etc), player was the preferred definition because it is all-encompassing. The audience is thus defined as an active agent, with scope for varying levels of activity. In playful media in the context of immersives, while different people will engage with different aspects of the immersive experience at different levels of interaction, everyone contributes as a player at some point, whether they’re participating, spectating, or co-creating.
What are the opportunities to frame and invite play in digital and physical spaces?
Game designers need to take both the active participant and the non-playing audience into account, especially when the immersive experience is set up in a gallery or museum. Much like attending a sports event, watching a player face challenges in an immersive experience is akin to watching a live performance in which the outcome is not predetermined. While many immersive experiences are designed primarily with an eye to the needs and desires of a player, and therefore focus on engaging the player, it is important to be respectful of those who are not playing and to create opportunities and mechanisms for them to participate at levels with which they are comfortable. In short, the roles and desires of a non-playing audience must also be taken into account by designers. For example, are the events of the game broadcasted to the spectators in some way so that they can be part of the immersion, or does the hardware prevent them from knowing what the player is experiencing? It becomes a more meaningful and memorable experience if both types of audiences are afforded agency in the experience.
Are we at a point of transformation or continuity?
It is currently difficult to experiment with VR because it is perceived as a commercial space, not as a struggling art-form, so funding to create truly experimental approaches is difficult to secure. The big transformations will happen when there are more instances of interactive media provided by streaming platforms, like Netflix’s Bandersnatch. Currently the equipment required is too expensive and the knowledge required too specialised for the democratisation of VR. Sandy Louchart advocated for a YouTube approach to VR, where virtual spaces created by individuals could be shared through a library. Overall, the consensus seemed to be that VR hasn’t reached its potential yet, and it won’t until these issues of isolation, restricted narratives, and democratisation of production are addressed.
Projects noted in Discussion
An exploration of the island Inchcolm and the performance of the stories within it: https://monabozdog.co.uk/2016/11/29/first-blog-post/
Pearson, Mike. 2010. Site Specific Performance. Palgrave.
Reflections from the Moderator, Robin Sloan
Moderating this panel was an excellent opportunity to discuss creative engagement with what are increasingly being described as 'immersive' platforms: VR, AR, and MR. It was interesting how frequently we found ourselves being drawn into discussion of VR, despite general agreement that VR can be quite limiting. At one stage a comment was made that VR can be something of a red herring. It is perhaps the standout platform that wider audiences associate with immersive technologies, and can appear the most futuristic. And yet there was wider discussion about how the other platforms, in AR and MR, as well as bespoke multi-technology installations, offer more prospects for innovation in play and in storytelling.
There were some differences in approach to working with immersive technologies for the design of playful media. We discussed putting the technology first and working within constraints, developing story and concepts first and then selecting appropriate technologies for audience delivery, and also looking sideways to developments in AI, machine learning and 5G networks as being key to new immersive play experiences. But there was broad agreement that 'immersive' is a somewhat misleading description, the implication being that these new and emerging technologies are somehow more 'immersive' than games and interactive stories that are delivered in other ways, such as on consoles, computer screens, tabletops, or in the theatre.
Robin Sloan, Senior Lecturer in Game Design and the Creative Industries theme leader at Abertay University
REFLECTIONS FROM OUR SPEAKERS
The panel was a great opportunity to exchange ideas, thoughts, inspirations and insights with practitioners and academics who share the passion for storytelling and playing in ‘im/material’, immersive and mixed-reality worlds. Discussing immersion was particularly interesting to me - coming from a theatre background I approach immersive experiences irrespective and independent of technology. It was also interesting to discuss how the other panellists approach experience design, whether their processes are predominately story- or technology-driven, and how this approach impacts the overall design.
I found the discussion of material and immaterial worlds fascinating, particularly finding ways for the two to collaborate, coexist and enhance or amplify each other. It is here that technology could make a real contribution through augmentation, through AR and MR rather than VR, adding rather than replacing, substituting and/or isolating. And if VR is sometimes limiting in this respect, what can we do to make it more welcoming, more inclusive and more accessible? VR possessed a whack-a-mole-ish persistence and kept popping up in the conversation, despite all of us agreeing that it is reductive to equate immersive experiences with VR.
And finally we discussed the places and the people, where is the im/material located and who is it for? We had a stimulating conversation about the importance of audience/players agency, the power of imagination and the transformative power of sharing an experience.
The panel provided a rich discussion around the role of technology in mediating our real world experiences, with thought provoking contributions from my panellists which have given me a great deal to consider. I was struck during the discussion that at many times we leaned towards a technological focus, drawing how virtual reality in particular relates to the immaterial. Technology can too often define thought and design processes when working in digital or mixed reality fields, acting as a constraint which is both helpful and potentially limiting. We discussed this leaning during the panel, reflecting that AR and MR may provide more playful potential to create new mediated experiences of real/fictional worlds, however, this concern has stayed with me since the panel.
In my own practice, I often begin making from a point of constraints, I believe these encourage and foster creativity, however, the constraint of technology (often a “go to” in my practice) does, I now realise pre-determine modes of interaction. To return to the panel, VR, as Simon Meek pointed out, is a “neck controller” where the player/participant is able to use their head position to re-orient themselves in and explore digital worlds. This affordance drives many VR works which create spectacles for navigation by the neck. Looking more broadly, modes of interaction are the bridge for a participant between the real and the immaterial providing frameworks for what they are and are not able to do in these spaces. Virtual worlds, particularly video games, often claim to be open world, sandbox or open to player imagination, however, the frameworks for interaction coded into them provide very clear limitations. This again points to and supports our claim that AR and MR may have more creative potential - they more dramatically blur the lines between the material and immaterial making the boundary less visible and perhaps more malleable for the participant. They also, in making use of real-world space, invite the richness and randomness of the real world into the experience, creating unique happenings, unpredictable encounters and enriching the experience to create a unique moment made up of a cocktail of real and virtual interactions for each player/participant. They are perhaps a rich test bed for “making good” on claims about player agency, control and freedom in virtual spaces.
The panel provided further food for thought in relation to the different roles and modes people take within participative work in public space, the potential for artists working in the medium to draw from theatrical processes which have already tackled issues around working in immersive spaces and problems with funding experimental work which seek to blur boundaries between the physical and digital. The panel clearly showcased that we as academics and practitioners in this field have an incredible opportunity to experiment within and push the boundaries of practices in this space to help to define, redefine and redefine again what it is to make work that is immaterial.
The discussions at the IM/Material meet up in Abertay provided me with an interesting set of propositions as to how we view AR/VR. Whilst the thematic of the discussion was very much oriented towards Play in VR/AR, the contributions by the panel offered a reflection on the nature of the medium and its current position in terms of mainstream acceptance. I found particularly interesting the set of discussions that revolved around the trade-off between increased immersion and lack of tangible interface. As such it was interesting to discuss the fundamental differences between AR and VR and relate to their relevance towards practical applications in terms of sharing virtual and real space. AR allows for a collective to observe from different viewpoints in the real space, whereas VR offers the possibility of greater interactions but is essentially removing the audience from the real space and one another. The context of interaction is key when considering each technology and as such, it is essential for the designer of im/material worlds to carefully consider the relationships between physical space/objects and virtual constructs. Some of the work discussed in the panel offered some great insight as to how these can co-exist and the nature of such co-existence should be part of the design space in which the designer operates. If we look at it from a Serious Games perspective, the decision to incorporate materiality and the extent in which it should be incorporated is key to the realisation of learning or play outcomes. As such, my take away from the panel is that the decision between VR and AR takes place later in the process than I would have previously thought. Such a decision can only occur once the requirements for an application are clear and the relationship between material and immaterial has been thoroughly explored and considered.