Simulating Empathy: Are first-person experiences in VR documentary capable of eliciting true empathy?
Shaquilla Alexander is a 2017 Ravensbourne graduate from the BA (Hons) Editing and Postproduction course, who has gone on to work in the film and advertising industry.
Upon winning Ravensbourne’s Distinguished Dissertation Award 2017, Shaquilla established film theory website, So The Theory Goes, as a way to make film research more accessible to students. Her findings are available online at www.sothetheorygoes.com
This research seeks to explore whether first-person experiences using virtual reality technology can make users more empathetic to the feelings and experiences of others – simply by immersing themselves into virtual environments. Finding out more about VR documentary and empathy peaked my interest because it brings together my professional interest in film and technology with my personal interest in humanitarianism.
Virtual reality (VR) has been described as ‘an immersive media experience that replicates either real or imagined environment and allows users to interact with this world in ways that feel as if they are there’ (Owen et al, 2015). It works by replacing users’ visual and aural connections to the real world with connections to a virtual environment (VE), through the use of headphones and a VR headset (otherwise known as a head-mounted display, or HMD). Virtual environments are created either using computer-generated imagery (CGI) or using live footage captured with 360-degree cameras. Its scope now extends farther than gaming, flight simulation (Heap et al, 1994, pp. 371–391), literature (Weinbaum, 2010), and film (TRON, DVD,1982; Brainstorm, DVD, 1983; The Thirteenth Floor, DVD, 1999; The Matrix, DVD, 1999).
PopCap founder John Vechey now refers to virtual reality as offering the possibility of detaching our physical bodies and our reality from our presence, marking an important step in our current information revolution (TEDx Talks, 2015b). David Sackman, CEO of AppliedVR, builds upon this by suggesting that the information VR can provide can be used to create acceptance across societies and races; reduce poverty; and build self-esteem and confidence (TEDx Talks, 2015a). It is this promise — delivered almost two years ago by VR creator Chris Milk — that has captured the attention of filmmakers, journalists and humanitarians alike:
It’s not a video game peripheral. It connects humans to other humans in a profound way that I’ve never seen before in any other form of media. And it can change people’s perception of each other. […] So, it’s a machine, but through this machine, we become more compassionate, we become more empathetic, and we become more connected. And ultimately, we become more human.
(TED, online video, 2015a)
This presents the unique idea that virtual reality technology can help restore humanity; the thought that by putting on a HMD and immersing ourselves in a virtual world, we can become more connected to the real world. This empathy machine has since been used by many media outlets including the BBC, the Guardian, The Huffington Post, VICE and The New York Times to shed light on current new stories and to promote humanitarian efforts for the United Nations. VR documentary Clouds over Sidra was commissioned by the UN to tell the story of a Syrian refugee in a way that had never been through traditional journalistic mediums. The Displaced — one of the New York Times’ first virtual reality projects — allowed their 1.5 million print subscribers to witness firsthand the consequences of war and displacement on children in the Middle East (Howe, 2016, p.15).
Although plenty of academics believe that VR allows us to step into another person’s shoes and become more understanding in the process, the question remains… are first-person experiences in VR documentary capable of eliciting true empathy?
Much has been said about empathy; a term so commonly used and still so hard to define. It has been the topic of much debate and discussion within the fields of social and clinical psychology, phenomenology, philosophy and cognitive neuroscience for many years. Psychologist Theodor Lipps equated empathy to the term Einfühlung, meaning ‘feeling into’ (Coplan and Goldie, 2014, p. XIII). To Lipps, empathy is ‘a process of inner imitation or inner resonance that is based on a natural instinct and causes us to imitate the movements and expressions we perceive in physical and social objects’ (Coplan and Goldie, 2014, p. XIII). Whereas phenomenologists like Edith Stein believe that empathy is how we experience ‘foreign consciousness’, allowing us to ‘understand others but also to understand ourselves as others experience us’ (Coplan and Goldie, 2014, p. XVI). Other sources identify empathy as being ‘the ability to intuit the distress of another, or to feel a fait echo of their excitement, and therefore respond in ways which bring the other person closer, rather than alienate themt (Watt-Smith, 2015, p. 94).
In most cases, each definition (although differing slightly) make reference to a few key components:
1) simulation, recreation or reconstruction of another person’s experiences
2) sharing in another person’s feelings
3) social understanding
For the sake of this research into empathy, I am inclined to agree with Amy Coplan’s concise views on empathy within her 2014 essay Understanding Empathy: Its Features and Effects: ‘…empathy is a complex [affective and cognitive] imaginative process in which an observer simulates another person’s situated psychological state while maintaining clear self-other differentiation’ (Coplan, 2014, p.5). In other words, empathy involves both feeling (affective) and thought (cognitive) processes whereby an observer can recreate another person’s state of mind while maintaining their individual sense of self. Coplan also refers to three concepts, each required for true empathy to be possible. These include affective matching, other-oriented perspective-taking, and self-other differentiation (Coplan, 2014, p.5).
Using this understanding of empathy and other key texts in psychology and philosophy, I will discuss the proposed use (and capability) of virtual reality documentary to elicit empathy. Looking into VR documentary from multiple angles, this dissertation will debate the viewpoints of those both for and against the use of VR to foster empathy.
Chapter 1 compares immersive virtual reality to other visual storytelling media (including photojournalism and documentary), and contemplates their abilities to promote empathy. Chapter 2 delves deeper into the research conducted on VR as well as audience reactions to the content; theorising whether its first-person simulation can impact human emotion in the real world. Chapter 3 discusses the possible ethical ramifications of utilising constructed VR experiences for social understanding.
Through this analysis, I aim to demonstrate that although first-person experiences in VR documentary might appear to elicit empathy, VR (as it stands) is not capable of eliciting the long term, altruistic behaviour promised by many practitioners within this industry.
Chapter 1: Visual Immersion
In photography, viewers witness still glimpses of a larger story captured from the photographer’s lens. In film and documentary, audiences take in the narrative created by filmmakers — measured in frames and projected onto a 16 x 9 frame. In virtual reality film, there is no limit to their peripheral vision and no distance placed between the user and the story unfolding around them in the first-person.
Through the use of sound design and spatial storytelling, VR films actively work to build up an experience around the user to make them part of the narrative. As Matt Ratcliffe, co-founder of VR design studio Masters of Pie, so acutely says in an interview on virtual reality storytelling, ‘people are no longer the audience, they are the participant’ (Ratcliffe, interview, 2016).
The technology has presented a new opportunity to journalists, filmmakers and storytellers to engage audiences. VR pioneer Nonny de la Peña has been a strong advocate for the use of VR for factual storytelling — creating immersive journalism films that use virtual reality technology and CG animation to reconstruct the news. She argues that ‘immersive journalism offers the opportunity of uniquely different level of understanding contrasted to reading the printed page or passively watching audiovisual material’ (De la Peña et al, 2010, p. 299).
This notion is echoed by several within the VR industry. In a TED talk about the potential of virtual reality technology, Chris Milk discusses the connection between the framing of the story and user reaction. He argues that although film allows its audience to feel empathy for people from very different worlds, it was only when he started experimenting with storytelling inside of the frame (and then removing the frame altogether), that he was able to get a deeper emotional reaction from the people who witnessed his stories (TED, online video, 2015a).
The idea being that if one can proverbially see through another’s eyes, he/she is more likely to empathise with them. However, there is reason to suggest otherwise; it is not the act of literally putting us in the visual perspective of another person that can elicit empathy. Empathy requires three key components: affect matching, other-oriented perspective taking and self-other differentiation (Coplan, 2014, p. 6).
As recent science has shown, we are soft-wired to experience another’s feelings as if we were experiencing them ourselves, simply by observing their expressions (The RSA, online video, 2010). We possess what researchers refer to as mirror neurones, and it is through mirroring that we can empathise with others in everyday life. Mirroring is referred to in Coplan’s framework as affective matching (Coplan, 2014, p.6).
In theory, virtual reality documentary brings users face-to-face with others in the same way, and in seeing the plight of others first-hand, they can feel just as strong an emotion as we would in real life. However, this is not the case. In VR we are more often told how the other person(s) are feeling rather than being allowed to see this for ourselves. As Adam Cohen, Head of VR Production at OffAbbott, aptly points out in our interview, there are several other layers between the user and the subject including the screen, the virtual avatar and the processing of virtual space (Cohen, interview, 2016). Theses layers can limit the effectiveness of the affective match.
Take, for example, the VR documentary Clouds Over Sidra (film, 2015). It shows users the life of 12-year old Syrian girl Sidra who has been living in Zaatari refugee camp in Jordan for over a year. The narration tells users about her family and the circumstances that forced them to leave Syria. Music and 360-degree sound design help them navigate the virtual space. The freedom of movement within this virtual space also gives VR users room to explore the virtual refugee camp locations for themselves.
One of the most poignant moments in the documentary comes at the very end when Sidra says, I will not be twelve forever, and I will not be in Zaatari forever. My teacher says the clouds moving over us also came here from Syria. Someday the clouds and me are going to turn around and go back home’ (Clouds Over Sidra, film, 2015). However, rather than seeing how her circumstances might be making her feel, users are provided with a 360-degree view of her surroundings. This approach can provoke sympathy, but a much more personal approach centred around the expression of feeling by the subject is needed for empathy to be possible.
Researchers also refer to the role perspective-taking plays when empathising. Coplan describes perspective-taking as being ‘an imaginative process through which one constructs another person’s subjective experience’ (Coplan, 2014, p. 9). It can consist of either self-oriented or other-oriented perspective taking, though there is a clear distinction between the two. Self-oriented perspective-taking is when someone represents themselves when imagining another person’s situation. On the other hand, other-oriented perspective taking requires a person to ‘simulate the target’s individual’s experiences as though she were the target individual’ (Coplan and Goldie, 2014, p. 10). In other words, the emphasis is on what the target or subject might be feeling in a given situation rather than on how the user might feel in the same situation.
VR can facilitate self-oriented perspective taking, whereby VR users can imagining themselves as the subject and object of the documentaries. By providing an intimate, first-person perspective, users are made to feel as though they are involved in this narrative. They are one of the kids in the Zaatari refugee camp, like Sidra in Clouds Over Sidra. They are one of the passengers on a dinghy making the perilous journey to Europe in BBC’s VR experience, We Wait (film, 2016). This is intended to make users identify with and mirror the emotions of the subjects within the film, but what the technology is doing is creating a singular experience centred around the user, whereby one focuses more on what he/she is feeling and thinking, distracting from what the subjects might be feeling.
In an interview with VR filmmaker Darren Emerson, we discuss his film Invisible (2015) and why he feels that VR can make people more empathetic. When discussing the new medium, he made an interesting argument about empathy and user experience. He states that VR creators are asking for users to ‘bring something deeper into the experience, isolating them and allowing them to bring an essence of their human condition and understanding to the work’ (Emerson, interview, 2016).
Darren Emerson also suggests that VR is capable of inducing empathy thanks to immersion, narrative and the subjective context of the viewer. However, there is reason to suggest that this immersion and subjective context inhibits true empathy. They work to create a self-oriented experience. Rather than understanding what it is like to live someone else’s experiences as our own from their perspective, we understand what it is like to be ourselves in these situations, based on our knowledge, behaviours and biases. This prevents us from truly understanding the situation from the other person’s psychological point of view.
One could argue that the first-person storytelling in VR limits self-other differentiation, meaning it is harder to maintain a clear emotional distance between self and other. In documentary film and photography, audiences have the chance to see the subjects and how they are feeling, thus allowing for affect matching. They are also able to take an other-oriented perspective and are given enough narrative distance (due to the presence of the frame) to maintain a clear self-other differentiation. In VR filmmaking, it is encouraged for users to ‘lose any sense of themselves as separate from the medium or its simulated world’ (Lister et al, 2008, p. 424), in what is commonly referred to in both the VR and gaming industry as immersion. Although immersion is possible in other forms when the audience can cross the gap between themselves and the story, mentally placing themselves in the position of the protagonists, VR technology has made it possible to build the world around them in the first-person — taking away that gap altogether.
This has been carefully summed up in an opinion piece written by Michael W. Clune for The Atlantic. Clune writes:
Art activates our human capacity to use a visible image to unlock an invisible reality. […] VR interprets the phrase “to see through another’s eyes” literally. [It] immerses your senses in another place. But the mind of another person isn’t a place, and it can’t be entered through your senses alone
(Clune, online article, 2016)
As Clune explains, the gap between the narrative and ourselves was never the problem; in fact, this is what largely gives those other forms their power and ability to help with true empathy and understanding. It is their two-dimensionality that ‘forces us to use our imagination to pass through the visible to the invisible’ (Clune, online article, 2016). We stop seeing others’ experiences through our own eyes and come to appreciate them through the eyes of another.
Take, for example, a scene that is present in both BBC’s documentary series Exodus: A Journey to Europe (TV programme, 2016) and BBC’s VR experience We Wait. The production company Keo Films gave smartphones to migrants and refugees to help document their stories. The resulting three part documentary series Exodus presents the stories of Tarek, Isra’a and their family, teacher Hassan and student Ahmed, as well as many others, all of whom have had to flee their homes due to ongoing conflict and war. In Episode 1 of Exodus, Hassan recounts the lead up to and the journey on the dinghy travelling from Turkey to Europe. He recounts several harrowing experiences, including being saved from a capsized dinghy by the coast guard. Audiences are even shown smartphone footage of the experience, giving them a first-person account of what happened (Exodus: A Journey to Europe, TV programme, 2016).
A similar situation appears in VR film We Wait, piloted at the end of last year for use with the Oculus Rift headset. The film is made up of animated reconstructions based on real-life migration stories. Users follow along with a family attempting to make this trip to safety. In We Wait, screams from the animated women and children, flashing lights and the chaotic movement from within the dinghy are meant to make users feel a sense of artificial anxiety. However, in episode one of Exodus, when we are watching smartphone footage from a similar first-person perspective, it is intercut with talking heads from Hassan talking about how he was feeling in that moment. While the smartphone footage is made to make us feel the same sense of fear as We Wait, Hassan’s words remind us that this was his experience, not ours. They reaffirm exactly what he was feeling, which in this case was a genuine sense of helplessness (Exodus: A Journey to Europe, TV programme, 2016).
As Coplan so rightly says, ‘only empathy that combines affective matching, other-oriented perspective-taking, and self-other differentiation provides experiential understanding’ (Coplan, 2014, p.16). In other words, true empathetic understanding is not possible without the presence of all three of these components. As we will discuss in Chapters 2 and 3, VR’s self-oriented perspective-taking and limited self-other differentiation can lead to personal distress, reactive emotions, false consensus effects and prediction errors — none of which are conducive with true empathy.
Chapter 2: VR Research and Reactions
The interest in virtual reality’s capacity to elicit empathy extends out ever farther than journalism and film production. Social psychologists are looking into whether virtual environments can impact social behaviour in audiences. There have been numerous reports published over the past couple of years by the Virtual Human Interaction Lab at Stanford University, to add to the existing and extensive research conducted into VR and the management of pain and trauma (TEDx Talks, online video, 2015a). MxR Lab and MIT’s Open Documentary Lab are also exploring the possibilities within VR storytelling (Kasson, 2016, p. 19). Researchers from The Virtual Human Interaction Lab have gone on to investigate the impact of embodied perspective-taking on the reduction of negative stereotypes (Yee and Bailenson, 2006) and the use of virtual reality superheroes to encourage pro-social behaviour (Szolnoki et al, 2013).
While there has been evidence to suggest that simulation can promote pro-social behaviour (i.e. helping behaviour), the question here is whether this can be qualified as empathy. Yes, users are being affected by the films, but what exactly is affecting them? Is it the medium itself? Moreover, are they experiencing a mirrored feeling, or are they merely reaching to seeing the plight of others so intimately? As discussed in Chapter 1, VR documentary provides users with a self-oriented perspective in its attempt towards empathy. It does not rely heavily on affective matching, a proven process we use for empathising in the real world. As a result of this, users attempting to empathise with subjects within these films are more likely to experience what Coplan refers to as personal distress and reactive affects (Coplan, 2014, p. 12).
Personal vs Empathetic Distress
Just like the researchers from Stanford, MIT and MxR Lab, psychologist Maria Panagiotidi is also investigating virtual reality technology. Her project, The Empathy Station, was featured as part of the Alternative Realities Summit at Sheffield Doc Fest 2016. It worked by conducting neuroscience tests on visitors of the annual film festival.
The Empathy Station had three parts. Firstly, the participants took an implicit association test (IAT) aimed at detecting any previous biases participants may have towards particular groups of people (i.e. Arab-Muslims, non-Arab-Muslims…). The test involved linking names and terms that appeared on screen with either a smiling (positive) or frowning (negative) face (British Council Arts, online video, 2016b). Then, visitors participated in a Robert-Hunt style experiment using a rubber hand. The participant has their real hand hidden, replaced with a fake rubber hand laid out on the table. Someone tickles both the real and fake hands, then just the fake hand and participants are still able to feel sensation in his or her actual hand even though he or she are aware that the tickling has stopped. This experiment focused on participants’ perception of reality and the assumptions the brain can make (Panagiotidi, interview, 2017). Lastly, participants are shown Clouds Over Sidra as an example of how VR could be used to induce empathy.
When asked whether she believes VR technology itself can induce empathy, Panagiotidi argued against it, saying that it is the novel technology causing audiences to react so strongly towards VR films (British Council Arts, online video, 2016b). She is not alone in this assumption. Ana Serrano, Chief Digital Officer of the CFC in Canada and founder of their interactive storytelling institute CFC Media Lab, echoed this sentiment in our recent interview discussing the subject:
I think when people feel like they are either scared or a little bit discomforted in VR or slightly amazed or what have you, which can happen because the environment is so intimate and so immediate […] I think those feelings become the touchstones for what people think the engine is doing to them, and because they have been told that the engine elicits empathy, they think they then have developed empathy
(Serrano, interview, 2016)
They both highlight an important point about the impact the actual medium could be having on audiences. Unlike other media, like photography or film, virtual reality is much more immediate and much more intimate. Not only is the user being exposed to something they have likely never experienced before, but within VR documentaries, they are intentionally placed at a much closer distance to subjects than we have seen in any other mediums. It would not be unusual for their reactions to be psychological or physiological. However, seeing as users are told that these responses equate to empathy, there has been nothing to convince them otherwise.
Gabo Arora, who directed Clouds over Sidra, has previously stressed the importance of minimal physical distance between the subject and the viewer, arguing that ‘with less distance is more understanding and more engagement’ (Howe, 2016, p.16). Chris Milk also acknowledged the power of proximity within VR, by saying that users can feel a sense of emotional closeness to the characters or subjects of the film due to what they feel to be physical closeness (TED, online video, 2016). However, as Serrano points out, this proximity may be getting people to react, but it is a reaction to discomfort. In other words, this could merely be personal distress. Coplan discusses the difference between empathetic distress and personal distress — the latter being the result of maintaining a self-oriented perspective as opposed to an objective, other-oriented perspective (Coplan, 2014, p. 12).
Personal distress, as Coplan describes, is when ‘one observes another person in distress and reacts by becoming distressed himself’ (Coplan, 2014, p.12). With empathetic distress, the focus remains solely on the actual person in distress. There is a clear distinction between the two terms — a distinction that could explain why all feelings of distress after watching VR documentary content does not automatically equate to empathy. According to Coplan’s research, sometimes ‘a person experiencing personal distress will display pro-social behaviour but generally only when there is no alternative method of eliminating his discomfort’ (Coplan, 2014, p.12). This would explain the positive behaviour that may have been experienced as a result of being in a virtual environment, as was found in some experiments conducted by the Virtual Human Interaction Lab.
It is equally important to pay attention to note whether a user’s emotional state after having a VR experience matches what the subject may be feeling. Gabo Arora, Chris Milk, Nonny de la Peña and several other VR producers have described scenes where users have taken off the VR headsets with tears in their eyes (Howe, 2016, p.17); Grandon, online article, 2016). These are reactive affects, not to be confused with empathy experienced through affect matching.
Coplan mentions reactive affects in her research; a term found in the works of other psychologists including Mark H. Davis. Both Davis and Coplan refer to reactive affects as affects ‘resulting from an observer’s perception of a target, [which] fail to match even the valence of the target’s affect’ (Davis, 1996, quoted in: Coplan, 2014, p. 7). In other words, they are affects that stem from the user’s perception of the subjects experience, not from what the subject is truly experiencing. Coplan argues that reactive emotions like anger or sadness do not qualify as empathetic because they are not ‘sufficiently accurate representations of a target’s situated psychological state’ (Coplan, 2014, p. 7). They misrepresent the type of emotion experienced by the subject.
Although the context is different, this notion still applies to experiences with VR content. If a VR user watches The Displaced, sees the circumstances that the children protagonists are living through, and in turn feels fear or anger, this is a reactive affect. This does not count as empathy because the subject was not feeling neither fear nor anger. A key component of empathy is affect matching — whereby both the user and the subject affective states are identical — and by failing to mirror emotions, true empathy cannot be achieved.
Chapter 3: Ethics
The research linking VR to pro-social behaviour has provided an even greater reason for filmmakers (among others) to merge documentary, journalism, and VR technology to bring out empathy in audiences. Over the last two years, virtual reality technology has been used to bring audiences closer to people, places and events they may otherwise have never encountered. It has been used for various subjects, including migration, racial discrimination, world history and religion, providing what some believe to be a ‘Wikipedia of experiences’ (Kelly, online article, 2016) for users to learn from. The technology itself may generate strong reactions in users, but there are ethical implications to treating the insight provided by virtual reality experiences as true social understanding. Also, by providing a self-oriented perspective, VR documentary can prevent the experiential understanding VR producers are hoping to provide, giving way to false consensus effects and prediction errors on the part of the user (Coplan, 2014, p. 13).
There is a question of authenticity with regards to non-fiction VR content. At present, there is no official code of ethics governing the use and risks/responsibilities associated with VR documentary production. This lack of regulation means that journalistic ethics are being relied upon to maintain the authenticity and integrity of the work. When the urge to make people feel more is so great, it begs the question of how far VR content creators are willing to go to elicit empathetic feeling. The danger lies in the creation of these factual pieces. When 360-degree cameras are used to capture these films, there is the assumption that because the camera is set in place and records a full view of events, then, in turn, the perspective that it provides is somehow more truthful. This is not the case. Just like other forms, VR documentary utilises choreography, the influence of the camera, framing, the choice of focus and post-production editing (referred to in VR as stitching) to heighten the emotion of the pieces. Moreover, it is not above using illusions of reality to influence the reception to these experiences, despite these illusions going unnoticed by audiences.
Many factors can significantly influence VR documentaries. The Tow Centre for Digital Journalism report on Virtual Reality Journalism states that camera placement, framing/choice of subject focus and post-production editing can all have a medium to high influence on the message within VR journalistic work (Owen et al, 2015). As Darren Emerson points out with regards to his VR film Invisible, ‘when using 360, I think the perception of the audience is that what they are seeing is wholly real… when often it is not’ (Emerson, interview, 2016).
A prime example of this is the VR documentary, The Displaced (film, 2015). Created by VRSE.works and The New York Times, it tells the story of the displaced children forced to take care of themselves due to the devastation caused by civil war in Syria. The film intercuts the three stories to show similarities in their situations. Users can see how the children take care of themselves on their own, what their lives have been like, as well as their outlook on the future. It makes use of music, narration, sound design and graphics to forward the narrative. The film does make heavy use of editing techniques and received criticism for arranging the actors and actions within the documentary when it first came out in 2015 (Sullivan, online article, 2015).
As Tom Kent, editor of the Associated Press says with regards to the journalist’s role when using VR technology:
[…] creating empathy is a goal beyond just telling a story. If the ultimate aim is to create emotion, a journalist could be tempted to omit balancing or inconvenient information that could interfere with the desired emotional effect
(Kent, online article, 2015).
It begs the question of how accurate the representation of subjects are within VR documentary, and in turn how accurate their psychological state of mind is portrayed within these pieces.
The length of VR documentary content can vary, with most ranging anywhere from four to ten minutes. In that time, users are guided through the experiences with narration. In the case of films like Clouds over Sidra, The Displaced and We Wait, this narration will detail the lives of the subjects — their past, their daily routines, the events that led to their current circumstances. Also, the first-person aspect of VR storytelling means that users get to see a lot of what is described first-hand. When so much information is shared, it is easy to understand why users might feel like they have a complete picture of what the subjects’ lives are truly like. However, these experiences do not provide enough context to understanding someone else’s life. In other VR films like Notes on Blindness: Into Darkness (film, 2016), the objective is to get users to understand what an individual’s physical reality. While these experiences can provide great insight — insight that we may never have from other mediums — it is not possible to truly understand (and thus empathise) with an experience felt only for a few minutes.
Notes on Blindness: Into Darkness is a 360-degree immersive experience where users witness what it might be like to be blind. The film, currently hosted on VR app Within, is aimed at getting people to understand what it might be like to experience the world without one’s vision. The film features a 1983 recording from writer John Hull who talks about the sounds he hears when he is outside — his children playing, the sound of nearby water running, birds, cars passing and the sound of rustling trees to name a few. Blue lights show an outline of the sounds heard.
Hull himself has previously stated how difficult it is for sighted people to understand what it means to be blind. In an early article, he writes:
[W]hen you, a sighted person, close your eyes, do not imagine that what you see is comparable to what a blind person sees. After all, behind your closed eyelids you still have the brain of a sighted person, and your brain is full of the images, colours, shapes, movements and faces of the things and people around you, which you know are still there, and which you can recapture the moment you open your eyes.
(Feinberg, online article, 2016).
The same reasoning also goes for virtual reality experiences. Just because we have the same visual experience as the people captured in VR documentary films does not mean that we automatically know about their lived experience. We will still have in mind our previous personal experience. We are not able to relate to another person’s lived reality and personal history from a seven-minute immersive experience, and presuming that 360-degree capture can capture something this complex is misleading. There is also the question of how invasive is too invasive with VR documentary storytelling. VR offers insight into the lives of others, but the use of VR for social impact pieces can also trivialise victims suffering for the sake of entertainment (Feinberg, online article, 2016)
Effects and Errors
It is important to mention the impact this type of self-oriented perspective-taking can have on our reception and future understanding of the target of our empathising. This is discussed in great depth within Coplan’s research and is validated by the research of many other psychologists including Sara Hodges and Daniel Wegner (1997, quoted in: Coplan, 2014, p.11). By imagining ourselves in someone else’s situation (self-oriented perspective-taking), as Coplan argues, ‘it frequently results in inaccurate predictions and failed simulation of the other’s thoughts, feelings, and desires’ (Coplan, 2014, p.8) In other words, when we are self-oriented in our attempts at empathy, we make the assumption that the target of our empathising is feeling the way that we feel. This assumption of similarity can lead us to jump to the wrong conclusions about others, and in doing so our predictions of who they are and how they might behave can turn out to be wrong.
Virtual reality documentary presents a new and wholly immersive way of storytelling. This dissertation has aimed to prove whether the technology can, in fact, make its users more empathetic in the real world. Instead, this has proven to be a polarising debate within the emerging VR industry. Some would argue that empathy is possible with VR due to its immersive capabilities, while others insist that merely seeing through the eyes of another is not enough to truly understand their circumstances. There are those who hail the use of virtual reality technology for humanitarian works such as Clouds over Sidra and The Displaced, while others consider them to be examples of privilege and sample suffering for the sake of entertainment (Feinberg, online article, 2016).
As discussed in Chapter 1, VR experiences provide a self-oriented perspective and inhibit affect matching and self-other differentiation. In other words, it allows for greater emphasis on what the user is feeling (contrary to what the subject of the film is feeling), and its first-person perspective limits the ability to maintain clear differentiation between self and what one is experiencing emotionally in these documentaries. As Coplan points out, ‘only empathy that combines affective matching, other-oriented perspective taking, and self-other differentiation provides experiential understanding’ (Coplan, 2014, p. 17). By contrast, in traditional documentary and photojournalism, Users are given a third-person perspective. They are presented with subjects with which to identify, just as they are in VR. However, the frame allows users to maintain some distance between self and other, separating their feelings from that of the subjects, thus allowing for greater understanding of what the subject or target might be experiencing.
Although documentary does make use of many storytelling devices including sound design, camera placement and direction in its attempts to foster empathy, the medium does not distract from the message of identifying with the characters of the film. This does not mean that all documentary or photography allows for other-oriented perspective taking, nor is it implying that one cannot feel sympathy through self-oriented perspective-taking. What photojournalism and documentary film can do is allow us to see tragedies from the target’s eyes, not our own. We have context and characters with which to associate. The emphasis is taken off of self and placed on the subject where it belongs.
By providing users with such an emotionally impactful first-person experience with novel technology — an experience centred around them — VR creators are facilitating self-oriented perspective taking, and the consequences of this include personal distress and reaction affects, neither of which are reflective of what the other is feeling.
Equally, how can we identify that what we are experiencing is in fact empathy, and not another form of pro-social behaviour (like sympathy or compassion)? One could argue, at present, that there is not enough hard evidence to link VR to empathy, as opposed to sympathetic, compassionate or helping behaviour, all of which are very different pro-social behaviours.
VR has the potential to provide us with real insight into other cultures and social experiences. However, VR documentary (as it stands) cannot elicit empathy and skirts a dangerous ethical line in its attempt to create strong emotion for audiences. My research over the course of this dissertation has truly made me see how effective a tool VR can be for storytelling. It has taught me that not every storyline is suited for this technology, and when used inappropriately can distract from the actual message of the piece. Most importantly, it has shown me that that there is still plenty more to learn, and more research is needed to be able to understand VR’s ability to promote empathetic action.
Bailenson, J.N. and Yee, Nick (2006) ‘Walk a mile in digital shoes: The impact of embodied perspectivetaking on the reduction of negative stereotyping in immersive virtual environments’, PRESENCE 2006: The 9th Annual International Workshop on Presence, August 24-26 2006, Cleveland: The Virtual Human Interaction Lab. Pp. 1-9. Available: https://vhil.stanford.edu/pubs/2006/walk-a-mile-in-digital-shoes-theimpact-of-embodied-perspective-taking-on-the-reduction-of-negative-stereotyping-in-immersive-virtualenvironments/ (Accessed 07/02/17)
Boyacioglu, Beyza, Ding, Sue, Uricchio, William and Wolozin, Sarah (2016) ‘Virtually There: Documentary Meets Virtual Reality’, MIT Open Doc Lab: Virtually There Conference, May 6-7 2016, Cambridge: MIT Open Documentary Lab. Pp. 1-32. Available: http://opendoclab.mit.edu/wp/wp-content/uploads/2016/11/MIT_OpenDocLab
_VirtuallyThereConference.pdf (Accessed 10/03/17)
Brainstorm (1983) Directed by Douglas Trumbull [DVD, 1 hour 46 minutes] AJF Productions, Metro-Goldwyn-Mayer and SLM Entertainment
British Council Arts (2016a) ‘Alternate Realities at Sheffield Doc Fest 2016: exploring empathy and Virtual Reality‘, Soundcloud, [Podcast], 12 June. Available: https://soundcloud.com/artsbritishcouncil/
alternaterealities-at-sheffield-doc-fest-2016-exploring-the-world-of-virtual-reality (Accessed 12/10/16)
British Council Arts (2016b) Alternate Realities: The Empathy Station at Sheffield Doc Fest 2016, [Youtube], 6 July. Available: https://www.youtube.com/watch?v=KdwaG0chb-0 (Accessed 17/09/16)
Clouds Over Sidra (2015) Directed by Gabo Arora and Barry Pousman [VR Film] Los Angeles: VRSE.works
Clune, M. W. (2016) ‘Virtual Reality reminds users what it’s like to be themselves’, The Atlantic, Wednesday 20 April [Online] Available: http://www.theatlantic.com/technology/archive/
2016/04/thevirtual-world-in-a-real-body/478956/ (Accessed 17/09/16)
Cohen, Adam (2016) Head of VR Production at OffAbbot. Interview with Shaquilla Alexander. London: 23 September 2016
Coplan, Amy (2014) ‘Understanding Empathy: Its Features and Effects’. In: A. Coplan and P. Goldie [eds.] Empathy: Philosophical and Psychological Perspectives, Chapter 1, Oxford: Oxford University Press. Pp.3-18
Coplan, Amy and Goldie, Peter (2014) ‘Introduction’. In: A. Coplan and P. Goldie [eds] Empathy: Philosophical and Psychological Perspectives, Oxford: Oxford University Press. Pp. IV-XLVII
De la Peña, Nonny, Well, Peggy, Pomés, J. L., Spanlang, Bernhard, Friedman, Doron, Sanchez-Vives, M. V. and Slater, Mel (2010) ‘Immersive Journalism: Immersive Virtual Reality for the First-Person Experience of New, Presence [Online] Issue 19. Available: http://www.mitpressjournals.org/doi/pdf/10.1162/
PRES_a_00005 (Accessed 19/02/2017)
Emerson, Darren (2016) Co-Founder & CCO at VR City. Interview with Shaquilla Alexander. London: 2 October 2016
Exodus: Our Journey to Europe (2016) Series 1, Episode 1, BBC2, 11/07/16
Feinberg, P. P. (2016) ‘Empathy or insight? Reflections on virtual reality’, Phi Centre, Monday 15 August [Online] Available: https://phi-centre.com/en/post/en-empathy-sensory-stories-2016/ (Accessed 18/09/16)
Grandon, Marisol (2016) Fantastic voyages: how VR became a reality, British Film Institute, Friday 8 July [Online] Available: http://www.bfi.org.uk/news-opinion/sight-sound-magazine/features/fantasticvoyages-how-vr-became-reality (Accessed 13/03/17)
Harris, B. J. (2015) How the United Nations is using virtual reality to tackle real-world problems, Fast Company, Wednesday 12 October [Online] Available: https://www.fastcompany.com/3051672/
techforecast/how-the-united-nations-is-using-virtual-reality-to-tackle-real-world-problems (Accessed 24/07/16)
Heap, Nick, Thomas, Ray, Einon, Geoff, Mason, Robin and Mackay, Hugh (1994) Information Technology and Society: A Reader, London: Sage Publications Ltd
Howe, C.F. (2016) ‘VR becoming an actual reality for Documentarians’, documentary Magazine, Vol. 35, No. 2, Spring 2016, Pp. 15-18
Invisible (2016) Directed by Darren Emerson [VR Film] London: East City Films
Jones, Sarah (2016) Coventry University researcher. Interview with Shaquilla Alexander. London: 12 October 2016
Kasson, E.G. (2016) ‘Where Virtual Reality Prototypes Are Hatched’, documentary Magazine, Vol. 35, No. 2, Spring 2016, Pp. 18-21
Kelly, Kevin (2016) ‘The untold story of Magic Leap, the world’s most secretive startup’, Wired, Tuesday 26 April [Online] Available: https://www.wired.com/2016/04/magic-leap-vr (Accessed 17/09/16)
Kent, Tom (2015) ‘An ethical reality check for virtual reality journalism’, Medium, Monday 31 August [Online] Available: https://medium.com/@tjrkent/an-ethical-reality-check-for-virtual-realityjournalism-8e5230673507#.v9e62wtuk (Accessed 13/03/2017)
Kingston, Samantha (2016) Founder of Virtual Umbrella. Interview with Shaquilla Alexander. London: 3 October 2016
Light, Ken (2010) Witness in our time, Second Edition: Working lives of documentary photographers, Washington: Smithsonian Books
Lister, Martin, Dovey, Jon, Giddings, Seth, Grant, Iain and Kelly, Kieran (2008) New Media: A Critical Introduction, New York: Taylor & Francis
Mashable (2015) Virtual reality, explained with some trippy optical illusions, [Youtube], 26 March. Available: https://www.youtube.com/watch?v=qD3w3cAhEYU (Accessed 31/10/16)
McAllister, Sean (2016) Documentary filmmaker. Interview with Shaquilla Alexander. London: 30 September 2016
Miller, Edward (2016) Co-Founder & CEO at Scape Technologies. Interview with Shaquilla Alexander. London: 21 September 2016
Nicholls, Johanna (2016) Senior Digital Producer at National Theatre. Interview with Shaquilla Alexander. London: 24 August 2016
Notes on Blindness: Into Darkness (2016) Directed by Arnaud Colinart, Amaury La Burthe, Pete Middleton and James Spinney [VR Film] Strasbourg: ARTE France, AudioGaming and Ex Nihilo
Oberhaus, Daniel (2016) ‘We’re Already Violating Virtual Reality’s First Code of Ethics’, Motherboard, Sunday 6 March [Online] Available: https://motherboard.vice.com/en_us/article/vr-code-of-ethics (Accessed 12/10/16)
Owen, Taylor, Pitt, Fergus Aronson-Rath, Raney and Milward, James (2015) Virtual Reality Journalism, New York: Tow Centre for Digital Journalism
Oxford Dictionary of Psychology (2006) Oxford: Oxford University Press
Panagiotidi, Maria (2017) Research Psychologist and Lecturer at Staffordshire University. Interview with Shaquilla Alexander. London: 10 March 2017
Polzer, Lydia and Machin, David (2015) Visual journalism, Basingstoke: Palgrave Macmillan
Ratcliffe, Matt (2016) Co-Founder of Masters of Pie. Interview with Shaquilla Alexander. London: 6 October 2016
Rosenberg, R.S., Baughman S.L., Bailenson J. N. (2013) ‘Virtual Superheroes: Using superpowers in virtual reality to encourage Prosocial behaviour’, PLoS ONE [Online] Issue 8. Available: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0055003 (Accessed 19/02/17)
Schudson, Michael (2011) The sociology of news, New York: W. W. Norton & Company
Serrano, Ana (2016) Chief Digital Officer at the CFC and Founder of CFC Media Lab. Interview with Shaquilla Alexander. London: 6 October 2016
Seymour, Tom (2016) ‘Intelligence: Documentary’, British Journal of Photography, Vol. 163, No. 7851, September, Pp. 75-78
Shields, Rob (2002) ‘Digital Virtualities’. In: R. Shields [ed] The Virtual, Chapter 3, London and New York: Routledge. Pp. 45-80
Singer, Jane (2016) Professor of Journalism Innovation at City University. Interview with Shaquilla Alexander. London: 30 September 2016
Sullivan, Margaret (2015) ‘The Tricky Terrain of Virtual Reality’, The New York Times, Sunday 15 November [Online] Available: https://www.nytimes.com/2015/11/15/public-editor/new-york-times-virtual-realitymargaret-sullivan-public-editor.html (Accessed 13/03/17)
The Displaced (2015) Directed by Imraan Ismail [VR Film] Los Angeles: VRSE.works
The Matrix (1999) Directed by Lana Wachowski and Lilly Wachowski [DVD, 2 hours 16 minutes] Warner Bros and Village Roadshow Pictures
The RSA (2010) RSA Animate: The Empathetic Civilisation, [Youtube], 6 May. Available: https://www.youtube.com/watch?v=l7AWnfFRc7g (Accessed 07/03/17)
TED (2015a) Chris Milk: How virtual reality can create the ultimate empathy machine, [TED], 9 March. Available: https://www.ted.com/talks/chris_milk_how_virtual_reality
_can_create_the_ultimate_empathy_machine?language=en (Accessed 12/10/16)
TED (2015b) Nonny de la Peña: The future of news? Virtual Reality, [TED], 17 November. Available: https://www.ted.com/talks/nonny_de_la_pena_the_
future_of_news_virtual_reality (Accessed 18/09/16)
TED (2016) Chris Milk: The birth of virtual reality as an art form, [TED] 7 February. Available: https://www.ted.com/talks/chris_milk_the_birth_of_
virtual_reality_as_an_art_form (Accessed 31/10/16)
TEDx Talks (2015a) Real change through virtual reality | David Sackman | TEDxEastEnd, [Youtube], 20 February. Available: https://www.youtube.com/watch?v=gpOZIdqYMlU&t=7s (Accessed 31/10/16)
TEDx Talks (2015b) Virtual Reality: The Future Is Coming | John Vechey | TEDxOrcasIsland, [Youtube], 14 September. Available: https://www.youtube.com/watch?v=IJf7BfpKTOY (Accessed 31/10/16)
The Thirteenth Floor (1999) Directed by Josef Rusnak [DVD, 1 hour 40 minutes] Columbia Pictures
TRON (1982) Directed by Steven Lisberger [DVD, 1 hour 36 minutes] Walt Disney Productions
Watt-Smith, Tiffany (2015) Book of Human Emotions: An encyclopaedia of feeling from anger to wanderlust, London: Profile Books
Weinbaum, Stanley (2010) Pygmalion’s spectacles, Montana: Kessinger Publishing
Wells, Liz (2002) The photography reader, London: Routledge
Wen, Tiffanie (2014) ‘Can virtual reality make you a better person?’, BBC, Wednesday 1 October [Online] Available: http://www.bbc.com/future/story/20141001-the-goggles-that-make-you-nicer (Accessed 28/02/17)
We Wait (2016) Directed by Daniel Efergen and Darren Dubicki [VR Film] London: BBC Connected Studios and Aardman
list of illustrations
Cover image: Dietrich, Reinhard (2011) Somali Refugee camp, photograph
Fig. 1: Dietrich, Reinhard (2011) Somali Refugee camp, photograph
Fig. 2: Ghedini, Paolo (2016) Spherical 360 degree photographs
Fig. 3: 20019 (2005) untitled photograph
Fig. 4: Dr Derek Ham (2017) I am a Man, VR experience. Derek’s work can be found on his website: www.derekham.com
Fig. 5: Mabusalah (2016) untitled photograph