Interactive storytelling: Types of cutscenes in games

Video game cutscenes are like little movies, played between gameplay segments. They, like movies and TV show episodes, are a sequence of one or more scenes, where each scene is a sequence of one or more shots.

There are 1st-person shots and scenes, which show the action from the perspective of a particular character. And there are 3rd-person shots and scenes, which don't show the action from the perspective of a particular character, but rather the view from where the camera is located and facing[1]. 

[1] there are also scenes rendered in Virtual Reality, in VR games and movies. These add an extra dimension of immersion.

In movies and TV shows most shots are 3rd-person shots, with the occasional 1st-person shot thrown in. In games, 1st-person (cut)scenes are more common. For example, in the Metro games, the Dishonored games, Halo 3: ODST, Halo: Reach and Cyperpunk 2077. The reasons for this needn't concern us in this post.

Video game cutscenes can include interaction. This post looks at such cutscenes, and how the interactivity in them affects the strength of the storytelling in them.

 

Visual-action storytelling

Here we introduce the notion of "visual-action storytelling". It will help us discuss the affect of interactivity on cutscenes in games.

The primary form of storytelling found in movies and TV is "visual-action storytelling", in which we see the story events occur. By "action" I don't mean in the sense of an "action movie", with weapons, fights, and chase-scenes. I don't mean something that has to be highly dynamic. The "action" is just what the viewer sees unfold over time, and that includes very still scenes where very little is happening.

Visual action is made up of components like the actor's performances, the sets, the cinematography, and the editing.

If there was a movie that was just 1.5 hours of a character recounting a story to some others, where all the footage was just of the storyteller and their audience, this would be a very weak form of visual action. It would contain visual action of the storyteller and their audience, but no visual action of the story being told. In that situation, the "real" story details are being told, not shown.

Whereas if instead of focusing totally on the storyteller and their audience, there were also visual-action scenes showing the events of the story that's being recounted, that would be a stronger form of visual action. In that version, the movie's viewers would be shown the visual action of the story details.

 

Interactivity in visual-action storytelling

Normal cutscenes are non-interactive. In video games, some degree of interactivity can be introduced into the visual-action storytelling of cutscenes. Though, as we'll see, it's only limited forms of interactivity, and their addition can lessen the strength of the visual-action storytelling.

The following looks at the different ways interactivity can play a role in the visual-action of cutscenes.
 

QTE-and-choice cutscenes

Two forms of interactions that may occur within cutscenes are QTEs (Quick-time Events) and making choices.

Imagine a story-focused game, where, after much journeying the player makes it to the castle, and gains an audience with the king at the king's court. There's a cutscene of the player character entering the court and talking to the king. As the cutscene continues, the situation goes south, and a fight breaks out between the player's character and the guards.

One way that fight could be implemented would be with normal gameplay, where the player moves their character about, attacks with their weapons, and blocks with their shield.

Alternatively, the fight could be a continuation of the cutscene. That way, the fight could be made to look very cinematic. It could contain dedicated character animation and performances, be 'shot' in a cinematic way, use various camera angles and movements, and be edited to look spectacular. However, the player would lose direct control over their character.

Imagine a moment during the cutscene where an enemy swings their sword at the player, and the player dodges to the left, just in time to avoid the blade. A QTE (Quick-time Event) could be used for making the character dodge the blade, to add some interaction into the cutscene.

Here's how the QTE would work. As the enemy goes to swing their sword, time will slow down a bit, and the screen will show a prompt, telling the player to press left on their joystick. The player would (usually) have a small window of time to enter that input. If they enter the correct input within the time limit, they succeed at that QTE -- and successfully dodge the sword -- otherwise they fail at it. The penalties for failure depend on the game and the particular situation in it. The penalties could be minor, all the way up to player's character dying.

As another example, during the fight the player could be grabbed from behind by some of the guards, and there could be a QTE prompt telling the player to quickly tap (and keep tapping) one of the controller buttons. If the player taps the button fast enough, they'll escape the grips of the guards.

Remember that in both of these examples, the action would be shown in a cinematic fashion, just like in a movie.

Other types of inputs used in QTEs include moving a joystick in a particular path (e.g. in a full circle, or to right and then anticlockwise, to up). And on touch-screen devices, the player may need to tap on hotspots on the screen, or slide their finger along a path (e.g. a circular path). The QTEs can be action-mirroring inputs.

QTEs are divisive. Many players do not like them. They see QTEs as a fairly pointless attempt at including a bit of gameplay here and there in cutscenes. They may not find QTEs enjoyable.

One thing that I think all could agree on is that QTEs are a fairly simple form of input. The game tells the player what to do, and when to do it, and the player just needs to follow the instructions properly. Performing the input(s) for a QTE is a fairly rote task. The player doesn't have much agency when it comes to QTEs. (We'll talk about choice below, and choices may be implemented as part of the QTEs, so in this sense they can provide some agency).

Choices provide another means for there to be player-interactions within cutscenes. At points in cutscenes the player can be presented with a choice from a small menu of options, and -- like with QTEs -- there will usually be a time limit for making the choice. The choice might be between a small number (2 to 4) of dialog options or action options.

As an example of choices between action options, there might be a oncoming zombie horde, where the player has to quickly choose between saving one of their companions, Joe or Jackie. Especially in timed action choices, we may consider these kinds of choices to be a kind of QTE.

We can call the kinds of cutscenes just described "QTE-and-Choice cutscenes" (or "Q&C cutscenes" for short).

The addition of QTEs and choices don't weaken the strength of the storytelling in these cutscenes. They're just like normal cutscenes, except for the introduction of some basic forms of interaction (QTEs and choices).

 

During-Gameplay Cutscenes

There are kinds of cutscenes that can play during the gameplay scenes in games. With these, the player still has some control over their character, while, at the same time, cutscene-like details play out around them. We'll call them "during-gameplay cutscenes" (or "D-G cutscenes" for short).

Half-Life 2 is the classic example of a game that includes during-gameplay cutscenes. Almost all of its cutscenes are of this sort. The Dishonored games are another example of games with during-gameplay cutscenes. The Metro games also contain a fair number of D-G cutscenes.

Half-Life 2 is a first-person shooter in which some hostile aliens have invaded earth. The game starts with you disembarking a train, and soon after you see an alien guard, holding a large baton, shove a person they're overseeing. As you walk past some of the humans there, they say things to you. Soon, you reach a checkpoint, where you're taken away to an interrogation room, where the helmeted alien who led you there takes his helmet off, to reveal themselves to be a human -- a person that your character knows. The entire time all this unfolds, the player has control over their character's movement (though of course where they can move is constrained by the walls etc in their environment), and can control where their character is looking.

Such during-gameplay cutscenes consist of scripted dialog and events (animated occurrences), that are triggered by the player's actions. For example, an NPC saying something to the player might be triggered by the player walking close-enough to the NPC. Or the player performing actions, like picking up a gun on the ground, might trigger some during-gameplay cutscene.

The cutscene elements of D-G cutscenes can consist of any scripted details. They could be as simple as some characters talking. D-G cutscenes can also be of any length. Some may quite brief, lasting only a couple of seconds.

One form of D-G cutscene is where the player's character and NPC(s) are having a conversation as they walk along. These are epitomised by Naughty Dog's Uncharted games and The Last of Us games. Yahtzee Crowshaw calls these "Walk and talk sequences".

We wouldn't normally think of these Walk-and-talk scenes as kinds of cutscenes, but they match the criteria laid out above, of being scripted details that play out during gameplay. In this case, there is the scripted movements of your NPC companions, and the scripted dialog said by your character and the NPCs.

These conversation-focused D-G cutscenes don't have to take place while the characters are walking. They can occur during any kind of activity the characters are undertaking, like climbing a structure, or fighting against enemies.

Games like Rockstar's Grand Theft Auto (GTA) games have a huge amount of D-G cutscene detail going on all around the character during gameplay. There's vehicles on the roads, people walking about, and so on. We wouldn't normally think of these as kinds of cutscenes, but they meet our criteria. In this case, the scripted details are more "background details", that are in-part there to give the cities a lived-in feel. They're always rolling, and aren't there to play a specific role in the plot.

Other well-known games with D-G cutscenes are the Metro games, and the recent God of War games.

In a D-G cutscene, the player's control may be restricted in certain ways. They may be able to look around, but not move around, such as in the opening sequence of Skyrim, where the player wakes up as a prisoner, in a horse-drawn cart. Or in Metro 2033 where the player is sitting at table, with other characters, in a bar.

Arguably, the greatest implementation of during-gameplay cutscenes thus far, is the gang's camp in Red Dead Redemption 2. The camp seems to have a life of its own, outside of the existence of the player's interaction with it. Characters go about chores, have conversations amongst themselves, sit around the campfire at night drinking, chatting, and playing musical instruments, before retiring to their tent once it gets late enough for them. And the player can choose to, or not, take part in the goings on, such as the conversations.

 

Mixing during-gameplay cutscenes and other forms of cutscenes

During-gameplay cutscenes and other kinds of cutscenes can be seamlessly mixed together. For example, in the aforementioned scene in Metro 2033, where you're sitting at a table with a few others, having drinks and conversation. Most of the time it's a D-G cutscene, where characters are talking and the player can freely look around. But at times, NPCs will propose a toast, at which point it will show a short non-interactive 1st-person cutscene, where you grab your drink and take as swig of it, and put it back down.

It's possible to mix all three types of cutscene -- non-interactive, QTE-and-choice, and during-gameplay -- within the one overall cutscene.

 

Downsides and limitations of during-gameplay cutscenes

During-gameplay cutscenes sound good on paper. They enable gameplay/control and visual-action storytelling to be integrated. They appear to be the best of both worlds. We may wonder why they aren't used more in games. One reason is that they have a number of limitations and downsides. We'll look at these now.

 

During-Gameplay Cutscenes Weaken and Constrain the Visual-Action Storytelling

Movie/TV scenes and cutscenes use cinematic techniques, through the use of camera angles, camera movements, and editing (cuts). In D-G cutscenes, because the player has control, the same camera that's used in gameplay -- whether first- or third- person -- is still used in the D-G cutscene. That lessens the strength of the storytelling, to some degree.

Movie/TV scenes and cutscenes involve real/virtual actor's performances. These performances include the

• blocking (including the marks where they stand and move at each moment of each shot)
• physical interactions between characters
• body language, including posture, how they move, facial expressions, gestures, etc.
• dialogue

And their timing is crucial to their performances.

And of course the performances involve interactions between characters. Eye-contact between characters. Where a person is standing and facing in relation to others, and how people walk around, amongst others. When two people meet for the first time. When a person approaches another to ask them for assistance. Or when there is flirting between people. When there's a group of people, and someone is speaking, there's how others are listening, and their reactions. Vigorous agreement, vs indifference. And so on.

All those body language and performance details have meaning, meaning that they communicate. A lot, for example, is communicated by the nature of the eye-contact between people. For example, the differences between avoiding eye contact, confident eye contact, aggressive staring, and flirty eye contact.
 
The usual control schemes in games are too simple, to be able to control such details. The player's character might be able to move in different directions, look in different directions, jump, and crouch. Those options pale in comparison to the sort of control an actor has in their performance. So the player's character can only have a 'passive' role in D-G cutscenes, that does not include those specific performance details.

As a different way of seeing this point, consider that in D-G cutscenes, the player could only communicate to other characters via moving forwards, backwards, left and right, turning to the side, and, by what direction they're looking in. The reader can imagine if they, themselves, were in a situation with other people and these were the only ways they could communicate to others. Or imagine if they were controlling a robot with such a control scheme, that was amongst those people, and the robot had zero ability to express facial expressions or body language, just those kinds of movement.

There's a fundamental difficulty here, in that the game can't know the player's intentions. The player may want to be friendly, or dismissive, towards another character. But the game can't infer those intentions from the player's control over their character. There's a paucity of information about the player's intentions, and only so much that can be inferred. This couldn't be overcome by having AI to have NPCs react more realistically.

To accommodate these control limitations, the player might be seeing some event taking place on the other side of a chain link fence. Or seeing large alien machines moving around in the distance. In such cases the player is distanced from the action, and are more like spectators.

There could be D-G cutscenes where the player's character can choose where to stand amongst a number of people, as a conversation plays out. In such situations, it doesn't matter where the player character is standing in the scene, because all that matters is that they hear the dialog. In such cases the player is more of a passive spectator of what's going on.

That the player tends to be relegated to a passive spectator is why D-G cutscenes are so often focused on auditory details, like conversations between characters, or messages over loudspeakers. The player can be a passive participant in these, because audio is (somewhat) omni-directional, it doesn't matter exactly where the player character is facing and positioned at anytime during the D-G cutscene -- they'll still be able to hear the details.

Imagine a D-G cutscene where the player was an active participant in it. E.g. if player is up close to another character, in defiance of them, gets shoved back, and then darts back towards them, looks quickly to the sides at the other people there, and then turns to one of them, puts a hand on their shoulder, and whispers something to them. That would have to be done with a normal cutscene (which could possibly include QTEs).

Or imagine game developers trying to make a D-G cutscene where the player is showing another around their study, pointing out different objects to them. Where the player picks up a rare bottle of whiskey and shows that to the other character, handing the bottle to them. This couldn't be done either, if the player character is being controlled in the usual fashion.

One possible way of making the player character's performance include what was required for such scenes would be forcing them to do exactly what is required (and thus give up freedom of control).

A control scheme that would give the player control over all those nuances would have to be very complex. Perhaps it could be done with a mouse and keyboard, using all the keys on the keyboard, and use of those keys in combination with modifier keys. But then it'd be too complex for any normal person to use. VR with facial and body tracking could be a more realistic option.

Or something like QTEs could be used, though this wouldn't really give the player control over the character, it would be more just the player entering in the correct inputs to satisfy each of the QTEs.

A potential option would be to give the player the freedom, and have the other characters respond to the player's actions in a realistic way. The problem with this is that it'd be too complex to script in the different branching possibilities, and we don't have the means to simulate the responses of the other characters. And even if we could simulate them, we'd also have to also simulate the plot implications of some of the possibilities. And even if we could do that, many of those other possibilities would result in weaker storytelling (because strong stories don't lie near to each other in story space).

The player's character having to play a fairly passive role in D-G cutscenes means that such cutscenes could not be used for the storytelling of most scenes from movies/TV -- scenes that don't usually contain a passive role. Given that, it's fair to say that D-G cutscenes generally contain weaker storytelling than can be present in non-interactive cutscenes.

To me, the way the D-G cutscene details are always happening at a distance from the player character makes them feel a bit like a theme park attraction. As if the player is moving along through a set path, and there's scenery/sets and animatronics on either side of the path. The start of Half-Life 2, and the metro stations where the people live, in the Metro games, have this feel to me. I feel there's an uncanny valley feeling to the interactions with NPCs in during-gameplay cutscenes.

 

In conclusion, non-interactive cutscenes enable strong visual-action storytelling, but are at odds with gameplay. Interaction allows QTE & Choice cutscenes, and During-Gameplay cutscenes. QTE & Choice cutscenes involve weak gameplay as part of strong visual-action storytelling. During-Gameplay cutscenes enable stronger gameplay, at the expense of weaker and more-constrained visual-action storytelling.

Interactive storytelling: Activity-and-Choice Story-Games

In this post I'll define a genre of video games that I'll call 'Activity-and-Choice Story-Games'. These are exemplified by games like Detroit: Become Human, Life is Strange: True Colors, The Wolf Among Us, and Florence.

These games can be seen as an evolution[1] of Point-and-Click Adventure Games, in which the puzzles of those games are replaced by 'activities' the player undertakes and choices they have to make. We'll get to what activities and choices are in a moment. These changes have been made to make this genre more story-focused.

[1] that B evolved from A doesn't automatically make B superior to A. Bs could be simpler than As, not necessarily more complex. Evolution does not have a direction. So we're not saying this genre is superior to Point-and-Click Adventure Games.

Here are some other prominent examples of 'Activity and Choice Story-Games' (A&C Story-Games): Life is Strange 2, Telltale's The Walking Dead[2], and Tales from the Borderlands. 

[2] The first season of Telltale's The Walking Dead is a transitional form, that still involved some puzzles. The subsequent seasons of it became more pure A&C Story-Games.

Traditional Point-and-Click (P&C) Adventure Games are puzzle-focused. The player solves puzzles to progress through the game and its story. The player can pick up objects, look at objects, use objects (e.g. turn on a light switch), or use one object on another (e.g. putting a key in a lock, to open the lock). These actions are primarily means for the player to solve the puzzles.

This affects the kinds of stories that can be told in P&C Adventure Games. One, what the player is primarily doing is solving puzzles, and thus that is the main thing the player's character is doing in the game's story -- going about, solving puzzles. And two, the story can only[3] progress after a puzzle is solved. This second effect slows down the pacing of the story. It might take the player quite some time to solve a puzzle. All of these things restrict the kinds of stories that can be told in P&C Adventure Games.

[3] This isn't strictly true, some story events can happen while the player is going about trying to solve a puzzle. However, that can't be a story event that would change the game world in any way that would prevent the player from solving the current puzzle. Such story events are uncommon.

The story has to be one in which the main character is going around solving puzzles, where it still has to make at least some sense if the story is put on hold for potentially a fair while, till the player has solved the puzzle. The focus on puzzles thus detracts from the ability of these games to tell a story.

Activities

In Activity-and-Choice Story-Games, one of the elements that replaces puzzles are 'Activities'. An 'activity' is some simple task that the player needs to undertake, where there is minimal challenge for completing the task. An activity is there to provide the player with ways to interact with the game, and they tend to be story-focused.

Activities are usually straightforward tasks, and present either no or minimal challenge. In many of the games, the player will be told what the current activity is (perhaps in the form of the activity's goal or as an item in a to-do list).

Here are some examples of activities, from the early parts of Life is Strange 2, in which you play as a teenager, Sean:

• After coming back home from school, and chatting with a friend on a deck outside the house, you go inside. Your dad and little brother are there, you have a conversation with them, and your dad says that you have to decide whether your little brother or him gets the last chocolate bar. You get to be the judge making the decision of who gets to have it (you can also choose to have it yourself).
• That night you're going to go to a party with some classmates, and before heading out, you need to grab some stuff to take over: drinks, snacks, a blanket, and some money for supplies. You have to walk around the house, and get the items. And you have options for what to choose as the drinks to bring, and what to choose as the snack. You have the option to take some of your dad's beer, and some money from your dad's wallet, or not.
• Later you and your brother are on the run, and need to find some place to stay overnight. You come across a national park area, and the activity is going down the paths from the entrance to where the campsite areas are. During this there's conversations and some things you can look at. And after you've found some shelter, there's the activity of collecting some firewood for a fire. That's just a matter of exploring around and finding pieces of suitable wood.
• Then there's lighting the fire, then sitting there engaging in conversation with each other.
• Later on in the game, you stop at a roadside gas station and store, and get some supplies for the trip there. There's various options for what to buy, and you've got limited cash, so you have to think about what are the priorities.
• Sometimes an activity can be as simple as walking from location A to location B. Along the way there might be things the player can look at, and people they can talk to.

Activities are usually something relevant to the story, and they're more like scenes in a movie than any interactive parts of P&C Adventure Games are. Often they're 'slice of life' kinds of details.

Another example of an activity, from the start of Life is Strange: True Colors, is unpacking your bag when you arrive from out-of-town to the place you're staying at. You unpack the bag item by item, and for some items she might reflect on the role it had in her life, or some other items are textual (like some letters), which also provide a bit of a window into her life. Thus it provides some backstory. At the end of the process she goes to put her bag under her bed, and in doing so finds the guitar that her brother had gotten her as a present, which leads to a cutscene of her taking it out, commenting on it, and playing a bit of it.

While undertaking an activity the player may be able to look at things in their surrounds (including other characters), and the player character may comment on them in a way that reveals their thoughts on what is happening at the current point in the story. The player may also be able to talk to characters in their surrounds, about things relevant to what's currently happening in the story.

Activities may be very basic, easy-to-solve puzzles. That is, puzzles that can be quickly solved, and which present very little challenge to the player. Because A&C Story-Games de-emphasise puzzles, if they do contain some puzzles, they will (in addition to being easy) contain much fewer of them, compared to the number in a P&C adventure game.

Activities may also involve Quick-time Events (QTEs), to add small moments of interactivity and/or challenge in cutscenes. Successfully completing a QTE is usually easier than solving a puzzle (and of course it is a quite different kind of difficulty), and QTEs are compatible with a story-focus, because they can be embedded in cutscenes.

Choice

The second component of A&C Story-Games are choices.

We've already mentioned how choice can play a role in Activities. For example, in Life is Strange 2, when Sean and his brother are on the run, they come to a gas station and store, and with their limited money they have to choose which supplies to buy (if I recall correctly, there's also an option to steal some items). The player may also have choices in what to say to other characters, when they're talking to them.

These games may also have more explicit choices. Where the game presents the player with between 2-4 options for what to do, in the vein of the "Choose Your Own Adventure" books. For example, there may be an argument going on between person A and person B. Should your character

• Take person A's side
• Take person B's side
• Stay out of it

The first season of Telltale's The Walking Dead (2012) innovated on this formula in a way that proved to be very successful, and which also helped to popularise the use of choice in games. The main part of this innovation was to make the choices timed.

They applied this formula to both "Choose Your Own Adventure"-style choices and dialogue choices. The player was given a short amount of time to make each decision. If they failed to make a choice in this time, it'd either result in inaction from their character, or them choosing a default action.

Second, they made the choices very difficult -- where there were pros and cons to all the options.

Having to make a quick decision about a difficult choice turned out to be quite immersive. It forces the player to put themselves in their character's shoes.

Since that first season of TellTale's The Walking Dead, the use of choice in story-focused games has become quite popular. Interestingly, though, the use of timed choices hasn't been widely taken up. For example, the Life is Strange games have choices, but they're not timed.

Sometimes the choices are about personal expression rather than being difficult choices. For example, early in Life is Strange: True Colors, your character can choose which of two potential bad-guy characters will be the villain for the LARP another character is setting up.

A&C story-games replaced puzzles with activities and choice to make the games more story-focused. Let's look at the consequences of this for the storytelling in these games.

Because the focus in P&C is on puzzles, the interactable objects in the environment, the descriptions, and character conversations, are all primarily oriented towards solving puzzles. That is, to give the player clues for solving puzzles, and the means to solve puzzles. That is not to say that there's no story focus regarding these things. Only that such has to be secondary.

If puzzle-solving isn't put first, then the details presented to the player could hinder their ability to solve the puzzles. E.g. providing details that don't at all help with the puzzles may mislead the player about the solutions to the puzzles.

Whereas in A&C Story-Games, the objects in the environment, looking at them, and talking to characters, can all have a story-focus.

When the player looks at an object, their response may reflect their state of mind at this point in the story. It may also involve their thoughts about a character who has some connection to that object (maybe it's something they own). When the player talks to other characters, it is more about story-relevant details, than to get information for solving puzzles.

We could, if we wish, consider the acts of talking to other characters, and looking at objects, in A&C Story-Games, to be kinds of activity.

What we're calling "Activity-and-Choice Story-Games" would most often be called "Narrative Adventure Games". However, that latter term is often used to refer to a much broader set of games than A&C Story-Games. Some of the kinds of games called "Narrative Adventure Games" include P&C Adventure Games that have more of a focus on story than the typical P&C Adventure Game, like Kathy Rain, Walking Simulators, like Gone Home and Everybody's Gone to the Rapture, Visual Novels, like Doki Doki Literature Club, and film-like games that involve only choices, like Last Stop.

(Why do we not consider Walking Simulators like Gone Home to be A&C Story-Games? That game doesn't have any choices, though there's no reason why a Walking Simulator couldn't have choices. It's that pure Walking Simulators, like those games, don't have anything like the Activities we have been talking about. In them, you walk around, perhaps look at objects, open drawers and doors, and perhaps find audio-logs or audio-visual-logs. In literal terms these are 'activities', but they are not activities where the player is given a particular goal, and where the activity(s) that satisfy that goal have a particular role within the game's story.)

There are also "Activity Story-Games", which don't involve any choices, only activities. These are much less common. The only example that comes to mind is Rainswept.

Having easy/fewer (or no) puzzles, means that the narrative in A&C Story-Games is less constrained than they are in P&C Adventure Games.

• 'Solving puzzles' will play little or no part in what the player's character is doing during the story.
• There will be less "dead time" while the player is trying to figure out how to solve puzzles, so the story's pacing won't be slowed down as much. Which means the story isn't constrained to be one where it "makes sense" for there to potentially be large amounts of time between each story beat.

The shift from puzzles, to activities and choices, means that each scene in the game can be a specific narrative situation. This is actually quite different to P&C Adventure Games. In them, there tend not to be specific scenes during gameplay segments, but rather a set of locations accessible to the player, for the current set of puzzles they can work on.

Some examples of situations the player might find themselves in in a P&C adventure game. They've been locked inside an apartment, and have to figure out how to get out. Or they need to find a way to distract a guard, so they can go in and talk to the person in the office. In story-terms, these are fairly low-level kinds of details. They're not particularly meaningful.

As we've mentioned before, with a puzzle-focus, the story has to be put on hold until the player has completed puzzles. And because the player usually has multiple puzzles open to them at any point in time, the story can't advance much until the current set of puzzles have all been solved. So there can't be much in the way of story-relevant happenings while the player is working on puzzles. There can't even be much that's strongly associated with the story, that happens within, or upon the completion of, most of the individual puzzles -- only on the completion of the set of puzzles after which the story can move forwards. The ability to work on multiple puzzles at once is good for gameplay, just not so great for story.

And this gives a "genericness" to lots of the details in the game. The object descriptions and conversation choices need to be generic. The player's predicament, for each puzzle open to them at that time, has to be "weak", in that it will also let them work on other puzzles at the same time. If the player has multiple puzzles open to them at once, then when they're walking around in and between locations they're in the same generic holding pattern, until all of those puzzles required for advancing the story have been solved.

With an activity-focus, each "scene" in the game can be about a certain story- event/situation, in a specific moment in time within the story. Within that scene the player is given one (or more, though it's usually one) activity to undertake (and potentially at some points will have to make some choices). That activity, and the player's interactions, have a stronger story-focus in A&C story-based games. So story occurrences can be happening within the scene.

 

Because activities and choices allow the games to be much more focused on story, A&C Story-Games are probably what most deserve to be called interactive movies. There are full-motion video (FMV) games that are often considered interactive movies, but these tend to focus only on choices, without any of the activities. As a result, they tend to be shallower experiences.

Interactive storytelling: Action-mirroring control schemes

A number of story-focused video games use what I'll call an "action-mirroring" control scheme.

When a player makes their game character walk or run in a particular direction by pushing the controller's left joystick in that direction, the action of walking in the direction is "mirrored" by an input of pushing the joystick to that direction.

In first-person games, the player's character looking upwards (an action) is performed by pushing the controller's right joystick upwards. The action of looking up is "mirrored" by an input of pushing the joystick upwards.

These are common examples of "action-mirroring" inputs. Such inputs try to embody something of the essential nature of the actions they're used to perform, like moving the joystick in the direction the character is to move in.

Some games -- usually story-focused ones -- use action-mirroring for more than just movement and looking around. They have an "action-mirroring control scheme". For example, in such games a sliding door that may be slid open to the left may be opened by pushing in the joystick to the left.

Here are some prominent examples of games that use action-mirroring control schemes.

The Quantic Dream (David Cage) games Heavy Rain (2010) and Detroit: Become Human (2018).

The Telltale Games games The Walking Dead (2012), The Wolf Among Us (2013), and Tales from the Borderlands (2014).

The Supermassive Games games Until Dawn (2015) and The Quarry (2022). And games in their Dark Pictures Anthology, such as Man of Medan (2019) and Little Hope (2020).

The Bloober Team game Observer (2017).

The Flavourworks games Erica (2019) and HUSH - Crane (2022).

The Mountains game Florence (2018).

 

Some more examples of action-mirroring inputs. A cupboard door in real life may be opened by pulling its handle out and to the right or left. A kitchen cutlery drawer may be opened by pulling it towards oneself. Milk added to some coffee may be stirred by lowering a teaspoon into the coffee mug and moving it in a circular motion. In these cases, the action involves movement along a particular path. That path may be traced out with a joystick by movement in a single direction, like for a door that slides open to the left. Or it may be a more complex path, like the circular path traced out to stir some coffee.

Typically, when the player needs to enter an action-mirroring input, the game will show the player a prompt, indicating what input they need to enter. And the player may need to first 'lock onto' an item before entering the input. E.g. they (while standing in front of the door, facing it), may need to then press a shoulder button on their controller to lock-onto the door, after which they can enter the action-mirroring input to open the door.

The movement paths (like the circular one for stirring the coffee) can be traced out using a variety of input devices, meaning that action-mirroring control schemes are compatible not just with controllers with joysticks, but also a keyboard and mouse, and touch screens. With a keyboard and mouse, the player can, for example, slide the mouse to the left to slide the door open to the left. On a touch screen, the player can swipe left to slide the door open to the left.

When compared to the actions they perform, action-mirroring inputs are a major simplification. Pressing left on a joystick is a major simplification of opening a cupboard door. But, the idea is that they approximate the core nature of an action, while still being a fairly simple and easy form of input.

 

More examples of action-mirroring inputs.

The input for opening a cupboard door could be pushing the joystick upwards and then tracing an arc down towards the right. The upwards movement represents pulling the door outwards, and the arc towards the right represents pulling the door to the right.
 

(the action-mirroring input for opening a wardrobe, in Heavy Rain)

For the side to side action of brushing teeth with a toothbrush, the player may need to input a repeated side-to-side motion (joystick repeatedly moved from side to side, like in Heavy Rain, or by sliding one's finger from side to side, like in the touch-screen version of Florence).

(using the joystick to brush teeth in Heavy Rain)

(using the mouse to brush teeth in Florence)

For when your character is shaving their face in Heavy Rain, and carefully shaving some spots, the player might have to move the joystick in the direction the character is going to shave. And, to mirror the care that the character is taking (to slowly and carefully move the razor, so as to not cut themselves), the player might have to very gently push the joystick in that direction.

To struggle, to get out of the grasp of a bad guy, might involve quickly tapping a button or, in touch-screen games, the screen. In this case, there's a more abstract relation between the action and the input. The input is intended to mirror the effort or struggle of getting out of the hold.

In Supermassive Game's horror games, there are times where your character is in danger, and you need to hide and keep quiet, so that an enemy doesn't notice you.

In Until Dawn (2015), the player needs to hold their controller completely still until the danger has passed (this makes use of the motion-sensors on modern controllers).

Whereas in The Quarry (2022), the player simply has to hold down a certain button until the danger has passed. During this time their character is holding their breath. The player also needs to release the button as soon as the danger has passed, so they can take a breath. If they hold it too long, they'll gasp for breath and alert the enemy to their presence.

They implemented such scenes in yet another way in their Dark Pictures Anthology games like Man of Medan (2019) and Little Hope (2020), where it's like a little rhythm game, where the player needs to tap a button in time with their character's heartbeats.

With these inputs there's an abstract relation between the action and the input. The input is intended to mirror the task of trying to stay still and make no noise to avoid detection.

In HUSH - Crane (2022) which is on touch-screen devices, there's a moment where your character is removing the camisole worn by another character (Note: this is safe for work, as there's no nudity involved). To slide the straps down, the player needs to put a finger on each of the tops of the camisole shoulder straps, and slide their fingers down from there, along the upper arms of the other character. This is an example of where more than one action-mirroring input may be entered at once. An analogous input could be achieved using a controller, where the player has to press a button to lock-onto the target object then press down on both joysticks at once (in this case, the joysticks effectively represent the player character's hands).

(An action-mirroring input from HUSH - Crane, involving two simultaneous inputs)

Here we'll look more at how action-mirroring control schemes can differ between games.

Direct- and indirect- manipulation of inputs

Say the player has to enter a 'left' input, to open a door that swings out to the left. In some games, the player enters that input, after which the door is opened. This is an indirect-manipulation input.

In other games, the door will swing to the left as the player continues to hold the left input (and swing back to the right if they hold the right input). The player may need to hold the left input down for about a second in total for the door to become fully opened. If the player were to only hold left down for half that time, the door would only end up being opened half way. This is a direct-manipulation input.

This latter form of input is more immersive. It feels more like directly manipulating the object (the door). At the same time, it is more involved for the player, which means that it can get tedious if the player ends up having to open many doors this way.

Another example is, does the player rotate the joystick in a circular motion, after which the coffee gets stirred, or does the coffee get stirred as the player is entering the circular input?

Where on a touch-screen the player has to enter the input

Consider the side-to-side motion the player may need to enter to brush their character's teeth. Can they place their finger anywhere on the screen to enter this input, or do they need to place their finger within a particular area of the screen? As an example of the latter option, in Florence there's a rectangular "zone" at the bottom of the screen for entering the side-to-side input.

Sometimes, the player may need to place their finger on the object they are to interact with. For example, to stir a cup of coffee, the player might be presented with a top-down view of the cup, and may have to trace circles around the circumference of its contents to do the stirring.

Challenge added to action-mirroring

To make some action-mirroring more challenging to enter, such as during a fight scene, the game might provide only a small time-window for entering each mirroring input. Or it could require an action-mirroring input, plus additional button presses. Heavy Rain does this.

How effective are action-mirroring control schemes?

Addressing this question is the main purpose of this post.

These control schemes are there to add immersion to the game experience. To make the player feel, to some extent, as if they are performing the actions themselves. Of course, no such control schemes can make the player feel like they're actually performing the actions. These control schemes aren't trying to do that. They're just trying to add a, perhaps small, feeling that the input is somewhat like the action.

Action-mirroring control schemes seem promising, and the only way to see how effective they are was to implement them in games. Having played several of the games that use an action-mirroring control scheme, I've come to the view that they're, mostly, not that effective.

Some cases of them can be fairly effective. Consider how in Observer the player opens doors. The player needs to be near the door and then press a shoulder button on their controller to 'lock onto' the door, then move the joystick to the left (or right, depending on which side the door opens to). As they continue to push the joystick to the side, the door continues to open, as opposed to the player entering the input to open the door, and then afterwards the door opening. This feels immersive. Locking onto the door feels like grabbing its handle, and it opening up more as you continue hold the joystick to the side makes it feel like you're directly manipulating the door.

In Until Dawn, I found it immersive to have to hold the controller completely still, in order for your character to hide and not be noticed by an enemy.

Action-mirroring inputs may add immersion, but this can come at the expense of it requiring more time and effort to perform interactions. In a game that does not have an action-mirroring control-scheme, opening a door may require a simple button-press. Whereas in a game with an action-mirroring control scheme, the player may have to lock-onto the door, then hold the joystick to the left until the door fully opens. If the player was only opening a door once, this wouldn't make any real difference, but if the player is repeatedly having to open doors like this throughout the game, then the extra time and effort adds up. Opening doors that way may end up getting a bit tedious.

And though action-mirroring inputs may be immersive, the actions being mirrored tend to be fairly menial ones, of manipulating things in the physical environment. Like opening doors. So their immersion tends to be immersion in menial tasks. The immersion isn't particularly meaningful.

Action-mirroring inputs mostly don't feel immersive. Tracing out certain patterns with the joystick or with your finger on a touch-screen, just doesn't feel that much like performing the action in the real world. And, to the extent that they are immersive, it's in menial tasks, so it isn't meaningful immersion.

And, there's the added time and effort of entering those inputs, as compared to button presses, so they can get a bit tedious.

One reason they're not terribly immersive, is that in reality we tend to do tasks, especially menial tasks like opening doors, on autopilot. We don't explicitly think "I'm going to do this action, and doing so requires using my hands in this way". We just do it. We operate more on the level of our intentions, than on level of the physical character of our actions. We know we want to stir the coffee, and just do it. We don't think about how we're going to do it with a circular motion. We don't think of the mechanics of how we're going to do it. Those details are mostly subconscious. Having to explicitly follow a procedure to stir the coffee doesn't feel realistic.

Ideally, we'd like a control scheme that more closely matches how we perceive and undertake actions.

Interestingly, Quantic Dream's second-last game, Beyond: Two Souls, did use a control scheme that I think more closely matches how we perceive and undertake actions. And I think it worked quite well, so it's surprising to me that they did not keep using it for their most recent game (Detroit: Become Human).

In Beyond: Two Souls, interactable objects near the player are highlighted by a white dot. To interact with them, the player presses the right joystick in the direction of the white dot, relative to where the player character is. So if the object is to the left of the player character, the player presses left on the joystick to interact with it. Whereas if the player moves to the other side of the object, the player will need to press right on the joystick to interact with it. The reason it feels effective, to me, is that it feels more on the level of our intentions -- specifying which object to interact with.

(Two screenshots from "Beyond: Two Souls". The player needs to move the joystick in the direction the white dot is, in relation to where the player character is. E.g. in the first screenshot, they'll need to move the joystick to the left, to pick up the bag)

But a full discussion of how we can have a control scheme that's more inline with how we perceive the world and act on our intentions, is a topic for another post.

 

Finally, there is virtual reality (VR). VR can provide immersive interactions with the game world, that very much mirror the nature of real-world actions. For example, a player can reach out to a kitchen drawer in the game world, grasp its handle and pull it towards themselves, to slide it open. This is a topic I can't speak about in depth, as I don't have a VR headset yet, and have very limited experience in using VR.

Interactive storytelling: audio and audiovisual logs

In an earlier post, I talked about how games are often divided between interactive gameplay segments and non-interactive storytelling segments like cutscenes.

I want to look at other techniques that are used to add storytelling elements into games. In this post I'll talk about audio logs, and what I'll call 'audiovisual logs'.

The first-person shooter BioShock (Wikipedia | Steam) popularised audio logs. In BioShock the player starts out by entering an underwater city built to a utopian vision, whose society has -- for reasons unknown -- broken down and most of its inhabitants have either been killed or are scattered.

As they traverse through the city, they may find objects that look like old-fashioned tape recorders. If the player picks one of these up, they can listen to the sound recorded on it, which is usually like a diary entry recorded by a character who was an inhabitant of the city (often they are one of the major figures who lived in the city).

Here's a short video of finding and playing an audio log from early on in the game. 

The audio logs the player finds are the main means by which the player learns about the city's backstory.

Audio logs are sound recordings that the player can come across, that are usually related in some way to the location they're found at. E.g. the player might find an audio log recorded by a character in that character's private quarters. They tend to be fairly brief (I would imagine that most of them are well under a minute long), and while listening to them the player can still move their character around and shoot or whatever.

I'm using the term 'audio log' in a broad way. Instead of them being objects that the character finds and can pick up, they can also be recordings that are automatically triggered when the player does something, such as entering a certain room for the first time, or looking at a certain object (like opening up a locker and looking inside). The game Gone Home (Wikipedia | Steam) has audio logs of this nature.

In Gone Home, the character comes back from an overseas trip, to the house that her parents and sister had moved to while she was away. For some mysterious reason, when she arrives no-one appears to be home. The audio logs that the player hears as they progress through the game are what primarily informs them about what has actually happened. (These audio logs are like journal entries that you sister wrote to explain the situation to you. During the game it is not explained why it is that you're hearing these journal entries as if they were read by your sister. It's only at the very end of the game that the player gets a kind of explanation of it).

Terminology note: "audio log" is the most common name for these kinds of things, but they actually go by different names in different games. For example, in BioShock they're called "audio diaries", and in Bioshock Infinite (the third Bioshock game. Wikipedia | Steam) they're called "Voxophones". And these are just some prominent examples of games that use audio logs.

There's also what I'll call "audiovisual logs", where the player will not only hear the other character(s) talk, but see a visual representation of them and what they were doing at the time.

Here's what an audiovisual log could be like: where a character had recorded a front-facing video on their smart phone, that the player watches that as if they were watching a video. I'm not sure if that sort of audiovisual log has been implemented in any game or not. I haven't come across them.

But the kind of audiovisual log I'm talking about are a bit different. Say that the log is of several characters having a conversation around the dinner table in the dining room of a house. In these audiovisual logs, the player would see that audiovisual log when they're actually in that dining room. They'd see the characters sitting on the chairs around the table, that's right in front of them. So these are an 'in situ' visual representation of the characters.

In Everybody's Gone to the Rapture (Wikipedia | Steam), the player is investigating what happened in a town where something strange has happened and all the people have gone missing. As you explore the town you'll come across audiovisual logs here and there, which show conversations between town inhabitants that were happening as the strange occurrences were happening.

The audiovisual logs are like 'traces' of the past, whose presence is somehow connected to the strange happenings. In them, the people are visually represented by golden glowing blobs of light that roughly represent the shape of their body and which follow their movements. Here's a short example from early in the game.

In the sci-fi game Tacoma (Wikipedia | Steam), you're investigating what happened in the space station Tacoma and why all the crew are missing. As you explore the space station you'll come across scattered audiovisual logs. These were recordings made by the space station's computer, and they show events that happened in the lead up to the crew going missing.

These are presented slightly differently, in that the player can control the playback, by pausing and scrubbing back and forth through the conversation - in order to better be able to catch all of its details. An example from early in the game can be found about 15 seconds into this trailer for the game.

With most audio and audiovisual logs, the player is free to move about while the log is playing.

How can we understand these audio and audiovisual logs from a storytelling point of view?

They provide backstory, and function somewhat like a flash-back in a movie. The audio logs that are like diary or journal entries serve basically the same function as finding a diary or journal entry written on paper, that the player can look at and read the text of.

They may also provide some world- or character-building.

In some of the games, it feels fairly unrealistic for there to be all these audio logs scattered around the place. And it sometimes feels unrealistic that the recording was made in the first place. E.g. an audio log recording of a doctor doing some surgery where they suddenly decide to do something horrific, and the nurse is horrified. Who decided to record audio of what was happening during that part of the surgery? These sorts of contrivances can detract a bit from the storytelling.

(From a gameplay perspective, that your character can be still moving around and doing things while you're listening to an audio log, makes a bit of a difference. Compared to you having no control while listening to the audio log. It means the storytelling details of the log are integrated a bit more smoothly with the gameplay).

In the four games that we've looked at -- BioShock, Gone Home, Everybody's Gone to the Rapture, and Tacoma -- there is a mystery as to what happened prior to the events in the game, and the logs are the primary way that the player can piece together a picture of that.

There is a kind of puzzle element to it. The player hears the pieces (individual logs), and those pieces may not explicitly spell out the overall picture of what has happened. The player may need to themselves connect the dots to figure out the overall picture.

This isn't to say that that's the only way these logs could be used in a game. However, what does seem to be pretty inherent to such logs is that they provide backstory.

There can be a gameplay challenge, of finding the logs. BioShock is like this. If a player doesn't search thoroughly enough, they might miss several of the audio logs. That's some interactivity that's associated with the logs (in finding them). And the game might require user input to play a log (more interactivity). But the recordings in the logs are non-interactive. In this sense they are a bit like a cutscene.

What I said in an earlier post about cutscenes also applies to these logs. The logs may together form a strong narrative, but the player's experience of that strong narrative is diluted by the very loose narrative of the gameplay segments between each log that the player finds. Also, because the logs are fairly short and are scattered about, it is more difficult to use them to form a strong narrative. The logs tend to work better as more stand-alone pieces, that don't have the strong connective-tissue between them that's required for a strong narrative.

But while cutscenes can be used for flashbacks, flashforwards, or what is happening in the present, the logs only seem usable for (what are effectively) flashbacks. So they can only contribute to strong narrative for the backstory, not the story that the player experiences first-hand.

Interactive storytelling: control over a character means an abstracted existence in the game world, which makes storytelling difficult

Here's a reason it's difficult to have strong storytelling when the player has control over a character in a game. 

Consider the way the player character may move about in the game world. In a typical 3D game, the player can control the direction the character moves in, and often the speed that they're moving at. In some games, they can make the character crouch down, jump, or climb.

In most games, the movement is always exactly the same. There's the same running animations that are always used. If the player walks into a part of the environment that blocks their movement, such as a kitchen table, they'll have to change direction to move around the table. And the whole time the same movement animations will be used.

In the real world, there are a lot of details to how a person moves through their environment. The kinds of steps they take, their posture, how they may put their hand on objects that they pass by. Those details are affected by the person's nature, their state of mind at the time, and also their intentions. While they are in the kitchen cooking something, their steps may be quite different to when they're on their way through their kitchen to go outside.

Some games, such as in the Uncharted series, do try to capture some of the actual ways characters move through an environment, while that character is under the player's control. But even in these cases, they only capture a small part of what actual movement is like.

So in general, the player character's movement is fairly abstracted and generic, compared to people's movement in real life. The player can make the character do things like walk in a particular direction, but they can't control the nature of that walking. This is not the criticise the games or the game developers. There's all sorts of good reasons why it is like that.

I am focusing on the player's control over the character. In non-interactive cutscenes, the character will act in this more natural way. And in gameplay segments, there will be elements of that, such as in the "bad ass" way your character in a third-person game might hold a big gun while they're firing it, or in the "tough" way they reload the gun. But the player has no control over those sorts of details.

The same abstraction is there in the other ways the player character exists in and interacts with the game world. It's common in games for characters to be able to look at, and interact with, objects in the game environment. Looking at the object might bring up a description, or a 3D model of it that can be rotated around. Interacting with an object will trigger the character to perform some sort of action, such as toggling a light switch, opening a cupboard, or talking to a person.

These things are handled in a very abstracted way. The player tells the character to interact with the object, but has no further control over how that interaction happens.

To illustrate why that's very abstracted, consider how, in real life, someone might make and eat their breakfast. They might be still quite sleepy, and slowly shuffle about, assembling the ingredients. They might absent-mindedly prepare it, like putting bread in the toaster, chucking the toasted bread on a plate, and spreading a bit of butter on it. Then taking that and the coffee they prepared out to the back steps and sitting down to have them. They might place the mug and plate to the side, and alternate between sips of coffee, bits of bread, and taking in their surrounds. They might give a small sigh as they finish, dust off their hands, grab the mug and plate, and head back inside.

It's not possible in a game to have the sort of nuanced control that would be required for the player to have control over those details of what that character did. The player has essentially zero control over such details.

This isn't to criticise the games or game developers, as there are good reasons why games are like that. It's just to note that the character exists in the game world in a pretty abstracted fashion.

Here's some other ways that the character exists in an abstracted way. Consider a JRPG where the party of characters is going on an adventure through some lands towards a far-off place. In real life, such journeying would affect their physical and mental state. They would get tired and sore. At times they might feel despondent at how much of the journey they have left. They would need to sleep. In most games these sorts of details are abstracted away (during the gameplay segments, at least).

And that's because abstracting away those sorts of details is better for the gameplay.

I want to have a name for the kinds of details that are abstracted away. We can think of acting. A good actor has nuanced control over their entire body, in terms of posture, movement, body language, vocal control, facial expressions, etc -- at every moment in time. The timing of these details is important. There is how they go about moving through and interacting with details in their environment. All of these details reflect the character's emotional state and their intentions.

Are their movements languorous, or skittish? How do they sit in a chair? What is their body language, and where are they facing, when they're talking to different people?

We can call all these details the "performance". The performance details are abstracted away when the player has control over a character.

These performance details are very important to the storytelling in movies, TV, etc. Their absence, while the player has control over a character, is a reason why it's difficult to have strong storytelling within the gameplay.

Interactive storytelling containing interactive gameplay segments and separate non-interactive, story-focused segments

A game might consist of sequence like:

  1) cutscene, 2) gameplay, 3) cutscene, 4) gameplay, ...

We can call each of the elements in the sequence a 'segment'.

Story-focused segments like cutscenes are non-interactive, and they can contribute to a strong narrative. Cutscenes are like a short animation or movie that plays.

Gameplay segments are interaction-focused and they have a loose narrative.

This basic structure, of some segments focused on gameplay, and others focused on non-interactive storytelling, is found in many different genres of game, such as point-and-click adventure games, JRPGs, Western RPGs, platform games, and first-person shooters.

In different game genres, the nature of the gameplay segments differ. In classic point-and-click games, they're primarily about looking at and manipulating objects, and talking to characters, to solve puzzles. In classic JRPGs or Western RPGs, the gameplay segments primarily involve exploring around and fighting monsters. In a classic platform game, they involve running and jumping, to avoid or take out enemies, and to traverse different platforms and gaps. In a classic first-person shooter, it's traversing through an environment, shooting enemies and trying to avoid getting shot.

In all these genres, a gameplay segment ends when the player does some particular thing, like solve a puzzle, defeat a boss enemy, or get to the end of a level. When the gameplay segment ends, a story segment starts.

Let's consider all this in the context of classic point-and-click adventure games.

In gameplay segments, the player can move their character around within their current location, and move to other locations. They can choose to look at different objects in the surrounds. They can interact with objects (e.g. turn on a tap), to use an object they have on another object (e.g. use a lighter on a pile of sticks, to create a fire). They may be able to talk to other characters and choose between a few different dialog options. They do these things to try to solve puzzles, and by doing so, advance the narrative.

The gameplay segments tend to have a very loose narrative. Under the player's control, their character walks about, looks at and interacts with various objects. They may go back and forth between locations, and repeatedly interact with the same objects, as the player tries to figure out a puzzle solution. The player may get stuck for tens of minutes or even hours, before they figure out how to solve a puzzle.

The sequence of all these events within a gameplay segment will likely form a very loose narrative. If these gameplay segments were made into a movie, they'd be very dull movies to watch (imagine the UI elements were removed, and the footage was given to someone to watch and they weren't told it was from a game).

(The gameplay segments in the other genres we've talked about -- JRPGs, Western RPGs, platform games, and first-person shooters -- all have very loose narratives, too. This isn't to say gameplay segments can't contribute anything to the storytelling. They can, for example, do a lot of world building or character building. In a future post, I'd like to look at some of the ways the gameplay segments can contribute to the storytelling. Still, I'd maintain that these gameplay segments have a loose narrative.)

While the non-interactive, story-focused segments can together form a strong narrative, that narrative gets "diluted" by the loose narrative in the gameplay segments that sit in between them. If the entire experience was just the story-focused segments, it could have a strong narrative. But because the entire experience also includes the gameplay segments, the narrative of the entire experience is loosened. And since the gameplay segments tend to comprise the majority of the time the player spends with the game, there's substantial loosening of the narrative.

There are other ways that the gameplay segments affect the narrative of the non-interactive, story-focused segments. The gameplay segments constrain the types of narratives that can work in the story-focused segments.

One, the overall plot has to be compatible with being repeatedly put on hold, for long periods of time, during the gameplay segments.

Consider point-and-click adventures again. In these, the reason the plot has to be put on hold during the gameplay segments is the following. In a gameplay segment, the player will usually need to complete one or more puzzles before the next story-focused segment (cutscene) is triggered. The player is usually given as much time as they need to finish those puzzle(s), and the state of the game world (and thus the story) needs to basically be put on hold until the player solves the puzzle(s). If it wasn't put on hold, it'd interfere with the player's ability to complete the puzzles.

A second way that gameplay segments constrain the overall plot is that the plot has to be compatible with the sorts of activities the player character undertakes during them. In a point-and-click adventure game, the character will usually be walking around, examining objects, using objects and combining objects together.

In principle, any sort of plot could be used in the story-focused segments. But, a lot of them wouldn't "work" very well with being put on hold for extended periods of time, or with the kinds of activities the player character is undertaking during the gameplay segments.

Think of it this way: the plot of many novels or movies wouldn't really work if they contained extended sections where the main character went about like a player character does a point-and-click adventure game.

And again, the same applies to all those other genres where you have gameplay segments sandwiched between non-interactive story-focused segments. Many stories wouldn't work very well if the majority of the time in the game is spent in gameplay segments where a character is running and jumping, like in a classic platform game, or if the character is shooting down hordes of bad guys like in a classic first-person shooter.

 

To summarise, the gameplay segments loosen the overall narrative of such games, and they won't be compatible with various kinds of plots.  

I think the best way to think of those games, from a storytelling perspective, is that the gameplay is the main focus, and the storytelling is used to enhance the experience of the gameplay. That is, as opposed to if the storytelling was the main focus, and the gameplay was used to enhance the storytelling.

And, if we're interested in interactive storytelling, that raises the question of, what ways can the gameplay enhance the experience of the storytelling?

Interactive storytelling: choice & narratives, and separating narratives from interaction

There's a curious phenomenon, regarding different people's views of interactive storytelling. There's quite a few works whose storytelling gets high praise from gamers. Yet it seems common, when non-gamers try some interactive storytelling, for them to be not very impressed with the storytelling, and to consider it to be lesser than what they’d find in a good movie or novel.

What's the explanation of these contrary views?

Here's some potential explanations. Maybe non-gamers are just not familiar-enough with interactive storytelling. Maybe they're biased against interactive storytelling because it's less familiar to them. Or perhaps games are (in general) targeting younger audiences, who aren’t as interested in mature storytelling. Or maybe gamers are used to a lower-quality of storytelling in games, so have lower expectations for them.

 

I think there's at least a small element of truth in all of these, but I think the main explanation is something else.

To help explain, I'll introduce the notions of 'strong' and 'loose' narratives. That was the topic of my last post. Since the distinction is central to this post, I'll quote the entire contents of that previous post here:

In a strong narrative, everything that happens in the sequence of events have a purpose that contributes to the overall details of the story. The overall story is there to create a satisfying experience for the viewer, reader, or game player (from here on I'll just use 'viewer').

The overall story details will involve elements like the following. There will be challenges put in front of the protagonists, that they will overcome. The tension may ratchet up, until the climax of the story, and there will be a resolution. The protagonist will likely experience some growth. There may be certain themes. The overall story may present unexpected surprises. There will likely be mysteries that are raised earlier in the story, that get answered later on, which serve draw the viewer (etc) through the story.

I'm not saying that all of these have to be present for the narrative to be strong. I'm saying that a strong narrative supports the overall story details, whatever they are for a particular story.

A strong narrative is a set of events, put in a deliberate sequence, in order to produce a strong set of the overall story details (a strong climax, a satisfying resolution, etc). Events that occur build on what came earlier. A strong narrative won't include superfluous events, that don't help contribute to the overall story details.

A problem for the protagonist may arise in one scene, and the events in the next scene may exacerbate those problems. Earlier scenes may have painted a certain picture of the protagonist's predicament, and a later scene may turn those appearances on their head. The sequence of events will be constructed to make the story's mysteries compelling.

In strong narratives, "the whole is greater than the sum of its parts", because the parts are designed to contribute to the overall details of the story.

In loose narratives, on the other hand, there may be little or no overall story details for the sequence of events to contribute to. It's just: this happened, then this other thing happened, then something else, and so on.

The events will likely be connected -- perhaps by occurring in temporal sequence, following what happens to a protagonist, like what they did and where they went. But there won't be much to the overall story details, and thus not much for the sequence of events to contribute towards.

Everyday life is like a loose narrative. A sequence of stuff happens, but there's not the dramatic arc and so forth found in a strong narrative.

Of course, this isn't really a binary of only "very strong" narratives and "very loose" narratives. It's a spectrum, with stronger narratives at one end and looser narratives at the other end.

I believe that, generally speaking, people tend to prefer stronger narratives. There's always exceptions of course, but I think that in the majority of cases, if all else is equal, people will find a stronger narrative more compelling than a looser narrative. And if people find strong narratives more compelling, then in that sense I think it is fair to say that, generally speaking, a stronger narrative is a better narrative.

And this doesn't seem to be merely a matter of people's personal preferences or opinions. Evolution seems to have given us brains that desire and enjoy strong narratives. Brains that are suited to consuming information in the form of strong narratives.

 

With this distinction between strong and loose narratives, I think I can explain what’s going on with the divergent views about how good the narratives are in interactive storytelling.

To help illustrate, I’ll focus on Inkle’s “80 Days” (Wikipedia | Steam). It’s a souped-up “choose your own adventure”, that re-imagines Jules Vern’s “Around the World in Eighty Days”. The player's goal is to get around the world in 80 days.

It's a highly-rated story-based game. Its writing has been praised, and its storytelling has been lauded. The Guardian said it was "Interactive storytelling as its best". The game won the Independent Games Festival’s "Excellence in Narrative" award.

It’s worth noting that the game provides the player with a lot of choice. When the player has stopped at a destination, they have several choices for what to do while they are there. This might include buying items available there, or selling (hopefully at a profit) items they had previously bought elsewhere. They can talk to other characters, and make choices at various stages in the conversation.

When they are ready to leave, they have several transport options, to different destinations. And some will be faster than others, but might be more expensive. So the player needs to manage their funds, to ensure they get around the world in time but also don’t run out of money.

(Disclaimer: it has been several years since I have played it, so I apologise for any inaccuracies in my description of how it works).

To get a sense of the degree of choice, “The developers estimate that on one complete circumnavigation of the globe players will see approximately 2% of the game's 750,000 words of textual content” (from the game’s Wikipedia entry).

80 Days has a fairly loose narrative. There are some overall details to the story, that are contributed to by events that happen. But a lot of what you’ll encounter in a playthrough is more just: this happened, then this other thing happened, then something else happened.

This is not because the developers and writer are unable to write a strong narrative! It’s because of all the choice and freedom the player has. I would guess that there are thousands of different (but interrelated) possible playthroughs. Some of these would only vary from each other in fairly minor ways, but other ones would vary from each other in fairly major ways. So lets imagine that there are a couple of hundred possible different playthroughs (that vary in non-trivial ways from each other). It’s just not feasible to write hundreds of different strong narratives for a single game.

It’s not simply that it’s difficult to construct that many different strong narratives. The fact that all of those different playthroughs are related to each other, and overlap to some extent, actually makes it even more challenging. It would be harder to construct X number of inter-related strong narratives than to simply write X completely-distinct strong narratives. This is because strong narratives don't "live close together" in "narrative space".

So I think this is the explanation for why non-gamers who might try out a game like 80 Days may come away not terribly impressed by the storytelling.

But then, why is it that a game like this is lauded by gamers? I think the distinction between strong and loose narratives can help us here.

 

Gamers like 80 Days as a combination of that storytelling with the interactivity.

What interactivity brings is that the player gets to feel, to some extent, like they have been on an adventure, with ups and downs, and some interesting happenings along the way. They might feel good about having made some good choices along the way. Since there is so much choice, they can feel that they truly shaped what happened in their adventure. There is also the puzzle-like element to the game of trying to find good routes while also managing your finances well, which some people will enjoy.

And while the narrative is fairly loose, that doesn’t mean it’s bad, nor does that mean it can't contain small pockets of stronger narrative scattered through it. Also, narrative is separate from the writing in the story. The writing in a story with a loose narrative can obviously still be good.

It’s the overall package that the gamers like.

But there’s a lack of precision in how they evaluate the work. They don’t distinguish precisely enough between the qualities of the overall package and the storytelling component of that.

If the overall package is great, and that overall package is “interactive storytelling”, then it seems that the storytelling must be great. But just because someone finds the overall package great, that doesn’t mean that all the components of that package are great, in absolute terms. If a person really likes interactivity and choice and freedom, they may still really like the overall package, even if the narrative isn’t terribly strong. In absolute terms, you have to compare the narrative to those in other media, such as the strong narratives such as in movies and novels.

Can you actually separate the storytelling from the interactivity? Not totally. But, in general, I think you can to a large extent. It depends on the nature of the interaction in the particular work. You can definitely separate out narrative from the interactivity. The interaction determines what happens, and thus what specific narrative the player experiences. The narrative is the result of the interactions, which aren't part of it.

Any particular screen of text that you see in the game will seem like what could appear in a novel with a strong narrative. So the lack of a strong narrative is far from obvious. A strong narrative is more “abstract”. It concerns much larger-scale details in the story than any individual passage of text. And unless you already have an appreciation of the distinction between strong and loose narratives, you won’t be well equipped to be aware of that component of the story. I would imagine that the majority of people playing games don’t have a clear idea of that distinction. This isn’t a criticism of them, just what I think is the case, without making any judgements of the people.

So I think the non-gamers are right about the storytelling in a lot of the interactive storytelling. The interactivity is used to give the player freedom and choice, which necessitates a looser narrative. And this is the case even in beloved interactive storytelling works, that are widely lauded for their storytelling.

A question raised by this is, is all interactivity at odds with having a strong narrative? That’s something I’d like to write about more in the future.