Interactive storytelling: why character control is at odds with movie-like storytelling

This post considers why it is that, during gameplay segments of video games, when the player has control over their character (like of their movements, and what they interact with), it's not possible to have storytelling like found in movies and tv. Of course it's possible to have such storytelling in these games, such as by using cutscenes -- just not within the gameplay segments. Why?

The answer to this may seem fairly obvious. Nonetheless, I have found pinpointing it quite difficult.


The player can't make their character act.

A major component of visual storytelling like in tv and movies is the 'acting' component. This refers to both live-action performances as well as the performances of animated characters. Acting includes things like movement, posture, gesture, facial expression, vocal performance.

Here we're not concerned with other parts of visual storytelling, like cinematography (lenses, camera positioning and movement, shot composition, lighting, etc), editing (selecting, ordering, and timing shots, etc), sound effects, and music. Just the acting component.

Acting's arguably the core of visual storytelling. Those other elements (cinematography, editing, sound effects, and music) can be mostly or fully stripped away, leaving just the acting -- as is found in theater.

At each moment of a scene, each character has very specific, deliberate acting performance details (movement, or lack thereof, their posture, gesture, facial expression, vocal performance, etc). It's through these that their actions are conveyed. They give the character psychological and behavioral verisimilitude. They reflect the characters intentions and mental state.

But when the player has control over a character (like of their movements, and what they interact with), they don't have the fine-grained control that would allow control of all those acting performance details. It's just not practical with a controller, or keyboard and mouse.

Because the player can't control those acting performance details, such details are effectively absent during gameplay. In typical games, when the player is controlling a character, that character always stands and moves in the exact same way. They effectively have no body language or facial expressions. Their eyes are always looking straight ahead. It's like the performance details have been abstracted away. The character's actions reflect "neutral" intentions and a "neutral" emotional/mental state.

So as far as storytelling goes, having control over a character means there's a very limited "palette" to work with. When there's full control over a character, it's not possible to implement the typical scenes you'd find in a movie.


Strong storytelling requires that there's no control.

Here I'm specifically concerned with control like over a character's movements and what they interact with.

Imagine a game trying to have movie-like scenes but where the player controls their character.

Either all the details of the scene and story (including potential branching paths) are worked out in advanced, or the player is given more freedom and the game can simulate how the NPCs and game world to react to what the player does.

If the scene and story details, including the moment-to-moment acting performance details, are worked out in advance, the player has no freedom to control the character's movements and actions (other than some branching choices). There's a rigid 'performance' the player character has to follow. Which makes the player having control over the character's movements and actions incompatible with such strong storytelling.

What if the player had control over their character and the game simulated the NPCs and games world's responses to the players actions?

At present we don't have the tech to do this. Not where the NPCs respond with verisimilitude.

If the player had the usual sort of control over their character -- which strips away most of the performance details reflecting the emotional and mental state of the character -- that makes it difficult for the game to respond to. Imagine a movie scenes where one of the characters had all their acting performance details stripped away, and could only act in the way a video game character under player control can.

The player would need to be able to control the acting performance. Perhaps this could be done with some sort of sophisticated body tracking technology, that also tracks facial expressions. Future AI may be able to simulate suitable responses to the player's actions. That could be an interesting and enjoyable experience. However, from a storytelling perspective, it wouldn't lead to good stories. One reason is that strong narratives don't live near each other in narrative space. That is, in the space of all possible narratives, most of them are weak. The strong ones are spread out. Changing any particular narrative (to one near it in narrative space) usually weakens the narrative.

Also, if the player has such freedom, much of what they could do wouldn't have verisimilitude for their character and the situation that character finds themselves in. Consider the character being in a situation where they would be scared of something, and would act accordingly (e.g. scared of standing up to a bully, or asking a girl out). Because the player is experiencing a fictional world, there's no reason for them to be scared (or scared much), so they could play the character in an unrealistically confident way.

A player could try to use their control to play their character with verisimilitude, and the game world could realistically respond to that. This could be quite interesting and enjoyable. But for the reasons I mentioned above, I don't think it will lead to strong storytelling.


How do games deal with the inability to have strong storytelling while the player is controlling a character?

Cutscenes are one option -- simply remove player control for the duration of the strong storytelling. These may allow for basic interactivity, in the form of QTEs and choices.

There are some middle grounds, where the storytelling isn't as strong as in a movie scene, but where there is some interactivity.

There are various kinds of environmental storytelling. Like if in a post apocalyptic story the player explores an empty house and finds two skeletons in an embrace in bed, with an empty bottle of pills next to them. Or letters the player can pick up and read, audiologs, and what I've called audiovisual-logs (like in Tacoma).

In a typical movie scene the viewer sees the story details unfold. This is 'direct storytelling'. Elsewhere I've called this 'visual action'. Most environmental storytelling, on the other hand, involves 'indirect storytelling'. It's indirect because the player doesn't see the story details unfold. Rather they're given information that tells them about some story details that (usually) happened earlier (and possibly elsewhere) -- like with audiologs. (The one exception is audiovisual logs). This gets around the issue of control and acting performance details, because in indirect storytelling there are no acting performance details. I've argued elsewhere that such storytelling is a weaker form of narrative.

There's also during-gameplay cutscenes, such as found in Half Life 2, the Metro games, and Dishonored. In these, NPCs act out a scene around the player.

With both environmental storytelling and during-gameplay cutscenes, the player still has control. With environmental storytelling, the player explores the environment to find things like audiologs, and they can continue moving around while listening to an audiolog. Or with like the audiovisual logs in Tacoma, the player can play, pause, rewind and fast-forward the audiovisual logs, and they can move around while viewing them to get a better angle on the details of interest. While  during-gameplay cutscenes are playing out, the player may still have control over their character's movements, or at least over where their character is looking.

However, in both environmental storytelling and during-gameplay cutscenes, while the player has some control, they can't use it to participate in the storytelling. Audiologs, for example, just play out a recording of something that happened earlier. In during-gameplay cutscenes, the player is effectively an audience member watching the details play out. The player character can't have an acting role within those story details playing out. If the acting details of the PC matter at any point in that during-gameplay cutscene, the game has to switch to a non-interactive cutscene for the required details.

Another means of addressing the issue of control and storytelling is through structuring the game as an Activity & Choice game. The choices part is that the player is given dialog choices and other explicit choices about what to do. Sometimes these can be difficult choices. Regarding the activities, when the player has control during gameplay, it is within a specific part of, or scene within, the story. Life is Strange 2 and Pentiment are examples of such games. In the former, in an early part of the game the player's character is preparing to go out to a party that evening. Within this scene, the player is given control to explore around the house to get some items to bring to the party (like some snacks, drinks, and money). The player can control their character to move them around, to look at various objects, and to pick up the ones they want to take to the party. So the player can't control an acting performance, but they can play a 'scene' to some extent.

Since it's a particular story scene being represented, the details can all be focused on being relevant to that scene. It could be that, when getting party supplies, the player character walks around in a carefree way. They might think certain thoughts relevant to that situation, as they go about. What they say when the player looks at different things could be suited to what's going on at that particular point in the story.

It allows more of a story focus than in something like, say, a typical point-and-click adventure game, where the player usually has several puzzles open and thus are going about in what is not a very specific story scene, but still most scenes that you'd find in a movie can't be recreated during the interactive parts of Activity & Choice games, as that'd require the player to have control over the acting performance details.