Interactive storytelling: Action-mirroring control schemes
A number of story-focused video games use what I'll call an "action-mirroring" control scheme.
When a player makes their game character walk or run in a particular direction by pushing the controller's left joystick in that direction, the action of walking in the direction is "mirrored" by an input of pushing the joystick to that direction.
In first-person games, the player's character looking upwards (an action) is performed by pushing the controller's right joystick upwards. The action of looking up is "mirrored" by an input of pushing the joystick upwards.
These are common examples of "action-mirroring" inputs. Such inputs try to embody something of the essential nature of the actions they're used to perform, like moving the joystick in the direction the character is to move in.
Some games -- usually story-focused ones -- use action-mirroring for more than just movement and looking around. They have an "action-mirroring control scheme". For example, in such games a sliding door that may be slid open to the left may be opened by pushing in the joystick to the left.
Here are some prominent examples of games that use action-mirroring control schemes.
The Quantic Dream (David Cage) games Heavy Rain (2010) and Detroit: Become Human (2018).
The Telltale Games games The Walking Dead (2012), The Wolf Among Us (2013), and Tales from the Borderlands (2014).
The Supermassive Games games Until Dawn (2015) and The Quarry (2022). And games in their Dark Pictures Anthology, such as Man of Medan (2019) and Little Hope (2020).
The Bloober Team game Observer (2017).
The Flavourworks games Erica (2019) and HUSH - Crane (2022).
The Mountains game Florence (2018).
Some more examples of action-mirroring inputs. A cupboard door in real life may be opened by pulling its handle out and to the right or left. A kitchen cutlery drawer may be opened by pulling it towards oneself. Milk added to some coffee may be stirred by lowering a teaspoon into the coffee mug and moving it in a circular motion. In these cases, the action involves movement along a particular path. That path may be traced out with a joystick by movement in a single direction, like for a door that slides open to the left. Or it may be a more complex path, like the circular path traced out to stir some coffee.
Typically, when the player needs to enter an action-mirroring input, the game will show the player a prompt, indicating what input they need to enter. And the player may need to first 'lock onto' an item before entering the input. E.g. they (while standing in front of the door, facing it), may need to then press a shoulder button on their controller to lock-onto the door, after which they can enter the action-mirroring input to open the door.
The movement paths (like the circular one for stirring the coffee) can be traced out using a variety of input devices, meaning that action-mirroring control schemes are compatible not just with controllers with joysticks, but also a keyboard and mouse, and touch screens. With a keyboard and mouse, the player can, for example, slide the mouse to the left to slide the door open to the left. On a touch screen, the player can swipe left to slide the door open to the left.
When compared to the actions they perform, action-mirroring inputs are major simplification. Pressing left on a joystick is a major simplification of opening a cupboard door. But, the idea is that they approximate the core nature of an action, while still being a fairly simple and easy form of input.
More examples of action-mirroring inputs.
The input for opening a cupboard door could be pushing the joystick upwards and then tracing an arc down towards the right. The upwards movement represents pulling the door outwards, and the arc towards the right represents pulling the door to the right.
For the side to side action of brushing teeth with a toothbrush, the player may need to input a repeated side-to-side motion (joystick repeatedly moved from side to side, like in Heavy Rain, or by sliding one's finger from side to side, like in the touch-screen version of Florence).
For when your character is shaving their face in Heavy Rain, and carefully shaving some spots, the player might have to move the joystick in the direction the character is going to shave. And, to mirror the care that the character is taking (to slowly and carefully move the razor, so as to not cut themselves), the player might have to very gently push the joystick in that direction.
To struggle, to get out of the grasp of a bad guy, might involve quickly tapping a button or, in touch-screen games, the screen. In this case, there's a more abstract relation between the action and the input. The input is intended to mirror the effort or struggle of getting out of the hold.
In Supermassive Game's horror games, there are times where your character is in danger, and you need to hide and keep quiet, so that an enemy doesn't notice you.
In Until Dawn (2015), the player needs to hold their controller completely still until the danger has passed (this makes use of the motion-sensors on modern controllers).
Whereas in The Quarry (2022), the player simply has to hold down a certain button until the danger has passed. During this time their character is holding their breath. The player also needs to release the button as soon as the danger has passed, so they can take a breath. If they hold it too long, they'll gasp for breath and alert the enemy to their presence.
They implemented such scenes in yet another way in their Dark Pictures Anthology games like Man of Medan (2019) and Little Hope (2020), where it's like a little rhythm game, where the player needs to tap a button in time with their character's heartbeats.
With these inputs there's an abstract relation between the action and the input. The input is intended to mirror the task of trying to stay still and make no noise to avoid detection.
In HUSH - Crane (2022) which is on touch-screen devices, there's a moment where your character is removing the camisole worn by another character (Note: this is safe for work, as there's no nudity involved). To slide the straps down, the player needs to put a finger on each of the tops of the camisole shoulder straps, and slide their fingers down from there, along the upper arms of the other character. This is an example of where more than one action-mirroring input may be entered at once. An analogous input could be achieved using a controller, where the player has to press a button to lock-onto the target object then press down on both joysticks at once (in this case, the joysticks effectively represent the player character's hands).
Here we'll look more at how action-mirroring control schemes can differ between games.
Direct- and indirect- manipulation of inputs
Say the player has to enter a 'left' input, to open a door that swings out to the left. In some games, the player enters that input, after which the door is opened. This is an indirect-manipulation input.
In other games, the door will swing to the left as the player continues to hold the left input (and swing back to the right if they hold the right input). The player may need to hold the left input down for about a second in total for the door to become fully opened. If the player were to only hold left down for half that time, the door would only end up being opened half way. This is a direct-manipulation input.
This latter form of input is more immersive. It feels more like directly manipulating the object (the door). At the same time, it is more involved for the player, which means that it can get tedious if the player ends up having to open many doors this way.
Another example is, does the player rotate the joystick in a circular motion, after which the coffee gets stirred, or does the coffee get stirred as the player is entering the circular input?
Where on a touch-screen the player has to enter the input
Consider the side-to-side motion the player may need to enter to brush their character's teeth. Can they place their finger anywhere on the screen to enter this input, or do they need to place their finger within a particular area of the screen? As an example of the latter option, in Florence there's a rectangular "zone" at the bottom of the screen for entering the side-to-side input.
Sometimes, the player may need to place their finger on the object they are to interact with. For example, to stir a cup of coffee, the player might be presented with a top-down view of the cup, and may have to trace circles around the circumference of its contents to do the stirring.
Challenge added to action-mirroring
If the games wishes to make some action-mirroring more challenging to enter, such as during a fight scene, it might provide only a small time-window for entering each mirroring input. Or it could require an action-mirroring input, plus additional button presses. Heavy Rain is an example of a game that does this.
How effective are action-mirroring control schemes?
Addressing this question is the main purpose of this post.
These control schemes are there to add immersion to the game experience. To make the player feel, to some extent, as if they are performing the actions themselves. Of course, no such control schemes can make the player feel like they're actually performing the actions. These control schemes aren't trying to do that. They're just trying to add a, perhaps small, feeling that the input is somewhat like the action.
Action-mirroring control schemes seem promising, and the only way to see how effective they are was to implement them in games. Having played several of the games that use an action-mirroring control scheme, I've come to the view that they're, mostly, not that effective.
Some cases of them can be fairly effective. Consider how in Observer the player opens doors. The player needs to be near the door and then press a shoulder button on their controller to 'lock onto' the door, then move the joystick to the left (or right, depending on which side the door opens to). As they continue to push the joystick to the side, the door continues to open, as opposed to the player entering the input to open the door, and then afterwards the door opening. This feels immersive. Locking onto the door feels like grabbing its handle, and it opening up more as you continue hold the joystick to the side makes it feel like you're directly manipulating the door.
In Until Dawn, I found it immersive to have to hold the controller completely still, in order for your character to hide and not be noticed by an enemy.
Action-mirroring inputs may add immersion, but this can come at the expense of it requiring more time and effort to perform interactions. In a game that does not have an action-mirroring control-scheme game, it may be a simple button-press to open a door. Whereas in a game with an action-mirroring control scheme, the player may have to lock-onto the door, then hold the joystick to the left until the door fully opens. If the player was only opening a door once, this wouldn't make any real difference, but if the player is repeatedly having to open doors like this throughout the game, then the extra time and effort adds up. Opening doors that way may end up getting a bit tedious.
And though action-mirroring inputs may be immersive, the actions being mirrored tend to be fairly menial ones, of manipulating things in the physical environment. Like opening doors. So their immersion tends to be immersion in menial tasks. The immersion isn't particularly meaningful.
Action-mirroring inputs mostly don't feel immersive. Tracing out certain patterns with the joystick or with your finger on a touch-screen, just doesn't feel that much like performing the action in the real world. And, to the extent that they are immersive, it's in menial tasks, so it isn't meaningful immersion.
And, there's the added time and effort of entering those inputs, as compared to button presses, so they can get a bit tedious.
One reason they're not terribly immersive, is that in reality we tend to do tasks, especially menial tasks like opening doors, on autopilot. We don't explicitly think "I'm going to do this action, and doing so requires using my hands in this way". We just do it. We operate more on the level of our intentions, than on level of the physical character of our actions. We know we want to stir the coffee, and just do it. We don't think about how we're going to do it with a circular motion. We don't think of the mechanics of how we're going to do it. Those details are mostly subconscious. Having to explicitly follow a procedure to stir the coffee doesn't feel realistic.
Ideally, we'd like a control scheme that more closely matches how we perceive and undertake actions.
Interestingly, Quantic Dream's second-last game, Beyond: Two Souls, did use a control scheme that I think more closely matches how we perceive and undertake actions. And I think it worked quite well, so it's surprising to me that they did not keep using it for their most recent game (Detroit: Become Human).
In Beyond: Two Souls, interactable objects near the player are highlighted by a white dot. To interact with them, the player presses the right joystick in the direction of the white dot, relative to where the player character is. So if the object is to the left of the player character, the player presses left on the joystick to interact with it. Whereas if the player moves to the other side of the object, the player will need to press right on the joystick to interact with it. The reason it feels effective, to me, is that it feels more on the level of our intentions -- specifying which object to interact with.
(Two screenshots from "Beyond: Two Souls". The player needs to move the joystick in the direction the white dot is, in relation to where the player character is. E.g. in the first screenshot, they'll need to move the joystick to the left, to pick up the bag)
But a full discussion of how we can have a control scheme that's
more inline with how we perceive the world and act on our intentions,
is a topic for another post.
Finally, there is virtual reality (VR). VR can provide immersive interactions with the game world, that very much mirror the nature of real-world actions. For example, a player can reach out to a kitchen drawer in the game world, grasp its handle and pull it towards themselves, to slide it open. This is a topic I can't speak about in depth, as I don't have a VR headset yet, and have very limited experience in using VR.
No comments:
Post a Comment