Cowboy Programming Game Development and General Hacking by the Old West

May 27, 2008

Programming Responsiveness

Filed under: Game Development — Mick West @ 1:53 pm

[Update: This is a technical article on the causes of lag.  If you want the article on measuring lag, then click here]

Responsiveness is something that can make or break a game at first impression. This is especially true in reviews where a game with poor responsiveness will be described as being “sluggish”, “unresponsive”, “floaty” or “sloppy”. A better game might be referred to as “tight” or “responsive”. There are several factors that contribute to perceived responsiveness. This article looks at some of them from a programmer’s perspective, and offers some routes to making your game more responsive.

RESPONSE LAG

Response lag is the delay between the player triggering an event, and the player getting feedback (usually visual) that the event has occurred. If the delay is too long, then the game will feel unresponsive. Several factors contribute to the length of this response lag. If your game is unresponsive, it may well be the cumulative effects of four or five different factors. Adjusting one factor alone may not make a perceptible difference, but addressing all the factors can lead to a noticeable improvement.

Players, and sometime even designers, cannot always put into words what they feel is wrong with a particular game’s controls. Often they will try to do something that requires some synchronization, but will fail, and they won’t be able to tell you “the event happened 0.10 seconds after my input”, instead they will just tell you that the game felt “slow” or “not tight”, or “difficult”. Or they might not be able to tell you anything, and simply say the game sucked, without really understanding why it sucked. Designers and programmers need to be aware of response lag, and the negative effect it has on a game, even if test players do not directly report it as a factor.

WHY LAG HAPPENS

To understand why lag occurs, you need to understand the sequence of events that occur from the time the user presses a button, to when the results appear on screen. To understand this, we need to look at the main loop structure of the game. The main loop performs two basic tasks: logic and rendering. The “logic” portion of a main loop updates the game state (the internal representation of the game objects and environment), while the “rendering” portion of the loop creates a frame that is displayed on the television. At some point in the main loop (usually at the start) we also get input from the user. This is sometimes considered a third task in the main loop, but it’s also commonly a part of the logic task. I’ve kept it separate here because it’s important to see in what order things happen

LISTING 1 – THE SIMPLEST MAIN LOOP

while (1) {
  Input();
  Logic();
  Rendering();
}

There are several ways a main loop can be structured. The simplest is shown in listing 1, where we simply alternate calling the logic and the rendering code. It is assumed there is some frame synchronization in the call to Rendering(), and that we are running at a fixed frame rate, usually 60 or 30 frames per second for an NTSC console game.

The main loop here is also only showing half the story. The call to Rendering() is doing the CPU side of the rendering task, which is iterating over the environment and the objects, culling, animating, sorting, setting up transforms, and building a display list for the GPU to execute. The actual GPU rendering is performed after the CPU rendering, and usually is asynchronous, so while the main loop is processing the next frame, the GPU is still rendering the previous frame.

So where does the lag come in? Look at the sequence of events that occur from the user pressing a button to there being some feedback for pressing that button. A the highest level, the user presses a button, the game logic reads that button press and updates the game state, the CPU render function sets up a frame with this new game state, then the GPU renders it, and finally this new frame is displayed on the television.

Figure 1 shows this in a more graphical manner. Sometime in frame 1, the player presses a button to fire his gun. Since the input processing has already been done for that frame, this input is read in frame 2. Frame 2 updates the logic state based on this button press (a shot is fired). Also in frame 2, the CPU side of rendering is performed with this new logic state. Then on frame 3, the GPU performs the actual rendering of this new logic state. Finally at the start of frame 4, the newly rendered frame is presented to the user by flipping the frame buffers.

So how long is the lag? It depends on how long a frame is (where a “frame” here is a complete iteration of the main loop). It takes up to three frames for the users input to be translated into visual feedback. So if we are running at 30 frames per second, then the lag is 3/30th or 1/10th of a second. If we are running at 60 frames per second, then the lag will be 3/60th or 1/20th of a second.

This illustrates a common misconception about the difference between 60 fps and 30fps games. Since the difference between these two frame rates is just 1/60th of a second, people assume that the difference in responsiveness will also be 1/60th. In actual fact, going from 60 to 30 does not just add a vsync to your lag, it acts as a multiplier, doubling the length of the process pipeline that is responsible for lag. In our ideal example in figure 1, it adds 3/60ths of a second, not 1/60th. If the event pipeline is longer, which it quite possible can be, then it can add even more.

Figure 1 actually illustrates the best possible sequence of events. The button press event is translated into visual feedback via the shortest path possible. We know this because we can clearly see the sequence of events. As a programmer, being familiar with the order in which things happen is a vital part of understanding why thing act the way they do in the game. It quite easy to introduce additional frames of lag (meaning an extra 1/60th or 1/30th of a second delay), by not paying careful attention to the order in which things happen.

As a simple example, consider what would happen if we switched the order of the Logic() and Rendering() calls in our main loop. Look at frame 2 of figure 1, although the input is being read at the start of frame 2, and affects the logic during frame 2, the rendering is performed first, and so is using the logic state from frame 1. This introduces an extra frame of lag. While this is a novice level mistake, it is still something you need to make absolutely sure is not happening.

Extra frames of lag can be introduced in a more subtle manner as a result of the order of operations within the game logic. In out example we are firing a gun. Now perhaps our engine is set up to increment the position of the objects in the world using a physics engine, and then handle events that are raised due to this update (such as collision events). So in this situation the sequence of input/logic looks like listing 2.

LISTING 2 – PHYSICS UPDATE IS FOLLOWED BY EVENT HANDLING

void Logic() {
  HandleInput();
  UpdatePhysics();
  HandleEvents();
}

Now event handling via messages is a very nice way of decoupling systems. So a programmer might decide to use it for the player control events. To fire a gun, the HandleInput() function will fire an event telling the gun to fire. The HandleEvents() function will take this event and cause the gun to actually fire. But because the physics update has already happened for this frame, the effect on the world state will not be incorporated until the next frame, hence introducing an extra frame of lag.

MORE LAG CAUSES

The lag can be extended even further by lower level action ordering. Consider a “jump” – the feedback comes when the character actually moves. To make something move in a game, you can either set the velocity directly, or you apply a force to it, either acceleration or, more likely, a momentary impulse. The problem arises here if in your physics engine then positions are updated before the velocity change is applied (a common situation in many introductory game programming tutorials). In this case, although the velocity of the jumping object will be set on the same frame as the input event is handled, the object will not actually begin to move until the next time around the loop, on the next game frame, and so introduces an additional frame of lag.

Remember these are cumulative problems that can be difficult to discern in isolation, but the combined effect can make your game controls turn to mush. Suppose you had made all three mistakes listed above: you do rendering before logic, you handle logic events after advancing the physics state, and you update position before velocity. That’s three additional full iteration of the main loop, in addition to the three you already have built in. Six frames of lag between the player pressing a button, and seeing the result on screen. At 60 frames per second that’s 1/10th of a second, which is bad enough, but if the game is running at 30 frames per second, then the lag is DOUBLED to an unbearable 1/5th of a second, or 200ms.

Other factors can contribute to lag, compounding even further the effects above. Movement can be driven by animations, with velocity changes built into specific time points in an animation. The obvious problem here is if the animator places the jump velocity impulse a fraction of a second into the animation, to better match the visuals. It might look better, but it feels bad. That’s easily dealt with by making sure the velocity impulse is on the first frame of animation when immediate feedback is needed. But then the question is: how does triggering an animation translate into actual movement? It’s quite likely that animation updating is handled by the Render() function, and so any events triggered by the animation will not be handled until the time around the loop, which adds another frame. In addition, triggering an animation might not make it advance a frame until the next frame, delaying the event firing for a frame. Our lag could potentially be increased from six to eight frames, which would be quite unplayable, even at 60 frames per second.

That’s not the end of it either. There are many ways in which extra frames of lag can be introduced. You might be pipelining your physics on a separate thread (or a physics processing unit). You might be using triple buffering to smooth your frame rate. You could be using abstract events that take a couple of passes through the system to resolve into real events. You could be using a scripting language that adds an additional frame in the way it waits for an event. It’s quite possible to make you game logic incredibly flexible by abstracting various concepts of time and events, and yet while doing this you loose sight of exactly what is going on under the hood, making it far easier for additional frames of delay to creep in.

RESPONSIVENESS IS NOT REACTION TIME

On of the great misconceptions regarding responsiveness is that it is somehow connected to human reaction time. Humans cannot physically react to a visual stimulus and then move their fingers in less than one tenth of a second, and peak reaction times vary from 0.15 seconds to 0.30 seconds, depending on how “twitchy” the gamer is. Figures such as these are often brought up in discussion of game responsiveness, but the connection is specious.

It’s not how fast you react to the game that is the issue; it’s how fast the game reacts to you. The issue is not one of reaction times, but of synchronization. In games such as Guitar Hero there are things coming towards you, and you have to hit the correct button at a very precise point in time, when the target object is within a particular region. The player is anticipating the future event, and there is no reaction involved at all. The problems of lack of responsiveness occur when the game does not react fast enough to the player, and the target object has moved beyond the target region by the time the event occurs. The player presses the button at the correct point, and they do not expect the object to move even a few more pixels before it explodes. But since objects generally move at least a few pixels per frame, having a few frames of lag can allow the object to drift past its target.

Many action games are based around this anticipation and button pressing. In a skateboarding game you want to jump just before you hit the end of a rail. In a shooter, you want to shoot the instant someone moves in front of your gun. Again this is not reaction time, you will usually have seen the target at least half a second before you shoot it, probably more, and will be either moving the gun, or waiting for the target to move in front of the gun.

Because of the somewhat unintuitive nature of these problems, in order to create a responsive game, it is important that you fully understand the issues involved. The most important thing is to be able to clearly describe the frame by frame path though your logic and rendering that the action a button being pressed will take in order to create visual feedback. Once you have this, you can optimize it as close to the optimal pathway as possible.

17 Comments »

  1. Did you upload this from a Mac or something? There are lots of weird formatting problems with this article.

    Comment by Bryan O'Sullivan — May 27, 2008 @ 3:32 pm

  2. … Should be fixed now.

    Comment by Mick West — May 27, 2008 @ 4:10 pm

  3. A game with simple logic and heavy rendering can have the read input and logic phases be much shorter than gpu rendering and placed immediately before the rendering of a frame starts.
    would save you ~ a frame

    Comment by Yair — May 27, 2008 @ 7:03 pm

  4. Indeed, and that’s the way that old (very old) console games used to be, everything was done in the same frame. Some theorize this is why the older console games seemed more responsive.

    Comment by Mick West — May 27, 2008 @ 7:06 pm

  5. I wonder if you could elaborate on the differences between joystick input and keyboard input, and accommodating both.

    For instance, I’ve got a little game I’ve been working on that’s sort of like the old arcade game, Defender — a side scrolling space shooter — with the joystick (logitech usb gamepad is what I’ve tested with) it’s works very smoothly indeed, because I can map the analog stick position quickly and nicely to movements.

    keypress/release events are a little trickier to deal with, as they (as I’ve implemented things anyway) amount to adjustments to velocities — sometimes you want slow, precise movements, sometimes quick movements. With the keyboard, as you press a key, it starts out slow, and gets faster, as you hold down the key. Releasing a key, things slow down…. it’s ok. But with the joystick, having that -32767 – 32767 analog value, you get a much better reading on what the player’s intentions are. So the joystick is just tons better to play with.

    Any tricks to making keyboard input work better? Or is keyboard input just not worth talking about for some things?

    — steve

    Comment by SteveC — May 27, 2008 @ 8:19 pm

  6. I’m assuming you are not talking about the standard key-repeat rate here – if so, you are reading the keyboard wrong, and should look at, for example, my “Pushing Buttons” code.

    https://cowboyprogramming.com/2007/01/02/pushhing-buttons/

    Basically, you have a key that is either pressed or not. When it is pressed, you accelerate in some manner, and when it is not pressed, you decelerate in some other manner.

    Remember all the great games that came out before analog sticks were around? They were all basically button pressing games – no different from keyboard input. It’s all in how you handle the acceleration.

    It’s not a simple thing to get right. There are several cases you have to handle. For example if you are moving and change direction, then you use a different acceleration than if you are stationary and start moving. You’ll also want some hard limits on velocity in addition to friction.

    Look at some existing games, and try to duplicate their behavior.

    Comment by Mick West — May 27, 2008 @ 8:35 pm

  7. “I’m assuming you are not talking about the standard key-repeat rate here…”

    No, I figured that much out (not on the first try, but the 2nd try.)

    “For example if you are moving and change direction, then you use a different acceleration than if you are stationary and start moving.”

    I think I did something along those lines (will have to go back and look and see what I did.)

    “You’ll also want some hard limits on velocity in addition to friction.”

    Yeah, got that. Somewhat arbitrarily chosen values, hard to figure, except by trial and error, I guess.

    My suspicion is that that the keyboard can never be as good as the joystick, simply because it provides less information. What few testers I’ve had have told me I’ve made improvements in the keyboard, but it’s hard to know if you’ve maxed out the “goodness” of the keyboard response.

    Comment by SteveC — May 27, 2008 @ 8:41 pm

  8. There’s a lot of room for tweaking – basically changing things based on context. Always be asking “what is the player’s intent here”, and try to accommodate that as quickly as possible.

    Take the immediate past button pressing history into account when deciding what to do. If they press left, then it can do a different style of velocity modification if they were pressing right withing the last 0.5 seconds.

    Comment by Mick West — May 28, 2008 @ 6:39 am

  9. […] Programming Responsiveness […]

    Pingback by Cowboy Programming » Measuring Responsiveness in Video Games — May 30, 2008 @ 9:59 am

  10. Wow, this is really neat. I’ve thought a lot about this type of thing myself, when writing out a simple display engine… It would always seem that the external console would display “button pressed” a few moments before the game responded. After much trial an error, it turned out to be similar to one of the approaches you’ve written about above.

    Have you put any thought into decoupling the input management from the framerate? I did something like that with the physics of the thing I was using, and it gave a much more responsive feeling game. http://gafferongames.wordpress.com/game-physics/fix-your-timestep/ (an article about physics timesteps.)

    The only step further would be to do input handling in a thread, but that gets a bit complicated.

    Comment by Ape-Inago — November 30, 2008 @ 6:55 pm

  11. I’ve been mostly a console programmer, so framerates are generally fixed at 30 or 60. Decoupling is not generally a good thing in that situation. Especially if it can introduce extra latency. If you have a fixed framerate, it’s generally better to run the physics in lockstep or a fixed multiple (or fraction) of the display frame rate.

    Comment by Mick West — November 30, 2008 @ 9:17 pm

  12. […] OnLive tiene que solventar es el lag entre que pulsas una tecla y ves la reacción. En el blog de Mick West hay un diagrama que muestra la secuencia de eventos que transcurre entre que pulsas el boton […]

    Pingback by Simfoony » Blog Archive » A propósito de OnLive — March 31, 2009 @ 2:02 pm

  13. […] lag in order to defeat it. While browsing I found this great article to explain all of it. To understand why lag occurs, you need to understand the sequence of events that occur […] Mick […]

    Pingback by Understanding Lag « c# to javascript, actionscript — July 20, 2009 @ 12:27 am

  14. The phrase, “in actual fact” made me giggle.

    Great article, Mick.

    Comment by EvaUnit02 — September 23, 2009 @ 2:49 pm

  15. […] Programming Responsiveness – “Responsiveness is something that can make or break a game at first impression. This is especially true in reviews where a game with poor responsiveness will be described as being “sluggish”, “unresponsive”, “floaty” or “sloppy”. A better game might be referred to as “tight” or “responsive”. There are several factors that contribute to perceived responsiveness. This article looks at some of them from a programmer’s perspective, and offers some routes to making your game more responsive…” […]

    Pingback by Interesting Reading… – The Blogs at HowStuffWorks — January 11, 2010 @ 10:54 am

  16. […] West wrote a great article on the causes of lag in games, followed up by another one in how to measure it. I’m going to apply some of that to the lag […]

    Pingback by Games from Within | Lag: The Bane Of Touch Screens — September 16, 2010 @ 12:39 pm

  17. Hey Mick.

    First of all, I would have to say thanks for this well written explanation on lag. – You have painted a very clear image with screenshots and even codes.

    I have also always thought that responsiveness has to do with reaction time.

    Having said that, what do you feel is the best way to bridge the lag time?

    Mark Derulo
    Wichita, KS

    Comment by Mark J. Derulo — April 20, 2011 @ 2:26 am

RSS feed for comments on this post. TrackBack URL

Leave a comment

You must be logged in to post a comment.

Powered by WordPress