Cowboy Programming Game Development and General Hacking by the Old West

May 30, 2008

Measuring Responsiveness in Video Games

Filed under: Game Development — Mick West @ 9:59 am

In this article I suggest that the specifications of a video game should always include a measure called “response time” (also called “lag”, “controller lag”, or “input latency”). Response time is defined as the time between the player using the controller, and the results appearing on the screen. Example: Pressing the trigger button on the controller fires a gun on the screen. Video game response time can be measured with a cheap digital camera, and I explain how.


The “feel” of a game is in large part described in terms of how “responsive” it is. Very often a game will be described as “laggy” or “sluggish”, and by contrast other games will be “tight” or “fast”. I have previously described the technical reasons behind games lacking responsiveness, but I offered no way of measuring the response time, and so the developers have to rely on their own assumptions about the way they read the controller and present the results, and combine that with the subjective assessments of the test department.

Having an accurate way of measuring response time allows the developer to both verify their own assumptions (hence detecting bugs that are adding to the response time), and to provide an objective reference to the claims of the testers regarding the “tightness” of the game. Perceptions of changes in small variables like response time can vary by individual, and being able to measure it objectively will allow you to see if it has actually changed, and by how much.

Game developers also have to make the decision of whether to go with 60fps or 30fps. 60fps will generally have half the response time of 30fps, which can be a deciding factor (along with the smoother motion, which is visually more appealing on fast moving games). However for some games there are other factors that influence the response time. Having an accurate way of measuring the response time allows the developer to more accurately and objectively make a decision on if 60fps is necessary, or if they simply need to tighten up their 30fps game.


Measuring response time is very simple, and consists merely of videoing the screen and the controller at the same time with a video camera that records at 60 fps, and then playing this back and counting the frames between button press and the screen response.

The camera I use is an old model Canon Powershot SD800IS, a relatively cheap camera. You need a camera that supports 60 fps (frames per second) recording. This may be listed as “Sports Mode”, or “Fast Frame Rate”. I believe many of the current line of Canon Powershot SD cameras have this feature, including the popular and cheap Powershot SD770. This is a very popular brand of camera, so it is highly likely that someone on your team will own one.

Set the camera to movie mode, change to to the appropriate mode for 60fps (press the “func” button on the Canon SD and select “Fast Frame Rate”).

Then set it up to point at the television. You don’t need to get the whole screen in shot, just enough so you can see the game actions that will result from your button presses. Make sure that the controller is roughly in focus and hold it angled so that the button presses can clearly be seen.


Plasma and LCD televisions introduce lag of their own, as they frequently do some additional processing to a picture before they display it. This can be as much as a few 60ths of a second, which can be quite significant. Ideally you would do your testing on a CRT Television, which should have no additional lag (the cheaper the better). Your flat panel TV might also have a “game mode” which will reduce (but not eliminate) the lag.

To find how much lag your TV adds, you need to compare an easily measured response with the same response on a CRT TV, or with a known value. To simplify things for you I’ve measured the response time of the PS3 system menus, which come out at a very solid 3/60ths on a CRT, and 5/60th on the plasma TV shown above. Hence I need to subtract 2/60th of a second to get an accurate measure of responsiveness when using the plasma for measurements. Make the appropriate adjustment for your TV.

This additional lag, however, should not be ignored, and it should be taken into account when developing the game. Many people will run your game on a TV that introduces additional lag. Steps should be taken to minimize the negative effects this unavoidable lag can have.

To measure the response time (lag) the simplest tool I found has been the free Quicktime viewer. With Quicktime installed, right click on the video file from your camera, and then click on “Open with Quicktime”. You can then use the arrow keys to frame advance 1/60th at a time. (On a Mac, it works best to use Quicktime Player 7, which can be found in the Utilities folder)

For a particular response time, navigate the video to the first frame that shows the button fully pressed, and then count additional frames from there until you see the first indication of a response on screen. The number of frames you counted is your response time in 1/60ths of a second. You can convert to milliseconds by dividing by 60 and multiplying by 1000 (12/60ths is 200 ms)

Here are the frames for the PS3 system menu on the CRT TV:

In this first image (above), the thumb is descending to the “down” button on the D-Pad. It’s important to press the buttons with the thumb starting off the pad and use a rapid hitting motion, so you can see the precise frame when the button is fully pressed. Note the slight motion blur on the thumb – this is also a useful indicator of if it is still moving or not.

Now the thumb is clearly stopped and is pressing the button. Since this is the first frame where we see the button pressed, then we start counting from here, at zero (if the response were visible here, you’d have zero lag, impossible with current systems).

Now (above) we are on the first frame of actual lag, note the same line on the menu is still highlighted.

Second frame of lag, still no change

Third frame. The menu begins to move. This means that the PS3 system menu has a response time of 3/60ths of a second, or 50 ms (milliseconds). This is VERY GOOD, and is the fastest you can realistically expect from the PS3. While it’s theoretically possible to go to 2/60ths, this has severe performance problems. I don’t think 1/60th is possible on the PS3. So – all games running at 60 fps should shoot for 3/60ths as an ideal response time. This also means games running at 30 fps should shoot for 6/60ths. That’s in a perfect world, and as we shall see, actual results vary.


The first thing I tried was shooting a gun in GTA-IV for the PS3 on a plasma television. This is done with the right trigger button (R2), which feels (slightly) like pulling the trigger of a real gun. In real life, there is zero delay between the trigger reaching the end of it’s travel, and the bullet leaving the gun. In GTA-IV, this delay is somewhat longer. Here’s the actual movie, so you can frame advance it yourself, and here are the relevant frames:

Note on the first frame (-1) , the finger is still moving, we start counting on the next frame (0) when the finger is fully on the button and the button is fully depressed. We then count until the first response, which comes at frame 12. This indicates a response time of 12/60ths. Since it’s measured on the plama TV, we adjust this to 10/60ths. This gives us a raw response time for GTA-IV of 166 ms (200 ms on flat panel TVs).

I repeated this test for turning and jumping, and got the same results. This is rather a long response time, and correlates with people reporting the game being sluggish and unresponsive. The delay in firing the gun after pulling the trigger is quite noticeable.


I tested various other games, with various results. I’ll list the results in 60ths of a second, and the results have been adjusted -2 to account for the plasma lag. All games are on the PS3 unless noted otherwise, and I’ve included the PS3 system menus and GTA for reference.

Games that run at 60 fps:

  • PS3 System menus: 3/60ths
  • Guitar Hero 3 (XBox 360): 3/60th
  • Ridge Racer 7: 4/60ths
  • Virtua Tennis: 4/60ths
  • Ninja Gaiden Sigma: 4/60ths
  • PixelJunk Racers 4/60ths

Games that run at 30 fps:

  • Genji: days of the Blade: 6/60ths
  • Tony Hawk’s Proving Ground: 8/60ths
  • Blacksite: Area51: 8/60ths
  • Halo 3 (XBox 360) : 8-10/60ths
  • EA’s “Skate”: 10/60ths
  • GTA-IV: 10/60ths
  • Harry Potter: 10-14/60ths
  • Heavenly Sword: 7-18/60ths

The first question that arises is: why are there no PS3 games with a response time of 3/60ths? The PS3 UI runs at 3/60ths response time, so it’s quite possible. Why do all the 60fps games run at 4/60ths? Still, that’s still pretty good, and those games all feel quite responsive. Guitar Hero on the XBox manages a very impressive 3/60ths – very important for that type of game, especially when it’s actually 5/60ths on many TVs.

Next, why so much variance between the 30fps games – ranging from 6/60ths to 18/60ths?

It’s worth noting that while EA’s Skate seems like it would be a bit sluggish at 10/60ths, it actually feels quite responsive, in large part due to its use of stick gestures for input, as the movement starts before the gesture is complete, yet the player mentally synchronizes it with the end of the gesture, and so it actually seems very responsive.

Then what about the games where the lag varies? Harry Potter is 10-14 which is bad enough, but Heavenly Sword is an astonishing 7-18. It takes 7/60ths to start an attack, but 18/60ths to start to turn around (See movie: mvi_4263w). Clearly something is wrong there. I would consider that a bug. It’s sad that the programmer made the effort that allowed for a 7/60th response time, but then someone else messed up down the line, making the turn take nearly a third of a second. Halo 3 is another example, with the shooting and moving being 8, but the jumping being 10.

Particularly interesting here in the 30fps category is Genji: days of the blade. This is a very similar game to Ninja Gaiden, and yet Genji runs at 30, while Ninja Gaiden runs at 60. However, the fact that Genji runs at 30 is barely noticeable, and does not detract from the game at all. In part this is because of the way the camera moves smoothly through the scene with very few rapid pans. It’s also because of the low contrast graphics and motion blur. But it’s also because Genji’s response rate is 6/60ths, very similar to Ninja Gaiden’s 4/60ths.

Ninja Gaiden is a faster paced game than Genji, with the main character jumping around rapidly, and the camera tightly following him – so the graphical benefits of 60fps are more apparent. The low response time feels good as well. However, since Genji also has a low response time, it would benefit very little from running at 60 fps. As it runs at 30fps, this gives the designers the opportunity to put more graphics, enemies and special effects on screen, and reduces the pressure on programmers and artists to constantly strive to maintain 60, which can be a difficult factor in development.


Games that run at 60fps all seem to have a response time to 4/60ths, and while 3/60ths is possible, 4/60ths is a very good response time.

Some games running at 30fps have a response time of 8/60ths or 10/60ths (and some peak even higher). Genji shows us that a response time of 6/60ths is possible while running at 30 fps. 10/60ths can be too long, especially when combined with the processing delays in flat panel TVs which can push it up to 12/60ths. 1/5th of a second (200ms) is too long to wait for a gun to fire, and introduces annoying sluggishness when moving around or steering a car.

Some games have an inconsistent response time. Heavenly Sword varies from 7 to 18. If the system is capable of 7, then all moves should start in 7. Developers should verify ALL their response times, as other factors, such as animation, might be creating lag in specific places.

I suggest that game developers use this simple technique to measure the response time in their games, at least to verify that their assumptions are correct. If they are running at 60fps, then they should not be above 4/60ths. If they are running at 30fps, then they strive to duplicate Genji’s responsiveness of 6/60ths, and certainly not slip below 8/60ths. I suggest that they keep in mind that flat panel TVs (which are probably a majority in gamers households, and certainly in game reviewers’ households) add an additional 2 frames of lag, which makes it EVEN MORE important to keep programmed lag to a minimum.

I also suggest that game reviewers begin to use this technique to measure lag, and to include the measurement of lag in their reviews. While the subjective views of the reviewer are important and valuable, an objective measurement of response time would be a very useful additional piece of information for the person considering buying the game. Placing this information in a game review would encourage developers to produce more responsive games, which benefits everyone.

[Update Oct 2009] See this article that discusses some more refined techniques for measuring controller la:


  1. I bet that the lag in games such as Heavenly Sword, GTA IV, and Harry Potter could be due to the team putting emphasis on smooth transitions between animations. It’s tough to balance responsiveness vs. an acceptable amount of animation popping. In a game with a “cinematic” intent, I’m guessing people err on the side of the animation looking good, to the detriment of gameplay. I think a team should prioritize a good feel, and that a good look should be achieved after the feel. Too often game art starts in previz and doesn’t get rebuilt when the engine comes online.

    Comment by Nick — May 31, 2008 @ 11:59 am

  2. Certainly animations transition are responsible for some lag. However, in all the tests I did here, the lag is from either the rest post, or the aiming pose – so there should be no transition needed.

    Plus, even if there WAS a transition, say from the resting animation to a turning animation, there would still be SOME movement. In the turning in Heavenly sword, I hit the stick, and then there is no movement from anything for 20 frames (1/3 of a second), then the character and the camera start to move. This is a consistent figure, so it’s not just waiting for some idle animation to finish.

    And in GTA, there’s no need for a transition to fire a gun. I think in that case there are just effectively two additional game loops in the pipeline (taking it from 6 to 10/60th). As I explained in my other article.

    Comment by Mick West — May 31, 2008 @ 12:12 pm

  3. Yes, I actually read this whole thing and learned a lot! Interestingly enough John and I just had a lag problem on Guitar Hero II on our PS2, so I was interested by this topic.

    I agree that including lag in a review should be mandatory for game publications. I hope you shopped this out for publication. What a great topic!

    John’s getting an XBox 360 Elite for Father’s day. Any tips on games I should get him? It comes with Halo 3.

    Comment by Antoinette — June 3, 2008 @ 12:23 pm

  4. Hmm… i’m sticking with standard def tv’s when it comes to my action games… some of those delays on 30fps games are atrocious!

    The heavenly sword example is strange, the motion lag can be attributed to momentum? There frame rate is far from 30 most of the time, i’d say 20 on average, quite the slide show that game…

    I agree with some actual FACTS in game reviews will be great… i mean what the hell does silky smooth 30 frames mean anyway???

    Comment by 60Hz — June 10, 2008 @ 9:46 am

  5. Even if there was momentum, there would be SOME movement. As it is, after moving the stick, the character remains rock solid for a full 18/60ths, and then begins a slight movement, which quickly speeds up. Seems like a bug to me.

    Comment by Mick West — June 10, 2008 @ 9:59 am

  6. I’ve added a movie of Heavenly Sword, so you can count the frames yourself:

    Comment by Mick West — June 10, 2008 @ 10:12 am

  7. I hear its because the PS3 is a quarter the power of the 360.

    Comment by james — June 26, 2008 @ 9:50 am

  8. It would be interesting to see if there is any difference in any games between the PS3 and 360 versions of the same game, like GTA, (assuming they are running at the same frame rate). It’s possible that one version (probably the PS3) might have an extra pipeline stage that adds lag.

    Purely speculation at this point though.

    Comment by Mick West — June 26, 2008 @ 9:56 am

  9. “The ‘feel’ of a game is in large part described in terms of how ‘responsive’ it is.”

    Yes. Very true. This responsiveness truly determines the quality of the game as we perceive.

    Do you think there will be any difference in games between PS3 and 360?

    Comment by Manix — June 29, 2008 @ 11:14 pm

  10. The PS3 and 360 have similar amounts of power, but it’s arranged differently on each system. So the same game can run differently on each system, sometimes being at 30 on one (usually the PS3) and 60 on the other. Or a game might maintain a more steady 30 on one system, and drop (more) on the other. All things being equal, this would obviously result in a drop in responsiveness.

    It’s also possible that different engines might introduce an extra game frame of lag in the processing -> rendering pipeline. This might vary by platform.

    However, there is no inherent reason why games on one platform will always be less responsive than on another.

    Comment by Mick West — June 30, 2008 @ 5:55 am

  11. Oh..ok Mick.

    Anyway, thanks for this really informative article. I’ve already faved your website. :)

    Comment by Manix — July 2, 2008 @ 3:46 am

  12. I think it is absurd that you expect game reviews to do something useful rather than twitter on like the frustrated writers they are.

    Comment by Kriss — December 5, 2008 @ 5:36 am

  13. […] To read the original article that inspired the customer to have me create this, click here. […]

    Pingback by The Official Site of Benjamin J Heckendorn — December 5, 2008 @ 11:33 am

  14. Nice experiment ! I’m going to try it with a processional camera tomorrow at work – we have a xbox 360 and a bunch o cameras there.
    Thanks for the great read. Stumbled !

    Comment by Games Now — January 7, 2009 @ 1:52 pm

  15. Thanks for explanation Mick West.

    Comment by biodiesel forum — January 16, 2009 @ 3:15 pm

  16. Slowly but surely the top LCD and Plasma Televisions are getting to grips with combining TV PIctures and gaming technology. Something that should have been straight forward to do for some reason always got held back. Interesting article

    Comment by Best Flat Screen TV — March 25, 2009 @ 2:12 am

  17. I think the software developers are not taking the Playstation 3 to the limit that is why most of games don’t reach 60 frames per a second. Also maybe the games that run on 1080p have a better chance of hitting 60 frames per a second.

    Comment by James — June 13, 2009 @ 2:08 am

  18. I agree that including lag in a review should be mandatory for game publications. I hope you shopped this out for publication. What a great topic

    Comment by Pimp My Nintendo — August 2, 2009 @ 8:38 am

  19. Learn how to play a guitar today. Lots of video tutorials and guides…

    Cowboy Programming » Measuring Responsiveness in Video Games…

    Trackback by Learn how to play a guitar today. Lots of video tutorials and guides — August 7, 2011 @ 11:30 pm

  20. […] West wrote a great article on the causes of lag in games, followed up by another one in how to measure it. I’m going to apply some of that to the lag I was experiencing in my […]

    Pingback by Lag: The Bane Of Touch Screens | Games from Within — August 23, 2011 @ 7:55 pm

RSS feed for comments on this post. TrackBack URL

Leave a comment

You must be logged in to post a comment.

Powered by WordPress