I’ve had some good feedback from my Measuring Responsiveness in Video Game article, which explains how to use a cheap camera to measure the lag between pressing a button and the result appearing on screen.   One of the problems with the method I was using is that it does not let you measure responsiveness in typical gameplay situations, since you have to set up the controller so you can see the button being pressed, which means your finger or thumb has to start moving while not touching the button – generally not the case in normal gameplay.

The guys at Infinity Ward saw this problem, and their solution was to commission modder Benjamin Heckendorn to build a custom joystick that had a seperate display with an individual LED for each button.  Here’s the result:

With this setup, the board can just be placed next to the monitor, and both videoed together in normal gameplay.   Specific moves and events can then be isolated, and the frames counted in the video to measure the responsiveness. 

Frame-by-frame analysis of the button presses might also be useful for some analysis along the line of what I suggest in Pushing Buttons, although that’s still better done in code.

[UPDATE]  Ben now has a more detailed post on the controller on his site, with videos of it in action:

http://benheck.com/12-05-2008/controller-monitor-built-for-infinity-ward-game-testing