Cowboy Programming Game Development and General Hacking by the Old West

October 7, 2007

Get and Build Ruby 1.9 for Windows

Filed under: Cowboy Programming,Ruby — Mick West @ 1:29 pm

Today’s “programming” exercise: get the source for Ruby 1.9, and build a working version under Visual Studio, just like I did for version 1.8.

Step 1: Search Google, which gives me this starting point:

http://www.megasolutions.net/ruby/Help-building-Ruby-1_9-on-Windows-50037.aspx

> I am looking for some advice on building Ruby 1.9 on windows.

Here’s what I did on XP SP2 running under Parallels.

– Download and install MSFT Visual C++ 2005 Express Edition (gratis)
– Get the Windows Platform SDK
– Install Ruby 1.8.x using 1-click Windows Installer
– Grab Ruby 1.9 from Subversion

– Click through: Start Menu / Platform SDK / Open Build Environment
Window / Windows XP 32-bit … / Set Windows XP … Build Environment
– In the command window that opens, run “c:\Program Files\Microsoft
Visual Studio 8\Common7\Tools\vsvars32.bat”
– win32/configure.bat
– nmake

Result:

C:\dev\ruby\win32>ruby -v
ruby 1.9.0 (2007-04-12 patchlevel 0) [i386-mswin32_80]

Now I’ve already done the first three steps, so my first stumbling point is:

Grab Ruby 1.9 from Subversion

I don’t have Subversion installed, and I’ve never used it before. I’ve been using Perforce for years, and I use the free version of that for my home projects. So, I go to: http://subversion.tigris.org/project_packages.html, which is rather intimidating for the poor Windows user. All I want is the client, but there are a range of download options, none of which look trivial. One mentions a “friendly installer program”, so I click on that, and phew, a nicer list of install packages, I choose Subversion 1.4.5, download and run, accepting all defaults.

So, I feel like I should be able to go to the command line and type something in to get the Ruby 1.9 source, and with a few seconds of browsing, I find the correct instructions at http://www.ruby-lang.org/en/community/ruby-core/ :

svn co http://svn.ruby-lang.org/repos/ruby/trunk ruby

Imagine my surprise when this actually worked! Download seems a little slow, but it’s chugging away, downloading rather a lot of stuff (most of which I’m sure I do not need). It took about five minutes, and finally informed me it had checked out version 13655.

I’d run the above command line inside a directory \ruby-1.9, which then gives me the source tree in \ruby-1.9\ruby\, so I open a visual studio command prompt and do:
cd \ruby-1.9
md \build
cd build
\ruby-1.9\ruby\win32\configure

which gives me:
Creating Makefile
confargs.c
type `"C:\Program Files\Microsoft Visual Studio 8\VC\BIN\nmake.exe"' to make ruby for mswin32.

So I type “nmake”, and sadly after a nice start (several files are compiled) I get a useful error:
bison -o y.tab.c /ruby-1.9/ruby/parse.y
'bison' is not recognized as an internal or external command,
operable program or batch file.
NMAKE : fatal error U1077: 'bison' : return code '0x1'
Stop.

Hmm, I don’t remember having to have bison installed for 1.8. Bison is the GNU parser generator, which takes a .y file (the grammar definition) and makes a .c file (the parser). It seems like a simple case of “missing tools” from the “why won’t it build” list. But I’m a little confusled that 1.8 actually has a file parse.c. If I try nmake parse.c in the 1.8 tree I get “‘byacc’ is not recognized ….”, which shows they have switched tools from byacc (Berkeley Yacc) to Bison (both the same kind of thing). But that’s not the problem, as I don’t have byacc either. Nope, the problem is that parse.c is not part of the subversion repository, which is very reasonable for a development source tree, as it’s dependent on parse.y. Unfortunately I now have to install Bison. I’m beginning to fear I might have to install Cygwin, eek!

So, I download the Bison 2.1 Windows install package and run it -which seems to do everything except set the path, so I add C:\Program Files\GnuWin32\bin to my PATH variable, restart my Visual Studio 2005 command prompt, an re-run nmake, and shucks if there is not another error:

bison -o y.tab.c /ruby-1.9/ruby/parse.y
sed -e "s!^ *extern char \*getenv();!/* & */!;s/^\(#.*\)y\.tab/\1parse/"
y.tab.c > parse.c
'sed' is not recognized as an internal or external command, operable program or batch file.
NMAKE : fatal error U1077: 'sed' : return code '0x1'

Great, another GNU tool that is missing. Luckily it seems like all these tools are available under GnuWin32, so I can simple download the Sed installer, and it goes in the same bin directory as above, and like magic my build continues. Until I get to the next error – actually many errors, in the link stage for miniruby.exe there are a load of unresolved externals like _rb_id2name, and _rb_intern
, which are labels defined in parse.y/parse.c.

So, I look at parse.c in \ruby-1.9\build, and it’s zero bytes! It seems like even though the build failed with the missing sed.exe, it still made a zero length file, which make nmake think it’s all up to date, and a zero length file is a valid C program, so it compiles. I delete parse.c, and re-run nmake, and it correctly rebuilds parse.c and continues building, and it works!

C:\ruby-1.9\build>ruby -v
ruby 1.9.0 (2007-10-07 patchlevel 0) [i386-mswin32_80]

Okay, now I’ve got it built and running from the command line, how about building it from Visual Studio, like I did with 1.8.6. So I’m going to follow my own instructions, and see what is different, specifically in how miniruby.exe is built.

First of all, there are a couple of extra libraries, shell32.lib and ws2_32.lib, and one that has been removed: winsock32.lib. Seems like those two replace this one. Ws2_32 is windows sockets version 2 (replacing winsock). That’s all simple enough, but we’ll get to that later.

The main library is msvcr80-ruby19-static.lib, which is built with:

lib -nologo -machine:x86 -out:msvcr80-ruby19-static.lib array.obj ascii.obj bignum.obj class.obj compar.obj dir.obj dln.obj encoding.obj enum.obj enumerator.obj error.obj euc_jp.obj eval.obj eval_load.obj proc.obj file.obj gc.obj hash.obj inits.obj io.obj marshal.obj math.obj numeric.obj object.obj pack.obj parse.obj process.obj prec.obj random.obj range.obj re.obj regcomp.obj regenc.obj regerror.obj regexec.obj regparse.obj regsyntax.obj ruby.obj signal.obj sjis.obj sprintf.obj st.obj string.obj struct.obj time.obj unicode.obj utf8.obj util.obj variable.obj version.obj blockinlining.obj compile.obj debug.obj iseq.obj vm.obj vm_dump.obj thread.obj cont.obj id.obj prelude.obj acosh.obj crypt.obj erf.obj strlcpy.obj strlcat.obj win32.obj dmyext.obj

Well, that looks like a whole load of new stuff (marked in bold), some stuff for character encoding, a new regular expression library split over several modules, and a new virtual machine and associated compile and debug stuff. So where is all this stuff? Well, it looks like it’s all in the root again. Let’s see if we can knock up a project roughly following my old instructions:

Create a win32 console project, no precompiled headers, add oldnames.lib user32.lib advapi32.lib shell32.lib ws2_32.lib, add all the .c files in the root (ruby\) except dmydnl.c, add acosh.c, crypt.c, erf.c and (now) strlcat.c and strlcpy.c from ruby\missing.

Now where’s config.h? In 1.8.6 a file config.h was created in the build directory, but does not seem to be there now. Instead there’s one in \ruby-1.9\build\.ext\include\i386-mswin32_80\ruby\config.h, so I’ll just copy that over.

(NEW) Add the include path: c:\ruby-1.9\ruby\include

Now we still can’t find config.h, as it’s included as “ruby/config.h”, meaning it’s looking in c:\ruby-1.9\ruby\include\ruby\config.h, but we don’t want to put anything there, as that’s our raw source. So I just put it in \ruby-1.9\mickruby\mickruby\ruby\config.h, and added \ruby-1.9\mickruby\mickruby to the include paths, and it’s compiling away.

Okay, just four compile errors now, seems like it can’t find “insns.inc”, which is a file generated in \build. It looks like an enum of the YARV vm op-codes, and says it’s generated by insns2vm.rb from template/insns.inc.tmpl. It seems a bit odd that a generated file ends up in the build directory after a configure. What if you add a new VM instruction? Anyway, copy it (and insns_info.inc) into \ruby-1.9\mickruby\mickruby and then we are missing optinsn.inc, node_name.inc, vm.inc, vmtc.inc, so i just copy all 8 .inc files from build to mickruby\mickruby, and now we just have one error:

Error 255 error C2099: initializer is not a constant c:\ruby-1.9\ruby\regenc.c 32
OnigEncoding OnigEncDefaultCharEncoding = ONIG_ENCODING_INIT_DEFAULT;

Which reminds me, I was supposed to change the character set to “not set”, and add some preprocessor defns. It seems like there is unicode support now, so I’m not so sure about the character set thing. But I’ll give it a go …. nope.

So what’s with this error? ONIG_ENCODING_INIT_DEFAULT should be ONIG_ENCODING_ASCII, which should be (&OnigEncodingASCII), which is a structure defined in ascii.c. Now I’m tempted to blame Visual C here, but it compile fine from the command line, so instead I’m blaming myself, and something that I did. Looks like something to do with character encoding. But then my computer start acting weird, and visual studio tells me it’s out of memory. So I reboot, and the error GOES AWAY. Huh?

No matter, I’m now faced with new errors, lots of unresolved external symbols, I think I’m missing win32\win32.c, yup, but then that’s missing dln.h, which is in \ruby, so add that to the include path (just for this file), so now I’m just missing all the parse.c stuff, which is now in build, but since I’m not going to change the language right now, I’m just going to add it from there. Slightly dodgy, if I’m ever going to change parse.y. Anyway, seems like it includes parse.y in some odd way, which needs #include “regenc.h” which is in the root, so added c:\ruby-1.9\ruby to the include paths, and now we have a new error:

Error 8714 error LNK2019: unresolved external symbol _Init_prelude referenced in function _rb_call_inits inits.obj

Init_prelude comes from \build\prelude.c, which I’ll add directly, and FINALLY….


C:\ruby-1.9\mickruby\debug>mickruby -v
ruby 1.9.0 (2007-10-07 patchlevel 0) [i386-mswin32_80]

What a palaver! And what a mess I made of it, shoehorning it in Visual Studio. Still, it’s up and running, with the various hacks documents above, so that’s a start if I want to make a more professional version of it later.

September 1, 2007

Compiling Ruby as a Visual Studio Solution

Filed under: Cowboy Programming,Ruby — Mick West @ 10:22 am

(This is Ruby 1.8.6, for Ruby 1.9, see here)

I decided to see what it would take to compile Ruby as a Visual Studio project. Now you could just include it as a makefile project, but I figure that doing this would give me a better understanding of exactly what is involved in a build of Ruby. Essentially I would want something that would compile miniruby.exe, so I could then use that as a base for a stripped down version I can incorporate it into a full project.

Miniruby.exe is linked from

  • main.obj (built from main.c)
  • dmydln.obj (built from dmydln.c)
  • msvcr80-ruby18-static.lib

And the following windows system libraries

  • oldnames.lib
  • user32.lib
  • advapi32.lib
  • wsock32.lib

So obviously the big one here is msvcr80-ruby18-static.lib, which is built from:

array.obj bignum.obj class.obj compar.obj dir.obj dln.obj enum.obj error.obj eval.obj file.obj gc.obj hash.obj inits.obj io.obj marshal.obj math.obj numeric.obj object.obj pack.obj parse.obj process.obj prec.obj random.obj range.obj re.obj regex.obj ruby.obj signal.obj sprintf.obj st.obj string.obj struct.obj time.obj util.obj variable.obj version.obj acosh.obj crypt.obj erf.obj win32.obj dmyext.obj

These are mostly the platform independent files that live in the root source folder, there are three files at the end (acosh.obj crypt.obj erf.obj) that live in the \missing folder. Then there is win32.obj and dmyext.obj. Win32.obj simply comes from compiling /win32/win32.c, and dmyext.obj is from dmyext.c, which again is in the root (not sure why it’s not in with the rest of them).

So, in theory I should simply be able to compile all the constituent files of msvcr80-ruby18-static.lib, and then link in main.obj, dmydln.obj and the four windows libs, and I’ll be set.

Using Ruby 1.8.6 source, in c:\ruby-src\ruby-1.8.6, I started out by:

Create a Win32 console project, c:\ruby-src\mickruby. Do not use precompiled header. Add oldnames.lib;user32.lib;advapi32.lib;wsock32.lib to the additional dependancies in the project

Drag into the project (or just add) all the .c files from c:\ruby-src\ruby-1.8.6, except for lex.c, and dmydln.c (dmydln.c, just sets a flag and includes dln.c, I’m unsure why the normal build seems to include both)

Drag in (or just add) the extra files acosh.c, crypt.c and erf.c from c:\ruby-src\ruby-1.8.6\missing

Drag in (or just add) win32.c from c:\ruby-src\ruby-1.8.6\win32

Add the include paths c:\ruby-src\ruby-1.8.6 and c:\ruby-src\ruby-1.8.6\missing

You now need an appropriate config.h, which I temporarily copied from a “configured” \build directory, into the root source folder. This kind of assumes you were building Ruby from the command line, which involves running the batch file c:\ruby-src\ruby-1.8.6\win32\configure.bat from the build folder. Or, more explictly: md c:\ruby-src\build then cd c:\ruby-src\build then c:\ruby-src\ruby-1.8.6\win32\configure.bat (you’ll need to type the full path). This will create the file config.h in c:\ruby-src\build, and you can just copy it into c:\ruby-src\mickruby\

Change configuration\general\character set to “not set”

Add the preprocessor definitions:

  • RUBY_EXPORT
  • _CRT_SECURE_NO_DEPRECATE
  • _CRT_NONSTDC_NO_DEPRECATE

In the project entry point file (mickruby.cpp), just comment out all the source code for now. We want to use main.c as the project entry point for now. (You could also obvious copy over the code from main.c into mickruby.cpp, and exclude main.c from the project.
At this point, I can build the project, with 0 errors, and 4735 warnings, and IT WORKS, in release mode I get a 659K executable (mickruby.exe) that runs ruby code. The normal build gives me a 593K executable for miniruby.exe – probably a bunch of extra windows crap? Or maybe some options are different.

If I turn off link-time code generation, and twiddle a few options, the executable goes down to 602,112. Setting the optimizations as closely as possible to -O2b2xty-, gives us 598,016. Close enough.

Adding all this to my procedural tree sample takes the release build from 104K to 452K – that’s with nothing being called. If I add a call to ruby_init() in the initialization code, then it jumps up to 712K, which sounds exactly as would be expected from the above. That’s a chunk of code, half a meg, but in the grand scheme of things games typically need 512MB to run (XBox 360 and PS3 both have 512MB, PCs have that as a minimum). So while an extra 600K might have been crippling to a PS2 game, next-gen (which is really current-gen now) platforms are less impacted.

August 31, 2007

Why won’t it build?

Filed under: Cowboy Programming — Mick West @ 12:27 pm

It’s a timeless tale; you find some useful looking code sample on the internet, you download it, you try to build it and it does not build. This is such a common occurrence that non-building code samples seem to greatly outnumber correctly building code samples on the internet by a factor of ten to one.

So I was incredibly surprised when yesterday I attempted to build the programming language Ruby from scratch on my Vista Desktop. As was expected, it did not build, but the amount of fiddling I had to do was actually incredibly small – pretty all I had to do was figure out one cryptic line in the instructions: “Execute win32\configure.bat on your build directory”, and I was nearly good to go.

Then I tried the same thing on my XP laptop, and I got an error (“cl error 0x2”, which is not incredibly helpful), and I’m stuck (well, I was programming in bed, so I was not strongly motivated to figure it out).

NMAKE : fatal error U1077: ‘cl’ : return code ‘0x2’

But it got me thinking, why do hardly any code samples build? Well, in my experience it is always because something is either A) The Wrong Version or B) Missing. Problem B is generally the easier to solve, except in the rather unfortunately common case where something being missing means that another version is used, and it ends up being A – wrong.

1) Wrong or Missing Paths
Paths tell the tools where to look for things. If the paths are wrong, or missing, then the tools will look for things in the wrong place, and might find the wrong version, or they might find nothing. Unless your paths are correct, it does not matter what else it correct. The order of paths is very important, as the build tools (or Windows, if just running a tool) will look through them in that order.

2) Wrong or Missing Tools
Tools build things in a particular way. If you have the wrong tools, then things might build differently, or not at all. If a tool is missing, then hopefully the build will just stop, but sadly it might just find another (older, newer, or just wrong) version of the tool, and use that.

3) Wrong or Missing source
No every sample includes all the source you need. You might need another bunch of code, perhaps DirectX, TK, or the Windows SDK. Not everyone tells you this. The code might even require a particular version. If it does not find what it is looking for, it might use nother version. This can depend on your paths, but the source you need (usually header files), might simply be missing.

4) Wrong or Missing Library
Likewise with libraries, it might not be linking because it’s the wrong version. Symbol not found? What exactly is being linked here?

5) Wrong or Missing Environment Variables
Paths are one specific example here, but you often get things like compile and link options in environment variable, and symbols may be defined, and various other build flags. Perhaps a batch file is included that sets these? Perhaps there is a text file that describes how you should set them manually.

So of these, what is going on with my laptop build of Ruby? I’d only got as far as trying to do the “configure” step when I got the error. Now in this case I’m rather luck in that I have a working version, and an non-working version, so I can hopefully just play “what’s the difference”. Let’s check things one at a time:

1) Paths: I build from the “Visual Studio 2005 Command Prompt” that is part of Visual C++ 2005 Express edition. This sets up various paths and variables for me. Any difference here? Well, I have a different version of the DirectX SDK, but it’s after the visual studio stuff which is all the same.

2) Tools, check the version numbers, usually you can just run the tool and it will tell you, or run it with /h, –help, -v, /v or somesuch option.
Version for cl.exe – (desktop) 14.00.50727.762, (laptop) 14.00.50727.42 So that’s different. looks like I have a slightly older version of Visual Studio on my Laptop. Try to install the latest version? (Hmm, looks like Visual Studio 2008 Betas are out – must resist the urge to try it…). 200MB of downloads and two dubious restarts later and nothing has changed.

At this point I’m suspecting some configuration problem, based on scant clues from the internet, so I go straight for the environment variable, and realize I missed off the paths that are set in the environment variable, so I check those (just run SET in the command prompt you are using to build) and……

Laptop has
INCLUDE=C:\Program Files\Microsoft Visual Studio 8\VC\INCLUDE;

Desktop has:
Include=C:\Program Files\Microsoft Visual Studio 8\VC\INCLUDE;C:\Program Files\Microsoft Platform SDK\Include

It looks at first like the Microsoft Platform SDK is missing, but it turns out I had it. I just need to hard wire in the missing path to INCLUDE (and another for LIB) and it worked!

(Note you’ll get the same error if you simply did not have Microsoft Platform SDK installed, make sure you have that first. )

You know, it would be very helpful if the ruby configuration batch file (win32\configure.bat) had attempted to diagnose some common problems, instead of simply letting it run until a cryptic error. And why are other errors being suppressed? There would probably have been a more useful error, like “file xxxxxx not found”.

February 22, 2007

What is invalid parameter noinfo and how do I get rid of it?

Filed under: Cowboy Programming — Mick West @ 6:40 pm

_invalid_parameter_noinfo shows up as a problem most often as an “Unresolved external symbol” when you get some mix up between DEBUG and RELEASE modes. But it’s also a performance problem in situations where it’s compiled in unnecessarily.

So what is it? Well, it’s basically the function that gets called when the Microsoft Visual C++ compiled code detects a type of run-time error, such as a vector index out of range, or incrementing an iterator past the end of a container.

There are two functions: _invalid_parameter, which gets called in debug mode, and _invalid_parameter_noinfo, which gets called in non-debug mode. If you have a module compiled without _DEBUG defined, and you link with the debug library, then you will get the “Unresolved external symbol” error. This could be a problem with inconsistent #includes. See this thread on devmaster.net for some practical experiences.

What is _invalid_parameter_noinfo then? Well, first ask what _invalid_parameter is. _invalid_parameter is basically an assert. When an error is detected, _invalid_parameter is called, and is passed debug information, like the file name and line number pointers. It then calls _crt_debugger_hook and _invoke_watson. In non-debug mode (release mode), the debug info is not available, so invalid_parameter_noinfo simply calls invalid_parameter with NULL parameters. It’s actually an optimization, saving having the NULL parameters passed from every bit of your code, instead you code “just” needs to call this one function

You might also be having problems with _invalid_parameter_noinfo if you have code that crashes (on this function) on in release mode. That’s most likely some form of release-only bug, such as uninitialized memory, and the call to _invalid_parameter_noinfo is the end result. DO NOT IGNORE IT, or try to work around it. You need to find out exactly why it is being called.

But suppose your code works fine, and _invalid_parameter_noinfo is never called, you might be peeking through the disassembly, trying to figure out why it is so slow, and you see all these calls to _invalid_parameter_noinfo

Consider this code:

void	CVerletPoint::SatisfyConstraints()
{

	vector::iterator i;
	for (i = mp_constraints.begin(); i != mp_constraints.end(); i++)
	{
		(*i)->Satisfy(this);
	}

}

A nice simple bit of code that just iterates over a vector of CVerletConstraint objects, and calls some function on each one. It compiles to this:

00405C20  push        ebx  
00405C21  push        ebp  
00405C22  push        esi  
00405C23  mov         ebp,ecx 
00405C25  mov         esi,dword ptr [ebp+20h] 
00405C28  cmp         esi,dword ptr [ebp+24h] 
00405C2B  push        edi  
00405C2C  lea         edi,[ebp+1Ch] 
00405C2F  jbe         CVerletPoint::SatisfyConstraints+16h (405C36h) 
00405C31  call        _invalid_parameter_noinfo (407A81h) 
00405C36  mov         ebx,dword ptr [edi+8] 
00405C39  cmp         dword ptr [edi+4],ebx 
00405C3C  jbe         CVerletPoint::SatisfyConstraints+23h (405C43h) 
00405C3E  call        _invalid_parameter_noinfo (407A81h) 
00405C43  test        edi,edi 
00405C45  je          CVerletPoint::SatisfyConstraints+2Bh (405C4Bh) 
00405C47  cmp         edi,edi 
00405C49  je          CVerletPoint::SatisfyConstraints+30h (405C50h) 
00405C4B  call        _invalid_parameter_noinfo (407A81h) 
00405C50  cmp         esi,ebx 
00405C52  je          CVerletPoint::SatisfyConstraints+5Fh (405C7Fh) 
00405C54  test        edi,edi 
00405C56  jne         CVerletPoint::SatisfyConstraints+3Dh (405C5Dh) 
00405C58  call        _invalid_parameter_noinfo (407A81h) 
00405C5D  cmp         esi,dword ptr [edi+8] 
00405C60  jb          CVerletPoint::SatisfyConstraints+47h (405C67h) 
00405C62  call        _invalid_parameter_noinfo (407A81h) 
00405C67  mov         ecx,dword ptr [esi] 
00405C69  mov         eax,dword ptr [ecx] 
00405C6B  mov         edx,dword ptr [eax] 
00405C6D  push        ebp  
00405C6E  call        edx  
00405C70  cmp         esi,dword ptr [edi+8] 
00405C73  jb          CVerletPoint::SatisfyConstraints+5Ah (405C7Ah) 
00405C75  call        _invalid_parameter_noinfo (407A81h) 
00405C7A  add         esi,4 
00405C7D  jmp         CVerletPoint::SatisfyConstraints+16h (405C36h) 
00405C7F  pop         edi  
00405C80  pop         esi  
00405C81  pop         ebp  
00405C82  pop         ebx  
00405C83  ret

Yikes! What exactly is going on there? Lots of run time error checking, that’s what. Why is it doing this? Well, it’s to make your code “secure”, it makes it so that if something goes out of bounds, then the program will halt, preventing it from doing any harm (or being exploited by a hacker), and allowing you to debug it.

Is this a good thing? It depend on what you want. If you are writing code with lots of convoluted data structures and container interactions, then maybe this is something you want. But for code that operates on a data structure that does not change from frame to frame, then this code is just getting in the way. If it works the first frame, it will work forever. In release mode I do not expect this kind of error checking, and it certainly look like it would hurt performance. It is tests that always return true, skipping over a function that is never called.

So, you can turn it off with:

#ifndef _DEBUG
#define _SECURE_SCL 0
#endif

You will need that either defined in every file, or have _SECURE_SCL defined as 0 in the release build process.

Effect: Our code from above now shrinks to:

00405AF0  push        esi  
00405AF1  push        edi  
00405AF2  mov         edi,ecx 
00405AF4  mov         esi,dword ptr [edi+20h] 
00405AF7  cmp         esi,dword ptr [edi+24h] 
00405AFA  je          CVerletPoint::SatisfyConstraints+21h (405B11h) 
00405AFC  lea         esp,[esp] 
00405B00  mov         ecx,dword ptr [esi] 
00405B02  mov         eax,dword ptr [ecx] 
00405B04  mov         edx,dword ptr [eax] 
00405B06  push        edi  
00405B07  call        edx  
00405B09  add         esi,4 
00405B0C  cmp         esi,dword ptr [edi+24h] 
00405B0F  jne         CVerletPoint::SatisfyConstraints+10h (405B00h) 
00405B11  pop         edi  
00405B12  pop         esi  
00405B13  ret

Much better. six tests have been eliminated. saving at least 12 lines of assembly code. And the big news, the framerate of my blob program goes from 150fps to 170fps.

Check here for an investigation of different ways of iterating over STL vectors:
http://einaros.blogspot.com/2007/02/iteration-techniques-put-on-benchmark.html

So, is turning off _SECURE_SCL a bad cowboy practice? Well for games I think it’s quite reasonable to switch it off for a “FINAL” build (i.e. the one you are going to release to the consumer). Leaving it on might be a useful debugging tool. Just be aware of the potential for significant performance degradation in instances like the one above. That kind of case might be ripe for some kind of refactoring with error checking that is only performed when the container changes.

February 14, 2007

Comments vs. Self Documenting Code

Filed under: Cowboy Programming,Game Development — Mick West @ 10:20 am

I was reading a thread over on GameDev about the use of comments. There were a variety of opinions, ranging from the extremes of “comment every line of code” to “code should be self documenting and have no comments”.

Personally I use quite a few comments in my code. Game code tends to involve the interaction of a large number of complex systems. Self documenting code is really only a reality if you are insanely familiar with the context of that code. That’s plausible for a period of time, on smaller projects. But for large game projects, especially console games, that often use code that has evolved over several years, and interface with several third-party middleware libraries, it’s really not practical.

What do real code gurus say about commenting code? Steve McConnell has a whole chapter on “Self Documenting Code” in Code Complete 2, which covers the cases for and against comments, concluding (page 817:

The question of whether to comment is a legitimate one. Done poorly, commenting is a waste of time and sometimes harmful. Done well, commenting is worthwhile

In The Pragmatic Programmer Hunt and Thomas say (page 249)

In general, comments should discuss why something is done, its purpose and its goal. The code already shows how it is done, so commenting on this is redundant.

In The C++ Programming language, 2nd ed., Bjarne Stroustrup (the creator of C++) says (page 139)

A well-chosen and well-written set of comments is an essential part of a good program. Writing good comments can be as difficult as writing the program itself

Jef Raskin, Creator of the Apple Mac project, says (here)

Do not believe any programmer, manager, or salesperson who claims that code can be self-documenting or automatically documented. It ain’t so. Good documentation includes background and decision information that cannot be derived from the code.

The question of “Comments vs. Self Documenting Code” is a false dichotomy. You want both. If your code is really so messed up that it needs lots of comments, then it’s smelly code, and may be ripe for refactoring. Obviously it’s a good thing to have meaningful identifiers in code, and to avoid stating the obvious in comments. But if you can clarify something with a comment, then you should.

Unfortunately dogma is something you find often in programmers, even in good programmers. “Goto considered harmful”, “premature optimization is evil”, “comments indicate sloppy code”, “pointers are evil”, “singletons are evil”, “multiple check-outs are evil”.

All these things: goto, early optimization, comments, pointers, etc., are useful tools. They can also be dangerous tools. They can be over-used, and mis-used by bad programmers. Hence simple minded programmers (and managers) rail against them, and impressionable programmers regurgitate the dogma without really understanding the reasons behind it.

Read Code Complete 2, it’s all you need to know (which is quite a lot).

Older Posts »

Powered by WordPress