Cowboy Programming Game Development and General Hacking by the Old West

Today’s “programming” exercise: get the source for Ruby 1.9, and build a working version under Visual Studio, just like I did for version 1.8.

Step 1: Search Google, which gives me this starting point:

http://www.megasolutions.net/ruby/Help-building-Ruby-1_9-on-Windows-50037.aspx

> I am looking for some advice on building Ruby 1.9 on windows.

Here’s what I did on XP SP2 running under Parallels.

– Download and install MSFT Visual C++ 2005 Express Edition (gratis)
– Get the Windows Platform SDK
– Install Ruby 1.8.x using 1-click Windows Installer
– Grab Ruby 1.9 from Subversion

– Click through: Start Menu / Platform SDK / Open Build Environment
Window / Windows XP 32-bit … / Set Windows XP … Build Environment
– In the command window that opens, run “c:\Program Files\Microsoft
Visual Studio 8\Common7\Tools\vsvars32.bat”
– win32/configure.bat
– nmake

Result:

C:\dev\ruby\win32>ruby -v
ruby 1.9.0 (2007-04-12 patchlevel 0) [i386-mswin32_80]

Now I’ve already done the first three steps, so my first stumbling point is:

Grab Ruby 1.9 from Subversion

I don’t have Subversion installed, and I’ve never used it before. I’ve been using Perforce for years, and I use the free version of that for my home projects. So, I go to: http://subversion.tigris.org/project_packages.html, which is rather intimidating for the poor Windows user. All I want is the client, but there are a range of download options, none of which look trivial. One mentions a “friendly installer program”, so I click on that, and phew, a nicer list of install packages, I choose Subversion 1.4.5, download and run, accepting all defaults.

So, I feel like I should be able to go to the command line and type something in to get the Ruby 1.9 source, and with a few seconds of browsing, I find the correct instructions at http://www.ruby-lang.org/en/community/ruby-core/ :

svn co http://svn.ruby-lang.org/repos/ruby/trunk ruby

Imagine my surprise when this actually worked! Download seems a little slow, but it’s chugging away, downloading rather a lot of stuff (most of which I’m sure I do not need). It took about five minutes, and finally informed me it had checked out version 13655.

I’d run the above command line inside a directory \ruby-1.9, which then gives me the source tree in \ruby-1.9\ruby\, so I open a visual studio command prompt and do:
cd \ruby-1.9
md \build
cd build
\ruby-1.9\ruby\win32\configure

which gives me:
Creating Makefile
confargs.c
type `"C:\Program Files\Microsoft Visual Studio 8\VC\BIN\nmake.exe"' to make ruby for mswin32.

So I type “nmake”, and sadly after a nice start (several files are compiled) I get a useful error:
bison -o y.tab.c /ruby-1.9/ruby/parse.y
'bison' is not recognized as an internal or external command,
operable program or batch file.
NMAKE : fatal error U1077: 'bison' : return code '0x1'
Stop.

Hmm, I don’t remember having to have bison installed for 1.8. Bison is the GNU parser generator, which takes a .y file (the grammar definition) and makes a .c file (the parser). It seems like a simple case of “missing tools” from the “why won’t it build” list. But I’m a little confusled that 1.8 actually has a file parse.c. If I try nmake parse.c in the 1.8 tree I get “‘byacc’ is not recognized ….”, which shows they have switched tools from byacc (Berkeley Yacc) to Bison (both the same kind of thing). But that’s not the problem, as I don’t have byacc either. Nope, the problem is that parse.c is not part of the subversion repository, which is very reasonable for a development source tree, as it’s dependent on parse.y. Unfortunately I now have to install Bison. I’m beginning to fear I might have to install Cygwin, eek!

So, I download the Bison 2.1 Windows install package and run it -which seems to do everything except set the path, so I add C:\Program Files\GnuWin32\bin to my PATH variable, restart my Visual Studio 2005 command prompt, an re-run nmake, and shucks if there is not another error:

bison -o y.tab.c /ruby-1.9/ruby/parse.y
sed -e "s!^ *extern char \*getenv();!/* & */!;s/^\(#.*\)y\.tab/\1parse/"
y.tab.c > parse.c
'sed' is not recognized as an internal or external command, operable program or batch file.
NMAKE : fatal error U1077: 'sed' : return code '0x1'


Great, another GNU tool that is missing. Luckily it seems like all these tools are available under GnuWin32, so I can simple download the Sed installer, and it goes in the same bin directory as above, and like magic my build continues. Until I get to the next error – actually many errors, in the link stage for miniruby.exe there are a load of unresolved externals like _rb_id2name, and _rb_intern
, which are labels defined in parse.y/parse.c.

So, I look at parse.c in \ruby-1.9\build, and it’s zero bytes! It seems like even though the build failed with the missing sed.exe, it still made a zero length file, which make nmake think it’s all up to date, and a zero length file is a valid C program, so it compiles. I delete parse.c, and re-run nmake, and it correctly rebuilds parse.c and continues building, and it works!

C:\ruby-1.9\build>ruby -v
ruby 1.9.0 (2007-10-07 patchlevel 0) [i386-mswin32_80]


Okay, now I’ve got it built and running from the command line, how about building it from Visual Studio, like I did with 1.8.6. So I’m going to follow my own instructions, and see what is different, specifically in how miniruby.exe is built.

First of all, there are a couple of extra libraries, shell32.lib and ws2_32.lib, and one that has been removed: winsock32.lib. Seems like those two replace this one. Ws2_32 is windows sockets version 2 (replacing winsock). That’s all simple enough, but we’ll get to that later.

The main library is msvcr80-ruby19-static.lib, which is built with:

lib -nologo -machine:x86 -out:msvcr80-ruby19-static.lib array.obj ascii.obj bignum.obj class.obj compar.obj dir.obj dln.obj encoding.obj enum.obj enumerator.obj error.obj euc_jp.obj eval.obj eval_load.obj proc.obj file.obj gc.obj hash.obj inits.obj io.obj marshal.obj math.obj numeric.obj object.obj pack.obj parse.obj process.obj prec.obj random.obj range.obj re.obj regcomp.obj regenc.obj regerror.obj regexec.obj regparse.obj regsyntax.obj ruby.obj signal.obj sjis.obj sprintf.obj st.obj string.obj struct.obj time.obj unicode.obj utf8.obj util.obj variable.obj version.obj blockinlining.obj compile.obj debug.obj iseq.obj vm.obj vm_dump.obj thread.obj cont.obj id.obj prelude.obj acosh.obj crypt.obj erf.obj strlcpy.obj strlcat.obj win32.obj dmyext.obj

Well, that looks like a whole load of new stuff (marked in bold), some stuff for character encoding, a new regular expression library split over several modules, and a new virtual machine and associated compile and debug stuff. So where is all this stuff? Well, it looks like it’s all in the root again. Let’s see if we can knock up a project roughly following my old instructions:

Create a win32 console project, no precompiled headers, add oldnames.lib user32.lib advapi32.lib shell32.lib ws2_32.lib, add all the .c files in the root (ruby\) except dmydnl.c, add acosh.c, crypt.c, erf.c and (now) strlcat.c and strlcpy.c from ruby\missing.

Now where’s config.h? In 1.8.6 a file config.h was created in the build directory, but does not seem to be there now. Instead there’s one in \ruby-1.9\build\.ext\include\i386-mswin32_80\ruby\config.h, so I’ll just copy that over.

(NEW) Add the include path: c:\ruby-1.9\ruby\include

Now we still can’t find config.h, as it’s included as “ruby/config.h”, meaning it’s looking in c:\ruby-1.9\ruby\include\ruby\config.h, but we don’t want to put anything there, as that’s our raw source. So I just put it in \ruby-1.9\mickruby\mickruby\ruby\config.h, and added \ruby-1.9\mickruby\mickruby to the include paths, and it’s compiling away.

Okay, just four compile errors now, seems like it can’t find “insns.inc”, which is a file generated in \build. It looks like an enum of the YARV vm op-codes, and says it’s generated by insns2vm.rb from template/insns.inc.tmpl. It seems a bit odd that a generated file ends up in the build directory after a configure. What if you add a new VM instruction? Anyway, copy it (and insns_info.inc) into \ruby-1.9\mickruby\mickruby and then we are missing optinsn.inc, node_name.inc, vm.inc, vmtc.inc, so i just copy all 8 .inc files from build to mickruby\mickruby, and now we just have one error:

Error 255 error C2099: initializer is not a constant c:\ruby-1.9\ruby\regenc.c 32
OnigEncoding OnigEncDefaultCharEncoding = ONIG_ENCODING_INIT_DEFAULT;

Which reminds me, I was supposed to change the character set to “not set”, and add some preprocessor defns. It seems like there is unicode support now, so I’m not so sure about the character set thing. But I’ll give it a go …. nope.

So what’s with this error? ONIG_ENCODING_INIT_DEFAULT should be ONIG_ENCODING_ASCII, which should be (&OnigEncodingASCII), which is a structure defined in ascii.c. Now I’m tempted to blame Visual C here, but it compile fine from the command line, so instead I’m blaming myself, and something that I did. Looks like something to do with character encoding. But then my computer start acting weird, and visual studio tells me it’s out of memory. So I reboot, and the error GOES AWAY. Huh?

No matter, I’m now faced with new errors, lots of unresolved external symbols, I think I’m missing win32\win32.c, yup, but then that’s missing dln.h, which is in \ruby, so add that to the include path (just for this file), so now I’m just missing all the parse.c stuff, which is now in build, but since I’m not going to change the language right now, I’m just going to add it from there. Slightly dodgy, if I’m ever going to change parse.y. Anyway, seems like it includes parse.y in some odd way, which needs #include “regenc.h” which is in the root, so added c:\ruby-1.9\ruby to the include paths, and now we have a new error:

Error 8714 error LNK2019: unresolved external symbol _Init_prelude referenced in function _rb_call_inits inits.obj

Init_prelude comes from \build\prelude.c, which I’ll add directly, and FINALLY….


C:\ruby-1.9\mickruby\debug>mickruby -v
ruby 1.9.0 (2007-10-07 patchlevel 0) [i386-mswin32_80]

What a palaver! And what a mess I made of it, shoehorning it in Visual Studio. Still, it’s up and running, with the various hacks documents above, so that’s a start if I want to make a more professional version of it later.