Rubyinstaller devel - Apr 2008 - Gem based Mingw build environment

I''ve been rereading the threads from the last few months, and in
"Windows Compilation madness"[1], a gem based Windows development
environment was discussed.  I think that being able to gem install a
mingw development environment is a really great idea.  In that thread
there were three reasons against this approach, and I want to discsuss
them:

Luis Lavena wrote:
> Roger Pack suggested me to bundle MinGW in a gem...:
> 1) a 8MB gem (!!!)
Ok, I understand that ''gem instal''l would have a problem with
this,
but I think if we (or I, if no one else is interested) abstract out
the necessary bits from Luis''  installer3 rake recipes, we could just
distribute all ruby code.  We just need to make it fairly simple for
the user to get msys and mingw set up.
> 2) because mingw is inside the gem, will not be easy hook it into PATH
We could have a command in the gem''s bin dir that would help us set
this up.  It would ask the user if he has msys and mingw installed,
and if so prompt for the path.  If not, it could download it into the
location of the user''s choosing.  Then it could create a config file
in $HOME or $USERPROFILE, similar to what Hoe or Autotest or Rubyforge
uses.   This file would contain the paths to msys and mingw and any
other global configuration needed to make this work.  From there, the
users could require this gem into their rakefile, and have all the
tools they need.
> 3) that didn''t solve the rbconfig issues.
This issue is one I''m not sure I understand, and it may be the
showstopper.  Anyway, those are my ideas I''m interested to hear what
people think.

Thanks,

Gordon


[1] 
http://groups.google.com/group/comp.lang.ruby/browse_frm/thread/6b299623e53edbd9/71760b19769067e6?lnk=gst&q=lavena+mingw+gem#71760b19769067e6

On Tue, Apr 1, 2008 at 1:04 PM, Gordon Thiesfeld <gthiesfeld at gmail.com>
wrote:
> I''ve been rereading the threads from the last few months, and in
>  "Windows Compilation madness"[1], a gem based Windows
development
>  environment was discussed.  I think that being able to gem install a
>  mingw development environment is a really great idea.  In that thread
>  there were three reasons against this approach, and I want to discsuss
>  them:
>
hey Gordon, great feedback!

>  Luis Lavena wrote:
>  > Roger Pack suggested me to bundle MinGW in a gem...:
>
>  > 1) a 8MB gem (!!!)
>
>  Ok, I understand that ''gem instal''l would have a problem
with this,
>  but I think if we (or I, if no one else is interested) abstract out
>  the necessary bits from Luis''  installer3 rake recipes, we could
just
>  distribute all ruby code.  We just need to make it fairly simple for
>  the user to get msys and mingw set up.
>
At that time I was negative to that idea, but after thinking it a bit,
it started to sound good. Is something like apt-get build-essentials
or similar.

What I didn''t liked at that time was that rubygems use open-uri and is
not possible keep track of partial downloads or similar, something we
achieved in installer3 adapting some code from Buildr project [1].

In any case, our next step is work on the MSI scripts (Microsoft
Installer build with WiX) to generate the Ruby 1.8 module and also
some modules for MinGW and MSYS.

Then create a Developer Kit that could be installed and hooked into
the $PATH, as I commented in my blog post [2] a few days back.
>  > 2) because mingw is inside the gem, will not be easy hook it into
PATH
>
>  We could have a command in the gem''s bin dir that would help us
set
>  this up.  It would ask the user if he has msys and mingw installed,
>  and if so prompt for the path.  If not, it could download it into the
>  location of the user''s choosing.  Then it could create a config
file
>  in $HOME or $USERPROFILE, similar to what Hoe or Autotest or Rubyforge
>  uses.   This file would contain the paths to msys and mingw and any
>  other global configuration needed to make this work.  From there, the
>  users could require this gem into their rakefile, and have all the
>  tools they need.
I have played a bit under that road, but there is a small problem
cannot easily fix, see the following IRB and command line output:

D:\Users\Luis>set FOOBAR
Environment variable FOOBAR not defined

D:\Users\Luis>irb
irb(main):001:0> ENV[''FOOBAR'']
=> nil
irb(main):002:0> ENV[''FOOBAR''] = "Something"
=> "Something"
irb(main):003:0> ENV[''FOOBAR'']
=> "Something"
irb(main):004:0> exit

D:\Users\Luis>set FOOBAR
Environment variable FOOBAR not defined

D:\Users\Luis>SET FOOBAR=Something

D:\Users\Luis>irb
irb(main):001:0> ENV[''FOOBAR'']
=> "Something"
irb(main):002:0> ENV.delete(''FOOBAR'')
=> "Something"
irb(main):003:0> ENV[''FOOBAR'']
=> nil
irb(main):004:0> exit

D:\Users\Luis>SET FOOBAR
FOOBAR=Something

==
As you can see, we cannot alter any environment variable inside Ruby
and make the change "persist" outside the Ruby VM. What we could do is
wrap around and inherit the new environment, that is just nasty and
feel dirty...

What Curt Hibbs and Andy Hunt ended doing with installer-win2 on
subversion repository is wrap the environment variable changes into a
batch file that was the responsible to call nmake.

Will be better if we install MinGW+MSYS outside the controller Ruby VM
and environment and provide a windows shortcut like "MinGW command
prompt" or something like that.
>  > 3) that didn''t solve the rbconfig issues.
>
>  This issue is one I''m not sure I understand, and it may be the
>  showstopper.  Anyway, those are my ideas I''m interested to hear
what
>  people think.
The rbconfig hack was related to the subject of keep using VC6 builds
and use MinGW as compiler for the extensions, since mkmf, the library
used to generate the makefiles for ruby extension is highly coupled
with the compiler used to generate the ruby installation, example:

Ruby 1.8 built with VC6 (require rbconfig):

RbConfig::CONFIG[''CC'']
=> "cl -nologo"
RbConfig::CONFIG[''CFLAGS'']
=> "-MD -Zi -O2b2xg- -G6"
RbConfig::CONFIG[''AR'']
=> "lib -nologo"

Now, Ruby 1.8 with MinGW

irb(main):001:0> require ''rbconfig''
=> true
irb(main):002:0> RbConfig::CONFIG[''CC'']
=> "gcc"
irb(main):003:0> RbConfig::CONFIG[''CFLAGS'']
=> "-g -O2 "
irb(main):004:0> RbConfig::CONFIG[''AR'']
=> "ar"

As you can see, we need to tweak the RbConfig options to make them
work with MinGW command tools, but there are other issues, like the
platform RubyGems uses (RUBY_PLATFORM) and it will try to fire
''nmake''
instead of ''make'' when building the extensions bundled into a
gem...

It get too much complicated if we go down that road, so instead of
"patching and hacking" current build of ruby, jump into MinGW
completely sounds more logical :-D
>  Thanks,
Thanks to you Gordon, for taking the time to analyze this and provide
your feedback. I wish we could do some of these stuff without hack
around Ruby, but avoiding ruby sometimes is healthier :-)

[1] http://incubator.apache.org/buildr/
[2]
http://blog.mmediasys.com/2008/03/29/progress-of-one-click-installer-rubyinstaller/

-- 
Luis Lavena
Multimedia systems
-
Human beings, who are almost unique in having the ability to learn from
the experience of others, are also remarkable for their apparent
disinclination to do so.
Douglas Adams

On Tue, Apr 1, 2008 at 11:46 AM, Luis Lavena <luislavena at gmail.com>
wrote:
>
>  Thanks to you Gordon, for taking the time to analyze this and provide
>  your feedback. I wish we could do some of these stuff without hack
>  around Ruby, but avoiding ruby sometimes is healthier :-)
>
I appreciate you taking the time to explain all of this to me.  So,
the package size issue and the rbconfig issues are resolved.  That
leaves us with the issue of setting env variables.  Then, it''s just a
matter of which packaging mechanism to use (msi, or rubygems).

I don''t have a lot of experience with compiling extensions.  I have
compiled the win32console extension with VC6, and with MinGW.  With
VC6, I had to download the Windows platform SDK, run two batch files
to set env vars, and finally I got it to build.  It took me hours to
get it working.  Maybe that''s because I''d never done it before
- for
someone with a lot of C/C++ Windows experience it may have been
trivial.  With Mingw, I hacked into your installer project''s rake
recipes, used the msys_sh method, and I was done.  It took me 20
minutes tops.  I guess my question is, since you can provide a package
that builds ruby and all of it''s stdlib extensions via a rakefile
without setting env vars, is it something that we need to worry about?

Maybe you''re just making it look too easy, I don''t know :-)

Thanks,

Gordon