Rubyinstaller devel - Jan 2008 - Building Ruby + extensions with modern Visual C++

Hi,

There doesn''t seem to be any discussion on this list right now, so let
me kick something off. I contacted Curt and Luis earlier this week to
ask what their plans were with regard to migrating the OCI to a new
compiler. Luis seems to be working on a bootstrapping approach using
mingw, which is fine, but I still think that the "best" thing to do
would be to build with the Microsoft compiler on Windows, it being the
native solution and all. Also, according to what I''ve read, linking
with the newer versions of the MS C runtime has some advantages in
terms of security and stability compared to the old msvcrt.dll that
mingw (and vc6) links to.

I started trying to build everything necessary for the OCI using
VC2005. I quickly realized that finding, installing, updating, and
configuring VC2005 Express is a gigantic pain in the ass. It does not
work with the Platform SDK out of the box, which means you have to
edit some config files by hand to build win32 applications. This is
far from ideal.

Getting set up with VC2008 Express, however, is easy. You just have to
download one file from http://www.microsoft.com/express/ and install
it. Everything works out of the box. I also see some advantage to
using the latest version (it will probably be available for longer, at
the least).

So, on to the actual compiling.

I started with Ruby 1.8.6-p111 and zlib. Both build flawlessly with
vc2008. Easy. I continued with openssl, building with NASM 2.00 +
vc2008. I had to copy the nasm.exe binary to nasmw.exe for the openssl
makefile to work, but then it also built fine.

Then I hit readline. Readline is a mess. There isn''t really an
up-to-date, well-maintained windows port, as far as I can tell. I
think there needs to be a reliable port of this library to really
depend on the Microsoft compiler. I''m willing to try to learn how to
port it, but I don''t really have the existing expertise to do it
quickly. If anyone has some ideas or pointers here, I''m all ears.

What other dependencies does the OCI have? iconv, tcl?

I want to be clear that I''m not challenging Luis''s work on the
mingw-based bootstrapping setup. I think it''s valuable to work on both
in parallel, to learn more about what''s possible. I will have some
time this weekend to work on getting further with the vc2008 builds.


A good night to all,

- Will

Remember that you also have to build all of the extensions and RubyGems
because the binaries produced with VC2008 use a different runtime dll than
VC6 and you can''t mix them without causing problems.

Curt

On Jan 11, 2008 8:14 PM, Will Rogers <wjrogers at gmail.com> wrote:
> Hi,
>
> There doesn''t seem to be any discussion on this list right now, so
let
> me kick something off. I contacted Curt and Luis earlier this week to
> ask what their plans were with regard to migrating the OCI to a new
> compiler. Luis seems to be working on a bootstrapping approach using
> mingw, which is fine, but I still think that the "best" thing to
do
> would be to build with the Microsoft compiler on Windows, it being the
> native solution and all. Also, according to what I''ve read,
linking
> with the newer versions of the MS C runtime has some advantages in
> terms of security and stability compared to the old msvcrt.dll that
> mingw (and vc6) links to.
>
> I started trying to build everything necessary for the OCI using
> VC2005. I quickly realized that finding, installing, updating, and
> configuring VC2005 Express is a gigantic pain in the ass. It does not
> work with the Platform SDK out of the box, which means you have to
> edit some config files by hand to build win32 applications. This is
> far from ideal.
>
> Getting set up with VC2008 Express, however, is easy. You just have to
> download one file from http://www.microsoft.com/express/ and install
> it. Everything works out of the box. I also see some advantage to
> using the latest version (it will probably be available for longer, at
> the least).
>
> So, on to the actual compiling.
>
> I started with Ruby 1.8.6-p111 and zlib. Both build flawlessly with
> vc2008. Easy. I continued with openssl, building with NASM 2.00 +
> vc2008. I had to copy the nasm.exe binary to nasmw.exe for the openssl
> makefile to work, but then it also built fine.
>
> Then I hit readline. Readline is a mess. There isn''t really an
> up-to-date, well-maintained windows port, as far as I can tell. I
> think there needs to be a reliable port of this library to really
> depend on the Microsoft compiler. I''m willing to try to learn how
to
> port it, but I don''t really have the existing expertise to do it
> quickly. If anyone has some ideas or pointers here, I''m all ears.
>
> What other dependencies does the OCI have? iconv, tcl?
>
> I want to be clear that I''m not challenging Luis''s work
on the
> mingw-based bootstrapping setup. I think it''s valuable to work on
both
> in parallel, to learn more about what''s possible. I will have some
> time this weekend to work on getting further with the vc2008 builds.
>
>
> A good night to all,
>
> - Will
> _______________________________________________
> Rubyinstaller-devel mailing list
> Rubyinstaller-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rubyinstaller-devel
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/rubyinstaller-devel/attachments/20080112/6ef033ea/attachment.html

On Jan 12, 2008 4:55 AM, Curt Hibbs <curt.hibbs at gmail.com>
wrote:
> Remember that you also have to build all of the extensions and RubyGems
> because the binaries produced with VC2008 use a different runtime dll than
> VC6 and you can''t mix them without causing problems.
>
That will not be a problem, since the changes I introduced in RubyGems
(before they released version 1.0.x)  make a strong difference between
VC6 and VC8/VC9.

So gems pre-built for VC6 can co-exist with ones for VC8 (on
rubyforge), but you can''t install VC8 gems on top of VC6 ruby (or vice
versa).

-- 
Luis Lavena
Multimedia systems
-
A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams

On Jan 12, 2008 12:14 AM, Will Rogers <wjrogers at gmail.com>
wrote:
> Hi,
>
> There doesn''t seem to be any discussion on this list right now, so
let
> me kick something off. I contacted Curt and Luis earlier this week to
> ask what their plans were with regard to migrating the OCI to a new
> compiler. Luis seems to be working on a bootstrapping approach using
> mingw, which is fine, but I still think that the "best" thing to
do
> would be to build with the Microsoft compiler on Windows, it being the
> native solution and all. Also, according to what I''ve read,
linking
> with the newer versions of the MS C runtime has some advantages in
> terms of security and stability compared to the old msvcrt.dll that
> mingw (and vc6) links to.
I used to think that "all native tools" was better than MinGW path,
security and performance related, but if you take a closer look at the
build process of ruby, you will find the _CRT_SECURE_NO_DEPRECATE
being defined for every file being compiled.

That define is used to avoid errors and warning of code being using
the unsafe CRT functions found in MSVCRT.dll

They suggest you switch to the safest version, which requires another
set of parameters to the functions that are signature incompatible
with other CRT implementations.

Also I''ll like to point that, if you take a closer look to the
MSVCRT.dll store in windows\system32, you will find is not version 6.0
(which was buggy) but is 7.0.2600.2180

So, Windows XP SP2 is using version 7.0! and ruby was linked with CRT 6.0!

Hehehe, the function signatures of both dlls match, so there is no
problem. And 7.0 CRT is safer than 6.0...

Anyway...
> I started trying to build everything necessary for the OCI using
> VC2005. I quickly realized that finding, installing, updating, and
> configuring VC2005 Express is a gigantic pain in the ass. It does not
> work with the Platform SDK out of the box, which means you have to
> edit some config files by hand to build win32 applications. This is
> far from ideal.
I''ve used the Updated Windows SDK for Windows Vista (1GB, free
download), which was VC8 and is the same as VS2005. It shipped with
SDK so no need to change things to get it working.

Also you get the properly configured console to run all the tools from there.
> Getting set up with VC2008 Express, however, is easy. You just have to
> download one file from http://www.microsoft.com/express/ and install
> it. Everything works out of the box. I also see some advantage to
> using the latest version (it will probably be available for longer, at
> the least).
Yes, VC2008 Sounds interesting, I''ll need to give it a whirl...

The idea with the bootstrap process is that we can create, with the
same set of tasks, the interpreter and the dependencies to have it
working, compiler-agnostic.

So far, I''ve only implemented MinGW+MSYS, but the idea is we can add
some recipes or maybe a "rake/compiletask" that can determine the
available compiler and build the dependencies and Ruby.

As I''ve commented to James Tucker, first we must get a complete stable
environment, then we can take over the world :)
> So, on to the actual compiling.
>
> I started with Ruby 1.8.6-p111 and zlib. Both build flawlessly with
> vc2008. Easy. I continued with openssl, building with NASM 2.00 +
> vc2008. I had to copy the nasm.exe binary to nasmw.exe for the openssl
> makefile to work, but then it also built fine.
>
That change should be pushed upstream (to openssl guys) to allow them
be compatible with newer versions of the compiler).
> Then I hit readline. Readline is a mess. There isn''t really an
> up-to-date, well-maintained windows port, as far as I can tell. I
> think there needs to be a reliable port of this library to really
> depend on the Microsoft compiler. I''m willing to try to learn how
to
> port it, but I don''t really have the existing expertise to do it
> quickly. If anyone has some ideas or pointers here, I''m all ears.
>
Oh, now you know my pain. Not event the MinGW port is up to date, I''ve
tried 4.3, 5.0 and 5.2 that I don''t remember where I found to my VC8
experiment.

At that time (May 2007), I bundled everything in the same SVN repo, so
after adding the dependencies the thing was big (150MB svn repo).

I need to undust that form my backup and switch these changes to
bazaar, which helped me keep track of upstream changes and provide
better set of patches to the maintainers of some libraries.
> What other dependencies does the OCI have? iconv, tcl?
iconv, tk and tcl. The last two I think will not worth the effort...

This paste contains the list of the extensions that are bundled with
official (VC6, garbagecollect) build of Ruby 1.8.6:

http://rafb.net/p/Wyx6mG28.html

And this was my status about them:

http://rafb.net/p/1GUz5L68.html

If you take a closer look, tk and tkutils didn''t get marked as OK...
Every time I tried couldn''t get them working, every time I failed.

But it seems lot of folks rely on Tk to do some GUIs, so... maybe we
can keep them "enabled".
> I want to be clear that I''m not challenging Luis''s work
on the
> mingw-based bootstrapping setup. I think it''s valuable to work on
both
> in parallel, to learn more about what''s possible. I will have some
> time this weekend to work on getting further with the vc2008 builds.
The job to get all the dependencies working with VC8 a bit stressing,
mostly due lack of insterest from upstream.

Anyway, what do you think if we put the mingw+msys recipes on hold and
implement some for VC8? it seems it''s worth we give it a try but not
in the manual way, but somehow automated.

Maybe we can came up with a CompilerUtils that can #compile #make,
#link or #configure disregard of it''s GCC or VC in the background.

The idea of build the dependencies from scratch is good too, since
some of the binaries are outdated.

In any case, we could have a series of patches for these tools until
upstream maintainers get them included.

A good thing about Bazaar is that we can branch and later merge back
without too much problems :-D

I''ll be hanging on ruby-lang later today, but couldn''t promise
I''ll
have Express 208 installed, since I hate these tools modify my file
associations ;-)
> A good night to all,
Night to you too,

-- 
Luis Lavena
Multimedia systems
-
A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams