win32utils devel - Nov 2008 - Binaries dependencies (shared_libraries) for Windows.

Hello Guys,

This issue was raised originally back in 2007:

http://rubyforge.org/pipermail/rubygems-developers/2007-March/002646.html

And raised again by Charlie Savage in July 2008:

http://rubyforge.org/pipermail/rubygems-developers/2008-July/003978.html

I''ve explored the different alternatives under several environments
and the following are my findings:

1) Alteration of PATH to prepend the DLL location.

This was one of the proposed alternatives, and I believe is the one
implemented in libxml package. This was also mimicked due the lack of
this functionality for DataObjects SQLite3 adapter during the Merb 1.0
release.

While this has proven to work, activation of several gems that bundle
binaries using this technique make this PATH alteration to child
processes, thus could end with alteration of the expected environment
for those processes.

As example, I bundled libmySQL.dll from MySQL 5.0.51a in mysql gem,
while system wide I have installed 5.1.29.

The gem executed and connected properly to the 5.1 server, but
executing of command line application "mysql" (common practice found
in several projects) generated segfaults for these applications and
failure due worng exported symbols (and missing ones).

There is another issue in relation to PATH size limititations.

PATH under XP and 2003 can be up to 2048 bytes, under certains
circumstances, this PATH get mangled and only 1024 bytes get copied to
spawned child process or by the the process altering the PATH too.

There is a patch for 2003 and XP:

http://support.microsoft.com/kb/906469

In any case, the folder structure used by rubygems and the location
where users install ruby will generate, after activation of 3 or 4
gems compromise the PATH integrity.

As example, I have 450 bytes used for my PATH, with ruby Installed in
C:\Ruby\bin.

Adding "c:/rubylib/ruby/1.8/gems/gem_name-version/lib" to the PATH for
each gem activated.

2) Usage of SetDllDirectory

Another solution proposed was the usage of Win32API in combination
with SetDllDirectory to append the path for the dll, altering the
LoadLibrary search order to be able to pick the specified dll.

SetDllDirectory was introduced in XP SP1 and with Windows 2003, so has
no presence previous in Windows 2000 or NT.

Moreover, SetDllDirectory can only add ONE path a time, and doesn''t
allow concatenation like PATH as search.

This means you can SetDllDirectory and inmediately after you''re
required to load the extension that depend on the DLL. You need to
repeat those steps for each binary that depends on external DLLs.

It seems SetDllDirectory considered multiple paths in their specs, but
no implementation can be used for it right now.

RubyGems is not designed to produce that kind of modification or
awareness of the dependencies, rendering this method useless for our
purposes.

3) LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH

The last technique involves explicitly loading the DLL dependency at
runtime into the process, instead of waiting for the extension to fire
the sequence.

The usage of LOAD_WITH_ALTERED_SEARCH_PATH option allow explicit call
to a DLL without affecting the PATH and without being affected by
SetDllLibrary limitations.

The only drawback is that these dlls will be loaded into the process
at gem activation and not when the extensions get loaded, making your
process a big heavier from startup.

This similar approach is being used by Google Chrome to locate and
load plugins and by other projects with small differences.

==
Those are the scenarios I''ve evaluated. I''m CC''ing
win32utils list
since I believe Park and Daniel can provide some feedback on these
findings prior me hacking something for RubyGems.

Thanks in advance for any feedback provided to this.

Regards,
-- 
Luis Lavena
AREA 17
-
Human beings, who are almost unique in having the ability to learn from
the experience of others, are also remarkable for their apparent
disinclination to do so.
Douglas Adams