win32utils devel - Aug 2009 - Comments for Daniel Berger blog post

Hey Daniel,

I''m trying to leave you a comment on this article you wrote:

http://djberg96.livejournal.com/168403.html

But since you disabled OpenID and anonyous, I''m not fond to register
for another account just to reply your comments.

Quoting your post:

""
Unfortunately, I''ve slammed into the cold hard fact that FFI just
isn''t the grand solution we all hoped it would be. The first problem
is that libffi, the underlying source for C based implementations,
isn''t going to build without the gcc toolchain. That pretty much
leaves everyone but Linux, FreeBSD and OS X in the dust, including two
heavy hitters, MS Windows and Solaris (if you''re using the Sun Studio
compiler).
""

That is incorrect. Ruby-FFI team worked and integrated rake-compiler
project in the build process to be able to deliver a binary gem that
install and run flawlessly on both mswin32 (VC6) and mingw32 (GCC).

http://github.com/luislavena/rake-compiler/

If you''re asking to build it against Visual Studio, of course it will
fail, but I suggest you send patches to improve that process, it is
now on GitHub:

http://github.com/ffi/ffi

"""
Then there''s the issue of JRuby''s lack of support for certain
parts of
C, such as file descriptors, as my attempt to port file-temp
demonstrated.
"""

You have a point over there, but transfering file descriptors across
implementations would be impossible, because that would remove the
safe lock around the whole managed idea.

What about IronRuby? They would face the same issue with file descriptors.

"""
So now I''m in a dilemma. If I want to write cross-platform code that
will work with JRuby and C based implementations, I''m relegated to
keeping two (or more) separate source files, one for MRI and one for
JRuby. It would more likely be 3, as I''ll still need a separate source
file for Windows, since the code is radically different from its *nix
counterpart most of the time.
"""

Why so? I mean, you can have several files in the same gem that, based
on the platform, get required independently. Perhaps I''m missing your
point, but would love to explore the issue you''re facing to help you
find the balance and a solution

"""
On top of that I''ve heard disturbing reports that there is little
interest in supporting FFI on Windows, in which case we may as well
declare it dead in the water. Whether you like it or not, Windows is a
major player and its here to stay. If it''s not going to work on
Windows you may as well chuck it now and stick with C extensions.
"""

You can''t impose the platform to anyone, even less in the OpenSource
community. That''s why I''ve created rake-compiler to help them
support
us.

Now that FFI builds with RubyInstaller and DevKit, bug and patches can
be provided by other developers. FFI team announced there will be no
official support since the core team doesn''t run Windows, but that
doesn''t mean they are not fond to integrate these changes.

Wonder why your negative comment about it or if you haven''t heard the
news about Ruby-ffi 0.4.0 release with native gems for mingw32 and
mswin32.
-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

Hi Luis,

2009/8/8 Luis Lavena <luislavena at gmail.com>:
> Hey Daniel,
<snip>
>
> You can''t impose the platform to anyone, even less in the
OpenSource
> community. That''s why I''ve created rake-compiler to help
them support
> us.
>
> Now that FFI builds with RubyInstaller and DevKit, bug and patches can
> be provided by other developers. FFI team announced there will be no
> official support since the core team doesn''t run Windows, but that
> doesn''t mean they are not fond to integrate these changes.
>
> Wonder why your negative comment about it or if you haven''t heard
the
> news about Ruby-ffi 0.4.0 release with native gems for mingw32 and
> mswin32.
I guess I''m the source of the negative comment :)
I''m always watching ruby-ffi and rubyinstaller mailing list. And I
know FFI builds with RubyInstaller and DevKit.
Do you have tried to build ruby-ffi 0.4.0 with mswin32 compiler? I
tried and it failed.
As you now, I have a patched version ruby-ffi-2.0.0 and binary gem.
Porting to mswin32 compiler of GCC oriented project is a pain
especially in system programming.

When I wrote a message on the ruby-ffi forum about the mswin32 patched
version, they just commented that why not try mingw32 version. No one
ever give an interest on my patch.

I guess you dislike mswin32 compiler and have a bias towards ming32.
Recently all your posts are focused on mingw32.

Don''t you want to see mswin32 version of Ruby anymore? Would you
summarize why you decide mingw32 version as rubyinstaller platform?

Regards,

Park Heesob

On Fri, Aug 7, 2009 at 10:36 PM, Heesob Park<phasis at gmail.com>
wrote:
> Hi Luis,
>
> 2009/8/8 Luis Lavena <luislavena at gmail.com>:
>> Hey Daniel,
> <snip>
>>
>> [snip]
>>
>> Wonder why your negative comment about it or if you haven''t
heard the
>> news about Ruby-ffi 0.4.0 release with native gems for mingw32 and
>> mswin32.
>
> I guess I''m the source of the negative comment :)
> I''m always watching ruby-ffi and rubyinstaller mailing list. And I
> know FFI builds with RubyInstaller and DevKit.
> Do you have tried to build ruby-ffi 0.4.0 with mswin32 compiler? I
> tried and it failed.
No, I no longer have installed or try Visual Studio for any C project anymore.
> As you now, I have a patched version ruby-ffi-2.0.0 and binary gem.
> Porting to mswin32 compiler of GCC oriented project is a pain
> especially in system programming.
Indeed I''m fully aware of that particular issue, I''ve tried
several
times to build not Ruby but all the dependencies related to it,
including GNU ones with VC8 and failed in the process.
> When I wrote a message on the ruby-ffi forum about the mswin32 patched
> version, they just commented that why not try mingw32 version. No one
> ever give an interest on my patch.
I believe because there is no way for them to reproduce it or test
that it works properly. If they integrate that patch and someone else
system installation breaks, then noone will look at you for fixes, but
look at FFI team instead.

That''s how OSS works.
> I guess you dislike mswin32 compiler and have a bias towards ming32.
> Recently all your posts are focused on mingw32.
I dislike unneeded complexity. I dislike spent 3 full weekends getting
a handful of Ruby dependencies working just to find out that the
performance gain was marginal. A complete waste of time.

And all the pain,headaches and blindness caused by staring at Ruby C
code to get it work proper didn''t made me feel good after.
> Don''t you want to see mswin32 version of Ruby anymore? Would you
> summarize why you decide mingw32 version as rubyinstaller platform?
I don''t have a problem summarizing it, but perhaps I''m not
good with
summaries :-P

VC8/VC9 links to MSVCR80/90, not MSVCRT.dll, because of that, none of
the dependencies Ruby uses can be used.

This has been raised every time on Ruby-core, and segfaults because
mixing of CRT got my tired.

For mswin32 compilers to work, I would have to go, on every release,
over the Ruby dependencies and get those build and work with mswin32
compiler, since already provided executables are not going to work,
since they link to a different CRT.

Going VC8/9 will also break Ruby binary gem compatibility, since gems
for VC6 would fail on VC8/9 due the exact same CRT issue and even
worse, due incorrect msvcr80-ruby18.dll or msvcr90-ruby18.dll

Usage of freely available VC8/9 has not proven advantage over old and
obsolete GCC 3.4.5. To be able to use PGO you need paid Visual Studio,
so is a negative point to use it as official compiler.

Even more complicated is explain to developers that the CRT is no
longer a simple DLL but is now an SxS assembly that needs to be
referenced and you need to install a Runtime package every time you
want to run it.

This becomes a issue and put higher the entry point barrier for people
wanting to install or use a "packaged" version of a tool.

Leave out of the equation 3rd party developers, outside of
RubyInstaller team that creates tools that links to binaries which we
don''t have control, like MySQL, PostgreSQL, Oracle, etc.

Even worse, consider how many of those do not care about Windows at
all? Believe me, takes a great deal of effort and time to prove them
that they can get their projects working on Windows.

I''ve written about this stuff for the past 4 years, been ignored over
Ruby-core, ignored over private email and been considered a noob
ranting all over Ruby current Windows situation.

I''ve even explored VC8 back in 2007/2008 and even before getting Ruby
to build with it instead of GCC.

Ruby has not been born to properly interact with Win32API, not even
with Windows... 4 layers of functions to be able to get into IO
elements? C files fully covered of conditional macros to determine
which OS you''re in? That sucks.

But can''t do anything to change that, tried, but I''m alone.

Perhaps I''m wrong, or all the above points I''ve mention means
nothing to you.

I''m not trying to prove that "my solution" is the the right
one, heck,
maybe I''m completely wacko and I just do crazy stuff, but had worked
for me until now.
> Regards,
>
Regards and have a nice weekend
-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

Hi,

2009/8/9 Luis Lavena <luislavena at gmail.com>:
> On Fri, Aug 7, 2009 at 10:36 PM, Heesob Park<phasis at gmail.com>
wrote:
>> Hi Luis,
>>
>> 2009/8/8 Luis Lavena <luislavena at gmail.com>:
>>> Hey Daniel,
>> <snip>
>>>
>>> [snip]
>>>
>>> Wonder why your negative comment about it or if you
haven''t heard the
>>> news about Ruby-ffi 0.4.0 release with native gems for mingw32 and
>>> mswin32.
>>
>> I guess I''m the source of the negative comment :)
>> I''m always watching ruby-ffi and rubyinstaller mailing list.
And I
>> know FFI builds with RubyInstaller and DevKit.
>> Do you have tried to build ruby-ffi 0.4.0 with mswin32 compiler? I
>> tried and it failed.
>
> No, I no longer have installed or try Visual Studio for any C project
anymore.
>
>> As you now, I have a patched version ruby-ffi-2.0.0 and binary gem.
>> Porting to mswin32 compiler of GCC oriented project is a pain
>> especially in system programming.
>
> Indeed I''m fully aware of that particular issue, I''ve
tried several
> times to build not Ruby but all the dependencies related to it,
> including GNU ones with VC8 and failed in the process.
>
>> When I wrote a message on the ruby-ffi forum about the mswin32 patched
>> version, they just commented that why not try mingw32 version. No one
>> ever give an interest on my patch.
>
> I believe because there is no way for them to reproduce it or test
> that it works properly. If they integrate that patch and someone else
> system installation breaks, then noone will look at you for fixes, but
> look at FFI team instead.
>
> That''s how OSS works.
>
>> I guess you dislike mswin32 compiler and have a bias towards ming32.
>> Recently all your posts are focused on mingw32.
>
> I dislike unneeded complexity. I dislike spent 3 full weekends getting
> a handful of Ruby dependencies working just to find out that the
> performance gain was marginal. A complete waste of time.
>
> And all the pain,headaches and blindness caused by staring at Ruby C
> code to get it work proper didn''t made me feel good after.
>
>> Don''t you want to see mswin32 version of Ruby anymore? Would
you
>> summarize why you decide mingw32 version as rubyinstaller platform?
>
> I don''t have a problem summarizing it, but perhaps I''m
not good with
> summaries :-P
>
> VC8/VC9 links to MSVCR80/90, not MSVCRT.dll, because of that, none of
> the dependencies Ruby uses can be used.
>
> This has been raised every time on Ruby-core, and segfaults because
> mixing of CRT got my tired.
>
> For mswin32 compilers to work, I would have to go, on every release,
> over the Ruby dependencies and get those build and work with mswin32
> compiler, since already provided executables are not going to work,
> since they link to a different CRT.
>
> Going VC8/9 will also break Ruby binary gem compatibility, since gems
> for VC6 would fail on VC8/9 due the exact same CRT issue and even
> worse, due incorrect msvcr80-ruby18.dll or msvcr90-ruby18.dll
>
> Usage of freely available VC8/9 has not proven advantage over old and
> obsolete GCC 3.4.5. To be able to use PGO you need paid Visual Studio,
> so is a negative point to use it as official compiler.
>
> Even more complicated is explain to developers that the CRT is no
> longer a simple DLL but is now an SxS assembly that needs to be
> referenced and you need to install a Runtime package every time you
> want to run it.
>
> This becomes a issue and put higher the entry point barrier for people
> wanting to install or use a "packaged" version of a tool.
>
> Leave out of the equation 3rd party developers, outside of
> RubyInstaller team that creates tools that links to binaries which we
> don''t have control, like MySQL, PostgreSQL, Oracle, etc.
>
> Even worse, consider how many of those do not care about Windows at
> all? Believe me, takes a great deal of effort and time to prove them
> that they can get their projects working on Windows.
>
> I''ve written about this stuff for the past 4 years, been ignored
over
> Ruby-core, ignored over private email and been considered a noob
> ranting all over Ruby current Windows situation.
>
> I''ve even explored VC8 back in 2007/2008 and even before getting
Ruby
> to build with it instead of GCC.
>
> Ruby has not been born to properly interact with Win32API, not even
> with Windows... 4 layers of functions to be able to get into IO
> elements? C files fully covered of conditional macros to determine
> which OS you''re in? That sucks.
>
> But can''t do anything to change that, tried, but I''m
alone.
>
> Perhaps I''m wrong, or all the above points I''ve mention
means nothing to you.
>
> I''m not trying to prove that "my solution" is the the
right one, heck,
> maybe I''m completely wacko and I just do crazy stuff, but had
worked
> for me until now.
>
Thanks for your a little long summary :)
I understand your situation and despair.

Now I really want to know is that the mingw32 compiled dll is
compatible with mswin32 version of ruby. Is there any way to make a
dll build with mingw32 compiler to compatible both mingw32 and mswin32
version?

Regards,

Park Heesob

On Sat, Aug 8, 2009 at 9:06 PM, Heesob Park<phasis at gmail.com>
wrote:
> Hi,
>
> 2009/8/9 Luis Lavena <luislavena at gmail.com>:
>> [LONG SNIP]
>
> Thanks for your a little long summary :)
Apologizes, needed to get it out of my chest.

I posted about VC8 back in January 2008:

http://rubyforge.org/pipermail/rubyinstaller-devel/2008-January/000230.html

Taking in consideration was doing what "Average Joe" will do, not
being a VS expert or anything like that.

Wonder how PGO compilation after running the test can improve those values.
> I understand your situation and despair.
>
> Now I really want to know is that the mingw32 compiled dll is
> compatible with mswin32 version of ruby. Is there any way to make a
> dll build with mingw32 compiler to compatible both mingw32 and mswin32
> version?
>
Yes, GCC links and target by default to MSVCRT.DLL, which makes
mingw32 binaries compatible with mswin32 ones.

Everything will be great as long both target and use the same CRT.

Now, the issues are Ruby and Ruby code. There are lot of RUBY_PLATFORM
evaluations, so while binaries may be compatible, the Ruby code
perhaps not.

rake-compiler has been created and improved to allow targeting mswin32
and mingw32 platforms, in the aim to maintain backward compatibility
until mingw32 version gets more adopted.

Regards,
-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

2009/8/9 Luis Lavena <luislavena at gmail.com>:
> On Sat, Aug 8, 2009 at 9:06 PM, Heesob Park<phasis at gmail.com>
wrote:
>> Hi,
>>
>> 2009/8/9 Luis Lavena <luislavena at gmail.com>:
>>> [LONG SNIP]
>>
>> Thanks for your a little long summary :)
>
> Apologizes, needed to get it out of my chest.
>
> I posted about VC8 back in January 2008:
>
> http://rubyforge.org/pipermail/rubyinstaller-devel/2008-January/000230.html
>
> Taking in consideration was doing what "Average Joe" will do, not
> being a VS expert or anything like that.
>
> Wonder how PGO compilation after running the test can improve those values.
>
>> I understand your situation and despair.
>>
>> Now I really want to know is that the mingw32 compiled dll is
>> compatible with mswin32 version of ruby. Is there any way to make a
>> dll build with mingw32 compiler to compatible both mingw32 and mswin32
>> version?
>>
>
> Yes, GCC links and target by default to MSVCRT.DLL, which makes
> mingw32 binaries compatible with mswin32 ones.
>
If mingw32 binaries are compatible with mswin32, I think ffi-0.4.0
binary might be compatible with mswin32_90 version of ruby.
What''s the way to install ffi-0.4.0 mingw32 binary gem to mswin32_90
version of ruby?
> Everything will be great as long both target and use the same CRT.
>
I''m not sure what you mean by "as long both target and use the
same CRT".


Regards,

Park Heesob

On Sat, Aug 8, 2009 at 11:03 PM, Heesob Park<phasis at gmail.com>
wrote:
> 2009/8/9 Luis Lavena <luislavena at gmail.com>:
>> On Sat, Aug 8, 2009 at 9:06 PM, Heesob Park<phasis at gmail.com>
wrote:
>>> Hi,
>>>
>>> 2009/8/9 Luis Lavena <luislavena at gmail.com>:
>>>> [LONG SNIP]
>>>
>>> Thanks for your a little long summary :)
>>
>> Apologizes, needed to get it out of my chest.
>>
>> I posted about VC8 back in January 2008:
>>
>>
http://rubyforge.org/pipermail/rubyinstaller-devel/2008-January/000230.html
>>
>> Taking in consideration was doing what "Average Joe" will do,
not
>> being a VS expert or anything like that.
>>
>> Wonder how PGO compilation after running the test can improve those
values.
>>
>>> I understand your situation and despair.
>>>
>>> Now I really want to know is that the mingw32 compiled dll is
>>> compatible with mswin32 version of ruby. Is there any way to make a
>>> dll build with mingw32 compiler to compatible both mingw32 and
mswin32
>>> version?
>>>
>>
>> Yes, GCC links and target by default to MSVCRT.DLL, which makes
>> mingw32 binaries compatible with mswin32 ones.
>>
>
> If mingw32 binaries are compatible with mswin32, I think ffi-0.4.0
> binary might be compatible with mswin32_90 version of ruby.
Well no, if memory is allocated by mswin32 ffi, it cannot be GC by
mswin32_90 Ruby.
> What''s the way to install ffi-0.4.0 mingw32 binary gem to
mswin32_90
> version of ruby?
>
gem install ... --platform=i386-mswin32

But use it at your own risk.
>> Everything will be great as long both target and use the same CRT.
>>
> I''m not sure what you mean by "as long both target and use
the same CRT".
>
Memory allocation and free''d must be done by the same CRT functions,
on the exact same library, cannot be performed by different CRT set of
functions.

That leads to segfaults.

-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

Hi,

On Fri, Aug 7, 2009 at 5:16 PM, Luis Lavena<luislavena at gmail.com>
wrote:
> Hey Daniel,
>
> I''m trying to leave you a comment on this article you wrote:
>
> http://djberg96.livejournal.com/168403.html
>
> But since you disabled OpenID and anonyous, I''m not fond to
register
> for another account just to reply your comments.
I don''t see a way to allow OpenID on LJ. If there''s a way,
I''ll enable
that. I won''t allow anonymous comments, though.
> Quoting your post:
>
> ""
> Unfortunately, I''ve slammed into the cold hard fact that FFI just
> isn''t the grand solution we all hoped it would be. The first
problem
> is that libffi, the underlying source for C based implementations,
> isn''t going to build without the gcc toolchain. That pretty much
> leaves everyone but Linux, FreeBSD and OS X in the dust, including two
> heavy hitters, MS Windows and Solaris (if you''re using the Sun
Studio
> compiler).
> ""
>
> That is incorrect. Ruby-FFI team worked and integrated rake-compiler
> project in the build process to be able to deliver a binary gem that
> install and run flawlessly on both mswin32 (VC6) and mingw32 (GCC).
>
> http://github.com/luislavena/rake-compiler/
>
> If you''re asking to build it against Visual Studio, of course it
will
> fail, but I suggest you send patches to improve that process, it is
> now on GitHub:
>
> http://github.com/ffi/ffi
I''m not going to install mingw just so I can build FFI. If we can get
it to build with VS that would be great.
> """
> Then there''s the issue of JRuby''s lack of support for
certain parts of
> C, such as file descriptors, as my attempt to port file-temp
> demonstrated.
> """
>
> You have a point over there, but transfering file descriptors across
> implementations would be impossible, because that would remove the
> safe lock around the whole managed idea.
If I can''t get at low level details with JRuby, like file descriptors
and pointer function addresses, then JRuby is officially hopeless as a
system''s programming language.
> What about IronRuby? They would face the same issue with file descriptors.
We''ll have to wait and see, but somehow I doubt it.
> """
> So now I''m in a dilemma. If I want to write cross-platform code
that
> will work with JRuby and C based implementations, I''m relegated to
> keeping two (or more) separate source files, one for MRI and one for
> JRuby. It would more likely be 3, as I''ll still need a separate
source
> file for Windows, since the code is radically different from its *nix
> counterpart most of the time.
> """
>
> Why so? I mean, you can have several files in the same gem that, based
> on the platform, get required independently. Perhaps I''m missing
your
> point, but would love to explore the issue you''re facing to help
you
> find the balance and a solution
People keep telling me this, but I''ve yet to see it in practice. As
far as I can tell you have to generate the gems up front. I don''t see
a way to dynamically determine, at the point of installation, which
file should be considered the "real" file.

Please take a look at sys-proctable for the most extreme case of
variable source code, and tell me how you would handle it.
> """
> On top of that I''ve heard disturbing reports that there is little
> interest in supporting FFI on Windows, in which case we may as well
> declare it dead in the water. Whether you like it or not, Windows is a
> major player and its here to stay. If it''s not going to work on
> Windows you may as well chuck it now and stick with C extensions.
> """
>
> You can''t impose the platform to anyone, even less in the
OpenSource
> community. That''s why I''ve created rake-compiler to help
them support
> us.
>
> Now that FFI builds with RubyInstaller and DevKit, bug and patches can
> be provided by other developers. FFI team announced there will be no
> official support since the core team doesn''t run Windows, but that
> doesn''t mean they are not fond to integrate these changes.
>
> Wonder why your negative comment about it or if you haven''t heard
the
> news about Ruby-ffi 0.4.0 release with native gems for mingw32 and
> mswin32.
I didn''t know about it. I will try ruby-ffi 0.4.0 and see how it goes.

Honestly, though, if JRuby can''t support some of the features I want,
I don''t see any point in converting most of my current C extensions to
FFI. Then people can choose whatever compiler they wish. :)

Regards,

Dan