wxruby development - Nov 2008 - Any investigation on using FFI instead of SWIG?

This may be a misguided message, but I was wondering if any thought has been
given to making wxRuby work using the new native FFI (Foreign Function
Interface) interface in the place of SWIG?

Something like the FFI thing would be much easier for a non-C programmer
like me to assist with.

If you aren''t familiar with it (quite new) here are some links.
http://kenai.com/projects/ruby-ffi/pages/Home
http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html
http://wmeissner.blogspot.com/2008/11/more-on-ruby-ffi.html
http://pluskid.lifegoo.com/?p=370

Now, I realize that FFI is really, really new and probably not yet ready to
handle something as complex as the needs of wxRuby. However, I wanted to
inquire as to the principle of it.

I have worked on other scripting engines before (windows platform in other
languages). I have experience with generating code to help map complex
structures in a native language to a simplified one in a scripted language.
I''m just curious as to what you guys think.

Ideally, I would love to see a shared library that wxRuby and wxPython could
both use to simplify complex structures that needed to be passed to the
scripts. Then each project would use something like FFI to import those
functions into Ruby or Python. Then the work could be shared. :)

Anyway, I''m interested in your thoughts, ideas and problem points.

-MarkE
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/wxruby-development/attachments/20081109/5ddf8c5c/attachment-0001.html>

There are many good points to using something like FFI.  The major problem
though, that I see from it at this point, is that it would require a full
re-write of the wxRuby library.  Something we''re not planning to do
right
now with the 2.0 release.  We''re focusing on a 2.0 full release here
very
soon, that we can label stable, and freeze feature sets, and such.  So that
we can finally have a stable product out there for people to use in a
comerical setting.

We''ve already been looking at possibly switching from SWIG to something
else, can''t remember the name of the project off the top of my Head,
but
Alex would remember.  Sometimes, new can be good, and bad at the same time,
so we''ll just have to test the waters, and see what we can figure out.

The major problem that I can see with this, is that wxWidgets has their own
data types for just about everything, from Integers, to Strings, to Complex
data formats.  While I see struct being available with FFI (Thank god, that
makes wrapping stuff a lot easier), it still may be a problem when trying to
convert a Ruby String, over to a wxString.

But it will definately something to look at, when we get 2.0 out the door.

On Sun, Nov 9, 2008 at 1:33 PM, Mark E. <mnmfactory at gmail.com> wrote:
> This may be a misguided message, but I was wondering if any thought has
> been given to making wxRuby work using the new native FFI (Foreign Function
> Interface) interface in the place of SWIG?
>
> Something like the FFI thing would be much easier for a non-C programmer
> like me to assist with.
>
> If you aren''t familiar with it (quite new) here are some links.
> http://kenai.com/projects/ruby-ffi/pages/Home
> http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html
> http://wmeissner.blogspot.com/2008/11/more-on-ruby-ffi.html
> http://pluskid.lifegoo.com/?p=370
>
> Now, I realize that FFI is really, really new and probably not yet ready to
> handle something as complex as the needs of wxRuby. However, I wanted to
> inquire as to the principle of it.
>
> I have worked on other scripting engines before (windows platform in other
> languages). I have experience with generating code to help map complex
> structures in a native language to a simplified one in a scripted language.
> I''m just curious as to what you guys think.
>
> Ideally, I would love to see a shared library that wxRuby and wxPython
> could both use to simplify complex structures that needed to be passed to
> the scripts. Then each project would use something like FFI to import those
> functions into Ruby or Python. Then the work could be shared. :)
>
> Anyway, I''m interested in your thoughts, ideas and problem points.
>
> -MarkE
>
> _______________________________________________
> wxruby-development mailing list
> wxruby-development at rubyforge.org
> http://rubyforge.org/mailman/listinfo/wxruby-development
>
>

-- 
Mario Steele
http://www.trilake.net
http://www.ruby-im.net
http://rubyforge.org/projects/wxruby/
http://rubyforge.org/projects/wxride/
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/wxruby-development/attachments/20081109/5cbe62b1/attachment.html>

Mario,

Thanks for the response. I fully agree that getting 2.0 stable is more
important. I''m glad to hear that there are other considerations for
wrapping
libraries. I would love to be able to contribute at that level but I really
don''t like C. (Yes, I know, that''s my problem.)

So anything like FFI that can abstract it out more is a great help to me in
my ability to contribute. :)

Oh, Mario, I noticed you had a link in your sig to
"http://www.trilake.net".
I saw that it is in Council Bluffs, IA. Too funny! I grew up in Omaha, NE
and my wife and I have moved to Birmingham, AL only in the last 3 years or
so.

Hope the weather is nice there. My family is still in Omaha and I''ll be
visiting them for Christmas. It was just nice to notice a Ruby developer in
the true midwest. :)

-Mark E.

On Sun, Nov 9, 2008 at 2:29 PM, Mario Steele <mario at ruby-im.net> wrote:
> There are many good points to using something like FFI.  The major problem
> though, that I see from it at this point, is that it would require a full
> re-write of the wxRuby library.  Something we''re not planning to
do right
> now with the 2.0 release.  We''re focusing on a 2.0 full release
here very
> soon, that we can label stable, and freeze feature sets, and such.  So that
> we can finally have a stable product out there for people to use in a
> comerical setting.
>
> We''ve already been looking at possibly switching from SWIG to
something
> else, can''t remember the name of the project off the top of my
Head, but
> Alex would remember.  Sometimes, new can be good, and bad at the same time,
> so we''ll just have to test the waters, and see what we can figure
out.
>
> The major problem that I can see with this, is that wxWidgets has their own
> data types for just about everything, from Integers, to Strings, to Complex
> data formats.  While I see struct being available with FFI (Thank god, that
> makes wrapping stuff a lot easier), it still may be a problem when trying
to
> convert a Ruby String, over to a wxString.
>
> But it will definately something to look at, when we get 2.0 out the door.
>
> On Sun, Nov 9, 2008 at 1:33 PM, Mark E. <mnmfactory at gmail.com>
wrote:
>
>> This may be a misguided message, but I was wondering if any thought has
>> been given to making wxRuby work using the new native FFI (Foreign
Function
>> Interface) interface in the place of SWIG?
>>
>> Something like the FFI thing would be much easier for a non-C
programmer
>> like me to assist with.
>>
>> If you aren''t familiar with it (quite new) here are some
links.
>> http://kenai.com/projects/ruby-ffi/pages/Home
>> http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html
>> http://wmeissner.blogspot.com/2008/11/more-on-ruby-ffi.html
>> http://pluskid.lifegoo.com/?p=370
>>
>> Now, I realize that FFI is really, really new and probably not yet
ready
>> to handle something as complex as the needs of wxRuby. However, I
wanted to
>> inquire as to the principle of it.
>>
>> I have worked on other scripting engines before (windows platform in
other
>> languages). I have experience with generating code to help map complex
>> structures in a native language to a simplified one in a scripted
language.
>> I''m just curious as to what you guys think.
>>
>> Ideally, I would love to see a shared library that wxRuby and wxPython
>> could both use to simplify complex structures that needed to be passed
to
>> the scripts. Then each project would use something like FFI to import
those
>> functions into Ruby or Python. Then the work could be shared. :)
>>
>> Anyway, I''m interested in your thoughts, ideas and problem
points.
>>
>> -MarkE
>>
>> _______________________________________________
>> wxruby-development mailing list
>> wxruby-development at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/wxruby-development
>>
>>
>
>
> --
> Mario Steele
> http://www.trilake.net
> http://www.ruby-im.net
> http://rubyforge.org/projects/wxruby/
> http://rubyforge.org/projects/wxride/
>
> _______________________________________________
> wxruby-development mailing list
> wxruby-development at rubyforge.org
> http://rubyforge.org/mailman/listinfo/wxruby-development
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/wxruby-development/attachments/20081109/197ef378/attachment.html>

Hi Mark

Mark E. wrote:
> This may be a misguided message, but I was wondering if any thought 
> has been given to making wxRuby work using the new native FFI (Foreign 
> Function Interface) interface in the place of SWIG?
>
> Something like the FFI thing would be much easier for a non-C 
> programmer like me to assist with.
Thanks for bringing this up. I followed the discussion about FFI on the 
ruby mailing list with interest, mainly for this reason. Our current 
SWIG & C++ base is a really high barrier to contributions. It''s
taken me
two or three years to go from being an experienced Ruby programmer to 
having a good grasp of all the stuff that''s going on. And there are 
still subtleties and bugs that surprise me.

<http://pluskid.lifegoo.com/?p=370>
> Now, I realize that FFI is really, really new and probably not yet 
> ready to handle something as complex as the needs of wxRuby. However, 
> I wanted to inquire as to the principle of it.
It''s a good time to be discussing this. As Mario says, we
wouldn''t be
looking to change the basis of wxRuby 2.0 - it''s, at last, pretty much 
finished. But I hope there will be a wxRuby 3.0 - probably matched to 
wxWidgets 3.0 - and I want to look at whether SWIG still is the best 
choice for wrapping.

At the moment I think the biggest shortcoming for FFI for this is that 
it''s aimed at C libraries, whereas wxWidgets is very much a C++
toolset. 
>From what I can see, FFI is mainly designed to put simple C types like 
int, char[], long and so on in and out of functions. Whereas what wxRuby 
does is closely tie Ruby object instances to C++ typed pointers - 
instances of classes.

This ends up implying quite a lot of other tricky things that SWIG, plus 
our hacking on top of it, deals with. This is stuff like object tracking 
and object ownership - avoiding destroying either Ruby or C++ objects 
before time, but not leaking memory - and have the C++ core being able 
to be overridden by methods in Ruby.

What to me is an attractive possibility is if a much smaller core of 
wxRuby could be defined / run in custom C++ / SWIG / whatever code, and 
much more could be edited in Ruby, as with FFI. The sort of thing I''m 
thinking of is how the modern event mapping code (in 
lib/wx/classes/evthandler.rb) works. FFI might give us some ideas, or 
even be part of the mix.

By the way, the other new tool that Mario was thinking of is rbplusplus 
/ rbgccxml / rice. Rbgccxml uses gcc to parse the C++ headers to get the 
relevant class and method definitions. Rice provides a set of functions 
for safer conversions between C++ objects and ruby objects. And 
rbplusplus brings this all together to automatically generate glue code 
and compile it. This approach is similar to SWIG, which brings all those 
together, apart from the compiling bit.
> I have worked on other scripting engines before (windows platform in 
> other languages). I have experience with generating code to help map 
> complex structures in a native language to a simplified one in a 
> scripted language. I''m just curious as to what you guys think.
>
> Ideally, I would love to see a shared library that wxRuby and wxPython 
> could both use to simplify complex structures that needed to be passed 
> to the scripts. Then each project would use something like FFI to 
> import those functions into Ruby or Python. Then the work could be 
> shared. :)
The work that the wxWidgets team has been doing on 3.0 is partly aimed 
at this - they have created a "clean" set of C++ headers that define
the
public APIs as they should be used - and as they should be ported to 
Ruby/Python/Lua etc. The real headers are full of macros and 
base/implementation class complexities, but with the new clean headers 
it should be possible just to point a tool like SWIG or rbgccxml at them 
to generate code.

cheers
alex

On Sun, Nov 9, 2008 at 5:09 PM, Mark E. <mnmfactory at gmail.com> wrote:
> Mario,
>
> Thanks for the response. I fully agree that getting 2.0 stable is more
> important. I''m glad to hear that there are other considerations
for wrapping
> libraries. I would love to be able to contribute at that level but I really
> don''t like C. (Yes, I know, that''s my problem.)

Well, the problem with any library that''s done in C/C++, is that you
need to
have some kind of working knowledge of C/C++ to interface with it.  So in
that area, it would be kinda hard.

> So anything like FFI that can abstract it out more is a great help to me in
> my ability to contribute. :)

There are always other ways to contribute to the wxRuby development, aside
from doing the pure C/C++ Bindings.  We do have a layer of Ruby code, that
is layed over the C/C++ Extension, to make it more Ruby like, such as the
common attribute accessors, such as title, background_colour, text, value,
etc, etc, etc.  So, if your interested in helping out, and want to make the
stuff more Ruby-esque, then by all means, experiment with the code, and
check out the Current implementations, and see how it would be better to
create more Ruby-esque API.  But keep in mind, that you would also need to
document them to. ;-)

> Oh, Mario, I noticed you had a link in your sig to
"http://www.trilake.net".
> I saw that it is in Council Bluffs, IA. Too funny! I grew up in Omaha, NE
> and my wife and I have moved to Birmingham, AL only in the last 3 years or
> so.
>
> Hope the weather is nice there. My family is still in Omaha and
I''ll be
> visiting them for Christmas. It was just nice to notice a Ruby developer in
> the true midwest. :)

Yep, I grew up in Omaha, NE myself, and live in CB with my Fiancee. 
It''s
starting to get a bitter on the cold side of things.  And you''d be
surprised
how many True MidWest Ruby coders there are up here.  We have the Omaha
Dynamic language User Group (ODYNUG) up here, though there are ones from
different backgrounds, such as perl, python, Erlang, Java, there''s a
few of
us Ruby enthusiats here in the mid-west.

-- 
Mario Steele
http://www.trilake.net
http://www.ruby-im.net
http://rubyforge.org/projects/wxruby/
http://rubyforge.org/projects/wxride/
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/wxruby-development/attachments/20081109/09829874/attachment.html>

Alex,

Thanks for the response. I''m very glad to learn about the wxWidgets
header
file interface changes for 3.0. That just makes more sense.

On Sun, Nov 9, 2008 at 5:46 PM, Alex Fenton <alex at pressure.to> wrote:
> [...]
> By the way, the other new tool that Mario was thinking of is rbplusplus /
> rbgccxml / rice. Rbgccxml uses gcc to parse the C++ headers to get the
> relevant class and method definitions. Rice provides a set of functions for
> safer conversions between C++ objects and ruby objects. And rbplusplus
> brings this all together to automatically generate glue code and compile
it.
> This approach is similar to SWIG, which brings all those together, apart
> from the compiling bit.
>
Very interesting. Thanks for sharing that.

Thanks again for taking the time to explain this. I appreciate it.

-MarkE
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/wxruby-development/attachments/20081109/634e7335/attachment.html>