wxruby development - Jun 2007 - Splitting wxRuby2.so into multiple libraries

Hey Alex and fellow developers,

I figured I''d split this off into a seperate discussion, to get an idea
of
what to look at with this.  Ofcourse, with every first step, there needs  
to be some kind of outline set forth to split up wxRuby2 into multiple  
libraries, to ensure coherance amongst the developers, and future  
development, so people will know, this goes to this, this goes to that,  
etc, etc, etc.  Also, as a side note, I think the splitting up of wxRuby2  
into multiple files, to make it modular, will also encourage wrapping of  
3rd party controls that have been created for wxWidgets as well.  So this  
could be a good thing in the long run to.

Anyways, here is my suggestion for this, as I''ve helped another
developer
out with wrapping wxWidgets in another language, I saw how he designed the  
split up, and think this would be an ideal layout for this.  The way that  
it was split across the board, was there was a central file, to store the  
main stuff needed for wxWidgets to work, you know, Memory Management, main  
base classes to initialize the Application, so on and so forth.  Then each  
separate control would get it''s own file for usage, and then a wrapper
type file that basically included all of these files, for thoes that just  
want it to go, with one include/require.

So my proposal is this:

Core File:
   This will have the main static link of wxWidgets to it, so that the API  
can be exposed to other loaded modules, which can be dynamically loaded by  
user request.  Also, in this file, it will contain the main definitions  
for memory management between Ruby and wxWidgets, as well as the core  
classes, such as Wx::App, Wx::Events, Wx::Timer, so on and so forth,  
basically all the classes that do not have a graphical representation.   
Giving the developer a single file to work with for the base usability of  
wxRuby.

GUI Files:
   This part, involves setting up a single include, that would link to in  
some way (Dynamically, or Staticlly) to the Core file, to gain access to  
the wxWidgets API, to create each control / class defintion.  Such as it  
may be, Wx::Window, Wx::Frame, Wx::TextCtrl, Wx::TreeCtrl, Wx::Scintilla,  
Wx::Socket, so on and so forth.  This way, if someone wants to be mean and  
lean, they can just include the core file, then if they are only creating  
a simple text editor for an example, they would just need Wx::Frame, and  
Wx::TextCtrl or Wx::Scintilla, to create the controls that is needed.

   A part to keep this from seperating into to many seperate files, Such as  
one for Wx::Menu, Wx::ToolBar, so on and so forth, it would proabbly be  
best, to include these classes in Wx::Frame, so as to keep things  
centralized, and yet still have the modularity.  Seems a bit backwards,  
but this way, central controls will still remain with the class that they  
are used primarly for, and ensure no "Split into infinity" occurs.



Now, the fun part of this all, is how to get the splitting of the files  
setup, so that it can work together, and not cause redundancy in the core  
of wxWidgets be within each seperate library that provides the  
functionality for said control, and repeating what does not need to be  
done.  This, I don''t have an answer for, cause of two things.  One, I  
don''t know how far down you can break the wxWidgets library to just  
provide access to these controls, and allow the "Shared Memory Space"
of
the program, to allow other DLL/SO files to be able to access it.   
Secondly, I don''t know how SWIG interface works, never worked with it,
and
I hardly know C/C++ as it is, just enough to get by, and even then it''s
troublesome at most.  One solution that I could think of, is to have  
wxWidgets as it''s own library, then Dynamically link to it, instead of
statically linking it into the wxRuby2 SO File.  Which seems a bit  
redundant, but it''s just one way I would think of going about it, and
may
provide some speed improvements as well, considering that the DLL/SO that  
wxWidgets runs on, would not be directly impacted by the speed of Ruby  
itself.  Though that may be negligable in the long run as it is.

With this, I''m just laying out a basic idea of how to split these into
seperate parts, and see what people think.  If it''s a good idea, if
it''s a
bad idea, what opinions you have, maybe you have a better way.  All great,  
cause anything to bring this together would only benifit wxRuby in the  
long run, and open for expansion later on down the road.  Maybe looking at  
how 3rd Party wxWidget libraries are made up, may offer some insight into  
this delima?

I leave with that, and hope to discuss this further with you guys.

Mario Steele

-- 
Using Opera''s revolutionary e-mail client: http://www.opera.com/mail/

Hi Mario

Mario Steele wrote:
> I figured I''d split this off into a seperate discussion, to get an
idea of
> what to look at with this.  Ofcourse, with every first step, there needs  
> to be some kind of outline set forth to split up wxRuby2 into multiple  
> libraries, 
Thanks for this - they''re useful notes and I agree with most of what 
you''ve set out.
>  Also, as a side note, I think the splitting up of wxRuby2  
> into multiple files, to make it modular, will also encourage wrapping of  
> 3rd party controls that have been created for wxWidgets as well.  
That''s a good point - I hadn''t thought of this. One of
wxWidget''s
strengths is the third party controls available.
> The way that  
> it was split across the board, was there was a central file, to store the  
> main stuff needed for wxWidgets to work, you know, Memory Management, main
> base classes to initialize the Application, so on and so forth.
This corresponds to swig/wx.i + swig/App.i + swig/RubyConstants.i in wxRuby
>  Then each  
> separate control would get it''s own file for usage, and then a
wrapper
> type file that basically included all of these files, for thoes that just  
> want it to go, with one include/require.
>   
I''d go for this, or we could bundle groups of related classes into 
compiled groups. The model for the loader wrapper file is lib/wx.rb

One ruby feature that might be potentially very useful is
''autoload''.
With this users could still only have to require ''wx'', but
other classes
would be transparently loaded on demand when they were referenced by
code.
>    This will have the main static link of wxWidgets to it, so that the API
> can be exposed to other loaded modules, which can be dynamically loaded by
> user request.  Also, in this file, it will contain the main definitions  
> for memory management between Ruby and wxWidgets, as well as the core  
> classes, such as Wx::App, Wx::Events, Wx::Timer, so on and so forth,  
> basically all the classes that do not have a graphical representation.   
>
>   
(and also defining the autoloads for other classes, setting out where 
they are found).
>    A part to keep this from seperating into to many seperate files, Such as
> one for Wx::Menu, Wx::ToolBar, so on and so forth, it would proabbly be  
> best, to include these classes in Wx::Frame, so as to keep things  
> centralized, and yet still have the modularity.  Seems a bit backwards,  
> but this way, central controls will still remain with the class that they  
> are used primarly for, and ensure no "Split into infinity"
occurs.
>   
Agreed. If you''ve compiled your own wxRuby, it''s interesting
to look at
the obj/ directory sorted by size. The biggest is Grid.o, which along 
with the various Grid* events would make a large separate chunk. 
Similarly TextCtrl, TreeCtrl, ListCtrl, HtmlWindow and AuiNotebook, 
which are all heavyweights on that list and have a number of associated 
classes.

The current rework of EvtHandler and the mapping of wx Events to ruby 
objects is one piece of the puzzle that will make it possible to load 
new event-firing classes once an App has started.
> Now, the fun part of this all, is how to get the splitting of the files  
> setup, so that it can work together, and not cause redundancy in the core  
> of wxWidgets be within each seperate library that provides the  
> functionality for said control, and repeating what does not need to be  
> done.  This, I don''t have an answer for, cause of two things. 
One, I
> don''t know how far down you can break the wxWidgets library to
just
> provide access to these controls, and allow the "Shared Memory
Space" of
> the program, to allow other DLL/SO files to be able to access it.   
>   
Hehe - this is the fun part for me too. I don''t know very much at all 
about linkers and linking. I really knew nothing but ''./configure;
make;
make install'' about C++ and compiling before I got involved in wxRuby.
> Secondly, I don''t know how SWIG interface works, never worked with
it, and
> I hardly know C/C++ as it is, just enough to get by, and even then
it''s
> troublesome at most.  
There''s no doubt that SWIG is pretty complex, and learning a little C++
has driven me to frustration at times. On the other hand, I''m hoping 
that recent changes have made the build process a bit more standard and 
straightforward.
> One solution that I could think of, is to have  
> wxWidgets as it''s own library, then Dynamically link to it,
instead of
> statically linking it into the wxRuby2 SO File.  
>   
I''m not necessarily against this, but I''m very keen that we
keep wxRuby
as a single-step install. Having to install the base GUI library first 
seems to be a source of confusion for new users of Qt and GTK, and not 
having to a perceived strength of wxRuby.

Perhaps we could distribute a wxWidgets monolithic dll/bundle/so with 
the gems that is dynamically linked to?
> With this, I''m just laying out a basic idea of how to split these
into
> seperate parts, and see what people think.  If it''s a good idea,
if it''s a
> bad idea, what opinions you have, maybe you have a better way.  
As above, I broadly support all that you''ve suggested. In terms of 
timing, given that we''re not 100% sure about the specifics of the 
linking bit in particular, I''m inclined to set this as an objective for
wxRuby 2.2. I.e. aim to finish what we have now and release as a stable 
2.0, and then break into pieces for the next minor release in a way 
that''s compatible (eg using autoload).

cheers
alex

On Fri, 29 Jun 2007 04:05:22 -0500, Alex Fenton <alex at pressure.to>
wrote:
> Hi Mario
>
> Mario Steele wrote:
>> I figured I''d split this off into a seperate discussion, to
get an idea
>> of
>> what to look at with this.  Ofcourse, with every first step, there
needs
>> to be some kind of outline set forth to split up wxRuby2 into multiple
>> libraries,
> Thanks for this - they''re useful notes and I agree with most of
what
> you''ve set out.
No problem, figure the first step with anything, is to start out with a  
plan.
>>  Also, as a side note, I think the splitting up of wxRuby2
>> into multiple files, to make it modular, will also encourage wrapping
of
>> 3rd party controls that have been created for wxWidgets as well.
> That''s a good point - I hadn''t thought of this. One of
wxWidget''s
> strengths is the third party controls available.
Very much so.  GTK and Qt have the problem that they are pretty much set  
with the widgets they get with the library, unless you extend them  
yourself in your Ruby code, but wxWidgets has libraries for extended  
controls out there, that you can link to.  But right now, it''s just not
possible with wxRuby2.
>> The way that
>> it was split across the board, was there was a central file, to store  
>> the
>> main stuff needed for wxWidgets to work, you know, Memory Management,  
>> main
>> base classes to initialize the Application, so on and so forth.
> This corresponds to swig/wx.i + swig/App.i + swig/RubyConstants.i in  
> wxRuby
That''s good to hear, atleast now we got most of the core work out of
the
way. LOL
>>  Then each
>> separate control would get it''s own file for usage, and then a
wrapper
>> type file that basically included all of these files, for thoes that  
>> just
>> want it to go, with one include/require.
>>
> I''d go for this, or we could bundle groups of related classes into
> compiled groups. The model for the loader wrapper file is lib/wx.rb
>
> One ruby feature that might be potentially very useful is
''autoload''.
> With this users could still only have to require ''wx'',
but other classes
> would be transparently loaded on demand when they were referenced by  
> code.
This would be a very useful tool, and would allow for some good hits, but  
at the same time, we need to be careful, incase that part isn''t
included
for some reason, maybe a Wx::MessageDialog may be used in case of failure  
to find an associated library, instead of a simple crash, and burn case  
scenerio.  This is thinking way to far ahead, but it''s a start none the
less.
>>    This will have the main static link of wxWidgets to it, so that the
>> API
>> can be exposed to other loaded modules, which can be dynamically loaded
>> by
>> user request.  Also, in this file, it will contain the main definitions
>> for memory management between Ruby and wxWidgets, as well as the core
>> classes, such as Wx::App, Wx::Events, Wx::Timer, so on and so forth,
>> basically all the classes that do not have a graphical representation.
>>
>>
> (and also defining the autoloads for other classes, setting out where
> they are found).
>
Agreed again.
>>    A part to keep this from seperating into to many seperate files,  
>> Such as
>> one for Wx::Menu, Wx::ToolBar, so on and so forth, it would proabbly be
>> best, to include these classes in Wx::Frame, so as to keep things
>> centralized, and yet still have the modularity.  Seems a bit backwards,
>> but this way, central controls will still remain with the class that  
>> they
>> are used primarly for, and ensure no "Split into infinity"
occurs.
>>
> Agreed. If you''ve compiled your own wxRuby, it''s
interesting to look at
> the obj/ directory sorted by size. The biggest is Grid.o, which along
> with the various Grid* events would make a large separate chunk.
> Similarly TextCtrl, TreeCtrl, ListCtrl, HtmlWindow and AuiNotebook,
> which are all heavyweights on that list and have a number of associated
> classes.
>
> The current rework of EvtHandler and the mapping of wx Events to ruby
> objects is one piece of the puzzle that will make it possible to load
> new event-firing classes once an App has started.
>
On windows here, only ever used the Binary distro''s of wxRuby2.  But I
wouldn''t doubt the size of Object files created, mainly cause of the
fact
that when you compile the source code into object files, it''s not
stripped
of the redundancy factor.  It statically links everything together, so  
that way, if it''s missed in another file, it''s still readily
available.
This is part of the reason why compile processes take so long.  The last  
part of the linking procedure, is on the final output of the DLL/SO, where  
it strips all recurring object code, if it''s found in another file
that''s
being linked into the dll/so first.  So yeah, tends to be alot of  
unnesscary bloat.  But at the same time, SWIG tends to add to the bloat  
to, as well as Ruby''s includes, so there''s some things that
just cannot be
avoided.
>> Now, the fun part of this all, is how to get the splitting of the files
>> setup, so that it can work together, and not cause redundancy in the  
>> core
>> of wxWidgets be within each seperate library that provides the
>> functionality for said control, and repeating what does not need to be
>> done.  This, I don''t have an answer for, cause of two things. 
One, I
>> don''t know how far down you can break the wxWidgets library to
just
>> provide access to these controls, and allow the "Shared Memory
Space" of
>> the program, to allow other DLL/SO files to be able to access it.
>>
> Hehe - this is the fun part for me too. I don''t know very much at
all
> about linkers and linking. I really knew nothing but ''./configure;
make;
> make install'' about C++ and compiling before I got involved in
wxRuby.
As you can tell from my previous statement, I have a working knowledge of  
C++ and compiling, beyond the quick build commands for ''./configure;
make;
make install''.  But at the same time, there''s some things that
elude me.
In all experince, once a library is loaded into the Program''s Memory  
space, all DLL/SO/Machine Code has access to the routines stored within  
that library.  Which is why Dynamic Link is such a popular thing.  All is  
really needed, is a handle to the library in question, in order to access  
it''s functions, and ofcourse, handles to the functions within the
library
itself.  Which is why the dynamic link library protocol was created in the  
first place, so you didn''t need to have a big executable, with
everything
inside of it, and you can have universal feel on different machines.
>> Secondly, I don''t know how SWIG interface works, never worked
with it,
>> and
>> I hardly know C/C++ as it is, just enough to get by, and even then
it''s
>> troublesome at most.
> There''s no doubt that SWIG is pretty complex, and learning a
little C++
> has driven me to frustration at times. On the other hand, I''m
hoping
> that recent changes have made the build process a bit more standard and
> straightforward.
Attempting to learn C/C++ is like learning to bash your head into a brick  
wall repeatedly and willingly.  You know it''s going to hurt, you know
it''s
going to be hard, but there''s no way around it, so you go straight for
it.  Part of the reason why I like the higher level langauges (AKA  
Interpreted langauges), cause they take alot of the complexity out of the  
programming.
>> One solution that I could think of, is to have
>> wxWidgets as it''s own library, then Dynamically link to it,
instead of
>> statically linking it into the wxRuby2 SO File.
>>
> I''m not necessarily against this, but I''m very keen that
we keep wxRuby
> as a single-step install. Having to install the base GUI library first
> seems to be a source of confusion for new users of Qt and GTK, and not
> having to a perceived strength of wxRuby.
>
> Perhaps we could distribute a wxWidgets monolithic dll/bundle/so with
> the gems that is dynamically linked to?
I can understand that point of view as well, as it is the reason why I  
came back to wxWidgets when you guys started getting wxRuby2 out there.   
Ruby-Gnome2 is a very mature project, and there''s alot of great support
there, but at the same time, it requires the user to install this major  
runtime distrobution, and that tends to leave a bleak taste even in my  
mouth.  Don''t like having to have my users download another 8 meg  
installer file, that''s going to install a bunch of dll''s and
po files,
just to run my 2 meg program.

But, at the same time, wxWidgets isn''t as monolithic as GTK, or Qt.   
Thankfully, this is cause wxWidgets uses native libraries to the Operating  
System to make everything go.  And I like that about wxWidgets.  Last time  
I checked, the only two Operating Systems that seem to have more then one  
dll/so library to install, is wxMac and wxGTK.  But even then, I don''t
think it''s that many files, nor is it that big.  Could be wrong though.
I
think if we do go this route, then we could definatly set it up where  
there''s a single gem file for wxRuby2, that the users already have  
wxWidgets installed, and then one for the, how shall I put it, "Dummy in  
training" install, for thoes who don''t know anything better.
>> With this, I''m just laying out a basic idea of how to split
these into
>> seperate parts, and see what people think.  If it''s a good
idea, if
>> it''s a
>> bad idea, what opinions you have, maybe you have a better way.
> As above, I broadly support all that you''ve suggested. In terms of
> timing, given that we''re not 100% sure about the specifics of the
> linking bit in particular, I''m inclined to set this as an
objective for
> wxRuby 2.2. I.e. aim to finish what we have now and release as a stable
> 2.0, and then break into pieces for the next minor release in a way
> that''s compatible (eg using autoload).
I''m inclined to agree with you here.  The biggest obsticle for this, is
ensuring that we don''t have the full wxWidgets core library compiled
into
each seperate library that we split these off into.  The only thing that I  
can think of as of right now, is to break from the Static Linking, and go  
to the Dynamic Linking, which would allow us to keep the so/dll files  
small for the extensions in Ruby, while allowing a single DLL/SO file for  
the core wxWidgets to be used on the system.  While I''d love to have
this
for 2.0, I think 2.2 would be a more likely target for this objective.   
With 2.1 being the beta to break these down, and getting Sockets into the  
library.


I do know one thing though, it seems to me, that with a project like  
Ruby-Gnome2, they seem to seperate the files based apon what library they  
are working with, and they go the dynamic link route on it.  So I''m  
thinking we are proabbly going to need the same idea, only, instead of it  
being multiple dynamic link libraries that we are linking to, it''s the
same one.

Anyways, glad to hear you agree with what my thinking is on the subject.   
Anyone else is welcome to join in on this, cause there may be something me  
or Alex is missing on it.  Who knows.

L8ers,
Mario Steele

-- 
Using Opera''s revolutionary e-mail client: http://www.opera.com/mail/