wxruby users - May 2010 - need some advice : scientific scripting language project

Hello,

Ok, I am writing a (pretty big) project trying to use ruby as the main
scripting language for performing math computation, scientific display
and display interaction.
Here is roughly what I want

- a terminal window with a ruby interpreter in it (I want to be able to
control all the completions/editions capabilities)
- a bench of ruby classes to manipulate signals/images,etc...
- I want everything to be really really fast (I''ll be manipulating huge
signals/images)
- simple commands to display instances of these classes
- graphic (programable) interaction on these displays (e.g. moving the
mouse on a such a window should display image values, clicking should
zoom, etc....

Now my questions are the following

- Should I use irb ? Can I wrap it easily in a wxWidget window ? Can I
reprogram some new capabilities (e.g., history completion, multi-line
history commands...)

- A unix terminal window displays output text really really fast.. what
object should I use in wxRuby to display text as fast (wxTextCtrl is
pretty slow, even if I use a buffer)

- I want to be able to interrupt long computations (clicking on a stop
button or typing ctrl-c on the terminal window) ==> I should be able to
send SIGINT signal to the interpreter. Should I use 2 processes : one
for the terminal and one for the rest ?
The problem if I do that is that both processes will have a
wxMainLoop... now what will be the best way for them to communicate ?

Thank you very much

Emmanuel
-- 
Posted via http://www.ruby-forum.com/.

Hi Emmanuel

On 19/05/2010 07:47, Emmanuel Emmanuel wrote:
> Ok, I am writing a (pretty big) project trying to use ruby as the main
> scripting language for performing math computation, scientific display
> and display interaction.
>    
Sounds like a big and interesting project. I originally got interested 
in wxRuby building research tools (for sociology) so I''d certainly be 
interested to hear your progress.
> Here is roughly what I want
>
> - a terminal window with a ruby interpreter in it (I want to be able to
> control all the completions/editions capabilities)
>    
readline?
> - a bench of ruby classes to manipulate signals/images,etc...
> - I want everything to be really really fast (I''ll be manipulating
huge
> signals/images)
>    
ruby/gsl? r-ruby?
> - simple commands to display instances of these classes
> - graphic (programable) interaction on these displays (e.g. moving the
> mouse on a such a window should display image values, clicking should
> zoom, etc....
>    
wxRuby is well suited to this sort of application.
> Now my questions are the following
>
> - Should I use irb ? Can I wrap it easily in a wxWidget window ? Can I
> reprogram some new capabilities (e.g., history completion, multi-line
> history commands...)
>    
I don''t know of any examples of wrapping IRB in wxRuby, although it
does
provide a library interface. You might look into Wx::StyledTextCtrl 
which offers syntax highlighting and other more sophisticated support 
for programming code text.
> - A unix terminal window displays output text really really fast.. what
> object should I use in wxRuby to display text as fast (wxTextCtrl is
> pretty slow, even if I use a buffer)
>    
The text display classes are Wx::TextCtrl, Wx::LogTextCtrl, 
Wx::StyledTextCtrl, Wx::RichTextCtrl and Wx::HtmlControl. I''d expect 
simple TextCtrl or StyledTextCtrl to be fastest, but I''ve never tested.

Judicious use of #freeze and #thaw might help to manage updates on 
screen. Generally Ruby is fairly slow, and wxRuby''s bridge to C++ adds 
additional overhead to each Wx method call, so you might improve speed 
by batching calls to the Wx API to update the TextCtrl''s
context.
> - I want to be able to interrupt long computations (clicking on a stop
> button or typing ctrl-c on the terminal window) ==>  I should be able to
> send SIGINT signal to the interpreter. Should I use 2 processes : one
> for the terminal and one for the rest ?
> The problem if I do that is that both processes will have a
> wxMainLoop... now what will be the best way for them to communicate ?
>    
You have a couple of options:

- if the computational task can be broken down into small discrete 
elements, you can execute these in an evt_idle handler, which runs when 
the GUI isn''t doing anything else. This is the easiest route because
you
can allow interruptions using normal wxRuby event handling techniques.

- you could use Ruby threads and run the computation in a slave thread. 
This is somewhat more complex because the slave thread should not 
directly update the GUI, but should signal its state by posting Wx 
events to the main GUI thread. A class that does this sort of thing is 
here (afraid it''s embedded in a fairly complex Wx app):

http://weft-qda.rubyforge.org/svn/trunk/weft-qda/lib/weft/wxgui/gui_subscriber.rb

If you use wxRuby and threads in Ruby 1.8 you need to explicitly 
allocate time to the slave thread - there are numerous discussions of 
this in the list archives.

- you could fork / pipe and run the computation as a separate process. 
There is no need for this to be a GUI process, I presume, so you don''t 
need to worry about a separate GUI main_loop. One good thing about this 
route is that it will take advantage of multiple cores if they''re 
available, and will separate the GUI and analytic code. But it''ll 
involve working out how to pass data to the subsidiary process (eg 
popen3?) and manage and signal to it (SIGINT etc?). Given Ruby''s unix 
background, managing a child process is easier if you only need the 
program to work on *nix platforms, not Windows.

hth
alex

First of all, Thank you Alex for all your interesting comments !

>Alex Fenton wrote:
> Hi Emmanuel
> 
> Sounds like a big and interesting project. I originally got interested
> in wxRuby building research tools (for sociology) so I''d certainly
be
> interested to hear your progress.
> 
 I will let you know. Actually I already wrote a script language (called 
LastWave, http://www.cmap.polytechnique.fr/~bacry/LastWave/index.html) 
that has been quite used by a lot of persons... I started it a long time 
ago, so I did not use the modern script language (python or ruby)...
it is version 3.0 now and using wxWidget.... but it clearly needs to be 
rewritten using a good scripting language... it needs to be entirely 
redesigned....
However I learned a lot, and I am pretty clear about some of the issues 
(memory, speed, some graphics issue).

I know there exist some language such as Matlab ($$$) or R (free), but I 
do think these languages suffer from some drawbacks (I could develop 
that point but it is not the subject!).
Moreover, if I write such a soft using Ruby for instance, I could 
benefit from both Matlab an R (there are libraries to encapsulate their 
engine).....
>> Here is roughly what I want
>>
>> - a terminal window with a ruby interpreter in it (I want to be able to
>> control all the completions/editions capabilities)
>>    
> 
> readline?
I am not sure to understand... I want a window in which I can redirect 
any key being hit so that I can implement multi-line editing commands, 
history completion and so on .....
readline is a library ?
> 
>> - a bench of ruby classes to manipulate signals/images,etc...
>> - I want everything to be really really fast (I''ll be
manipulating huge
>> signals/images)
>>    
> 
> ruby/gsl? r-ruby?
nope and this is clear from my experience.... I know pretty precisely 
what I want ... and it is not on the market... I''ll have to do it (I 
alredy started and it is not the hardes part at all).
But yes, I want my soft to be able to communicate with the libraries 
that deal with the Matlab engine and the R engine


> 
>> - simple commands to display instances of these classes
>> - graphic (programable) interaction on these displays (e.g. moving the
>> mouse on a such a window should display image values, clicking should
>> zoom, etc....
>>    
> 
> wxRuby is well suited to this sort of application.
Yes I know it is, since LastWave has been done fully on wxWidget !
> 
>> Now my questions are the following
>>
>> - Should I use irb ? Can I wrap it easily in a wxWidget window ? Can I
>> reprogram some new capabilities (e.g., history completion, multi-line
>> history commands...)
>>    
> 
> I don''t know of any examples of wrapping IRB in wxRuby, although
it does
> provide a library interface. You might look into Wx::StyledTextCtrl
> which offers syntax highlighting and other more sophisticated support
> for programming code text.
Is there a irb documentation (of the class I mean) anywhere on the web ? 
I could not find it!
>> - A unix terminal window displays output text really really fast.. what
>> object should I use in wxRuby to display text as fast (wxTextCtrl is
>> pretty slow, even if I use a buffer)
>>    
> 
> The text display classes are Wx::TextCtrl, Wx::LogTextCtrl,
> Wx::StyledTextCtrl, Wx::RichTextCtrl and Wx::HtmlControl. I''d
expect
> simple TextCtrl or StyledTextCtrl to be fastest, but I''ve never
tested.
No wxTextCtrl is what I used on LastWave and I have been doing some test 
on wxRuby recently. It is fast, but really nothing compared to a unix 
terminal....
> 
> Judicious use of #freeze and #thaw might help to manage updates on
> screen. 
There are methods of which class ?
> Generally Ruby is fairly slow, and wxRuby''s bridge to C++ adds
> additional overhead to each Wx method call, so you might improve speed
> by batching calls to the Wx API to update the TextCtrl''s context.
Yes I agree and plan to do that (though I heard ruby 1.9 is much faster! 
Do you think I should use python instead?)
>> - I want to be able to interrupt long computations (clicking on a stop
>> button or typing ctrl-c on the terminal window) ==>  I should be
able to
>> send SIGINT signal to the interpreter. Should I use 2 processes : one
>> for the terminal and one for the rest ?
>> The problem if I do that is that both processes will have a
>> wxMainLoop... now what will be the best way for them to communicate ?
>>    
> 
> You have a couple of options:
> 
> - if the computational task can be broken down into small discrete
> elements, you can execute these in an evt_idle handler, which runs when
> the GUI isn''t doing anything else. This is the easiest route
because you
> can allow interruptions using normal wxRuby event handling techniques.
No, what you call the "comoutational part" is not only
computational....
It has graphics drawing (that must be fast) ... I want to write 
something "a la matlab" so there is not a clear cut in between 
terminal/graphics and computations

> 
> - you could use Ruby threads and run the computation in a slave thread.
> This is somewhat more complex because the slave thread should not
> directly update the GUI, but should signal its state by posting Wx
> events to the main GUI thread. A class that does this sort of thing is
> here (afraid it''s embedded in a fairly complex Wx app):
> 
>
http://weft-qda.rubyforge.org/svn/trunk/weft-qda/lib/weft/wxgui/gui_subscriber.rb
> 
> If you use wxRuby and threads in Ruby 1.8 you need to explicitly
> allocate time to the slave thread - there are numerous discussions of
> this in the list archives.
Yes I tried (maybe too naively) doing threads.... but it slowed down the 
computations a lot .. again I maight have done it the wrong way !
> 
> - you could fork / pipe and run the computation as a separate process.
> There is no need for this to be a GUI process, I presume, so you
don''t
> need to worry about a separate GUI main_loop. One good thing about this
> route is that it will take advantage of multiple cores if they''re
> available, and will separate the GUI and analytic code. But it''ll
> involve working out how to pass data to the subsidiary process (eg
> popen3?) and manage and signal to it (SIGINT etc?). Given Ruby''s
unix
> background, managing a child process is easier if you only need the
> program to work on *nix platforms, not Windows.
I want to work on ALL platforms. You think I sould not use 2 processes 
then ?
Now how would the 2 processes communicate if they are both running 
wxMainLoop (one for the terminal and the other one for all other 
graphics...) ?

There is one thing VERY important for me : I need to be able to stop a 
computation if the user decides it''s taking too long for instance...
and
I don''t want the soft to crash !


Thank you again

Emmanuel
> 
> hth
> alex
-- 
Posted via http://www.ruby-forum.com/.

On 19/05/2010 14:10, Emmanuel Emmanuel wrote:
>   I will let you know. Actually I already wrote a script language (called
> LastWave, http://www.cmap.polytechnique.fr/~bacry/LastWave/index.html)
> that has been quite used by a lot of persons... I started it a long time
> ago, so I did not use the modern script language (python or ruby)...
> it is version 3.0 now and using wxWidget.... but it clearly needs to be
> rewritten using a good scripting language... it needs to be entirely
> redesigned...
>    
Looks good, although I don''t understand the topic of analysis ;)
>>> Here is roughly what I want
>>>
>>> - a terminal window with a ruby interpreter in it (I want to be
able to
>>> control all the completions/editions capabilities)
>>>
>>>        
>> readline?
>>      
> I am not sure to understand... I want a window in which I can redirect
> any key being hit so that I can implement multi-line editing commands,
> history completion and so on .....
> readline is a library ?
>    
I was wondering if the library could be used to provide 
command-completion and history features - but on looking now maybe it''s
quite tied to the terminal which hosts it.
>> ruby/gsl? r-ruby?
>>      
> nope and this is clear from my experience.... I know pretty precisely
> what I want ... and it is not on the market... I''ll have to do it
(I
> alredy started and it is not the hardes part at all).
> But yes, I want my soft to be able to communicate with the libraries
> that deal with the Matlab engine and the R engine
>    
I''d be really interested to hear how you find the Ruby-R bindings, as 
I''ve been doing increasing amounts of stat analysis with R over the
past
year.
>> I don''t know of any examples of wrapping IRB in wxRuby,
although it does
>> provide a library interface. You might look into Wx::StyledTextCtrl
>> which offers syntax highlighting and other more sophisticated support
>> for programming code text.
>>      
> Is there a irb documentation (of the class I mean) anywhere on the web ?
> I could not find it!
>    
I can''t find anything either - it doesn''t appear on
ruby-doc.org even
though it''s part of the standard library. I think when I did a bit of 
work extending it I just relied on the source files.

The 1.9 module structure and method listing is here:
http://ruby-doc.org/ruby-1.9/classes/IRB.html
>>> - A unix terminal window displays output text really really fast..
what
>>> object should I use in wxRuby to display text as fast (wxTextCtrl
is
>>> pretty slow, even if I use a buffer)
>>>
>>>        
>> The text display classes are Wx::TextCtrl, Wx::LogTextCtrl,
>> Wx::StyledTextCtrl, Wx::RichTextCtrl and Wx::HtmlControl. I''d
expect
>> simple TextCtrl or StyledTextCtrl to be fastest, but I''ve
never tested.
>>      
> No wxTextCtrl is what I used on LastWave and I have been doing some test
> on wxRuby recently. It is fast, but really nothing compared to a unix
> terminal....
>    
No, I imagine not. Like to know whether there is a perceptible speed 
difference between wxTextCtrl (C++) and Wx::TextCtrl (Ruby) performing a 
comparable task.
>> Judicious use of #freeze and #thaw might help to manage updates on
>> screen.
>>      
> There are methods of which class ?
>    
Wx::Window, so applies to all things that are drawn on screen.
>> Generally Ruby is fairly slow, and wxRuby''s bridge to C++ adds
>> additional overhead to each Wx method call, so you might improve speed
>> by batching calls to the Wx API to update the TextCtrl''s
context.
>>      
> Yes I agree and plan to do that (though I heard ruby 1.9 is much faster!
> Do you think I should use python instead?)
>    
Ruby 1.9 brought particular speed improvements to some numeric 
operations, which might be relevant to your project.

Python?! I don''t know. It probably is faster. But it''s not so
much
faster that it should outweigh your preference of language, and tools. 
wxPython is of course very well established, with a longer history than 
wxRuby.
> No, what you call the "comoutational part" is not only
computational....
> It has graphics drawing (that must be fast) ... I want to write
> something "a la matlab" so there is not a clear cut in between
> terminal/graphics and computations
>    
It sounds like you have a decomposable task - ie there are interim or 
partial results from part of the whole computation. So any option is 
open to you. It''s a question of what''s fast enough, and how
it''s most
comfortable to pass data (results) back to the GUI layer to be drawn on 
screen.
>> If you use wxRuby and threads in Ruby 1.8 you need to explicitly
>> allocate time to the slave thread - there are numerous discussions of
>> this in the list archives.
>>      
> Yes I tried (maybe too naively) doing threads.... but it slowed down the
> computations a lot .. again I maight have done it the wrong way !
>    
Yes, it''ll be pretty inefficient because the Ruby thread scheduler is 
being forced to jump (rather clumsily) between the GUI event loop and 
the computational task in the slave thread. You might test evt_idle to 
see if it processes the same task faster.
>> - you could fork / pipe and run the computation as a separate process.
>> There is no need for this to be a GUI process, I presume, so you
don''t
>> need to worry about a separate GUI main_loop. One good thing about this
>> route is that it will take advantage of multiple cores if
they''re
>> available, and will separate the GUI and analytic code. But
it''ll
>> involve working out how to pass data to the subsidiary process (eg
>> popen3?) and manage and signal to it (SIGINT etc?). Given
Ruby''s unix
>> background, managing a child process is easier if you only need the
>> program to work on *nix platforms, not Windows.
>>      
> I want to work on ALL platforms. You think I sould not use 2 processes
> then ?
>    
Ordinary ruby will not take advantage of more than one core - but if you 
spawn a separate process it may run on a different core. This may be a 
big advantage for your use case.
> Now how would the 2 processes communicate if they are both running
> wxMainLoop (one for the terminal and the other one for all other
> graphics...) ?
>    
You could perhaps open a subprocess with IO#popen 
(http://ruby-doc.org/core/classes/IO.html#M002242), write the data to be 
processed to the process''s STDIN, and then read the results back as 
they''re available on the process''s STDOUT (and do any
graphical display
that''s wanted).
> There is one thing VERY important for me : I need to be able to stop a
> computation if the user decides it''s taking too long for
instance... and
> I don''t want the soft to crash !
>    
Several ways - perhaps just closing the pipes to the subprocess will 
kill it; you could send it a system signal; or kill it with Process. 
Afraid I don''t know what is needed to make this portable to Windows.

a

Thanks again for teh time you spend answering me !
> I''d be really interested to hear how you find the Ruby-R bindings,
as
> I''ve been doing increasing amounts of stat analysis with R over
the past
> year.
Ok, let me go into it further and I''ll develop that in an other post.
> No, I imagine not. Like to know whether there is a perceptible speed
> difference between wxTextCtrl (C++) and Wx::TextCtrl (Ruby) performing a
> comparable task.
No there is none
>>> Judicious use of #freeze and #thaw might help to manage updates on
>>> screen.
>>>      
>> There are methods of which class ?
>>    
> 
> Wx::Window, so applies to all things that are drawn on screen.
thanks !
> Ruby 1.9 brought particular speed improvements to some numeric
> operations, which might be relevant to your project.
> 
> Python?! I don''t know. It probably is faster. But it''s
not so much
> faster that it should outweigh your preference of language, and tools.
> wxPython is of course very well established, with a longer history than
> wxRuby.
ok
>> Yes I tried (maybe too naively) doing threads.... but it slowed down
the
>> computations a lot .. again I maight have done it the wrong way !
>>    
> 
> Yes, it''ll be pretty inefficient because the Ruby thread scheduler
is
> being forced to jump (rather clumsily) between the GUI event loop and
> the computational task in the slave thread. You might test evt_idle to
> see if it processes the same task faster.
I''ll try!
>> I want to work on ALL platforms. You think I sould not use 2 processes
>> then ?
>>    
> 
> Ordinary ruby will not take advantage of more than one core - but if you
> spawn a separate process it may run on a different core. This may be a
> big advantage for your use case.
> 
>> Now how would the 2 processes communicate if they are both running
>> wxMainLoop (one for the terminal and the other one for all other
>> graphics...) ?
>>    
> 
> You could perhaps open a subprocess with IO#popen
> (http://ruby-doc.org/core/classes/IO.html#M002242), write the data to be
> processed to the process''s STDIN, and then read the results back
as
> they''re available on the process''s STDOUT (and do any
graphical display
> that''s wanted).
yes, so for that, the receiving process should have idle events reading 
the stdin right to wait for inputs right?

Do you know if this works well on Windows platform ?

>> There is one thing VERY important for me : I need to be able to stop a
>> computation if the user decides it''s taking too long for
instance... and
>> I don''t want the soft to crash !
>>    
> 
> Several ways - perhaps just closing the pipes to the subprocess will
> kill it; you could send it a system signal; or kill it with Process.
> Afraid I don''t know what is needed to make this portable to
Windows.
This is the signal thing you are afraid it might be not portable or even 
more than that ?

Emmanuel

-- 
Posted via http://www.ruby-forum.com/.

Emmanuel Emmanuel wrote:
> Thanks again for teh time you spend answering me !
>   
You''re welcome.
>> No, I imagine not. Like to know whether there is a perceptible speed
>> difference between wxTextCtrl (C++) and Wx::TextCtrl (Ruby) performing
a
>> comparable task.
>>     
> No there is none
>   

That''s useful to know, thanks.
> yes, so for that, the receiving process should have idle events reading 
> the stdin right to wait for inputs right?
>   
Something like that, yes. Use evt_idle if possible since you can 
directly update the GUI state in that handler. Dealing with threads, 
posting events, synchronisation etc is much more hassle.
> Do you know if this works well on Windows platform ?
>   
IIRC Ruby''s own fork, popen and popen3 DON''T work on Windows.
However
there are definitely libraries which provide comparable functionality 
and similar API through the Win32 API.
> This is the signal thing you are afraid it might be not portable or even 
> more than that ?
I''m not sure but I think SIGINT / SIGKILL are *nix specific. Just
having
the pipe might be enough to control the slave process  and terminate 
when needed. Otherwise there are various other options for communicating 
between processes - have a search for ''ruby IPC''.

best
alex

Not to jump in and ruin the discussion, quite good stuff.  I just have one
thing to comment on:
popen3/popen/etc/etc.

When I was working on my Ruby IDE, I found the limitation of popen/popen3,
and the very many different implementations of how to properly utilize the
API to do what you wanted.  So I created my own Library, called xprocess.
 It utilizes the same methods that are used across all platforms, and
provides a singular, unified API, that is consistent, and the same across
all platform.

I don''t have any Gem build of it, or published on RubyForge, as I only
had
the need for the library at the time.  It will need to be compiled on any
platform you wish to implement it on, but you can get it from
http://repos.ruby-im.net/xprocess  There is currently no documentation for
it, but you can look in the test directory, and see a couple of samples of
how to use it.

hth,

Mario

-- 
Mario Steele
Lieutenant 3 - Geo 99
XO - STO IFT Fleet
Chief Engineer - Second Life
http://www.iftcommand.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/wxruby-users/attachments/20100519/4a0b1a5e/attachment.html>

Ok,

let me describe what I have done and working very well on a macos X....
I am going to sum up the code ... (just have in mind that the goal is to 
get an interaction between a terminal window and computations/graphics 
in the "same" way as R for instance).

I still have a few questions (to both of you !) and any comments will be 
great !

====The father process opens a child process (using popen mode "r+")
and
opens a window with a TextCtrl in it and just waits with wxMainLoop.
This process has basically two possible status : i) we are typing in a 
command (so this is a basic line-editor program) and as soon as we hit a 
return key, this command is sent to the child process using pipe.puts 
(pipe is the object returned by the popen command for now) and then it 
switches to the ii) waiting mode.
In this last mode, the process (every 10ms) reads the stdout of the 
child process (pipe.read_nonblock) and prints in the terminal window 
whatever is in it as long as a specific character sequence is found 
indicating that the command has been executed by the child process ==> 
return to the command mode i)

====The child process is very simple : a wxMainLoop (again!) and every 10ms 
an Idle procedure is ran.
It reads the $stdin using $stdin.read_nonblock. If there is something 
then it corresponds to a command that will be executed, e.g.,
$ans = eval(command,TOPLEVEL_BINDING)
(I take care of eventual errors of course)
And then, it just $stdout.puts the specific character sequence that 
indicates command execution is over and $stdout.flush

Notice that this process can open windows, draw graphics, handle 
interactions with these graphics ...

++++++++++++++

That''s it.
In this way, the display on the terminal is really fast (never got 
something as fast). Be aware that the (ruby) command that is executed 
with eval can use puts ... and it will print on the terminal.
The output buffering system is the one of the system....!

So for instance

100000.times {|i| puts i}

takes less that 2 seconds !
Which is really fast.

Moreover if a ctrl-C is typed in the terminal window I can send a SIGINT 
to the child process (I do it using C raise) which interrupts the 
execution of the ruby command !?!

I was really happy... till I realized that, it will not work on Windows 
XP!

==> First I looked on the web to see if there is anything that could 
replace the popen. I found popen4.. BUT
a- You have to pass a block (at the end of the block the process 
dies)... which is not good for me since I want the child process to live 
through the wxMainLoop
b- (which is related to a- I think) I want to do a read_nonblock (so I 
don''t reach the eof ... )

==> MARIO : does your library will help me ? Does it manage the 
stdin/stdout  of the child process in a standard way (buffering and 
all)? Is it non blocking ? Can I read and write (non blocking) 
alternatively without closing the process ?

==> Do you have any other idea (Alex, I guess you understand exactly 
what I want to do since you know R)

Thank you for everything !!

Emmanuel







-- 
Posted via http://www.ruby-forum.com/.

ouups :

==> what about using wxProcess (i.e., creating a process using wxWigets 
API) ??

E.
-- 
Posted via http://www.ruby-forum.com/.

Hello Emmanuel,

The XProcess library I developed, is asynchronous, and has methods in which
to detect if there is data available on a pipe object or not.  I setup the
library specifically for the need to be able to be async to keep the GUI
responsive while the sub-process is being executed.

As I said before, there are many different methods across Windows, Linux,
and OS X, in which to create a pipe to a sub-process, and Windows lacks the
major feature in which to allow sub-proceses naturally in ruby, which is
Fork.  What I have done, is created a single API interface in which to be
able to create processes, and send and receive data from the sub-process,
without blocking the wxWidgets API.

I did think about wrapping wxProcess, but after looking through the
documentation for it, I felt that it would just be simpler to create the API
myself in another class, then to go through the trouble of trying to wrap
wxProcess.

The simplest way I can explain the API for XProcess, is that you initialize
the class with the executable you wish to run.  EG:

xproc = XProcess.new("ruby")

Then you use the method execute to pass along command line arguments to the
sub-process, such as -e, -v, or whatever command line arguments your
sub-process needs.

Then the xproc object has 4 attributes, pid, which is the process id, in,
out, and err, which are the representations of STDIN, STDOUT, and STDERR for
the sub-process.

You can get output from the process by doing:

if xproc.out.is_data_avail?
  data = xproc.out.gets
  puts "Received: #{data}"
end

The method is_data_avail? on STDOUT and STDERR will not block the process,
it will either return true, or false.

If you want to see if the process has closed either STDOUT or STDERR, you
simply check by using the eof? method on the STDOUT or STDERR.

if xproc.out.eof?
  puts "The process has closed STDOUT"
end

XProcess doesn''t close the IO handles for input, output or error, so
you
will need to close them yourself, if you don''t want to  use them.  And
if
you want to kill the process if it''s taking to long, simply call the
kill
method.

I will need to write up some documentation for the XProcess library, but
it''s fairly straight forward, and simple to use.  I try to keep things
simplified, and unified across all platforms.

I have had confirmation that this library works on all 3 platforms, but if
you find any bugs, please feel free to let me know, off list, and I will see
what I can do to fix the problems.

hth,

Mario

-- 
Mario Steele
Lieutenant 3 - Geo 99
XO - STO IFT Fleet
Chief Engineer - Second Life
http://www.iftcommand.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/wxruby-users/attachments/20100520/9ede6f85/attachment.html>

Thanks Mario for this long answer !

Few questions

1- Is there  also a way to read a maximum number of bytes (non blocking, 
knowing that there are some data available) ?
(the equivalent of .read_nonblock()

2- Is it working with ruby 1.9 ?

3- How do I install it, is there a small doc ?

Thanks

Emmanuel
-- 
Posted via http://www.ruby-forum.com/.

Okay Mario,

I did install it, it seems to work .... except that, I am doing 
something wrong maybe, but in order to send to the child process some 
strings into the stdin, I do

cmd.in.puts(string)

Now, when the child process finds a string in the stdin, it processes it 
.... and really weired... but this cmd.in.puts is blocking as long as 
the child process has finished its processing ???

I really don''t understand what''s hapenning here

Could you help me ?

Emmanuel
-- 
Posted via http://www.ruby-forum.com/.

ouups sorry for this message (again!)

I was wrong in my last post... it is not blocking. Something really 
weired happens.
As I already explained both processes run a wxMainLoop and both have 
wxTimers that wake up the process regularly.

So when the child process is processing the stdin the parent process 
does not receive any alarm from its timer any more ??!
-- 
Posted via http://www.ruby-forum.com/.

Hey Emmanuel,

First off, I haven''t really tested Ruby 1.9, so if it''s
working for you,
that''s great.

Second off, once any output to stdin is executed, it should automatically
return, without any delays.  I will have to investigate this more, unless
it''s this third thing.

Third, are you using Ruby 1.9, Wx::Timer, and Threads in the setup?  Cause,
if your using Ruby 1.9, there is no need to utilize Wx::Timer, to wake up
your application, as Ruby 1.9 has native Thread support, which is compatible
with wxWidgets thread utilization setup.  The only problem you would have,
is attempting to update the GUI from any other thread, outside the Main
Thread.  Which is the base level thread, that your app runs in.  If you
attempt to update the GUI from any other thread, wxWidgets will panic, and
seg-fault on you, or lock your program up.

I will be in IRC, if you wish to discuss more about my library.  irc://
irc.freenode.net:6667/#wxruby

hth,

Mario

On Thu, May 20, 2010 at 5:22 PM, Emmanuel Emmanuel <lists at
ruby-forum.com>wrote:
> ouups sorry for this message (again!)
>
> I was wrong in my last post... it is not blocking. Something really
> weired happens.
> As I already explained both processes run a wxMainLoop and both have
> wxTimers that wake up the process regularly.
>
> So when the child process is processing the stdin the parent process
> does not receive any alarm from its timer any more ??!
> --
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> wxruby-users mailing list
> wxruby-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/wxruby-users
>


-- 
Mario Steele
Lieutenant 3 - Geo 99
XO - STO IFT Fleet
Chief Engineer - Second Life
http://www.iftcommand.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/wxruby-users/attachments/20100520/7bc9de33/attachment.html>

Hello Mario,

I identified the problem (first of all no I am NOT using ruby 1.9.. I 
was just asking ...) :


When the child process is writing quickly  to the stdout then the call 
(from the main process) to

cmd.out.data_avail?

basically waits for the child process to have finished writing....

---

Example :
parent process runs Idle every 10ms. Idle looks at the stdout of the 
child process and prints on a textCtrl whatever is in it

child process runs 100000.times {|i| puts i}

==> The whole thing takes 3 seconds to run (macos X) but the flush is 
done just once !
(Idle is called once and blocks at  cmd.out.data_avail?, wait for child 
process to finish writing and then displays all the numbers at once)

Let''s note that if I put a sleep after the puts then the display is 
updated very often during the loop....

Emmanuel
-- 
Posted via http://www.ruby-forum.com/.

hi

Glad you''re making progress; not surprised to hear that Windows is the 
sticking point. Mario knows a lot more than me about using pipes etc 
with wxRuby so just a few observations that might or might not be useful:


Emmanuel Emmanuel wrote:
> The child process is very simple : a wxMainLoop (again!) and every 10ms 
> an Idle procedure is ran.
>   
Are you sure child process needs to be a wx program at all? I would 
expect it to be a simple Ruby non-GUI program if it just uses STDIN and 
STDOUT to communicate in an efficient format. With that you can test 
different options, eg

- a long-running server to handle many jobs (which starts up and then 
enters an endless "loop do ... end") vs a per-job new process. If
it''s
like R, the overhead of starting a new process per-job is insignificant 
against the overall job computation time. And you can close the write 
pipe on each job and then read data incrementally, rather than the more 
complex popen3 model.

- I wonder if Drb (distributed ruby) might work? I have used it with 
some success in attaching a wxRuby GUI to an application running 
separately. One other bonus of this is it would then be easy to move to 
a client/server model on different pcs if wanted in the future.
> Notice that this process can open windows, draw graphics, handle 
> interactions with these graphics ...
>   
Is this in the child process? That seems an unusual arrangement but of 
course there might be very good reasons for this...
> ++++++++++++++
>
> That''s it.
> In this way, the display on the terminal is really fast (never got 
> something as fast). 

Great. I''m away on hols for a few weeks now so won''t be
picking up, but
best of luck with the project

alex