win32utils devel - Jun 2006 - [Fwd: Ruby Win32-Service]

Thoughts?

Dan
-------------- next part --------------
An embedded message was scrubbed...
From: "Patrick Hurley" <phurley at gmail.com>
Subject: Ruby Win32-Service
Date: Sun, 18 Jun 2006 12:46:01 -0400
Size: 2863
Url:
http://rubyforge.org/pipermail/win32utils-devel/attachments/20060618/cc346796/attachment.eml

On 6/18/06, Daniel Berger <djberg96 at gmail.com>
wrote:
> Thoughts?
Dan,

I faced the same issue of the guy pointed out.

Calling the EventHash from other native thread (not the one where ruby
interpreter is running) trash everything. Worse is that the
interpreter is currently "looping" in service_main

As I understand the idea of Patrick, we could emulate the threading of
Service_Ctrl in a green thread, something that will not work quite
right without us pulling status in a regular basis.

The second suggestions sounds good, but is far from where I could
understand the impact of implementing that.

I was looking at ruby source (signal.c) and scratching the surface,
IN_MAIN_CONTEXT (which is alias for rb_w32_main_context) offered,
based on conjectures from Nobuyoshi Nakada, could offer some context
switch for the rub interpreter, but couldn''t solve if the interpreter
is already running.

I could suggest this:

In Service_Ctrl, set the service status so service_main get the update
and could exit from it. and wait for event "ExitedFromServiceMain".

in the daemon_mainloop, after exiting rb_funcall "service_main",
signal the ExitedFromServiceMain so Service_Ctrl could call the
EventHash and don''t trash the ruby interpreter. (Here we could
implemented as IN_MAIN_CONTEXT as a safe-net).

Then signal the EventStop and set the service status as STOPPED.

Hope the "workflow" sounds good, for me it does (but don''t
have time
to hack it currently now).

Regards,


-- 
Luis Lavena
Multimedia systems
-
Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi
>
> ---------- Forwarded message ----------
> From: "Patrick Hurley" <phurley at gmail.com>
> To: "Daniel Berger" <djberg96 at gmail.com>
> Date: Sun, 18 Jun 2006 12:46:01 -0400
> Subject: Ruby Win32-Service
> First off I just want to pass on a big thank you, the wonderful win32
> packages have helped me numerous times. But (of course you knew it had
> to be coming :-)...
>
> I have just started doing some work with win32-service -- I was
> creating a wrapper around it and daemon.rb, so I could write one body
> of code that depending upon platform would provide a reasonable
> interface for both windows and Unix platforms. During my testing I
> noticed some odd behavior in my test program. Having written a couple
> win32 services before I suspected some threading issues, so I did some
> digging first just in Ruby using DL, I created a call to
> GetCurrentThreadID and in logging I displayed the Current TID -- and
> we are switching system threads in the ruby interpreter.
>
> Looking at service.c, I see you are using RegisterServiceCtrlHandler,
> there is no guarantee that it will be called from the main thread
> (matter of fact it generally won''t). In Service_Ctrl, you make an
> rb_funcall using the call back hash, which then calls back into the
> interpreter while the main thread is still running under mainloop.
>
> My side effect was some scrambled ruby file handles and some pipe
> closed on stopping errors. I would be happy to give back and fix this
> issue, but wanted you input on how to proceed. I see two options:
>
> 1. Only send windows events from Service_Ctrl, and set status flags.
> On calls to status, I could actually call service_stop,
> service_paramchange, etc... This bypasses threading, but trades it for
> a cooperative multi-tasking setup.
>
> 2. (Which I prefer, but have never done before). In the extension
> create a ruby thread (green, not OS) which waits on a call back queue)
> and then have Service_Ctrl maintain that queue. This would ensure that
> the Ruby interpreter only runs in one thread - but does involve mixing
> in ruby threading.
>
> If you think I am off my rocker and completely misunderstand the
> problem, sorry for wasting your time :-); however, if you think I
> understand the issue and would like me to take a stab at a patch,
> please let me know.
>
> Thanks again for all your work
> Patrick Hurley
>
>
>
> _______________________________________________
> win32utils-devel mailing list
> win32utils-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/win32utils-devel
>
>

On 6/18/06, Luis Lavena <luislavena at gmail.com>
wrote:
> I faced the same issue of the guy pointed out.
>
> Calling the EventHash from other native thread (not the one where ruby
> interpreter is running) trash everything. Worse is that the
> interpreter is currently "looping" in service_main
Exactly, ok so I am not off my rocker :-)

> As I understand the idea of Patrick, we could emulate the threading of
> Service_Ctrl in a green thread, something that will not work quite
> right without us pulling status in a regular basis.
Since I am going to use this for work, I can spend some paid time
tomorrow following up. I went through the C extension docs and it
looks pretty easy to create a green thread attached to an actual C
function. So my plan is as follows:

1. Create a Win32 Semaphore and mutex, that mirrors a simple list of
events (e.g. param change, stop, pause, etc).

2. In ServiceCtrl when ever a ruby call back is required: I will lock
the mutex, place an item in the list, increment the semaphor and
release the mutex.

3. In a green thread I will be waiting on the semaphor -- I am not
sure if this will block ruby''s green threads if it does I will have to
spin against the semaphor and a ruby sleep -- ugly, but very doable.

4. When events arrive in the list, I will make the call back into the
main class -- this will occur in the same system thread, but a
different "ruby thread".

The only thing I am unsure of is #3, If a Ruby thread blocks on a
Win32 object, I think it will hang ruby -- if anyone has any
experience with this I would love to hear it -- otherwise I may spend
some time looking through the Ruby source to see if there is an
internal to allow waiting without the spin.

Thanks
pth

Hi,

2006/6/19, Patrick Hurley <phurley at gmail.com>:
> On 6/18/06, Luis Lavena <luislavena at gmail.com> wrote:
> > I faced the same issue of the guy pointed out.
> >
> > Calling the EventHash from other native thread (not the one where ruby
> > interpreter is running) trash everything. Worse is that the
> > interpreter is currently "looping" in service_main
>
> Exactly, ok so I am not off my rocker :-)
>
As you know, using native thread is not recommended in the current ruby version.
>
> > As I understand the idea of Patrick, we could emulate the threading of
> > Service_Ctrl in a green thread, something that will not work quite
> > right without us pulling status in a regular basis.
>
> Since I am going to use this for work, I can spend some paid time
> tomorrow following up. I went through the C extension docs and it
> looks pretty easy to create a green thread attached to an actual C
> function. So my plan is as follows:
>
> 1. Create a Win32 Semaphore and mutex, that mirrors a simple list of
> events (e.g. param change, stop, pause, etc).
>
> 2. In ServiceCtrl when ever a ruby call back is required: I will lock
> the mutex, place an item in the list, increment the semaphor and
> release the mutex.
>
> 3. In a green thread I will be waiting on the semaphor -- I am not
> sure if this will block ruby''s green threads if it does I will
have to
> spin against the semaphor and a ruby sleep -- ugly, but very doable.
>
> 4. When events arrive in the list, I will make the call back into the
> main class -- this will occur in the same system thread, but a
> different "ruby thread".
>
> The only thing I am unsure of is #3, If a Ruby thread blocks on a
> Win32 object, I think it will hang ruby -- if anyone has any
> experience with this I would love to hear it -- otherwise I may spend
> some time looking through the Ruby source to see if there is an
> internal to allow waiting without the spin.
>
Although all the above might be possible, it will not work with
SCM(Service Control Manager). Thus you must call
"StartServiceCtrlDispatcher" API to connect with the SCM.
The main problem is "StartServiceCtrlDispatcher" blocks and returns
only when the serivce has terminated.
If you can make StartServiceCtrlDispatcher nonblocking, I guess there
is no problem at all.

Regards,

Park Heesob

On 6/18/06, Heesob Park <phasis at gmail.com>
wrote:
> As you know, using native thread is not recommended in the current ruby
version.
True, but it is safe to use native threads as long as they don''t call
any ruby functions. So my thought was use a native thread to deal with
the blocked StartServiceCtrlDispatcher, but do some sort of ugly spin
in a green ruby thread (unless I can find a better way to interface to
the native thread).

I have dealt with other non-thread safe libraries using similar
techniques in services with a fair degree of success in the past (but
admittedly never with Ruby).

I will give it a try tomorrow and let you know how it goes, if it goes
well, I will provide a patch.

Thanks
pth

2006/6/19, Patrick Hurley <phurley at gmail.com>:
> On 6/18/06, Heesob Park <phasis at gmail.com> wrote:
> > As you know, using native thread is not recommended in the current
ruby version.
>
> True, but it is safe to use native threads as long as they don''t
call
> any ruby functions. So my thought was use a native thread to deal with
> the blocked StartServiceCtrlDispatcher, but do some sort of ugly spin
> in a green ruby thread (unless I can find a better way to interface to
> the native thread).
>
That is just the current win32-service implementation.
When the green ruby thread sleeps, all native threads blocked. and
StartServiceCtrlDispatcher can''t accept the service control event.
> I have dealt with other non-thread safe libraries using similar
> techniques in services with a fair degree of success in the past (but
> admittedly never with Ruby).
>
> I will give it a try tomorrow and let you know how it goes, if it goes
> well, I will provide a patch.
>
I hope you will make a good patch.

Thanks
Park Heesob

Patrick,
> I have just started doing some work with win32-service -- I was
> creating a wrapper around it and daemon.rb, so I could write one body
> of code that depending upon platform would provide a reasonable
> interface for both windows and Unix platforms.
I am interested in this work and have been hoping to do something
similiar myself. On the *nix side I have been using the Daemons
library (available as a gem) with an API call like so:

# 1. Create a proc that launches the program you wish to daemonize:
proc = Proc.new do
  SomeClassOfYours.start
end

# 2. Configure (most of this is optional)
options = {
             :app_name   => "myd",
             :dir_mode   => :normal,
             :dir        => config.pid_dir,
             :multiple   => false,
             :ontop      => false,
             :backtrace  => true,
             :log_output => true,
             :monitor    => true,
             :mode       => :proc,
             :proc       => proc
           }

# 3. Create the Daemon process, etc.
controller = Daemons::Controller.new(options, ARGV)
controller.catch_exceptions do
  controller.run
end

In additon to daemonizing the supplied proc this creates a complete
init script with start, stop, restart, and status commands and handles
PID file management, etc. See http://daemons.rubyforge.org/.

But like you I''d like to be able abstract this and creation of the
win32 service away from developer so that we can using a single simple
API that will work across platforms. For the sake of the cross
platform support I''d happily trade some configurability and
flexability (at least in the short run) in the spirit of POpen4
(http://popen4.rubyforge.org/).

If you are interested in working together on this drop me a line.
-- 
John-Mason Shackelford

Software Developer
Pearson Educational Measurement

2510 North Dodge St.
Iowa City, IA 52245
ph. 319-354-9200x6214
john-mason.shackelford at pearson.com
http://pearsonedmeasurement.com

Patrick,
> I have just started doing some work with win32-service -- I was
> creating a wrapper around it and daemon.rb, so I could write one body
> of code that depending upon platform would provide a reasonable
> interface for both windows and Unix platforms.
I am interested in this work and have been hoping to do something
similiar myself. On the *nix side I have been using the Daemons
library (available as a gem) with an API call like so:

# 1. Create a proc that launches the program you wish to daemonize:
proc = Proc.new do
 SomeClassOfYours.start
end

# 2. Configure (most of this is optional)
options = {
            :app_name   => "myd",
            :dir_mode   => :normal,
            :dir        => config.pid_dir,
            :multiple   => false,
            :ontop      => false,
            :backtrace  => true,
            :log_output => true,
            :monitor    => true,
            :mode       => :proc,
            :proc       => proc
          }

# 3. Create the Daemon process, etc.
controller = Daemons::Controller.new(options, ARGV)
controller.catch_exceptions do
 controller.run
end

In additon to daemonizing the supplied proc this creates a complete
init script with start, stop, restart, and status commands and handles
PID file management, etc. See http://daemons.rubyforge.org/.

But like you I''d like to be able abstract this and creation of the
win32 service away from developer so that we can using a single simple
API that will work across platforms. For the sake of the cross
platform support I''d happily trade some configurability and
flexability (at least in the short run) in the spirit of POpen4
(http://popen4.rubyforge.org/).

If you are interested in working together on this drop me a line.
--
John-Mason Shackelford

Software Developer
Pearson Educational Measurement

2510 North Dodge St.
Iowa City, IA 52245
ph. 319-354-9200x6214
john-mason.shackelford at pearson.com
http://pearsonedmeasurement.com