I have been looking at the EventMachine code and style for a while now, and have
read that this is probably not the best arrangement, however, this is a working
reconnection strategy I''m using (on 0.8.1 and 0.9.0 win32 + *nix). Any
improvements would be seriously welcome.
require ''rubygems''
require ''eventmachine''
require ''thread''
Out = Queue.new
In = Queue.new
class MyClient
def initialize
@network_connector = Thread.new do
loop do
EventMachine::run {
EventMachine::connect ''localhost'', 9000, MyConnection
}
end
end
@gen_loop = Thread.new do
loop do
Out << "#{rand}"
$stdout.puts ''g''
sleep 1
end
end
@con_loop = Thread.new do
loop do
puts "<<< #{In.pop}"
end
end
end
end
module MyConnection
def post_init
start_tls
end
def connection_completed
@connected = true
@thread = Thread.new do
while @connected do
message = Out.pop
send_data message
end
end
end
def receive_data data
In << data
end
def unbind
puts ''unbind''
@connected = false
@thread.kill unless @thread.join(1) if @thread
EventMachine::stop_event_loop
end
end
MyClient.new
This would be relevant for something like a DataLogger where irregardless of
connection state, one would want to continue collecting data off of say, a
serial port, whilst queueing up any new data for future sending.
In production it would also have a preservation for a disconnected
"Out.pop", and also the exit trap would dump the In and Out buffers.
As above, any suggestions or guidance would be most welcome.
Regards,
James.
Try to use fewer threads, perhaps, would be a suggestion. -Roger On Nov 19, 2007 10:59 PM, James Tucker <jftucker at gmail.com> wrote:> I have been looking at the EventMachine code and style for a while now, and have read that this is probably not the best arrangement, however, this is a working reconnection strategy I''m using (on 0.8.1 and 0.9.0 win32 + *nix). Any improvements would be seriously welcome. > > require ''rubygems'' > require ''eventmachine'' > require ''thread'' > > Out = Queue.new > In = Queue.new > > class MyClient > def initialize > @network_connector = Thread.new do > loop do > EventMachine::run { > EventMachine::connect ''localhost'', 9000, MyConnection > } > end > end > @gen_loop = Thread.new do > loop do > Out << "#{rand}" > $stdout.puts ''g'' > sleep 1 > end > end > @con_loop = Thread.new do > loop do > puts "<<< #{In.pop}" > end > end > end > end > > module MyConnection > > def post_init > start_tls > end > > def connection_completed > @connected = true > @thread = Thread.new do > while @connected do > message = Out.pop > send_data message > end > end > end > > def receive_data data > In << data > end > > def unbind > puts ''unbind'' > @connected = false > @thread.kill unless @thread.join(1) if @thread > EventMachine::stop_event_loop > end > end > > MyClient.new > > > This would be relevant for something like a DataLogger where irregardless of connection state, one would want to continue collecting data off of say, a serial port, whilst queueing up any new data for future sending. > > In production it would also have a preservation for a disconnected "Out.pop", and also the exit trap would dump the In and Out buffers. > > As above, any suggestions or guidance would be most welcome. > > Regards, > > James. > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-- -Roger Pack For God hath not given us the spirit of fear; but of power, and of love, and of a sound mind" -- 2 Timothy 1:7
On Nov 20, 2007 11:29 AM, James Tucker <jftucker at gmail.com> wrote:> I have been looking at the EventMachine code and style for a while now, and have read that this is probably not the best arrangement, however, this is a working reconnection strategy I''m using (on 0.8.1 and 0.9.0 win32 + *nix). Any improvements would be seriously welcome. > > require ''rubygems'' > require ''eventmachine'' > require ''thread'' > > Out = Queue.new > In = Queue.new > > class MyClient > def initialize > @network_connector = Thread.new do > loop do > EventMachine::run { > EventMachine::connect ''localhost'', 9000, MyConnection > } > end > end > @gen_loop = Thread.new do > loop do > Out << "#{rand}" > $stdout.puts ''g'' > sleep 1 > end > end > @con_loop = Thread.new do > loop do > puts "<<< #{In.pop}" > end > end > end > end > > module MyConnection > > def post_init > start_tls > end > > def connection_completed > @connected = true > @thread = Thread.new do > while @connected do > message = Out.pop > send_data message > end > end > end > > def receive_data data > In << data > end > > def unbind > puts ''unbind'' > @connected = false > @thread.kill unless @thread.join(1) if @thread > EventMachine::stop_event_loop > end > end > > MyClient.newPlease, don''t write code like this. EventMachine already has a reconnect method, which one can use to reconnect to lost connections. You can either call immediately in unbind or schedule a timer, that would attempt reconnection on periodic basis. -- Let them talk of their oriental summer climes of everlasting conservatories; give me the privilege of making my own summer with my own coals. http://gnufied.org
Roger Pack wrote:> Try to use fewer threads, perhaps, would be a suggestion. > -RogerRight, the generator and consumer threads are just examples. In reality, as an event driven system "In" shoudl be replaced entirely by a function call.> > On Nov 19, 2007 10:59 PM, James Tucker <jftucker at gmail.com> wrote: >> I have been looking at the EventMachine code and style for a while now, and have read that this is probably not the best arrangement, however, this is a working reconnection strategy I''m using (on 0.8.1 and 0.9.0 win32 + *nix). Any improvements would be seriously welcome. >> >> require ''rubygems'' >> require ''eventmachine'' >> require ''thread'' >> >> Out = Queue.new >> In = Queue.new >> >> class MyClient >> def initialize >> @network_connector = Thread.new do >> loop do >> EventMachine::run { >> EventMachine::connect ''localhost'', 9000, MyConnection >> } >> end >> end >> @gen_loop = Thread.new do >> loop do >> Out << "#{rand}" >> $stdout.puts ''g'' >> sleep 1 >> end >> end >> @con_loop = Thread.new do >> loop do >> puts "<<< #{In.pop}" >> end >> end >> end >> end >> >> module MyConnection >> >> def post_init >> start_tls >> end >> >> def connection_completed >> @connected = true >> @thread = Thread.new do >> while @connected do >> message = Out.pop >> send_data message >> end >> end >> end >> >> def receive_data data >> In << data >> end >> >> def unbind >> puts ''unbind'' >> @connected = false >> @thread.kill unless @thread.join(1) if @thread >> EventMachine::stop_event_loop >> end >> end >> >> MyClient.new >> >> >> This would be relevant for something like a DataLogger where irregardless of connection state, one would want to continue collecting data off of say, a serial port, whilst queueing up any new data for future sending. >> >> In production it would also have a preservation for a disconnected "Out.pop", and also the exit trap would dump the In and Out buffers. >> >> As above, any suggestions or guidance would be most welcome. >> >> Regards, >> >> James. >> _______________________________________________ >> Eventmachine-talk mailing list >> Eventmachine-talk at rubyforge.org >> http://rubyforge.org/mailman/listinfo/eventmachine-talk >> > > >
hemant wrote:> On Nov 20, 2007 11:29 AM, James Tucker <jftucker at gmail.com> wrote: >> I have been looking at the EventMachine code and style for a while now, and have read that this is probably not the best arrangement, however, this is a working reconnection strategy I''m using (on 0.8.1 and 0.9.0 win32 + *nix). Any improvements would be seriously welcome. >> >> require ''rubygems'' >> require ''eventmachine'' >> require ''thread'' >> >> Out = Queue.new >> In = Queue.new >> >> class MyClient >> def initialize >> @network_connector = Thread.new do >> loop do >> EventMachine::run { >> EventMachine::connect ''localhost'', 9000, MyConnection >> } >> end >> end >> @gen_loop = Thread.new do >> loop do >> Out << "#{rand}" >> $stdout.puts ''g'' >> sleep 1 >> end >> end >> @con_loop = Thread.new do >> loop do >> puts "<<< #{In.pop}" >> end >> end >> end >> end >> >> module MyConnection >> >> def post_init >> start_tls >> end >> >> def connection_completed >> @connected = true >> @thread = Thread.new do >> while @connected do >> message = Out.pop >> send_data message >> end >> end >> end >> >> def receive_data data >> In << data >> end >> >> def unbind >> puts ''unbind'' >> @connected = false >> @thread.kill unless @thread.join(1) if @thread >> EventMachine::stop_event_loop >> end >> end >> >> MyClient.new > > > Please, don''t write code like this. EventMachine already has a > reconnect method, which one can use to reconnect to lost connections. > You can either call immediately in unbind or schedule a timer, that > would attempt reconnection on periodic basis. > >In eventmachine 0.8.1 and 0.9.0 on both Windows and *nix, calling Connection#reconnect or EventMachine::reconnect would cause send_data to no longer send data to the actual server. In effect, the reconnect is not actually working in either of these versions, which is entirely the reason for the original mail. Do you have examples of working reconnects?, because the above is the only one I could make work.
On Nov 20, 2007 12:18 PM, James Tucker <jftucker at gmail.com> wrote:> Do you have examples of working reconnects?, because the above is the only one I could make work.Swiftiply clients reconnect. Take a look at the swiftcore/Swiftiply/swiftiply_client.rb for the full source. Basically, though, the code is like this: class SwiftiplyClientProtocol < EventMachine::Connection attr_accessor :hostname, :port, :key, :ip, :id def self.connect(hostname = nil,port = nil,key = '''') key = key.to_s connection = ::EventMachine.connect(hostname, port, self) do |conn| conn.hostname = hostname conn.port = port conn.key = key ip = conn.ip = conn.__get_ip(hostname) conn.id = ''swiftclient'' << ip.collect {|x| sprintf(''%02x'',x.to_i)}.join << sprintf(''%04x'',port.to_i)<< sprintf(''%02x'',key.length) << key conn.set_comm_inactivity_timeout inactivity_timeout end end def self.inactivity_timeout @inactivity_timeout || 60 end def self.inactivity_timeout=(val) @inactivity_timeout = val end def unbind ::EventMachine.add_timer(rand(2)) {self.class.connect(@hostname, at port, at key)} end end Kirk Haines
> In eventmachine 0.8.1 and 0.9.0 on both Windows and *nix, calling Connection#reconnect or EventMachine::reconnect would cause send_data to no longer send data to the actual server. In effect, the reconnect is not actually working in either of these versions, which is entirely the reason for the original mail. > > Do you have examples of working reconnects?, because the above is the only one I could make work. >class ClientHandler < EventMachine::Connection def unbind @connection_status = false EventMachine.add_timer(10) { reconnect_back } end def reconnect_back begin @reconnection_attempts += 1 reconnect(backend[:ip],backend[:port]) rescue PushDebug.log $! PushDebug.log $!.backtrace end end end ^^ code, sure as hell, works on my Linux boxen. When you say, reconnect causes send_data no longer to "send_data", what exactly you mean? Are you calling send_data, when socket is in disconnected state? Or are you saying, even after reconnect is successful, send_data no longer, works for that instance of connection handler. If its so, then file a ticket, but it works for me here.
hemant wrote:> ^^ code, sure as hell, works on my Linux boxen. When you say, > reconnect causes send_data no longer to "send_data", what exactly you > mean? > Are you calling send_data, when socket is in disconnected state? Or > are you saying, even after reconnect is successful, send_data no > longer, works > for that instance of connection handler. If its so, then file a > ticket, but it works for me here.I was calling reconnect from directly inside of unbind, and although the connections object id was not changing, the send_data method would not work after reconnection. Similarly, if I remember correctly, connection_completed was not called then either. If this is not intended behaviour or usage, then maybe that''s fine? Thanks for the help! James.> _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >
Kirk Haines wrote:> Swiftiply clients reconnect. Take a look at the > swiftcore/Swiftiply/swiftiply_client.rb for the full source.Many thanks Kirk :) I now have the following as an example of how one might detatch (most of) the network code from an application which desires persistent outgoing connectivity: class MyConnection < EventMachine::Connection attr_accessor :host, :port, :controller def connection_completed @controller.connection = self end def self.connect host, port, controller ::EventMachine.connect(host, port, self) do |conn| conn.controller = controller conn.host = host conn.port = port end end def receive_data data @controller.receive_data data end def unbind ::EventMachine.add_timer( rand(2) ) { self.class.connect( @host, @port, @controller ) } end end The "controller" then defines: attr_accessor :connection def receive_data data puts data end def send_data data raise NotConnectedException unless @connection status = @connection.send_data( data ) raise SendFailedException unless status > -1 puts data end class SendFailedException < Exception; end class NotConnectedException < Exception; end And not a thread in sight! :) Thanks again, James.> _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >
On Nov 20, 2007 6:51 PM, James Tucker <jftucker at gmail.com> wrote:> > def unbind > ::EventMachine.add_timer( rand(2) ) { > self.class.connect( @host, @port, @controller ) > } > endHmm, this code doesn''t work for me when I wrap the connect() call in the add_timer() call. I always get a ConnectionNotBound exception. Any idea why? When I leave out add_timer() and just call connect() it works fine. --Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071123/bd6629cc/attachment.html
On Nov 24, 2007 12:39 AM, Michael S. Fischer <michael at dynamine.net> wrote:> On Nov 20, 2007 6:51 PM, James Tucker <jftucker at gmail.com> wrote: > > > > > > > def unbind > > ::EventMachine.add_timer( rand(2) ) { > > self.class.connect( @host, @port, @controller ) > > } > > end > > > Hmm, this code doesn''t work for me when I wrap the connect() call in the > add_timer() call. I always get a ConnectionNotBound exception. Any idea > why? When I leave out add_timer() and just call connect() it works fine. >ConnectionNotBound is a very deceptive error that can be thrown when an ordinary Ruby error (like a RuntimeError or an ArgumentError) is generated in user-written code. Can you make sure that your code isn''t doing that?
On Nov 24, 2007 4:01 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> > Hmm, this code doesn''t work for me when I wrap the connect() call in the > > add_timer() call. I always get a ConnectionNotBound exception. Any > idea > > why? When I leave out add_timer() and just call connect() it works > fine. > > > > ConnectionNotBound is a very deceptive error that can be thrown when > an ordinary Ruby error (like a RuntimeError or an ArgumentError) is > generated in user-written code. Can you make sure that your code isn''t > doing that?I think I''ve fixed it. Thanks! Still need the return value from popen, though :-) --Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071124/9476c55d/attachment.html
On Nov 24, 2007 10:55 AM, Michael S. Fischer <michael at dynamine.net> wrote: > >> > ConnectionNotBound is a very deceptive error that can be thrown when > > an ordinary Ruby error (like a RuntimeError or an ArgumentError) is > > generated in user-written code. Can you make sure that your code isn''t > > doing that? > > I think I''ve fixed it. Thanks! Still need the return value from popen, > though :-) >What did you do to fix it? Was it indeed some Ruby error being thrown out of your code? EM doesn''t use popen, it uses socketpair+fork. I just noticed that it oddly seems not be cleaning up the fork child. I''ll investigate that asap. i assume that you''re looking for the exit value returned by the child process. Correct?
On Nov 24, 2007 8:33 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> What did you do to fix it? Was it indeed some Ruby error being thrown > out of your code?Unfortunately, I''m not yet doing revision control on my project, and so I can''t tell you exactly what I changed. I suspect I fat-fingered something inside the block, because after a bunch of retyping and experimentation, it suddenly began working. EM doesn''t use popen, it uses socketpair+fork. I just noticed that it> oddly seems not be cleaning up the fork child. I''ll investigate that > asap.You''re right; I failed to notice the #if OBSOLETE pragma :-) i assume that you''re looking for the exit value returned by the child> process. Correct? >At a minimum, I need the exit value. However, having access to the entire stat_loc structure via Ruby equivalents of the usual W* macros (see the wait(2) man page) would be really cool. Thanks for your help, --Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071124/58f31267/attachment-0001.html
> > At a minimum, I need the exit value. However, having access to the entire > stat_loc structure via Ruby equivalents of the usual W* macros (see the > wait(2) man page) would be really cool. > >Ok, this was rather a hack but I think it works. Please sync to HEAD revision and try it. There is a new method: EM::Connection#get_status, which may ONLY be called from within an #unbind handler. require ''rubygems'' require ''eventmachine'' module Handler def receive_data text puts text end def unbind puts get_status end end EM.run { EM.popen( "your_subprocess", Handler ) } -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071127/92e3a747/attachment.html
On Nov 27, 2007 6:33 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> > > > > At a minimum, I need the exit value. However, having access to the > > entire stat_loc structure via Ruby equivalents of the usual W* macros (see > > the wait(2) man page) would be really cool. > > > > > > Ok, this was rather a hack but I think it works. Please sync to HEAD > revision and try it. There is a new method: EM::Connection#get_status, > which may ONLY be called from within an #unbind handler. >Awesome! Thanks! What is the value returned by get_status? All I need now is client SSL certificate verification capability and I''ll be ready to roll. Best regards, --Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071127/50f462e3/attachment.html
On Nov 27, 2007 6:33 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote:> > > > > At a minimum, I need the exit value. However, having access to the > > entire stat_loc structure via Ruby equivalents of the usual W* macros (see > > the wait(2) man page) would be really cool. > > > > > > Ok, this was rather a hack but I think it works. Please sync to HEAD > revision and try it. There is a new method: EM::Connection#get_status, > which may ONLY be called from within an #unbind handler. >Works great in Ubuntu (64-bit). Doesn''t work at all on OS X 10.5 (Intel); I get the dreaded "ConnectionNotBound" exception. Let me know if you need a machine to debug on. --Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071127/536272fa/attachment.html
Are you running multi threaded? This can cause ConnectionUnbound exceptions that we could look into. Cheers. -Roger On Nov 27, 2007 8:23 PM, Michael S. Fischer <michael at dynamine.net> wrote:> On Nov 27, 2007 6:33 AM, Francis Cianfrocca <garbagecat10 at gmail.com> wrote: > > > > > > > > > > > > > > > > At a minimum, I need the exit value. However, having access to the > entire stat_loc structure via Ruby equivalents of the usual W* macros (see > the wait(2) man page) would be really cool. > > > > > > > > > Ok, this was rather a hack but I think it works. Please sync to HEAD > revision and try it. There is a new method: EM::Connection#get_status, > which may ONLY be called from within an #unbind handler. > > > > Works great in Ubuntu (64-bit). Doesn''t work at all on OS X 10.5 (Intel); I > get the dreaded "ConnectionNotBound" exception. > > Let me know if you need a machine to debug on. > > --Michael > > _______________________________________________ > Eventmachine-talk mailing list > Eventmachine-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/eventmachine-talk >-- -Roger Pack For God hath not given us the spirit of fear; but of power, and of love, and of a sound mind" -- 2 Timothy 1:7
Nope, not running multithreaded. The test code looks like:
module Handler
def post_init
puts "in post_init"
end
def receive_data(data)
print data
end
def unbind
puts "in unbind"
puts "exit status = #{get_status}"
EventMachine::stop_event_loop
end
end
EventMachine::run {
EventMachine::popen("ls -l", Handler)
}
On Nov 29, 2007 6:10 PM, Roger Pack <rogerpack2005 at gmail.com> wrote:
> Are you running multi threaded?
> This can cause ConnectionUnbound exceptions that we could look into.
> Cheers.
> -Roger
>
> On Nov 27, 2007 8:23 PM, Michael S. Fischer <michael at dynamine.net>
wrote:
> > On Nov 27, 2007 6:33 AM, Francis Cianfrocca <garbagecat10 at
gmail.com>
> wrote:
> >
> > >
> > >
> > >
> > > >
> > > >
> > > > At a minimum, I need the exit value. However, having access
to the
> > entire stat_loc structure via Ruby equivalents of the usual W* macros
> (see
> > the wait(2) man page) would be really cool.
> > > >
> > >
> > >
> > > Ok, this was rather a hack but I think it works. Please sync to
HEAD
> > revision and try it. There is a new method:
EM::Connection#get_status,
> > which may ONLY be called from within an #unbind handler.
> > >
> >
> > Works great in Ubuntu (64-bit). Doesn''t work at all on OS X
10.5(Intel); I
> > get the dreaded "ConnectionNotBound" exception.
> >
> > Let me know if you need a machine to debug on.
> >
> > --Michael
> >
> > _______________________________________________
> > Eventmachine-talk mailing list
> > Eventmachine-talk at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/eventmachine-talk
> >
>
>
>
> --
> -Roger Pack
> For God hath not given us the spirit of fear; but of power, and of
> love, and of a sound mind" -- 2 Timothy 1:7
> _______________________________________________
> Eventmachine-talk mailing list
> Eventmachine-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/eventmachine-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/eventmachine-talk/attachments/20071129/20c69137/attachment-0001.html