I am just getting into web servers/web applications and rails as well so bear with me. I am trying to write a web app that, based on a users input from the browser, will perform some task, and update the browser (ala ajax style) as needed and/or provide a way for the user to control the background task. Now, I have more experience with java servlets, which makes this easy enough for a beginner. The user inputs some info and the server spawns a new thread to perform the task in the background while the user can continue using the app, until it is updated with the results. (in my case the background task is putting or getting messages on/from an MQ Series queue) I keep a reference to the background task in the session object for access as needed. What I am wondering is if the same concepts can be applied to a rails application. I started researching rails off and on recently and it looks like this is mostly possible, but the vague area I am wondering about is spawning a background thread and then maintaining a reference to it so the user can get the data back and/or manipulate it in various ways(start/stop/put to a different queue etc) I saw RailsCron mentioned in another thread, but am not sure that is exactly what I am looking for (maybe I''m wrong). Anyone with some experience with this sort of scenario that can provide some guidance would be greatly appreciated. I''ve asked the question a few times in various forums with little to no response. Thanks in advance, Eric -- Posted via http://www.ruby-forum.com/.
link_to_remote ? On 2/14/06, Eric Ching <bingopajama@hotmail.com> wrote:> > I am just getting into web servers/web applications and rails as well > so bear with me. I am trying to write a web app that, based on a > users input from the browser, will perform some task, and update the > browser (ala ajax style) as needed and/or provide a way for the user to > control the background task. > > Now, I have more experience with java servlets, which makes this easy > enough for a beginner. The user inputs some info and the server spawns > a new thread to perform the task in the background while the user can > continue using the app, until it is updated with the results. (in my > case the background task is putting or getting messages on/from an MQ > Series queue) I keep a reference to the background task in the > session object for access as needed. > > What I am wondering is if the same concepts can be applied to a rails > application. I started researching rails off and on recently and it > looks like this is mostly possible, but the vague area I am wondering > about is spawning a background thread and then maintaining a reference > to it so the user can get the data back and/or manipulate it in various > ways(start/stop/put to a different queue etc) > > I saw RailsCron mentioned in another thread, but am not sure that is > exactly what I am looking for (maybe I''m wrong). Anyone with some > experience with this sort of scenario that can provide some guidance > would be greatly appreciated. I''ve asked the question a few times in > various forums with little to no response. > > Thanks in advance, > Eric > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060214/f03fece5/attachment.html
You could start a new thread and put it in the session, which they could
then stop or start.
Like:
session[:thread] = Thread.new{process}
Look at http://www.ruby-doc.org/core/classes/Thread.html &
http://www.ruby-doc.org/stdlib/libdoc/thread/rdoc/index.html
Or threads in the Ruby PicAxe book.
Joey
--
Posted via http://www.ruby-forum.com/.
On Feb 14, 2006, at 2:30 PM, joey__ wrote:> You could start a new thread and put it in the session, which they > could > then stop or start. > > Like: > session[:thread] = Thread.new{process} > > Look at http://www.ruby-doc.org/core/classes/Thread.html & > http://www.ruby-doc.org/stdlib/libdoc/thread/rdoc/index.html > Or threads in the Ruby PicAxe book. > > > JoeyYou will find that you cannot put a thread in the session. To put an object in the session it needs to be serializable. This means no threads, singletons or procs/lambdas can go in the session. Cheers- -Ezra Zygmuntowicz Yakima Herald-Republic WebMaster http://yakimaherald.com 509-577-7732 ezra@yakima-herald.com
If it''s as simple as that I''ll be very happy. Initially I''d just be using WEBrick (for internal purposes), does that pose any problems? For some reason I thought this was not possible, or practical. I''ll have to try and remember why I thought so. -- Posted via http://www.ruby-forum.com/.
Bummer, didn''t see the last post. So what''s a good solution to this problem? Seems like it would be a common need, but then I''m kind of green to web programming. Any suggestions? -- Posted via http://www.ruby-forum.com/.
Ezra Zygmuntowicz wrote:> You will find that you cannot put a thread in the session. To put an > object in the session it needs to be serializable. This means no > threads, singletons or procs/lambdas can go in the session.So you know, I''ve seen this information (and I think I''ve even told someone this information), but it''s just now ocurred to me that I should ask if there''s an equivalent of java''s "transient" in ruby? In java, making a variable transient means that it won''t be serialized. If Ruby''s serialization pulls the whole graph, then there must be some way to tell it to skip certain children... or be very careful what you create references to in your objects. As for the OP''s question though, I would think you could store your threads or procs in a hash and put the key in the session. b
>From my experience, at least when running on Apache, putting work intoa background thread did not give me the results that I wanted. It appears that the thread will suspend itself unless the server gets activity. I ended up having to use a timer() and Ajax.Request to keep polling for whether the thread was done to, among other things, keep the thread running. I had better luck forking off a heavyweight process to do the background work. I''ve heard that the background processes work better in LigHTTP. On 2/14/06, Ben Munat <bent@munat.com> wrote:> Ezra Zygmuntowicz wrote: > > You will find that you cannot put a thread in the session. To put an > > object in the session it needs to be serializable. This means no > > threads, singletons or procs/lambdas can go in the session. > > So you know, I''ve seen this information (and I think I''ve even told someone this > information), but it''s just now ocurred to me that I should ask if there''s an equivalent > of java''s "transient" in ruby? In java, making a variable transient means that it won''t be > serialized. > > If Ruby''s serialization pulls the whole graph, then there must be some way to tell it to > skip certain children... or be very careful what you create references to in your objects. > > As for the OP''s question though, I would think you could store your threads or procs in a > hash and put the key in the session. > > b > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
Ben Munat wrote:> As for the OP''s question though, I would think you could store your > threads or procs in a hash and put the key in the session.Given that the session *is* a hash, how does that help? It just won''t work. Without being able to serialize the threads, what does the hash key reference? Bear in mind that the lookup might not happen on the same machine, let alone in the same process... -- Alex
On Feb 14, 2006, at 8:40 PM, Alex Young wrote:> Ben Munat wrote: >> As for the OP''s question though, I would think you could store >> your threads or procs in a hash and put the key in the session. > > Given that the session *is* a hash, how does that help? It just > won''t work. Without being able to serialize the threads, what does > the hash key reference? Bear in mind that the lookup might not > happen on the same machine, let alone in the same process... > > -- > > Alex > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >One way I handle longer running background processes in my rails apps is to use distributed ruby(drb). Drb is dead simple to use so its easy to make a drb server that contains the classes needed to do the long running work. Then you can kick off the long running task to the drb server from an ajax request. Then you can use a periodically call remote to poll the drb server for its percent done on the job you submitted. Once its 100% you can return the results. Doing it this way divorces the long running task from the rails request/ response cycle all together and makes it easy to add progress bars to the view for these long tasks. I have this functionality almost in a state to refactor it into a plugin if any one was interested in it. Cheers- -Ezra Zygmuntowicz WebMaster Yakima Herald-Republic Newspaper ezra@yakima-herald.com 509-577-7732
I''ve been solving this by using Rinda (it''s in the pick-ax book and is the Ruby version of Linda which is a robust scalable method) so that RoR posts messages for things it wants done, and then "worker agents" process the messages and post back another one giving updates or a "I''m done" message. If you investigate Rinda and are interested in taking this approach, then I can share some code that will save you a lot of time (the documentation for getting started on Rinda is a little sparse). I''d love to hear how other people are solving the problem. -Greg -----Original Message----- From: rails-bounces@lists.rubyonrails.org [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Ching Sent: Tuesday, February 14, 2006 12:55 PM To: rails@lists.rubyonrails.org Subject: [Rails] Rails and background tasks/threads I am just getting into web servers/web applications and rails as well so bear with me. I am trying to write a web app that, based on a users input from the browser, will perform some task, and update the browser (ala ajax style) as needed and/or provide a way for the user to control the background task. Now, I have more experience with java servlets, which makes this easy enough for a beginner. The user inputs some info and the server spawns a new thread to perform the task in the background while the user can continue using the app, until it is updated with the results. (in my case the background task is putting or getting messages on/from an MQ Series queue) I keep a reference to the background task in the session object for access as needed. What I am wondering is if the same concepts can be applied to a rails application. I started researching rails off and on recently and it looks like this is mostly possible, but the vague area I am wondering about is spawning a background thread and then maintaining a reference to it so the user can get the data back and/or manipulate it in various ways(start/stop/put to a different queue etc) I saw RailsCron mentioned in another thread, but am not sure that is exactly what I am looking for (maybe I''m wrong). Anyone with some experience with this sort of scenario that can provide some guidance would be greatly appreciated. I''ve asked the question a few times in various forums with little to no response. Thanks in advance, Eric -- Posted via http://www.ruby-forum.com/. _______________________________________________ Rails mailing list Rails@lists.rubyonrails.org http://lists.rubyonrails.org/mailman/listinfo/rails
Alex Young wrote:> Ben Munat wrote: > >> As for the OP''s question though, I would think you could store your >> threads or procs in a hash and put the key in the session. > > > Given that the session *is* a hash, how does that help? It just won''t > work. Without being able to serialize the threads, what does the hash > key reference? Bear in mind that the lookup might not happen on the > same machine, let alone in the same process... >The key can just be a string, so the session is once again serializable. You keep your threads in a hash in memory. When the thread is needed again, the code that handles this looks for the appropriate session value to get it''s key, and then uses that to get the thread from the hash. It''s basically just keeping the non-serializable stuff in your own hash that won''t be serialized. Hmm, although, I guess this wouldn''t work without a daemon process in which to keep the hash... there is a way to do that in rails, right? Well, still... one could start a separate ruby process for this storage... would need to hook into that from the rails code somehow. b
Ezra Zygmuntowicz wrote:> One way I handle longer running background processes in my rails > apps is to use distributed ruby(drb). Drb is dead simple to use so its > easy to make a drb server that contains the classes needed to do the > long running work. Then you can kick off the long running task to the > drb server from an ajax request. Then you can use a periodically call > remote to poll the drb server for its percent done on the job you > submitted. Once its 100% you can return the results. Doing it this way > divorces the long running task from the rails request/ response cycle > all together and makes it easy to add progress bars to the view for > these long tasks. I have this functionality almost in a state to > refactor it into a plugin if any one was interested in it. >Ezra, you''re just a fount of useful information! I should have read this before I replied to Alex. And I would defintely like to hear about that plugin! b PS: I reaaaallllly think it''s important that mongrel (or something like it) turn into a servlet-container-style daemonized process into which one can hook and unhook apps while running, spin off background threads, pool db connections, etc. I don''t think the java world will take rails seriously without that. Then again, maybe drb can grow into that, or hook into mongrel, etc. etc.
Greg Edwards (other box) wrote:> I''ve been solving this by using Rinda (it''s in the pick-ax book and is the > Ruby version of Linda which is a robust scalable method) so that RoR posts > messages for things it wants done, and then "worker agents" process the > messages and post back another one giving updates or a "I''m done" message. > If you investigate Rinda and are interested in taking this approach, then I > can share some code that will save you a lot of time (the documentation for > getting started on Rinda is a little sparse). > > I''d love to hear how other people are solving the problem. > -Greg >It''d be cool to see that code.... of course, I should probably learn some more ruby and get my rails training app into a respectable state first... but this would be more fun! b
Ben Munat wrote:> Alex Young wrote: >> Ben Munat wrote: >> >>> As for the OP''s question though, I would think you could store your >>> threads or procs in a hash and put the key in the session. >> >> >> Given that the session *is* a hash, how does that help? It just won''t >> work. Without being able to serialize the threads, what does the hash >> key reference? Bear in mind that the lookup might not happen on the >> same machine, let alone in the same process... >> > > The key can just be a string, so the session is once again serializable. > You keep your threads in a hash in memory. When the thread is needed > again, the code that handles this looks for the appropriate session > value to get it''s key, and then uses that to get the thread from the hash. >The session''s serializable, but that''s got you nothing because when you unserialize it on the next request, what your key points to may well not exist any more, or exist on another machine, or just generally not be visible.> It''s basically just keeping the non-serializable stuff in your own hash > that won''t be serialized.If that worked, you wouldn''t need a session *at all*, you could just keep everything in memory all the time.> Hmm, although, I guess this wouldn''t work without a daemon process in > which to keep the hash... there is a way to do that in rails, right? > Well, still... one could start a separate ruby process for this > storage... would need to hook into that from the rails code somehow.Yay! DRb! -- Alex
Ezra Zygmuntowicz wrote:> One way I handle longer running background processes in my rails > apps is to use distributed ruby(drb). Drb is dead simple to use so its > easy to make a drb server that contains the classes needed to do the > long running work. Then you can kick off the long running task to the > drb server from an ajax request. Then you can use a periodically call > remote to poll the drb server for its percent done on the job you > submitted. Once its 100% you can return the results. Doing it this way > divorces the long running task from the rails request/response cycle all > together and makes it easy to add progress bars to the view for these > long tasks. I have this functionality almost in a state to refactor it > into a plugin if any one was interested in it.I''d love to see that. I''m bodging that approach together piecewise at the moment, and a plugin would make it *much* simpler. Rather than long-running tasks, though, I''m actually talking to permanent daemons that happen to live in the same object space, so the concept of ''percent done'' doesn''t really apply. I''ve spotted where I would pluginize this, but I haven''t had the impetus to fully work it through. If you''re closer to it than I am, please, please release! :-) -- Alex
On Feb 15, 2006, at 2:29 AM, Greg Edwards (other box) wrote:> I''ve been solving this by using Rinda (it''s in the pick-ax book and > is the > Ruby version of Linda which is a robust scalable method) so that > RoR posts > messages for things it wants done, and then "worker agents" process > the > messages and post back another one giving updates or a "I''m done" > message. > If you investigate Rinda and are interested in taking this > approach, then I > can share some code that will save you a lot of time (the > documentation for > getting started on Rinda is a little sparse).I''d thank you if you did share. I''d like to use Rinda but keep putting it off until I have more time (as if that''ll ever happen). If you have a reliable way of starting/restarting the Rinda servers, or any ideas, I think that''d be very well received too :-) Cheers, Bob> > I''d love to hear how other people are solving the problem. > -Greg---- Bob Hutchison -- blogs at <http://www.recursive.ca/ hutch/> Recursive Design Inc. -- <http://www.recursive.ca/> Raconteur -- <http://www.raconteur.info/> xampl for Ruby -- <http://rubyforge.org/projects/xampl/>
Wow, thanks for all the great responses. I think I need to plug away at some more Rails and give the sugestions a try (rinda/drb). Any example would be greatly appreciated. I want to use Rails because there is so much about it I do like, but without being able to tightly control background threads/tasks I really cannot jump into it 100% - that''s the make or break for me. -- Posted via http://www.ruby-forum.com/.
Greg- I would love to see your rinda code. I have a great grasp of drb and have been using it heavily but I haven''t really dug into rinda yet and the docs are sparse. So please do share. Thanks -Ezra On Feb 14, 2006, at 11:29 PM, Greg Edwards (other box) wrote:> I''ve been solving this by using Rinda (it''s in the pick-ax book and > is the > Ruby version of Linda which is a robust scalable method) so that > RoR posts > messages for things it wants done, and then "worker agents" process > the > messages and post back another one giving updates or a "I''m done" > message. > If you investigate Rinda and are interested in taking this > approach, then I > can share some code that will save you a lot of time (the > documentation for > getting started on Rinda is a little sparse). > > I''d love to hear how other people are solving the problem. > -Greg > > > > -----Original Message----- > From: rails-bounces@lists.rubyonrails.org > [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Ching > Sent: Tuesday, February 14, 2006 12:55 PM > To: rails@lists.rubyonrails.org > Subject: [Rails] Rails and background tasks/threads > > I am just getting into web servers/web applications and rails as well > so bear with me. I am trying to write a web app that, based on a > users input from the browser, will perform some task, and update the > browser (ala ajax style) as needed and/or provide a way for the > user to > control the background task. > > Now, I have more experience with java servlets, which makes this easy > enough for a beginner. The user inputs some info and the server > spawns > a new thread to perform the task in the background while the user can > continue using the app, until it is updated with the results. (in my > case the background task is putting or getting messages on/from an MQ > Series queue) I keep a reference to the background task in the > session object for access as needed. > > What I am wondering is if the same concepts can be applied to a rails > application. I started researching rails off and on recently and it > looks like this is mostly possible, but the vague area I am wondering > about is spawning a background thread and then maintaining a reference > to it so the user can get the data back and/or manipulate it in > various > ways(start/stop/put to a different queue etc) > > I saw RailsCron mentioned in another thread, but am not sure that is > exactly what I am looking for (maybe I''m wrong). Anyone with some > experience with this sort of scenario that can provide some guidance > would be greatly appreciated. I''ve asked the question a few times in > various forums with little to no response. > > Thanks in advance, > Eric > > -- > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-Ezra Zygmuntowicz WebMaster Yakima Herald-Republic Newspaper ezra@yakima-herald.com 509-577-7732
It sounds like RailsCron might be a good fit, as it''s a good way to run rails background tasks in general. You might be able to get away with a simple daemonized script/runner. Assuming that you have figured out the best way to have a background task running satisfactorily, I would write it as a method that traps a signal, and whenever that signal is sent, it looks in the database (or a flat file) to see what has changed. Alternately, you could poll the database. -Kyle
Alex Young wrote:>> It''s basically just keeping the non-serializable stuff in your own >> hash that won''t be serialized. > > If that worked, you wouldn''t need a session *at all*, you could just > keep everything in memory all the time.Well, yeah, but all the other crap that you can serialize you might as well have in a session and let it get serialized. This whole thread was more or less a thought experiment anyway. I don''t personally currently have a need to stick threads, procs, or singletons in the session... but I expect I might some day. I''m just working through ruby''s rough spots mentally... Ultimately, my point is that ruby/rails needs a way to keep application-wide data in memory. This is very useful in the Java world. In fact, I''d say that a big chunk of the reason why Java did so well in the web world was that they went straight for the container model. Plopping a multi-threaded, long-running server process into a world of start-process-stop CGI insanity made people go "whoa... cool". b
On Feb 15, 2006, at 6:32 AM, Eric Ching wrote:> Wow, thanks for all the great responses. I think I need to plug > away at > some more Rails and give the sugestions a try (rinda/drb). Any > example > would be greatly appreciated. > > I want to use Rails because there is so much about it I do like, but > without being able to tightly control background threads/tasks I > really > cannot jump into it 100% - that''s the make or break for me.Eric- Here is a simple drb server/client for you to play with. This is an example of exposing an ActiveRecord model over drb. You can actually publish any object you like over drb. this example is a bit contrived but it can show you how drb works a bit. Run the server in a terminal and then open irb and paste in the client part and play with that. I am going to work on getting my drb+ajax progress bars plugin into releaseable shape this weekend so stay tuned. #!/usr/local/bin/ruby #server require ''rubygems'' require ''drb/drb'' require_gem ''activerecord'' ActiveRecord::Base.establish_connection( :adapter => "mysql", :username => "root", :host => "localhost", :password => "xxxxxxx", :database => "remote_drb" ) ActiveRecord::Schema.define(:version => 1) do create_table "remote_records", :force => true do |t| t.column :title, :string t.column :desc, :string end end class RemoteRecord < ActiveRecord::Base include DRb::DRbUndumped end DRb.start_service("druby://127.0.0.1:3500", RemoteRecord) puts DRb.uri DRb.thread.join __END__ #client run in irb require ''rubygems'' require ''drb/drb'' require_gem ''activerecord'' DRb.start_service RemoteRecord = DRbObject.new(nil, ''druby://127.0.0.1:3500'') [ [''foo'',''bar''], [''qux'', ''baz''], [''nik'', ''nuk''] ].each do |record| RemoteRecord.create :title => record[0], :desc => record[1] end test = RemoteRecord.find :all test.each{|o| p "#{o.title} : #{o.desc}" } # results in: # "foo : bar" # "qux : baz" # "nik : nuk" Cheers- -Ezra Zygmuntowicz Yakima Herald-Republic WebMaster http://yakimaherald.com 509-577-7732 ezra@yakima-herald.com
Greg Edwards (other box)
2006-Feb-16 00:44 UTC
Using Linda/DRb with Rails (was: [Rails] Rails and background tasks/threads)
Ezra (and everyone else :-)
Ok, here''s how I am using Linda (cleaned up as best I can to make
things
more clear). Basically, put all of these files into one directory to get
started, and then after things are working then you can start moving
different pieces to different servers.
First of all, the background that you should read (or, the background that I
read that was helpful):
http://en.wikipedia.org/wiki/Linda_%28coordination_language%29 (good
background on Linda (and why you''d want to use it), which Rinda
implements)
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/15023
http://segment7.net/projects/ruby/drb/rinda/ringserver.html
http://www.ruby-doc.org/stdlib/libdoc/rinda/rdoc/classes/Rinda/TupleSpace.ht
ml
http://www.ntecs.de/blog/Tech/ComputerScience
and, of course, the pickax has a page on Rinda.
And finally, my code (not very well documented, since it assumes that
you''ve
read all of the other URLs I pasted above and that now you just want an
example of how it works all together as well as an example of several things
that I couldn''t really find easily in the documentation).
(there are 4 files below)
--- the port where everyone talks to each other needs to be defined ---
In file, rinda_port_uri.rb
--------------------------
RINDA_PORT_URI = "druby://your.hostname.com:12345"
#(you can also use "localhost" if you are all running on the same
machine, but as soon as you put ANY process on a different server, then they
have to all have the exact same druby:// line (as far as I can tell... :-)
#ruby -rsocket -e ''p Socket.gethostname''
# this command was helpful for me to find out what ruby thinks my
server was named.
File: rinda_blackboard.rb # this is the main "server" that enables
agents
to get/send messages for things they want someone to do.
-------------------------
require ''drb/drb''
require ''drb/acl''
require ''rinda/tuplespace''
require ''rinda_port_uri'' # defines the constant used for the
Port
acl = ACL.new(%w(deny all # block out other servers from accessing your
Linda blackboard
allow
10.0.0.7
allow
10.0.0.4
allow
60.59.145.219 #the documentation says that you can put wildcards
''60.59.145.*'' but that didn''t work for me...)
allow
60.59.145.220
allow
localhost))
DRb.install_acl(acl)
#ruby -rsocket -e ''p Socket.gethostname''
DRb.start_service(RINDA_PORT_URI, Rinda::TupleSpace.new(10))
#puts DRb.uri
#(0..5).each {|x| # this block will start up 5 agents
# puts "Starting agent #{x}"
# IO.popen("ruby rinda_agent.rb")
#}
puts "blackboard is open on #{RINDA_PORT_URI}"
DRb.thread.join
#puts ''[return] to exit''
#gets
File: rinda_agent.rb # this is where you do the work... you can have
multiple agents running, and on different servers all pointing to the same
rinda_blackboard
--------------------
# this is a "example wrapper" thing... If I want an agent to do some
work
# for me, then I (personally) want to pass:
# - a message,
# - a hash containing variables and stuff
# - a reference ID that uniquely identifies this message request
# - the "return message" I will be looking for that I''d like
the agent to
# reply using (so that I know what to look for)
# - a return hash containing results from the agent
# (that''s just me, what I want...)
require ''drb/drb''
require ''rinda/tuplespace''
require ''rinda_port_uri'' # defines the constant used for the
Port
require ''json/lexer''
require ''json/objects''
# I like using JSON to tranfer info between agents, but you can use
a simple hash if you wish.
DRb.start_service
ts = Rinda::TupleSpaceProxy.new(DRbObject.new(nil,RINDA_PORT_URI))
class Gen
def worker1(var_in, ref_id, msg_out)
... do something here
an_amazing_result = ...
#return the results of the work
[msg_out,an_amazing_result]
end
end
loop do
puts
puts
puts
puts
puts "waiting...(for PPT stuff)"
# Read the next command
op, var_in, ref_id,msg_out,var_out = ts.take(
[%r{^(msg_1_I_listen_for|msg_2_I_listen_for)$},nil,nil,nil,nil],55) # time
out after 55 seconds if no activity... good for debugging initially (can be
omitted in production)
puts "Received! #{op} #{ref_id} #{var_in}"
gen = Gen.new
msg_out,var_out = gen.send(op, var_in, ref_id)
#--- DONE --------------------
ts.write([msg_out,ref_id, var_out]) unless (msg_out.nil? or
msg_out.length == 0)
puts "done! #{msg_out} #{ref_id} #{var_out}"
end
file: rinda_watcher.rb
-----------------------
#finally, once you''ve got your blackboard where you can post messages
to get
work done, I wanted to be able to SEE what the heck was being put on there
:-) so this file is a easy to implement, hacky, fast way of doing that...
require ''drb/drb''
require ''rinda/tuplespace''
require ''rinda_port_uri'' # defines the constant used for the
Port
require ''pp''
DRb.start_service
puts "trying to connect to #{RINDA_PORT_URI}..."
ts = Rinda::TupleSpaceProxy.new(DRbObject.new(nil,RINDA_PORT_URI))
puts "---------------------Watching"
option = 1
case option
when 1
result = ts.read_all([nil,nil])
pp result unless result.nil?
result = ts.read_all([nil,nil,nil])
pp result unless result.nil?
result = ts.read_all([nil,nil,nil,nil])
pp result unless result.nil?
result = ts.read_all([nil,nil,nil,nil,nil])
pp result unless result.nil?
result = ts.read_all([nil,nil,nil,nil,nil,nil])
pp result unless result.nil?
puts
when 2
result ts.read_all([%r{prepare_to_generate_ppt_.*},
nil, #session[:username],
nil, #@current_ppt_name,
nil, #@new_ppt_name,
nil, #@path_to_template,
nil, #json_msg
nil, #user password
])
pp result unless result.nil?
when 3
0.upto(10) {|ignore|
hit = 0
result = ts.read_all([nil,nil])
unless result.nil? or result.size ==0
pp result
ts.take([nil,nil])
hit +=1
end
result = ts.read_all([nil,nil,nil])
unless result.nil? or result.size ==0
pp result
ts.take([nil,nil,nil])
hit +=1
end
result = ts.read_all([nil,nil,nil,nil])
unless result.nil? or result.size ==0
pp result
ts.take([nil,nil,nil,nil])
hit +=1
end
result = ts.read_all([nil,nil,nil,nil,nil])
unless result.nil? or result.size ==0
pp result
ts.take([nil,nil,nil,nil,nil])
hit +=1
end
result ts.read_all([nil,nil,nil,nil,nil,nil])
unless result.nil? or result.size ==0
pp result
ts.take([nil,nil,nil,nil,nil,nil])
hit +=1
end
result ts.read_all([nil,nil,nil,nil,nil,nil,nil])
unless result.nil? or result.size ==0
pp result
ts.take([nil,nil,nil,nil,nil,nil,nil])
hit +=1
end
if hit > 0
puts ''----------------------------''
puts
end
}
End
-Greg
Greg.railslist@eyetools.com
916 792 4538
-----Original Message-----
From: rails-bounces@lists.rubyonrails.org
[mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Ezra Zygmuntowicz
Sent: Wednesday, February 15, 2006 8:11 AM
To: other@eyetools.com; rails@lists.rubyonrails.org
Subject: Re: [Rails] Rails and background tasks/threads
Greg-
I would love to see your rinda code. I have a great grasp of drb and
have been using it heavily but I haven''t really dug into rinda yet
and the docs are sparse. So please do share.
Thanks
-Ezra
On Feb 14, 2006, at 11:29 PM, Greg Edwards (other box) wrote:
> I''ve been solving this by using Rinda (it''s in the
pick-ax book and
> is the
> Ruby version of Linda which is a robust scalable method) so that
> RoR posts
> messages for things it wants done, and then "worker agents"
process
> the
> messages and post back another one giving updates or a "I''m
done"
> message.
> If you investigate Rinda and are interested in taking this
> approach, then I
> can share some code that will save you a lot of time (the
> documentation for
> getting started on Rinda is a little sparse).
>
> I''d love to hear how other people are solving the problem.
> -Greg
>
>
>
> -----Original Message-----
> From: rails-bounces@lists.rubyonrails.org
> [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Ching
> Sent: Tuesday, February 14, 2006 12:55 PM
> To: rails@lists.rubyonrails.org
> Subject: [Rails] Rails and background tasks/threads
>
> I am just getting into web servers/web applications and rails as well
> so bear with me. I am trying to write a web app that, based on a
> users input from the browser, will perform some task, and update the
> browser (ala ajax style) as needed and/or provide a way for the
> user to
> control the background task.
>
> Now, I have more experience with java servlets, which makes this easy
> enough for a beginner. The user inputs some info and the server
> spawns
> a new thread to perform the task in the background while the user can
> continue using the app, until it is updated with the results. (in my
> case the background task is putting or getting messages on/from an MQ
> Series queue) I keep a reference to the background task in the
> session object for access as needed.
>
> What I am wondering is if the same concepts can be applied to a rails
> application. I started researching rails off and on recently and it
> looks like this is mostly possible, but the vague area I am wondering
> about is spawning a background thread and then maintaining a reference
> to it so the user can get the data back and/or manipulate it in
> various
> ways(start/stop/put to a different queue etc)
>
> I saw RailsCron mentioned in another thread, but am not sure that is
> exactly what I am looking for (maybe I''m wrong). Anyone with some
> experience with this sort of scenario that can provide some guidance
> would be greatly appreciated. I''ve asked the question a few times
in
> various forums with little to no response.
>
> Thanks in advance,
> Eric
>
> --
>
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
-Ezra Zygmuntowicz
WebMaster
Yakima Herald-Republic Newspaper
ezra@yakima-herald.com
509-577-7732
_______________________________________________
Rails mailing list
Rails@lists.rubyonrails.org
http://lists.rubyonrails.org/mailman/listinfo/rails
Sweet... moving this to my "saved" folder! b Greg Edwards (other box) wrote:> Ezra (and everyone else :-) > > Ok, here''s how I am using Linda (cleaned up as best I can to make things > more clear). Basically, put all of these files into one directory to get > started, and then after things are working then you can start moving > different pieces to different servers. > > First of all, the background that you should read (or, the background that I > read that was helpful): > http://en.wikipedia.org/wiki/Linda_%28coordination_language%29 (good > background on Linda (and why you''d want to use it), which Rinda implements) > http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/15023 > http://segment7.net/projects/ruby/drb/rinda/ringserver.html > http://www.ruby-doc.org/stdlib/libdoc/rinda/rdoc/classes/Rinda/TupleSpace.ht > ml > http://www.ntecs.de/blog/Tech/ComputerScience > > and, of course, the pickax has a page on Rinda. > > And finally, my code (not very well documented, since it assumes that you''ve > read all of the other URLs I pasted above and that now you just want an > example of how it works all together as well as an example of several things > that I couldn''t really find easily in the documentation). > > (there are 4 files below) > > --- the port where everyone talks to each other needs to be defined --- > In file, rinda_port_uri.rb > -------------------------- > RINDA_PORT_URI = "druby://your.hostname.com:12345" > #(you can also use "localhost" if you are all running on the same > machine, but as soon as you put ANY process on a different server, then they > have to all have the exact same druby:// line (as far as I can tell... :-) > #ruby -rsocket -e ''p Socket.gethostname'' > # this command was helpful for me to find out what ruby thinks my > server was named. > > > File: rinda_blackboard.rb # this is the main "server" that enables agents > to get/send messages for things they want someone to do. > ------------------------- > require ''drb/drb'' > require ''drb/acl'' > require ''rinda/tuplespace'' > require ''rinda_port_uri'' # defines the constant used for the Port > > acl = ACL.new(%w(deny all # block out other servers from accessing your > Linda blackboard > allow > 10.0.0.7 > allow > 10.0.0.4 > allow > 60.59.145.219 #the documentation says that you can put wildcards > ''60.59.145.*'' but that didn''t work for me...) > allow > 60.59.145.220 > allow > localhost)) > > DRb.install_acl(acl) > > #ruby -rsocket -e ''p Socket.gethostname'' > DRb.start_service(RINDA_PORT_URI, Rinda::TupleSpace.new(10)) > #puts DRb.uri > #(0..5).each {|x| # this block will start up 5 agents > # puts "Starting agent #{x}" > # IO.popen("ruby rinda_agent.rb") > #} > puts "blackboard is open on #{RINDA_PORT_URI}" > DRb.thread.join > #puts ''[return] to exit'' > #gets > > > > File: rinda_agent.rb # this is where you do the work... you can have > multiple agents running, and on different servers all pointing to the same > rinda_blackboard > -------------------- > # this is a "example wrapper" thing... If I want an agent to do some work > # for me, then I (personally) want to pass: > # - a message, > # - a hash containing variables and stuff > # - a reference ID that uniquely identifies this message request > # - the "return message" I will be looking for that I''d like the agent to > # reply using (so that I know what to look for) > # - a return hash containing results from the agent > # (that''s just me, what I want...) > > require ''drb/drb'' > require ''rinda/tuplespace'' > require ''rinda_port_uri'' # defines the constant used for the Port > require ''json/lexer'' > require ''json/objects'' > # I like using JSON to tranfer info between agents, but you can use > a simple hash if you wish. > > DRb.start_service > ts = Rinda::TupleSpaceProxy.new(DRbObject.new(nil,RINDA_PORT_URI)) > > class Gen > > def worker1(var_in, ref_id, msg_out) > ... do something here > an_amazing_result = ... > #return the results of the work > [msg_out,an_amazing_result] > end > end > > loop do > puts > puts > puts > puts > puts "waiting...(for PPT stuff)" > > # Read the next command > op, var_in, ref_id,msg_out,var_out = ts.take( > > [%r{^(msg_1_I_listen_for|msg_2_I_listen_for)$},nil,nil,nil,nil],55) # time > out after 55 seconds if no activity... good for debugging initially (can be > omitted in production) > > puts "Received! #{op} #{ref_id} #{var_in}" > > gen = Gen.new > msg_out,var_out = gen.send(op, var_in, ref_id) > > #--- DONE -------------------- > ts.write([msg_out,ref_id, var_out]) unless (msg_out.nil? or > msg_out.length == 0) > puts "done! #{msg_out} #{ref_id} #{var_out}" > end > > > file: rinda_watcher.rb > ----------------------- > #finally, once you''ve got your blackboard where you can post messages to get > work done, I wanted to be able to SEE what the heck was being put on there > :-) so this file is a easy to implement, hacky, fast way of doing that... > > require ''drb/drb'' > require ''rinda/tuplespace'' > require ''rinda_port_uri'' # defines the constant used for the Port > require ''pp'' > > DRb.start_service > puts "trying to connect to #{RINDA_PORT_URI}..." > ts = Rinda::TupleSpaceProxy.new(DRbObject.new(nil,RINDA_PORT_URI)) > puts "---------------------Watching" > > option = 1 > case option > when 1 > result = ts.read_all([nil,nil]) > pp result unless result.nil? > result = ts.read_all([nil,nil,nil]) > pp result unless result.nil? > result = ts.read_all([nil,nil,nil,nil]) > pp result unless result.nil? > result = ts.read_all([nil,nil,nil,nil,nil]) > pp result unless result.nil? > result = ts.read_all([nil,nil,nil,nil,nil,nil]) > pp result unless result.nil? > puts > when 2 > result > ts.read_all([%r{prepare_to_generate_ppt_.*}, > nil, #session[:username], > nil, #@current_ppt_name, > nil, #@new_ppt_name, > nil, #@path_to_template, > nil, #json_msg > nil, #user password > ]) > > pp result unless result.nil? > when 3 > 0.upto(10) {|ignore| > hit = 0 > result = ts.read_all([nil,nil]) > unless result.nil? or result.size ==0 > pp result > ts.take([nil,nil]) > hit +=1 > end > > result = ts.read_all([nil,nil,nil]) > unless result.nil? or result.size ==0 > pp result > ts.take([nil,nil,nil]) > hit +=1 > end > > result = ts.read_all([nil,nil,nil,nil]) > unless result.nil? or result.size ==0 > pp result > ts.take([nil,nil,nil,nil]) > hit +=1 > end > > result = ts.read_all([nil,nil,nil,nil,nil]) > unless result.nil? or result.size ==0 > pp result > ts.take([nil,nil,nil,nil,nil]) > hit +=1 > end > > result > ts.read_all([nil,nil,nil,nil,nil,nil]) > unless result.nil? or result.size ==0 > pp result > ts.take([nil,nil,nil,nil,nil,nil]) > hit +=1 > end > > result > ts.read_all([nil,nil,nil,nil,nil,nil,nil]) > unless result.nil? or result.size ==0 > pp result > > ts.take([nil,nil,nil,nil,nil,nil,nil]) > hit +=1 > end > > if hit > 0 > puts ''----------------------------'' > puts > end > } > > End > > > -Greg > > Greg.railslist@eyetools.com > 916 792 4538 > > > -----Original Message----- > From: rails-bounces@lists.rubyonrails.org > [mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Ezra Zygmuntowicz > Sent: Wednesday, February 15, 2006 8:11 AM > To: other@eyetools.com; rails@lists.rubyonrails.org > Subject: Re: [Rails] Rails and background tasks/threads > > Greg- > > I would love to see your rinda code. I have a great grasp of drb and > > have been using it heavily but I haven''t really dug into rinda yet > and the docs are sparse. So please do share. > > Thanks > -Ezra > > On Feb 14, 2006, at 11:29 PM, Greg Edwards (other box) wrote: > > >>I''ve been solving this by using Rinda (it''s in the pick-ax book and >>is the >>Ruby version of Linda which is a robust scalable method) so that >>RoR posts >>messages for things it wants done, and then "worker agents" process >>the >>messages and post back another one giving updates or a "I''m done" >>message. >>If you investigate Rinda and are interested in taking this >>approach, then I >>can share some code that will save you a lot of time (the >>documentation for >>getting started on Rinda is a little sparse). >> >>I''d love to hear how other people are solving the problem. >>-Greg >> >> >> >>-----Original Message----- >>From: rails-bounces@lists.rubyonrails.org >>[mailto:rails-bounces@lists.rubyonrails.org] On Behalf Of Eric Ching >>Sent: Tuesday, February 14, 2006 12:55 PM >>To: rails@lists.rubyonrails.org >>Subject: [Rails] Rails and background tasks/threads >> >>I am just getting into web servers/web applications and rails as well >>so bear with me. I am trying to write a web app that, based on a >>users input from the browser, will perform some task, and update the >>browser (ala ajax style) as needed and/or provide a way for the >>user to >>control the background task. >> >>Now, I have more experience with java servlets, which makes this easy >>enough for a beginner. The user inputs some info and the server >>spawns >>a new thread to perform the task in the background while the user can >>continue using the app, until it is updated with the results. (in my >>case the background task is putting or getting messages on/from an MQ >>Series queue) I keep a reference to the background task in the >>session object for access as needed. >> >>What I am wondering is if the same concepts can be applied to a rails >>application. I started researching rails off and on recently and it >>looks like this is mostly possible, but the vague area I am wondering >>about is spawning a background thread and then maintaining a reference >>to it so the user can get the data back and/or manipulate it in >>various >>ways(start/stop/put to a different queue etc) >> >>I saw RailsCron mentioned in another thread, but am not sure that is >>exactly what I am looking for (maybe I''m wrong). Anyone with some >>experience with this sort of scenario that can provide some guidance >>would be greatly appreciated. I''ve asked the question a few times in >>various forums with little to no response. >> >>Thanks in advance, >>Eric >> >>-- >> >>_______________________________________________ >>Rails mailing list >>Rails@lists.rubyonrails.org >>http://lists.rubyonrails.org/mailman/listinfo/rails >> >>_______________________________________________ >>Rails mailing list >>Rails@lists.rubyonrails.org >>http://lists.rubyonrails.org/mailman/listinfo/rails >> > > > -Ezra Zygmuntowicz > WebMaster > Yakima Herald-Republic Newspaper > ezra@yakima-herald.com > 509-577-7732 > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails
Add me to the list of people eagerly awaiting the arrival of this plugin. :) On Feb 15, 2006, at 12:24 PM, Ezra Zygmuntowicz wrote:> > Here is a simple drb server/client for you to play with. This is > an example of exposing an ActiveRecord model over drb. You can > actually publish any object you like over drb. this example is a > bit contrived but it can show you how drb works a bit. Run the > server in a terminal and then open irb and paste in the client part > and play with that. I am going to work on getting my drb+ajax > progress bars plugin into releaseable shape this weekend so stay > tuned.-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060216/d9a8db59/attachment.html