Hi,
I''m not much of a developer but I''ve been using backgroundrb
for a
while now and with the release of 0.2.1 it looks like I can finally
upgrade from the old version (0.2.0 had some weird issues when jobs
just wouldn''t run, seems to have cleared up now - Thanks!).
Anyhow, while I''m doing this, I would like to tackle this problem.
Most tasks I run in the background involve image processing so
they''re rather processor-intensive, and it''s not very good to
have
them running on the same machine that''s running rails - it slows
things down a lot. I would like to keep running rails on my main
server (server1) but move backgroundrb to a secondary server (server2).
I looked through the source trying to figure out how to do this and I
pieced together this configuration file:
--
:port: 22222
:timer_sleep: 60
:pool_size: 15
:load_rails: true
:rails_env: development
:environment: development
:host: localhost
:uri: druby://localhost:22222/
:database_yml: config/database.yml
:protocol: druby
:acl:
:deny: all
:allow: localhost 127.0.0.1 server1
:order: deny,allow
autostart:
1:
job_key: chat_notifier
class: chat_notification_worker
Which runs on server2. (I know, some of the options probably don''t
belong there but when it''s set up on a standalone server it works,
so...)
I''m using the druby protocol, which I''m pretty sure works over
TCP
instead of UDP, which seems to be what I want.
This all runs fine, except that it seems like my acl is completely
ignored or maybe it''s just not written properly. If I try to connect
just using telnet to test whether or not the port is open, I can
connect to the IPv6 localhost (::1) port 22222 from server2 just
fine. However, I can''t connect to IPv4 localhost (127.0.0.1) port
22222 from server2, and I definitely can''t connect to server2 port
22222 from server1.
I can run `script/backgroundrb console` OK from server2 as long
as :uri is set to "druby://localhost:22222/" and :host is set to
"localhost". If I set :host to "server2" then backgroundrb
runs, but
the console fails with:
/usr/local/lib/ruby/1.8/drb/drb.rb:736:in `open'': druby://localhost:
22222 - #<Errno::ECONNREFUSED: Connection refused - connect(2)>
(DRb::DRbConnError)
from /usr/local/lib/ruby/1.8/drb/drb.rb:729:in `each''
from /usr/local/lib/ruby/1.8/drb/drb.rb:729:in `open''
from /usr/local/lib/ruby/1.8/drb/drb.rb:1189:in `initialize''
from /usr/local/lib/ruby/1.8/drb/drb.rb:1169:in `new''
from /usr/local/lib/ruby/1.8/drb/drb.rb:1169:in `open''
from /usr/local/lib/ruby/1.8/drb/drb.rb:1085:in
`method_missing''
from /usr/local/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend''
from /usr/local/lib/ruby/1.8/drb/drb.rb:1084:in
`method_missing''
... 8 levels...
from /web/trunk/vendor/plugins/backgroundrb/server/lib/
backgroundrb/console.rb:9:in `backgroundrb_console_start''
from /web/trunk/vendor/plugins/backgroundrb/server/lib/
backgroundrb/console.rb:39:in `init''
from /web/trunk/vendor/plugins/backgroundrb/server/lib/
backgroundrb_server.rb:296:in `run''
from script/backgroundrb:29
I thought maybe since it was pointing to localhost, and Mac OS X
(this is running on OS X Server 10.4, both machines) has ::1 set up
as the first resolved address for localhost, maybe backgroundrb was
being confused into going only over IPv6, but using the IPv4 address
completely doesn''t work? I''ve tried various combinations of
localhost, server2, and server2''s ip address in both the :host
and :uri options with no success, unless both point to localhost, and
no matter what I do, I can only connect via IPv6 localhost, despite
the fact that my ACL allows connections from 127.0.0.1 and from
server1 as well. Setting the acl to :allow => "all" did not change
a
thing.
I know next to nothing about drb so if I''m missing something
blatantly obvious I apologize. Would really like to get this set up
soon, even a cheap hack that''ll work for the next week or so would be
a godsend. TIA for any help!
--
Steve
(Sorry if this ends up being a double post, sent from the wrong address
initially.)
Hi Steve,
Not sure if this will help, but a few things to note:
1) I''ve had to turn completely off my Mac OS X firewall to get DRb
processes
to communicate to each other (if I don''t I get the DRb::DRbConnError
exception that you also got). I might''ve screwed something up, but
that''s
at least worth a shot.
2) I''m pretty sure (?) that you don''t explicitly set the :uri
in the config
file. Instead, it looks like it''s built from :host and :port.
3) Also, not sure if you''re already doing this, but you probably want
separate config files for server1 and server2. The Middleman proxy object
setup *and* the backgroundrb server setup both use the same config file.
The following isn''t tested, but something like this might work (note
that
you should probably just replace ''server1'' and
''server2'' with ''localhost''
since they''re the same thing in your devel env)
# server 1
:host: server2
:port: 22222
:timer_sleep: 60
:pool_size: 15
:load_rails: true
:rails_env: development
:environment: development
:database_yml: config/database.yml
:protocol: druby
# server 2
:host: localhost
:port: 22222
:timer_sleep: 60
:pool_size: 15
:load_rails: true
:rails_env: development
:environment: development
:database_yml: config/database.yml
:protocol: druby
:acl:
:deny: all
:allow: localhost 127.0.0.1 server1
:order: deny,allow
autostart:
1:
job_key: chat_notifier
class: chat_notification_worker
Hope that helps,
Jason
On 12/4/06, Stephen Weiss <sweiss at stylesight.com>
wrote:>
> Hi,
>
> I''m not much of a developer but I''ve been using
backgroundrb for a
> while now and with the release of 0.2.1 it looks like I can finally
> upgrade from the old version (0.2.0 had some weird issues when jobs
> just wouldn''t run, seems to have cleared up now - Thanks!).
>
> Anyhow, while I''m doing this, I would like to tackle this problem.
> Most tasks I run in the background involve image processing so
> they''re rather processor-intensive, and it''s not very
good to have
> them running on the same machine that''s running rails - it slows
> things down a lot. I would like to keep running rails on my main
> server (server1) but move backgroundrb to a secondary server (server2).
>
> I looked through the source trying to figure out how to do this and I
> pieced together this configuration file:
>
> --
> :port: 22222
> :timer_sleep: 60
> :pool_size: 15
> :load_rails: true
> :rails_env: development
> :environment: development
> :host: localhost
> :uri: druby://localhost:22222/
> :database_yml: config/database.yml
> :protocol: druby
> :acl:
> :deny: all
> :allow: localhost 127.0.0.1 server1
> :order: deny,allow
> autostart:
> 1:
> job_key: chat_notifier
> class: chat_notification_worker
>
> Which runs on server2. (I know, some of the options probably
don''t
> belong there but when it''s set up on a standalone server it works,
> so...)
>
> I''m using the druby protocol, which I''m pretty sure works
over TCP
> instead of UDP, which seems to be what I want.
>
> This all runs fine, except that it seems like my acl is completely
> ignored or maybe it''s just not written properly. If I try to
connect
> just using telnet to test whether or not the port is open, I can
> connect to the IPv6 localhost (::1) port 22222 from server2 just
> fine. However, I can''t connect to IPv4 localhost (127.0.0.1) port
> 22222 from server2, and I definitely can''t connect to server2 port
> 22222 from server1.
>
> I can run `script/backgroundrb console` OK from server2 as long
> as :uri is set to "druby://localhost:22222/" and :host is set to
> "localhost". If I set :host to "server2" then
backgroundrb runs, but
> the console fails with:
>
> /usr/local/lib/ruby/1.8/drb/drb.rb:736:in `open'':
druby://localhost:
> 22222 - #<Errno::ECONNREFUSED: Connection refused - connect(2)>
> (DRb::DRbConnError)
> from /usr/local/lib/ruby/1.8/drb/drb.rb:729:in `each''
> from /usr/local/lib/ruby/1.8/drb/drb.rb:729:in `open''
> from /usr/local/lib/ruby/1.8/drb/drb.rb:1189:in
`initialize''
> from /usr/local/lib/ruby/1.8/drb/drb.rb:1169:in `new''
> from /usr/local/lib/ruby/1.8/drb/drb.rb:1169:in `open''
> from /usr/local/lib/ruby/1.8/drb/drb.rb:1085:in
> `method_missing''
> from /usr/local/lib/ruby/1.8/drb/drb.rb:1103:in
`with_friend''
> from /usr/local/lib/ruby/1.8/drb/drb.rb:1084:in
> `method_missing''
> ... 8 levels...
> from /web/trunk/vendor/plugins/backgroundrb/server/lib/
> backgroundrb/console.rb:9:in `backgroundrb_console_start''
> from /web/trunk/vendor/plugins/backgroundrb/server/lib/
> backgroundrb/console.rb:39:in `init''
> from /web/trunk/vendor/plugins/backgroundrb/server/lib/
> backgroundrb_server.rb:296:in `run''
> from script/backgroundrb:29
>
> I thought maybe since it was pointing to localhost, and Mac OS X
> (this is running on OS X Server 10.4, both machines) has ::1 set up
> as the first resolved address for localhost, maybe backgroundrb was
> being confused into going only over IPv6, but using the IPv4 address
> completely doesn''t work? I''ve tried various combinations
of
> localhost, server2, and server2''s ip address in both the :host
> and :uri options with no success, unless both point to localhost, and
> no matter what I do, I can only connect via IPv6 localhost, despite
> the fact that my ACL allows connections from 127.0.0.1 and from
> server1 as well. Setting the acl to :allow => "all" did not
change a
> thing.
>
> I know next to nothing about drb so if I''m missing something
> blatantly obvious I apologize. Would really like to get this set up
> soon, even a cheap hack that''ll work for the next week or so would
be
> a godsend. TIA for any help!
>
> --
> Steve
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20061204/0fcd1fdb/attachment.html
On Dec 4, 2006, at 7:45 PM, Jason Sydes wrote:> Hi Stephen, > Not sure if this will help, but a few things to note:Thanks for the reply!> > 1) I''ve had to turn completely off my Mac OS X firewall to get DRb > processes to communicate to each other (if I don''t I get the > DRb::DRbConnError exception that you also got). I might''ve screwed > something up, but that''s at least worth a shot. >My firewalls are totally off, I have a PIX for that. The pix doesn''t affect any traffic between the two machines (they''re directly linked for file sharing, connections between the two servers go over this direct link).> 2) I''m pretty sure (?) that you don''t explicitly set the :uri in > the config file. Instead, it looks like it''s built from :host > and :port.Yeah I wasn''t sure about this either, but it doesn''t seem to like it makes a difference. If I take out the line for uri, it works just the same.> > 3) Also, not sure if you''re already doing this, but you probably > want separate config files for server1 and server2. The Middleman > proxy object setup *and* the backgroundrb server setup both use the > same config file. The following isn''t tested, but something like > this might work (note that you should probably just replace > ''server1'' and ''server2'' with ''localhost'' since they''re the same > thing in your devel env)Yes, I understood this part. There is a separate configuration file for server1. However, I don''t think the config file for server1 has much to do with the problem at hand - server2 is not allowing connections from ipv4 localhost, let alone remote servers, so the remote server(1)''s configuration is a moot point. First I''ll get it to the point where I can connect from server1 in the first place - then I''ll worry about how server1 is configured. That being said, my configuration for server1, I think, would be something like: --- port: 22222 timer_sleep: 60 load_rails: true environment: development host: server2 uri: druby://server2:22222/ database_yml: config/database.yml pool_size: 10 (again, maybe not the uri line) And yes, the database.yml''s are both set up correctly. But honestly I hadn''t even worried about it, without connectivity it won''t help much. I think I might try a different port, or just building a straight-up druby connection and see if that works, to eliminate any other trouble. I still feel like it''s something with the ACL though.... Again thanks for any help! -- Steve
So, I had some success. Basically, I totally eliminated the acl by commenting out the line: self.setup_drb_acl in method ''setup'' in backgroundrb_server.rb. Then, the connection opens up (I have a firewall and this machine is totally blocked from the outside, I''m really not worried about leaving this open). Important to note here is that you can''t just remove the ACL from your configuration - you have to edit the source, since backgroundrb_server.rb will install its own ACL otherwise. However, staying with port 22222 also proved to be a problem - even though I was able to connect, no data seemed to go through on port 22222. Switching to port 22223 completely solved this problem. Now my program is very happy. I can start backgroundrb on server2, and start rails on server1. Then, when I do script/console and type MiddleMan.jobs on server1, I get a response. If I search for a worker that isn''t there, I get the expected error message, labelled from server2. So, 2 basic things I''m taking away from this - port 22222 has some issue, at least on my machines - and it seems that applying any ACL blocks remote access, regardless of what''s actually in the ACL. One more thing - if I did set the ACL to :allow => "127.0.0.1" instead of :allow => "localhost 127.0.0.1", connections through 127.0.0.1 worked, while as expected connections through ::1 failed. No matter what, you have to completely disable the acl to get a connection from a remote machine. Sorry if this is all documented a million times somewhere, I couldn''t find it. Thanks again for such useful software, it''s almost perfect now! One more question - is there an issue now with calling one worker''s method from another worker? I have a chat notification worker that works fine if it''s called from rails or from the backgroundrb console, but if it''s called from another worker nothing seems to happen (like, the method never gets called, there''s no error or anything). It''s not a deal-breaker, but it did work in ver 1... -- Steve
> port: 22222 > timer_sleep: 60 > load_rails: true > environment: development > host: server2 > uri: druby://server2:22222/ > database_yml: config/database.yml > pool_size: 10what if you do :port: 22222 :host: server2 :protocol: druby and just not specify :uri: and let backgroundrb control that and then: :acl: :order: deny allow :deny: all :allow: server1 server2 (you might want to try the IP of server1 /skaar -- ---------------------------------------------------------------------- |\|\ where in the | s_u_b_s_t_r_u_c_t_i_o_n | | >=========== W.A.S.T.E. | genarratologies |/|/ (_) is the wisdom | skaar at waste.org ----------------------------------------------------------------------