search for: crawler02

.... We want around 8. Is there a way to put a hard ceiling on the server side thread pool? Here is our setup: 5 app boxes, app01 - app05, with 8 mongrel instances each (this is where the 40 comes from, I think) Each app box points to a load balancer in front of two backgroundrb boxes, crawler01 and crawler02 this is the backgroundrb.yml on each app box :host: backgroundrb :port: 2000 :rails_env: production :pool_size: 1 We have to keep killing the bgrb. But we''re ok, because all the state of the workers is stored in a record in the db. That also allows us to use the load balancer in front o...