Hi, I faced such issue - my app should run some delayed method. For example, I have Test model, which User can create. But to other users it should be visible only if it has :saved value in status attribute, which is assigned with special button. I want to delete all tests, which remained unpublished 24 hours after creating. And really don''t know where to start. Found Rails episode ''Delayed Jobs'', but it is not what I''m looking for, it is mostly about background methods. Can you give me any keys to topic, from where I can begin digging by myself? -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-talk/-/TO1j6Ma67TYJ. For more options, visit https://groups.google.com/groups/opt_out.
On 23 March 2013 14:10, Barry <burmanz-JGs/UdohzUI@public.gmane.org> wrote:> Hi, I faced such issue - my app should run some delayed method. > For example, I have Test model, which User can create. But to other users it > should be visible only if it has :saved value in status attribute, which is > assigned with special button. > I want to delete all tests, which remained unpublished 24 hours after > creating. > And really don''t know where to start. Found Rails episode ''Delayed Jobs'', > but it is not what I''m looking for, it is mostly about background methods. > Can you give me any keys to topic, from where I can begin digging by myself?You could write a rake task to do the work and then run that from cron daily or hourly or whatever is appropriate. Colin -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
for regularly scheduled jobs, I use a mixture of cron (to create a delayed
job), and the delayed_job itself
the crontab instance is very light, just a small (non-rails) rb script to
insert the delayed_job in the delayed_jobs table
then the delayed_job instance will pickup the job and run it
in your instance, I would create a class method on the Test model -
something like
def self.remove_old_unpublished
delete_all(["created_at < ? and state
in(''unpublished'')", 24.hours.ago])
end
cron entry like this:
05 1 * * * cd /path/to/current &&
RAILS_ENV=production
/path/to/current/lib/delayed_job_cron_jobs/create_delayed_job.rb
--model "Test" --method "remove_old_unpublished" --queue
"general"
--arguments "{:any_argument => 42}"
the following gist is a script to insert delayed_jobs from cron
https://gist.github.com/jshow/5228049
fyi, the reason to take this route over the simpler rake route (run rake
task from cron) is performance and memory usage - this method will save you
a bunch of both.
Jodi
On Sat, Mar 23, 2013 at 10:10 AM, Barry
<burmanz-JGs/UdohzUI@public.gmane.org> wrote:
> Hi, I faced such issue - my app should run some delayed method.
> For example, I have Test model, which User can create. But to other users
> it should be visible only if it has :saved value in status attribute, which
> is assigned with special button.
> I want to delete all tests, which remained unpublished 24 hours after
> creating.
> And really don''t know where to start. Found Rails episode
''Delayed Jobs'',
> but it is not what I''m looking for, it is mostly about background
methods.
> Can you give me any keys to topic, from where I can begin digging by
> myself?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msg/rubyonrails-talk/-/TO1j6Ma67TYJ.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.
On 23 March 2013 15:15, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote:> for regularly scheduled jobs, I use a mixture of cron (to create a delayed > job), and the delayed_job itself > > the crontab instance is very light, just a small (non-rails) rb script to > insert the delayed_job in the delayed_jobs table > > then the delayed_job instance will pickup the job and run it > > in your instance, I would create a class method on the Test model - > something like > > def self.remove_old_unpublished > delete_all(["created_at < ? and state in(''unpublished'')", 24.hours.ago]) > end > > cron entry like this: > 05 1 * * * cd /path/to/current && RAILS_ENV=production > /path/to/current/lib/delayed_job_cron_jobs/create_delayed_job.rb --model > "Test" --method "remove_old_unpublished" --queue "general" --arguments > "{:any_argument => 42}" > > the following gist is a script to insert delayed_jobs from cron > https://gist.github.com/jshow/5228049 > > fyi, the reason to take this route over the simpler rake route (run rake > task from cron) is performance and memory usage - this method will save you > a bunch of both.I am always suspicious of the idea of doing something in a more complex way in order to save resources. It is only worth spending the additional time developing the solution if computing resources are likely to be an issue. Computing resources are usually cheaper than human resources. Colin -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
that is a good thinking, just like normalization - then comes a time to denormalize we have millions of visitors per month - and about 50 asynch processes - having one rails process deal with all those asynchs rather than one per is not helpful in any way using a best practice such as my approach is not harder to implement and will scale - choosing an approach of equal complexity that won''t scale doesn''t hold water On Sat, Mar 23, 2013 at 11:27 AM, Colin Law <clanlaw-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote:> On 23 March 2013 15:15, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: > > for regularly scheduled jobs, I use a mixture of cron (to create a > delayed > > job), and the delayed_job itself > > > > the crontab instance is very light, just a small (non-rails) rb script to > > insert the delayed_job in the delayed_jobs table > > > > then the delayed_job instance will pickup the job and run it > > > > in your instance, I would create a class method on the Test model - > > something like > > > > def self.remove_old_unpublished > > delete_all(["created_at < ? and state in(''unpublished'')", > 24.hours.ago]) > > end > > > > cron entry like this: > > 05 1 * * * cd /path/to/current && RAILS_ENV=production > > /path/to/current/lib/delayed_job_cron_jobs/create_delayed_job.rb --model > > "Test" --method "remove_old_unpublished" --queue "general" --arguments > > "{:any_argument => 42}" > > > > the following gist is a script to insert delayed_jobs from cron > > https://gist.github.com/jshow/5228049 > > > > fyi, the reason to take this route over the simpler rake route (run rake > > task from cron) is performance and memory usage - this method will save > you > > a bunch of both. > > I am always suspicious of the idea of doing something in a more > complex way in order to save resources. It is only worth spending the > additional time developing the solution if computing resources are > likely to be an issue. Computing resources are usually cheaper than > human resources. > > Colin > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit https://groups.google.com/groups/opt_out. > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
Please bottom post (appending). It makes responses easier to find. On Sat, Mar 23, 2013 at 10:32 AM, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote:> that is a good thinking, just like normalization - then comes a time to > denormalize > > we have millions of visitors per month - and about 50 asynch processes - > having one rails process deal with all those asynchs rather than one per is > not helpful in any way > > using a best practice such as my approach is not harder to implement and > will scale - choosing an approach of equal complexity that won''t scale > doesn''t hold water > > > On Sat, Mar 23, 2013 at 11:27 AM, Colin Law <clanlaw-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote: >> >> On 23 March 2013 15:15, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: >> > for regularly scheduled jobs, I use a mixture of cron (to create a >> > delayed >> > job), and the delayed_job itself >> > >> > the crontab instance is very light, just a small (non-rails) rb script >> > to >> > insert the delayed_job in the delayed_jobs table >> > >> > then the delayed_job instance will pickup the job and run it >> > >> > in your instance, I would create a class method on the Test model - >> > something like >> > >> > def self.remove_old_unpublished >> > delete_all(["created_at < ? and state in(''unpublished'')", >> > 24.hours.ago]) >> > end >> > >> > cron entry like this: >> > 05 1 * * * cd /path/to/current && RAILS_ENV=production >> > /path/to/current/lib/delayed_job_cron_jobs/create_delayed_job.rb --model >> > "Test" --method "remove_old_unpublished" --queue "general" --arguments >> > "{:any_argument => 42}" >> > >> > the following gist is a script to insert delayed_jobs from cron >> > https://gist.github.com/jshow/5228049 >> > >> > fyi, the reason to take this route over the simpler rake route (run rake >> > task from cron) is performance and memory usage - this method will save >> > you >> > a bunch of both. >> >> I am always suspicious of the idea of doing something in a more >> complex way in order to save resources. It is only worth spending the >> additional time developing the solution if computing resources are >> likely to be an issue. Computing resources are usually cheaper than >> human resources. >> >> Colin >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Ruby on Rails: Talk" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit https://groups.google.com/groups/opt_out. > >If you''re working in a distributed, multi-server and multi-process environment, cron is a poor solution. DelayedJobs and several others work in a distributed environment much better. I have been using the gem clockwork in addition to DJ, which makes things very simple. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
I''m just about to scale to a second app server - so good timing in which ways did you find cron to be a poor choice ? on a single server they meet our needs nicely you''d only run one clock instance per cluster - so much like cron (ie. no interserver clock scheduling). Have you tried using clock driven from a schedule described in the db (like a centralized cron, useful for failover) ? Any thoughts you have on this topic appreciated As for the OP, I hope they can see the short and long term options before them. J On Sat, Mar 23, 2013 at 2:01 PM, tamouse mailing lists < tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Please bottom post (appending). It makes responses easier to find. > > On Sat, Mar 23, 2013 at 10:32 AM, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: > > that is a good thinking, just like normalization - then comes a time to > > denormalize > > > > we have millions of visitors per month - and about 50 asynch processes - > > having one rails process deal with all those asynchs rather than one per > is > > not helpful in any way > > > > using a best practice such as my approach is not harder to implement and > > will scale - choosing an approach of equal complexity that won''t scale > > doesn''t hold water > > > > > > On Sat, Mar 23, 2013 at 11:27 AM, Colin Law <clanlaw-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> > wrote: > >> > >> On 23 March 2013 15:15, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: > >> > for regularly scheduled jobs, I use a mixture of cron (to create a > >> > delayed > >> > job), and the delayed_job itself > >> > > >> > the crontab instance is very light, just a small (non-rails) rb script > >> > to > >> > insert the delayed_job in the delayed_jobs table > >> > > >> > then the delayed_job instance will pickup the job and run it > >> > > >> > in your instance, I would create a class method on the Test model - > >> > something like > >> > > >> > def self.remove_old_unpublished > >> > delete_all(["created_at < ? and state in(''unpublished'')", > >> > 24.hours.ago]) > >> > end > >> > > >> > cron entry like this: > >> > 05 1 * * * cd /path/to/current && RAILS_ENV=production > >> > /path/to/current/lib/delayed_job_cron_jobs/create_delayed_job.rb > --model > >> > "Test" --method "remove_old_unpublished" --queue "general" --arguments > >> > "{:any_argument => 42}" > >> > > >> > the following gist is a script to insert delayed_jobs from cron > >> > https://gist.github.com/jshow/5228049 > >> > > >> > fyi, the reason to take this route over the simpler rake route (run > rake > >> > task from cron) is performance and memory usage - this method will > save > >> > you > >> > a bunch of both. > >> > >> I am always suspicious of the idea of doing something in a more > >> complex way in order to save resources. It is only worth spending the > >> additional time developing the solution if computing resources are > >> likely to be an issue. Computing resources are usually cheaper than > >> human resources. > >> > >> Colin > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Ruby on Rails: Talk" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > >> To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > >> For more options, visit https://groups.google.com/groups/opt_out. > >> > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Ruby on Rails: Talk" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > If you''re working in a distributed, multi-server and multi-process > environment, cron is a poor solution. DelayedJobs and several others > work in a distributed environment much better. I have been using the > gem clockwork in addition to DJ, which makes things very simple. > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit https://groups.google.com/groups/opt_out. > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
On Sat, Mar 23, 2013 at 1:20 PM, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote:> I''m just about to scale to a second app server - so good timing > > in which ways did you find cron to be a poor choice ? on a single server > they meet our needs nicely > > you''d only run one clock instance per cluster - so much like cron (ie. no > interserver clock scheduling). Have you tried using clock driven from a > schedule described in the db (like a centralized cron, useful for failover) > ? > > Any thoughts you have on this topic appreciated > > As for the OP, I hope they can see the short and long term options before > them. > > J > > > > > On Sat, Mar 23, 2013 at 2:01 PM, tamouse mailing lists > <tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> >> Please bottom post (appending). It makes responses easier to find. >> >> On Sat, Mar 23, 2013 at 10:32 AM, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: >> > that is a good thinking, just like normalization - then comes a time to >> > denormalize >> > >> > we have millions of visitors per month - and about 50 asynch processes - >> > having one rails process deal with all those asynchs rather than one per >> > is >> > not helpful in any way >> > >> > using a best practice such as my approach is not harder to implement and >> > will scale - choosing an approach of equal complexity that won''t scale >> > doesn''t hold water >> > >> > >> > On Sat, Mar 23, 2013 at 11:27 AM, Colin Law <clanlaw-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> >> > wrote: >> >> >> >> On 23 March 2013 15:15, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: >> >> > for regularly scheduled jobs, I use a mixture of cron (to create a >> >> > delayed >> >> > job), and the delayed_job itself >> >> > >> >> > the crontab instance is very light, just a small (non-rails) rb >> >> > script >> >> > to >> >> > insert the delayed_job in the delayed_jobs table >> >> > >> >> > then the delayed_job instance will pickup the job and run it >> >> > >> >> > in your instance, I would create a class method on the Test model - >> >> > something like >> >> > >> >> > def self.remove_old_unpublished >> >> > delete_all(["created_at < ? and state in(''unpublished'')", >> >> > 24.hours.ago]) >> >> > end >> >> > >> >> > cron entry like this: >> >> > 05 1 * * * cd /path/to/current && RAILS_ENV=production >> >> > /path/to/current/lib/delayed_job_cron_jobs/create_delayed_job.rb >> >> > --model >> >> > "Test" --method "remove_old_unpublished" --queue "general" >> >> > --arguments >> >> > "{:any_argument => 42}" >> >> > >> >> > the following gist is a script to insert delayed_jobs from cron >> >> > https://gist.github.com/jshow/5228049 >> >> > >> >> > fyi, the reason to take this route over the simpler rake route (run >> >> > rake >> >> > task from cron) is performance and memory usage - this method will >> >> > save >> >> > you >> >> > a bunch of both. >> >> >> >> I am always suspicious of the idea of doing something in a more >> >> complex way in order to save resources. It is only worth spending the >> >> additional time developing the solution if computing resources are >> >> likely to be an issue. Computing resources are usually cheaper than >> >> human resources. >> >> >> >> Colin >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "Ruby on Rails: Talk" group. >> >> To unsubscribe from this group and stop receiving emails from it, send >> >> an >> >> email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Ruby on Rails: Talk" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > >> >> If you''re working in a distributed, multi-server and multi-process >> environment, cron is a poor solution. DelayedJobs and several others >> work in a distributed environment much better. I have been using the >> gem clockwork in addition to DJ, which makes things very simple. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Ruby on Rails: Talk" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit https://groups.google.com/groups/opt_out. > >The primary issue when you bring in multiple servers is synchronization of the workers. Cron can''t do that, as it only knows about one server. Having something that workers can distribute over requires something more sophisticated. That''s what DJ is really about. Using clockwork makes it all the easier coming from cron. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
On 23 March 2013 21:37, tamouse mailing lists <tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> ... > The primary issue when you bring in multiple servers is > synchronization of the workers. Cron can''t do that, as it only knows > about one server. Having something that workers can distribute over > requires something more sophisticated. That''s what DJ is really about. > Using clockwork makes it all the easier coming from cron.Since all the OP is trying to do (I believe) is run a task occasionally to purge obsolete data from the database are those issues actually a problem? Colin -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
yes - I know - I use DJ presently my question was about what advantages you see with clockwork over cron - other than syntax I don''t see any advantages now if the schedule was stored in the db, and I could run multiple clockworks on each server there would some inherent scheduling failover (DJ already provides job/worker failover) On Sat, Mar 23, 2013 at 5:37 PM, tamouse mailing lists < tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On Sat, Mar 23, 2013 at 1:20 PM, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: > > I''m just about to scale to a second app server - so good timing > > > > in which ways did you find cron to be a poor choice ? on a single server > > they meet our needs nicely > > > > you''d only run one clock instance per cluster - so much like cron (ie. no > > interserver clock scheduling). Have you tried using clock driven from a > > schedule described in the db (like a centralized cron, useful for > failover) > > ? > > > > Any thoughts you have on this topic appreciated > > > > As for the OP, I hope they can see the short and long term options before > > them. > > > > J > > > > > > > > > > On Sat, Mar 23, 2013 at 2:01 PM, tamouse mailing lists > > <tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > >> > >> Please bottom post (appending). It makes responses easier to find. > >> > >> On Sat, Mar 23, 2013 at 10:32 AM, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> > wrote: > >> > that is a good thinking, just like normalization - then comes a time > to > >> > denormalize > >> > > >> > we have millions of visitors per month - and about 50 asynch > processes - > >> > having one rails process deal with all those asynchs rather than one > per > >> > is > >> > not helpful in any way > >> > > >> > using a best practice such as my approach is not harder to implement > and > >> > will scale - choosing an approach of equal complexity that won''t scale > >> > doesn''t hold water > >> > > >> > > >> > On Sat, Mar 23, 2013 at 11:27 AM, Colin Law <clanlaw-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> > >> > wrote: > >> >> > >> >> On 23 March 2013 15:15, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: > >> >> > for regularly scheduled jobs, I use a mixture of cron (to create a > >> >> > delayed > >> >> > job), and the delayed_job itself > >> >> > > >> >> > the crontab instance is very light, just a small (non-rails) rb > >> >> > script > >> >> > to > >> >> > insert the delayed_job in the delayed_jobs table > >> >> > > >> >> > then the delayed_job instance will pickup the job and run it > >> >> > > >> >> > in your instance, I would create a class method on the Test model - > >> >> > something like > >> >> > > >> >> > def self.remove_old_unpublished > >> >> > delete_all(["created_at < ? and state in(''unpublished'')", > >> >> > 24.hours.ago]) > >> >> > end > >> >> > > >> >> > cron entry like this: > >> >> > 05 1 * * * cd /path/to/current && RAILS_ENV=production > >> >> > /path/to/current/lib/delayed_job_cron_jobs/create_delayed_job.rb > >> >> > --model > >> >> > "Test" --method "remove_old_unpublished" --queue "general" > >> >> > --arguments > >> >> > "{:any_argument => 42}" > >> >> > > >> >> > the following gist is a script to insert delayed_jobs from cron > >> >> > https://gist.github.com/jshow/5228049 > >> >> > > >> >> > fyi, the reason to take this route over the simpler rake route (run > >> >> > rake > >> >> > task from cron) is performance and memory usage - this method will > >> >> > save > >> >> > you > >> >> > a bunch of both. > >> >> > >> >> I am always suspicious of the idea of doing something in a more > >> >> complex way in order to save resources. It is only worth spending > the > >> >> additional time developing the solution if computing resources are > >> >> likely to be an issue. Computing resources are usually cheaper than > >> >> human resources. > >> >> > >> >> Colin > >> >> > >> >> -- > >> >> You received this message because you are subscribed to the Google > >> >> Groups > >> >> "Ruby on Rails: Talk" group. > >> >> To unsubscribe from this group and stop receiving emails from it, > send > >> >> an > >> >> email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > >> >> To post to this group, send email to > rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > >> >> For more options, visit https://groups.google.com/groups/opt_out. > >> >> > >> >> > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Ruby on Rails: Talk" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> > an > >> > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > >> > To post to this group, send email to > rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > >> > For more options, visit https://groups.google.com/groups/opt_out. > >> > > >> > > >> > >> If you''re working in a distributed, multi-server and multi-process > >> environment, cron is a poor solution. DelayedJobs and several others > >> work in a distributed environment much better. I have been using the > >> gem clockwork in addition to DJ, which makes things very simple. > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Ruby on Rails: Talk" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > >> To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > >> For more options, visit https://groups.google.com/groups/opt_out. > >> > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Ruby on Rails: Talk" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > The primary issue when you bring in multiple servers is > synchronization of the workers. Cron can''t do that, as it only knows > about one server. Having something that workers can distribute over > requires something more sophisticated. That''s what DJ is really about. > Using clockwork makes it all the easier coming from cron. > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit https://groups.google.com/groups/opt_out. > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
On Sat, Mar 23, 2013 at 4:46 PM, Colin Law <clanlaw-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote:> On 23 March 2013 21:37, tamouse mailing lists <tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> ... >> The primary issue when you bring in multiple servers is >> synchronization of the workers. Cron can''t do that, as it only knows >> about one server. Having something that workers can distribute over >> requires something more sophisticated. That''s what DJ is really about. >> Using clockwork makes it all the easier coming from cron. > > Since all the OP is trying to do (I believe) is run a task > occasionally to purge obsolete data from the database are those issues > actually a problem?Since the person I was answering directly asked about scaling to multipler servers, I''d say yes. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
> On Sat, Mar 23, 2013 at 5:37 PM, tamouse mailing lists > <tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> >> On Sat, Mar 23, 2013 at 1:20 PM, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: >> > I''m just about to scale to a second app server - so good timing >> > >> > in which ways did you find cron to be a poor choice ? on a single server >> > they meet our needs nicely >> > >> > you''d only run one clock instance per cluster - so much like cron (ie. >> > no >> > interserver clock scheduling). Have you tried using clock driven from a >> > schedule described in the db (like a centralized cron, useful for >> > failover) >> > ? >> > >> > Any thoughts you have on this topic appreciated >> > >> > As for the OP, I hope they can see the short and long term options >> > before >> > them. >> > >> > J >> > >> > >> > >> > >> > On Sat, Mar 23, 2013 at 2:01 PM, tamouse mailing lists >> > <tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> >> >> >> Please bottom post (appending). It makes responses easier to find. >> >> >> >> On Sat, Mar 23, 2013 at 10:32 AM, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> >> >> wrote: >> >> > that is a good thinking, just like normalization - then comes a time >> >> > to >> >> > denormalize >> >> > >> >> > we have millions of visitors per month - and about 50 asynch >> >> > processes - >> >> > having one rails process deal with all those asynchs rather than one >> >> > per >> >> > is >> >> > not helpful in any way >> >> > >> >> > using a best practice such as my approach is not harder to implement >> >> > and >> >> > will scale - choosing an approach of equal complexity that won''t >> >> > scale >> >> > doesn''t hold water >> >> > >> >> > >> >> > On Sat, Mar 23, 2013 at 11:27 AM, Colin Law <clanlaw-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> >> >> > wrote: >> >> >> >> >> >> On 23 March 2013 15:15, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote: >> >> >> > for regularly scheduled jobs, I use a mixture of cron (to create a >> >> >> > delayed >> >> >> > job), and the delayed_job itself >> >> >> > >> >> >> > the crontab instance is very light, just a small (non-rails) rb >> >> >> > script >> >> >> > to >> >> >> > insert the delayed_job in the delayed_jobs table >> >> >> > >> >> >> > then the delayed_job instance will pickup the job and run it >> >> >> > >> >> >> > in your instance, I would create a class method on the Test model >> >> >> > - >> >> >> > something like >> >> >> > >> >> >> > def self.remove_old_unpublished >> >> >> > delete_all(["created_at < ? and state in(''unpublished'')", >> >> >> > 24.hours.ago]) >> >> >> > end >> >> >> > >> >> >> > cron entry like this: >> >> >> > 05 1 * * * cd /path/to/current && RAILS_ENV=production >> >> >> > /path/to/current/lib/delayed_job_cron_jobs/create_delayed_job.rb >> >> >> > --model >> >> >> > "Test" --method "remove_old_unpublished" --queue "general" >> >> >> > --arguments >> >> >> > "{:any_argument => 42}" >> >> >> > >> >> >> > the following gist is a script to insert delayed_jobs from cron >> >> >> > https://gist.github.com/jshow/5228049 >> >> >> > >> >> >> > fyi, the reason to take this route over the simpler rake route >> >> >> > (run >> >> >> > rake >> >> >> > task from cron) is performance and memory usage - this method will >> >> >> > save >> >> >> > you >> >> >> > a bunch of both. >> >> >> >> >> >> I am always suspicious of the idea of doing something in a more >> >> >> complex way in order to save resources. It is only worth spending >> >> >> the >> >> >> additional time developing the solution if computing resources are >> >> >> likely to be an issue. Computing resources are usually cheaper than >> >> >> human resources. >> >> >> >> >> >> Colin >> >> >> >> >> >> -- >> >> >> You received this message because you are subscribed to the Google >> >> >> Groups >> >> >> "Ruby on Rails: Talk" group. >> >> >> To unsubscribe from this group and stop receiving emails from it, >> >> >> send >> >> >> an >> >> >> email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> >> To post to this group, send email to >> >> >> rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> >> >> >> >> > >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups >> >> > "Ruby on Rails: Talk" group. >> >> > To unsubscribe from this group and stop receiving emails from it, >> >> > send >> >> > an >> >> > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> > To post to this group, send email to >> >> > rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> > For more options, visit https://groups.google.com/groups/opt_out. >> >> > >> >> > >> >> >> >> If you''re working in a distributed, multi-server and multi-process >> >> environment, cron is a poor solution. DelayedJobs and several others >> >> work in a distributed environment much better. I have been using the >> >> gem clockwork in addition to DJ, which makes things very simple. >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "Ruby on Rails: Talk" group. >> >> To unsubscribe from this group and stop receiving emails from it, send >> >> an >> >> email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Ruby on Rails: Talk" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > >> >> The primary issue when you bring in multiple servers is >> synchronization of the workers. Cron can''t do that, as it only knows >> about one server. Having something that workers can distribute over >> requires something more sophisticated. That''s what DJ is really about. >> Using clockwork makes it all the easier coming from cron. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Ruby on Rails: Talk" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Ruby on Rails: Talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit https://groups.google.com/groups/opt_out. > >This is a bottom post. Please follow suit. On Sat, Mar 23, 2013 at 4:47 PM, Jodi Showers <jodi-vRiTP4Lz4TuakBO8gow8eQ@public.gmane.org> wrote:> yes - I know - I use DJ presently > > my question was about what advantages you see with clockwork over cron - > other than syntax I don''t see any advantagesThe advantage is that you''d need to be running cron on each of your servers managing workers, whereas clockwork only needs to run on one to keep things synchronized.> now if the schedule was stored in the db, and I could run multiple > clockworks on each server there would some inherent scheduling failover (DJ > already provides job/worker failover)If you do decide to go that route, DJ can handle the requeuing itself, not needing anything else. It''s just a bit more configuration and such. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
On Sun, Mar 24, 2013 at 12:04 AM, tamouse mailing lists <tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> If you do decide to go that route, DJ can handle the requeuing itself, > not needing anything else. It''s just a bit more configuration and > such.As much as I''ve seen you ask people not to top post, I''m going to remind you to trim your emails, nobody wants to read 10 miles worth of quotes when you could simply trim it, like I did... -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.
On 24 March 2013 05:00, tamouse mailing lists <tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On Sat, Mar 23, 2013 at 4:46 PM, Colin Law <clanlaw-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote: >> On 23 March 2013 21:37, tamouse mailing lists <tamouse.lists-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >>> ... >>> The primary issue when you bring in multiple servers is >>> synchronization of the workers. Cron can''t do that, as it only knows >>> about one server. Having something that workers can distribute over >>> requires something more sophisticated. That''s what DJ is really about. >>> Using clockwork makes it all the easier coming from cron. >> >> Since all the OP is trying to do (I believe) is run a task >> occasionally to purge obsolete data from the database are those issues >> actually a problem? > > Since the person I was answering directly asked about scaling to > multipler servers, I''d say yes.I understand that it is relevant to Jodi''s problem, I was enquiring whether it is also relevant in the case described by the OP. Colin -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit https://groups.google.com/groups/opt_out.