I''m working on a project where the non-techy site administrator wants the ability to send template-based emails to all his registered users (on the order of thousands), meaning that each of the emails generated is unique and needs its own SMTP envelope. When the admin has composed his email template and selected the members who are to receive the email, he wants to hit the "send now" button. I want to handle it so that the emails will then be sent in a background process, while the admin can immediately continue browsing the rails site. The admin will receive an "email run complete" email from the background process once run is complere. Has anybody tried something similar? What insights hve you gained? My current idea is to fork a background process that spawns some 20 sender threads which connect to an SMTP server on the same subnet as the rails host. I am mostly interested in how I can make the email sending process efficient on the Ruby/Rails side of things, as a socket connection to the mail server is probably not possible. I use WEBrick for development and will be deploying on Apache2/FastCGI for production. cheers Gerret
On Oct 20, 2005, at 1:46 AM, Gerret Apelt wrote:> I''m working on a project where the non-techy site administrator wants > the ability to send template-based emails to all his registered users > (on the order of thousands), meaning that each of the emails generated > is unique and needs its own SMTP envelope. > > When the admin has composed his email template and selected the > members who are to receive the email, he wants to hit the "send now" > button. I want to handle it so that the emails will then be sent in a > background process, while the admin can immediately continue browsing > the rails site. The admin will receive an "email run complete" email > from the background process once run is complere. > > Has anybody tried something similar? What insights hve you gained? My > current idea is to fork a background > process that spawns some 20 sender threads which connect to an SMTP > server on the same subnet as the rails host. I am mostly interested in > how I can make the email sending process efficient on the Ruby/Rails > side of things, as a socket connection to the mail server is probably > not possible. >Garret- If I were going to do something like this I would avoid actionmailer as it seems to not like to send that many emails very fast. I would probably fork a ruby script that uses Tmail or Rhizmail to build the mime messages and the use threads in that process to send the mails with net/smtp. You could also look at drb to run a ruby dbr server that would just wait for a request from your rails app and then have that take care of the mail build and send process. It would probably work even better if you had a mail server on the same machine and you could push the emails into the mailer daemon''s queue. The queue of the smtp daemon is really built to queue up many emails like that so the more you can push down into the mail daemon the better off you will be. Cheers- -Ezra Zygmuntowicz WebMaster Yakima Herald-Republic Newspaper ezra-gdxLOakOTQ9oetBuM9ipNAC/G2K4zDHf@public.gmane.org 509-577-7732 _______________________________________________ Rails mailing list Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org http://lists.rubyonrails.org/mailman/listinfo/rails
On Oct 20, 2005, at 8:16 AM, Ezra Zygmuntowicz wrote:> On Oct 20, 2005, at 1:46 AM, Gerret Apelt wrote: > >> I''m working on a project where the non-techy site administrator wants >> the ability to send template-based emails to all his registered users >> (on the order of thousands), meaning that each of the emails >> generated >> is unique and needs its own SMTP envelope. > > If I were going to do something like this I would avoid > actionmailer as it seems to not like to send that many emails very > fast. I would probably fork a ruby script that uses Tmail or > Rhizmail to build the mime messages and the use threads in that > process to send the mails with net/smtp. You could also look at drb > to run a ruby dbr server that would just wait for a request from > your rails app and then have that take care of the mail build and > send process. It would probably work even better if you had a mail > server on the same machine and you could push the emails into the > mailer daemon''s queue. The queue of the smtp daemon is really built > to queue up many emails like that so the more you can push down > into the mail daemon the better off you will be.ActionMailer is a wrapper around Tmail, which is included in the source. Why would you expect it to be slow? I''ve done some high volume mailing systems before (not spam, but event notification!) and the solution to sending a lot of emails is: 1) Avoid serial latencies, which will kill you. Do NOT connect/ disconnect from the SMTP server for each message. 2) Avoid serial latencies, which will kill you. :-) Devise a scheme to have parallel delivery. 3) Avoid serial latencies, which will kill you. :-) :-) Do not deliver directly to recipients mail servers, which makes 1 & 2 impossible. 4) Avoid serial latencies, which will kill you. :-) :-) :-) Do implement a local mail server (separate machine is ideal if you have very high traffic) and use the fastest mail server available (which may not be sendmail). It should be noted that if you configure your mail server in a way that it is only accessible internally, then you can pretty much shut off all spam measures (RBL, etc) which introduce nasty serial latencies. This means that you should likely NOT use an existing mail server accessible to the outside world. -- -- Tom Mornini _______________________________________________ Rails mailing list Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org http://lists.rubyonrails.org/mailman/listinfo/rails
Hi, I''m new to Ruby on Rails and from the PHP world. I am wondering if there is an implementaion for Rails of the Phrasebook pattern for complex db queries that must be written in SQL. I''m thinking about portablility and isolating SQL to a single XML file. http://www.perl.com/pub/a/2002/10/22/phrasebook.html Maybe this is not "the rails way" of thinking? Thanks, Peter __________________________________ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
> Maybe this is not "the rails way" of thinking?Nope. The ''rails'' way of thinking is to use domain objects. Lets say you have a user model. You use the same query to list minors in 72 controller actions. Great scott! User.find :all, :conditions => ''age < 21'' Instead do this: class User < ActiveRecord::Base def self.find_minors(options = {}) find(:all, options.merge(:conditions => ''age < 21'')) end end User.find_minors User.find_minors :limit => 15, :offset => 30 Now when the US lowers the drinking age to 18, just change your model to reflect that. Here''s something more advanced... class User < ActiveRecord::Base def self.only_minors(&block) constrain(:conditions => ''age < 21'', &block) end end # find minors that start w/ ''b'' User.only_minors do User.find :all, :conditions => [''name LIKE ?'', ''b%''] end http://rails.rubyonrails.com/classes/ActiveRecord/Base.html#M000718 Careful with #constrain, multiple constrains tend to clobber each other. -- rick http://techno-weenie.net
peter michaux wrote:> [snip]If you want to start a NEW THREAD, please compose a NEW MESSAGE to rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org rather than replying to another thread and changing the subject. Due to the "References:" header, your message is still threaded under the original subject. Thanks, Steve
> I''m working on a project where the non-techy site administrator wants > the ability to send template-based emails to all his registered users > (on the order of thousands), meaning that each of the emails generated > is unique and needs its own SMTP envelope.We do this all the time, for the last 6 years. Over 100,000 users on our mailing list. We just make form letters like: Hi {firstname} - Your account, {username}, is coming up for renewal Then just do a quick loop through your database, and send the merged mails. If you have something like qmail (http://www.lifewithqmail.org/) for your outgoing mail, you can send 100,000 emails in no time at all, because it turns it into a background process itself. (Just make sure your /var/qmail partition has plenty of inodes or that''s what''ll kill it!) You could do it immediately from a web-click, or just have the 100,000 emails put into a database queue to be be sent by a cron process or something. No need to get into ActionMailer for that. Just use the most base-level Ruby/PHP/etc mail command.