We''re seeing that calling an AM deliver_* method in a controller is
introducing an unacceptable delay, ~1 second, as it connect via smtp and
sends the mail on it''s way.
So we''re wrapping all calls to deliver_* in
Thread.new { ... }
Any issues w/ this approach? Any other suggestions?
What would folks think about a patch to ActionMailer::Base that allowed
for a configuration option, something like
ActionMailer::Base.delivery_method = :async_smtp
that used the smtp delivery method, but wrapped in the Thread.new {} to
make it asynchronous?
On Aug 21, 2005, at 12:31 AM, Michael Schoen wrote:> We''re seeing that calling an AM deliver_* method in a controller is > introducing an unacceptable delay, ~1 second, as it connect via > smtp and sends the mail on it''s way. > > So we''re wrapping all calls to deliver_* in > > Thread.new { ... } > > Any issues w/ this approach? Any other suggestions? > > What would folks think about a patch to ActionMailer::Base that > allowed for a configuration option, something like > > ActionMailer::Base.delivery_method = :async_smtp > > that used the smtp delivery method, but wrapped in the Thread.new > {} to make it asynchronous?Given that the solution is as simple as you posed--simply wrapping the call in Thread.new--I don''t know that there needs to be a specific option in the API for this. I think I''d rather keep the API as lean as possible and just add something to the docs demonstrating how to do async calls. - Jamis
I am not sure I understood this the right way. I haven''t used AM yet. But Jamis, does not changing the API to allow for this mean that I have to modify my Rails installation to do this? Or will it only affect the way I am using AM in my code? Because - if it means that I have to change the Rails installation I really don''t like it. Because that means nightmare in terms of updating Rails later. If it means changing *my code only* then I don''t have a problem with just adding it in the docs. FWIW: First time I experienced the AM stuff (included by using the salted_login_generator), I found it weird that I had to wait so "long" before feedback was given. I understood that it was actually sending the mail though. The upside is that we will know that the mail was delivered or not before giving feedback to the user. However, I am not certain that the long waiting time is expected by the general user. It might *become* the de facto standard, but I don''t think it is yet... A problem with changing the *default* to sending in another thread is that existing apps are already using the old way. Many feedback screens may have text telling that the mail *was* sent, period. Well, just my 0.02$... ------------------------------------------------------------------------ /*Ronny Hanssen*/ Jamis Buck wrote:> On Aug 21, 2005, at 12:31 AM, Michael Schoen wrote: > >> We''re seeing that calling an AM deliver_* method in a controller is >> introducing an unacceptable delay, ~1 second, as it connect via smtp >> and sends the mail on it''s way. >> >> So we''re wrapping all calls to deliver_* in >> >> Thread.new { ... } >> >> Any issues w/ this approach? Any other suggestions? >> >> What would folks think about a patch to ActionMailer::Base that >> allowed for a configuration option, something like >> >> ActionMailer::Base.delivery_method = :async_smtp >> >> that used the smtp delivery method, but wrapped in the Thread.new {} >> to make it asynchronous? > > > Given that the solution is as simple as you posed--simply wrapping the > call in Thread.new--I don''t know that there needs to be a specific > option in the API for this. I think I''d rather keep the API as lean as > possible and just add something to the docs demonstrating how to do > async calls. > > - Jamis > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails
On Aug 21, 2005, at 4:12 PM, Ronny Hanssen wrote:> I am not sure I understood this the right way. I haven''t used AM yet. > But Jamis, does not changing the API to allow for this mean that I > have > to modify my Rails installation to do this? Or will it only affect the > way I am using AM in my code? Because - if it means that I have to > change the Rails installation I really don''t like it. Because that > means > nightmare in terms of updating Rails later. If it means changing *my > code only* then I don''t have a problem with just adding it in the > docs.Ronny, by "changing the API", I meant "adding an option" (as described by the OP). Thus, the existing AM interface would not be modified in any non-backwards-compatible way, it would only add the option to specify that sending mail could be done asynchronously. However, I don''t like the idea of adding that option. Instead, just doing "Thread.new { ... }" around your deliver calls is sufficient, and I proposed possibly adding documentation to that effect. - Jamis> A problem with changing the *default* to sending in another thread is > that existing apps are already using the old way. Many feedback > screens > may have text telling that the mail *was* sent, period.I would never suggest we change the default. Especially this late in the game. We''re really bending over backwards at this point to preserve backwards compatibility. - Jamis
I can only see this long wait as using a remote smtp server. i cant imagine AM taking that long if its connecting to an smtp server on the same computer or locally on a high speed network. Jamis, is it possible that AM objects can sustain a connection to the SMTP server, or does this break SMTP Protocol specification (i dont think so.) i wouldnt see a problem with this especially as most (all!) applications would be using a local smtp server. i cant really see a problem with this as we already persist the database connection, holding smtp could help too ;) On 8/21/05, Jamis Buck <jamis-uHoyYlH2B+GakBO8gow8eQ@public.gmane.org> wrote:> On Aug 21, 2005, at 4:12 PM, Ronny Hanssen wrote: > > > I am not sure I understood this the right way. I haven''t used AM yet. > > But Jamis, does not changing the API to allow for this mean that I > > have > > to modify my Rails installation to do this? Or will it only affect the > > way I am using AM in my code? Because - if it means that I have to > > change the Rails installation I really don''t like it. Because that > > means > > nightmare in terms of updating Rails later. If it means changing *my > > code only* then I don''t have a problem with just adding it in the > > docs. > > Ronny, by "changing the API", I meant "adding an option" (as > described by the OP). Thus, the existing AM interface would not be > modified in any non-backwards-compatible way, it would only add the > option to specify that sending mail could be done asynchronously. > > However, I don''t like the idea of adding that option. Instead, just > doing "Thread.new { ... }" around your deliver calls is sufficient, > and I proposed possibly adding documentation to that effect. > > - Jamis > > > A problem with changing the *default* to sending in another thread is > > that existing apps are already using the old way. Many feedback > > screens > > may have text telling that the mail *was* sent, period. > > I would never suggest we change the default. Especially this late in > the game. We''re really bending over backwards at this point to > preserve backwards compatibility. > > - Jamis > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- Zachery Hostens <zacheryph-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
I''m on thin ice, but: Isn''t the delay caused by the fact that it waits for the email to be sent? Which means that persisting the connection won''t help at all. The actual transmission of the email to the remote postoffice is going to take an "undefined" amount of time anyway. The smtp-server on a local machine only means that the net-connection is faster - to the smtp-server. The smtp-server still have to send the mail. Also - the smtp-server may also have a high load, the web-server itself may have a high load. And the remote postoffice may be slow in answering queries. Right? ------------------------------------------------------------------------ /*Ronny Hanssen*/ Zachery Hostens wrote:> I can only see this long wait as using a remote smtp server. i cant > imagine AM taking that long if its connecting to an smtp server on the > same computer or locally on a high speed network. > > Jamis, > > is it possible that AM objects can sustain a connection to the SMTP > server, or does this break SMTP Protocol specification (i dont think > so.) > > i wouldnt see a problem with this especially as most (all!) > applications would be using a local smtp server. i cant really see a > problem with this as we already persist the database connection, > holding smtp could help too ;) > > On 8/21/05, Jamis Buck <jamis-uHoyYlH2B+GakBO8gow8eQ@public.gmane.org> wrote: > >>On Aug 21, 2005, at 4:12 PM, Ronny Hanssen wrote: >> >> >>>I am not sure I understood this the right way. I haven''t used AM yet. >>>But Jamis, does not changing the API to allow for this mean that I >>>have >>>to modify my Rails installation to do this? Or will it only affect the >>>way I am using AM in my code? Because - if it means that I have to >>>change the Rails installation I really don''t like it. Because that >>>means >>>nightmare in terms of updating Rails later. If it means changing *my >>>code only* then I don''t have a problem with just adding it in the >>>docs. >> >>Ronny, by "changing the API", I meant "adding an option" (as >>described by the OP). Thus, the existing AM interface would not be >>modified in any non-backwards-compatible way, it would only add the >>option to specify that sending mail could be done asynchronously. >> >>However, I don''t like the idea of adding that option. Instead, just >>doing "Thread.new { ... }" around your deliver calls is sufficient, >>and I proposed possibly adding documentation to that effect. >> >>- Jamis >> >> >>>A problem with changing the *default* to sending in another thread is >>>that existing apps are already using the old way. Many feedback >>>screens >>>may have text telling that the mail *was* sent, period. >> >>I would never suggest we change the default. Especially this late in >>the game. We''re really bending over backwards at this point to >>preserve backwards compatibility. >> >>- Jamis >> >>_______________________________________________ >>Rails mailing list >>Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >>http://lists.rubyonrails.org/mailman/listinfo/rails >> > > >
this is generally not how email servers work. qmail i know for a fact does not work this way, smtpd accepts mail and closes. and later another application picks it up from the queue and sends it. postfix im pretty sure accepts mail for delivery, and then closes the connection. there should be no waiting on the part of the sender. the smtp is there for a reason, to send and recieve mail, not make the client sit there and wait to find out if it was sent. if mail worked this way you would not get mail-daemon bounced emails in reply, your client would simply say "cant send email". On 8/23/05, Ronny Hanssen <ronnyh-7tg5dEkr+6sdnm+yROfE0A@public.gmane.org> wrote:> I''m on thin ice, but: > > Isn''t the delay caused by the fact that it waits for the email to be > sent? Which means that persisting the connection won''t help at all. The > actual transmission of the email to the remote postoffice is going to > take an "undefined" amount of time anyway. The smtp-server on a local > machine only means that the net-connection is faster - to the > smtp-server. The smtp-server still have to send the mail. Also - the > smtp-server may also have a high load, the web-server itself may have a > high load. And the remote postoffice may be slow in answering queries. > > Right? > ------------------------------------------------------------------------ > /*Ronny Hanssen*/ > > > Zachery Hostens wrote: > > I can only see this long wait as using a remote smtp server. i cant > > imagine AM taking that long if its connecting to an smtp server on the > > same computer or locally on a high speed network. > > > > Jamis, > > > > is it possible that AM objects can sustain a connection to the SMTP > > server, or does this break SMTP Protocol specification (i dont think > > so.) > > > > i wouldnt see a problem with this especially as most (all!) > > applications would be using a local smtp server. i cant really see a > > problem with this as we already persist the database connection, > > holding smtp could help too ;) > > > > On 8/21/05, Jamis Buck <jamis-uHoyYlH2B+GakBO8gow8eQ@public.gmane.org> wrote: > > > >>On Aug 21, 2005, at 4:12 PM, Ronny Hanssen wrote: > >> > >> > >>>I am not sure I understood this the right way. I haven''t used AM yet. > >>>But Jamis, does not changing the API to allow for this mean that I > >>>have > >>>to modify my Rails installation to do this? Or will it only affect the > >>>way I am using AM in my code? Because - if it means that I have to > >>>change the Rails installation I really don''t like it. Because that > >>>means > >>>nightmare in terms of updating Rails later. If it means changing *my > >>>code only* then I don''t have a problem with just adding it in the > >>>docs. > >> > >>Ronny, by "changing the API", I meant "adding an option" (as > >>described by the OP). Thus, the existing AM interface would not be > >>modified in any non-backwards-compatible way, it would only add the > >>option to specify that sending mail could be done asynchronously. > >> > >>However, I don''t like the idea of adding that option. Instead, just > >>doing "Thread.new { ... }" around your deliver calls is sufficient, > >>and I proposed possibly adding documentation to that effect. > >> > >>- Jamis > >> > >> > >>>A problem with changing the *default* to sending in another thread is > >>>that existing apps are already using the old way. Many feedback > >>>screens > >>>may have text telling that the mail *was* sent, period. > >> > >>I would never suggest we change the default. Especially this late in > >>the game. We''re really bending over backwards at this point to > >>preserve backwards compatibility. > >> > >>- Jamis > >> > >>_______________________________________________ > >>Rails mailing list > >>Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > >>http://lists.rubyonrails.org/mailman/listinfo/rails > >> > > > > > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- Zachery Hostens <zacheryph-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Hm... True, true... So it means that it is only waiting to see the mail delivered to the smtp-server. When I was testing on my machine I could see the virus-scanner scan the message and then I got a response back. So I just automatically based my conclusion on that experience. But, of course you''re right about the way the mail-server works. Which means that if the default behavior changed it would not have any effect at all on rails users. I for one would love to see this delay gone... ------------------------------------------------------------------------ /*Ronny Hanssen*/ Zachery Hostens wrote:> this is generally not how email servers work. qmail i know for a fact > does not work this way, smtpd accepts mail and closes. and later > another application picks it up from the queue and sends it. > > postfix im pretty sure accepts mail for delivery, and then closes the > connection. there should be no waiting on the part of the sender. > the smtp is there for a reason, to send and recieve mail, not make the > client sit there and wait to find out if it was sent. if mail worked > this way you would not get mail-daemon bounced emails in reply, your > client would simply say "cant send email". > > > On 8/23/05, Ronny Hanssen <ronnyh-7tg5dEkr+6sdnm+yROfE0A@public.gmane.org> wrote: > >>I''m on thin ice, but: >> >>Isn''t the delay caused by the fact that it waits for the email to be >>sent? Which means that persisting the connection won''t help at all. The >>actual transmission of the email to the remote postoffice is going to >>take an "undefined" amount of time anyway. The smtp-server on a local >>machine only means that the net-connection is faster - to the >>smtp-server. The smtp-server still have to send the mail. Also - the >>smtp-server may also have a high load, the web-server itself may have a >>high load. And the remote postoffice may be slow in answering queries. >> >>Right? >>------------------------------------------------------------------------ >>/*Ronny Hanssen*/ >> >> >>Zachery Hostens wrote: >> >>>I can only see this long wait as using a remote smtp server. i cant >>>imagine AM taking that long if its connecting to an smtp server on the >>>same computer or locally on a high speed network. >>> >>>Jamis, >>> >>>is it possible that AM objects can sustain a connection to the SMTP >>>server, or does this break SMTP Protocol specification (i dont think >>>so.) >>> >>>i wouldnt see a problem with this especially as most (all!) >>>applications would be using a local smtp server. i cant really see a >>>problem with this as we already persist the database connection, >>>holding smtp could help too ;) >>> >>>On 8/21/05, Jamis Buck <jamis-uHoyYlH2B+GakBO8gow8eQ@public.gmane.org> wrote: >>> >>> >>>>On Aug 21, 2005, at 4:12 PM, Ronny Hanssen wrote: >>>> >>>> >>>> >>>>>I am not sure I understood this the right way. I haven''t used AM yet. >>>>>But Jamis, does not changing the API to allow for this mean that I >>>>>have >>>>>to modify my Rails installation to do this? Or will it only affect the >>>>>way I am using AM in my code? Because - if it means that I have to >>>>>change the Rails installation I really don''t like it. Because that >>>>>means >>>>>nightmare in terms of updating Rails later. If it means changing *my >>>>>code only* then I don''t have a problem with just adding it in the >>>>>docs. >>>> >>>>Ronny, by "changing the API", I meant "adding an option" (as >>>>described by the OP). Thus, the existing AM interface would not be >>>>modified in any non-backwards-compatible way, it would only add the >>>>option to specify that sending mail could be done asynchronously. >>>> >>>>However, I don''t like the idea of adding that option. Instead, just >>>>doing "Thread.new { ... }" around your deliver calls is sufficient, >>>>and I proposed possibly adding documentation to that effect. >>>> >>>>- Jamis >>>> >>>> >>>> >>>>>A problem with changing the *default* to sending in another thread is >>>>>that existing apps are already using the old way. Many feedback >>>>>screens >>>>>may have text telling that the mail *was* sent, period. >>>> >>>>I would never suggest we change the default. Especially this late in >>>>the game. We''re really bending over backwards at this point to >>>>preserve backwards compatibility. >>>> >>>>- Jamis >>>> >>>>_______________________________________________ >>>>Rails mailing list >>>>Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >>>>http://lists.rubyonrails.org/mailman/listinfo/rails >>>> >>> >>> >>> >>_______________________________________________ >>Rails mailing list >>Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >>http://lists.rubyonrails.org/mailman/listinfo/rails >> > > >
On 8/26/05, Ronny Hanssen <ronnyh-7tg5dEkr+6sdnm+yROfE0A@public.gmane.org> wrote:> Hm... True, true... So it means that it is only waiting to see the mail > delivered to the smtp-server. When I was testing on my machine I could > see the virus-scanner scan the message and then I got a response back. > So I just automatically based my conclusion on that experience. But, of > course you''re right about the way the mail-server works. Which means > that if the default behavior changed it would not have any effect at all > on rails users. I for one would love to see this delay gone...More and more mailers do stuff in the smpt phase that they really shouldn''t. Why? Because they would rather just deny the message at the smtp level then bounce it later. Technically this isn''t correct behavior, but it''s understandable. Not much you can do unless you run your own mail server and know how to configure it. Personally I think AM should have an option to use a local delivery agent. That way you can just inject it right into the queue. Chris
You might want to take a look at this wiki page: http://wiki.rubyonrails.com/rails/show/HowToRunBackgroundJobsInRails It mentions an incompatibility with Apache and Thread.new, along with some other things about running background tasks. On 27/08/05, snacktime <snacktime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> On 8/26/05, Ronny Hanssen <ronnyh-7tg5dEkr+6sdnm+yROfE0A@public.gmane.org> wrote: > > Hm... True, true... So it means that it is only waiting to see the mail > > delivered to the smtp-server. When I was testing on my machine I could > > see the virus-scanner scan the message and then I got a response back. > > So I just automatically based my conclusion on that experience. But, of > > course you''re right about the way the mail-server works. Which means > > that if the default behavior changed it would not have any effect at all > > on rails users. I for one would love to see this delay gone... > > More and more mailers do stuff in the smpt phase that they really > shouldn''t. Why? Because they would rather just deny the message at > the smtp level then bounce it later. Technically this isn''t correct > behavior, but it''s understandable. Not much you can do unless you > run your own mail server and know how to configure it. > > Personally I think AM should have an option to use a local delivery > agent. That way you can just inject it right into the queue. > > Chris > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >