I need to send about 10Mbyte files to users. Am using Apache + mongrel_cluster. The straight forward way would be using acts_as_attachment with file system storage and just send it away. Would there be any big performance problem with doing it straight forward? Should I do something more adventurous via an Apache mod? Thanks, Helzer --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
atpunkt-TM7Dk29TcfyELgA04lAiVw@public.gmane.org
2007-Sep-04 22:51 UTC
Re: Sending files from RoR application
You could forward them to a different URL, esp. one that''s not proxied through to the mongrel. If you need authentication you should think about using a combined php/ ror setup, where you have just one php-script, that checks for some cookie-based authentication and then delivers the file by readfile(). I may be wrong here, but afaik you will be blocking one mongrel- process for the duration of the download if you don''t hand it off to a different delivery-method. so it depends on how many mongrel-processes you can spare for analog- modem-users :-) hope this helps, phil On 5 Sep., 00:12, helzer <amir.hel...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> I need to send about 10Mbyte files to users. Am using Apache + > mongrel_cluster. > The straight forward way would be using acts_as_attachment with file > system storage and just send it away. > > Would there be any big performance problem with doing it straight > forward? > > Should I do something more adventurous via an Apache mod? > > Thanks, > Helzer--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On 9/4/07, atpunkt-TM7Dk29TcfyELgA04lAiVw@public.gmane.org <atpunkt-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote:> > You could forward them to a different URL, esp. one that''s not proxied > through to the mongrel. > If you need authentication you should think about using a combined php/ > ror setup, where you have just one php-script, that checks for some > cookie-based authentication and then delivers the file by readfile(). > I may be wrong here, but afaik you will be blocking one mongrel- > process for the duration of the download if you don''t hand it off to a > different delivery-method. > so it depends on how many mongrel-processes you can spare for analog- > modem-users :-)If properly set up, static files should be served directly through Apache and not touched by Mongrel at all, which means that a file being downloaded will not cause the mongrel process to block. In fact, you should be able to shut down your mongrel server and still have the files available for download (Apache will serve them directly) The only complication is if you require file downloads to be authenticated, then they must either be sent through mongrel/rails using send_file, or you can use the X-Sendfile header with Lighttpd to send the file (http://blog.lighttpd.net/articles/2006/07/02/x-sendfile) Adam --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Hi Phil,> I may be wrong here, but afaik you will be blocking one mongrel- > process for the duration of the download if you don''t hand it off to a > different delivery-method.That can''t be true. Mongrel received uploaded files when Apache finishes collecting them (all parts of a multi-part upload) and delivers files to Apache as a whole, for Apache to slowly send to the user. That''s the way all content gets delivered (not just sent files). I''m just thinking about the resources required by Mongrel to read the large files from disk, buffer them (fully) in memory and deliver to Apache, whereas Apache will probably read them chunk by chunk from the drive as users download. Amir --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Hi Adam, I''m using Apache and want to send the files only to registered users (meaning, it needs authentication, and can''t just sit in public/). Do you think that the resources required by Mongrel to read these files and send to Apache might become a big deal? Thanks, Amir --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
atpunkt-TM7Dk29TcfyELgA04lAiVw@public.gmane.org
2007-Sep-04 23:33 UTC
Re: Sending files from RoR application
Ah .. I see your problem but: - do you really expect to have 50 users downloading these files simultaneously ? (rough calculation: 50 * (10 M (file) + 12M (Apache thread-overhead)) ~ 1.1 GB Ram ( I''d say you need at least 2G anyway to be serving 50 people at the same time - even without file downloads) - do you really think using that much memory would slow down the server less than having to read from 50 files simultaneously ? - if you still want to ensure native delivery by apache use one of the many mod_rewrite-setups that will serve the content statically form the same virthost - will they be the same file or a different one for every user ? (because of apche/linux-filecaching) - will there be authentication needed ? if yes, i doubt you will get by without some mongrel/rails-call I know I''m not helping much, but you really should benchmark your different solutions by using jmeter or similar. Phil On 5 Sep., 01:17, helzer <amir.hel...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Hi Phil, > > > I may be wrong here, but afaik you will be blocking one mongrel- > > process for the duration of the download if you don''t hand it off to a > > different delivery-method. > > That can''t be true. Mongrel received uploaded files when Apache > finishes collecting them (all parts of a multi-part upload) and > delivers files to Apache as a whole, for Apache to slowly send to the > user. That''s the way all content gets delivered (not just sent files). > > I''m just thinking about the resources required by Mongrel to read the > large files from disk, buffer them (fully) in memory and deliver to > Apache, whereas Apache will probably read them chunk by chunk from the > drive as users download. > > Amir--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On 9/4/07, helzer <amir.helzer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > Hi Adam, > > I''m using Apache and want to send the files only to registered users > (meaning, it needs authentication, and can''t just sit in public/). > Do you think that the resources required by Mongrel to read these > files and send to Apache might become a big deal?If you need authentication then you''ll be using rails and send_file which will tie up a process until the download completes. Like I mentioned before, you can use the X-Sendfile header as an alternative, which will send the file using your web server rather than rails. There''s an article and a rails plugin available from this site: http://john.guen.in/past/2007/4/17/send_files_faster_with_xsendfile/ I''ve never actually used the X-Sendfile header method, since it''s been a while since I''ve needed to serve protected data, so I''m by no means an expert on this, just passing along some information that I''ve come across which you might find useful. Adam --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Thanks Adam, This looks like exactly what I need. I''ll run it tomorrow and let you know how it works for me. Amir --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
A little update... x_send_file is just fine. The install instructions are a little broken, but eventually it works. Get it from: http://tn123.ath.cx/mod_xsendfile/ After create the extension with: apxs2 -cia mod_xsendfile.c the extension gets copied to some default apache modules directory, and not necessarily to where you need it. In my case it went to: /usr/lib/apache2/modules/mod_xsendfile.so So, I copied that to my Apache modules directory, loaded it, and it runs just fine. Next, I need to change the Ruby plugin so that it only loads in production. Otherwise, my development and test environments don''t download anymore (as I don''t have xsendfile via mongrel directly). That should be simple enough to do with the RAILS_ENV variable. The bottom line is, download contents gets served just fine with zero Mongrel effort. Amir --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
helzer wrote:> A little update... > > x_send_file is just fine. > The install instructions are a little broken, but eventually it works. > Get it from: > http://tn123.ath.cx/mod_xsendfile/ > > After create the extension with: > apxs2 -cia mod_xsendfile.c > > the extension gets copied to some default apache modules directory, > and not necessarily to where you need it. In my case it went to: > /usr/lib/apache2/modules/mod_xsendfile.so > > So, I copied that to my Apache modules directory, loaded it, and it > runs just fine. > Next, I need to change the Ruby plugin so that it only loads in > production. Otherwise, my development and test environments don''t > download anymore (as I don''t have xsendfile via mongrel directly). > That should be simple enough to do with the RAILS_ENV variable. > > The bottom line is, download contents gets served just fine with zero > Mongrel effort. > > AmirAmir but how configure apche httpd.conf file so as apache handle this request not mongel in my case rails is handling request.. http://stackoverflow.com/questions/3778360/how-to-i-force-apache-to-handle-download-request-running-apachemongrel is this what i post to stackoverflow my problem is mongrel handling this request not apche?? -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.