Rails - Dec 2005 - 2005 a send_file odyssey (or Rails and Apache don''t always play well)...

Hi,

I''m developing a Rails application that (amongst other things)
lets users download media files.  These files range in size
from a few kilobytes up to several gigabytes.

All this development is taking place with Rails 1.0 on a
Solaris host with Ruby 1.8.2 and Rails mode set to
"development".

I''m using the Rails send_file method to send each file with
these settings:

      send_file(my_path_to_file, :filename => my_file_name,
                                 :type => my_file_mime_type,
                                 :disposition => "attachment",
                                 :stream => true,
                                 :buffer_size => 4096)

With appropriate values for my_path_to_file, my_file_name,
my_file_mime_type which are logged so I can check them in the
logs.

During development I''m downloading two test files, one around
10MB and another around 350MB.  Using WEBrick as the server
everything seems to work okay:

    Firefox (OS/X) : both downloads successful
    Safari (OS/X)  : both downloads successful
    IE 6 (Win XP)  : both downloads successful

Flushed with success I switch to using the deployment web
server Apache 1.3.x and retry the downloads with the 10MB
file:

     Firefox (OS/X) : file type and file name not found by Firefox
     Safari (OS/X)  : file type and file name not found by Firefox

Okay, so I have a look at the request/response headers going
back and forth with the excellent Live HTTP headers plugin for
Firefox and it shows that the Content-Type header is set to plain
text and the Content-Length header is missing. What gives?

Curious, I try the 350MB download:

     Firefox (OS/X) : 500 internal server error

Looking at the Apache error logs I see (long line wrapped):

/opt/ruby-1.8.2/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/
     action_controller/streaming.rb:71: warning: syswrite for
     buffered IO

Which makes me think Rails or Ruby are trying to do low level
system writes to a buffered IO stream.  My next step is to
copy the method referred to in the log message and change it
to ensure it doesn''t use low level writes:

         def my_send_file(path, options = {})
             raise MissingFile, "Cannot read file #{path}"
                unless File.file?(path) and File.readable?(path)

         options[:length]   ||= File.size(path)
         options[:filename] ||= File.basename(path)
         send_file_headers! options

         @performed_render = false

         if options[:stream]
           render :text => Proc.new { |response, output|
             logger.info "Streaming file #{path}" unless logger.nil?
             len = options[:buffer_size] || 4096
             File.open(path, ''rb'') do |file|
               if false # changed this line
                 begin
                   while true
                     output.syswrite(file.sysread(len))
                   end
                 rescue EOFError
                 end
               else
                 while buf = file.read(len)
                   output.write(buf)
                 end
               end
             end
           }
         else
           logger.info "Sending file #{path}" unless logger.nil?
           File.open(path, ''rb'') { |file| render :text =>
file.read }
         end
       end

I also need to take a copy of the private method send_file_headers!
who''s code is completely unchanged.

Now change my controller code to use the modified function:

      my_send_file(my_path_to_file, :filename => my_file_name,
                                    :type => my_file_mime_type,
                                    :disposition => "attachment",
                                    :stream => true,
                                    :buffer_size => 4096)

Testing with the 10MB file shows:

    Firefox (OS/X) : successful
    Safari (OS/X)  : successful

Testing with the 350MB file shows:

    Firefox (OS/X) : gets within 1MB or so of the total, then hangs
    Safari (OS/X)  : gets within 1MB or so of the total, then hangs

So still no luck getting Rails to send moderate sized files with
Apache.

I can''t be the first person to ever want to use send_file in a Rails
application with Apache, so can any more knowledgeable Ruby and Rails
people share their experiences and let me know what I''m doing wrong?

The next step is to search the Rails trac site to see if this behavior
is already registered as a bug, in the meantime, any advice would
be much appreciated.


Stu