Mongrel users - Sep 2010 - problem in downloading large file with apache+mongrel

hii all,
am using apache as frontend to mongrel and loaded xsendfile
module in apache ,am looking for 1 GB of download .this is download
routine

def download

@stream = Stream.find(params[:id])

send_file(@stream.location,:filename => @stream.name,:disposition >
''attachment'',:x_sendfile => true )

and

And this is how it is processed

Processing StreamsController#download (for 127.0.0.1 at 2010-09-22
12:32:10) [GET] Parameters: {"id"=>"6596"}

<-[4;36;1mSQL (0.0ms)?[0m ?[0;1mSET NAMES ''utf8''?[0m

<-[4;35;1mSQL (0.0ms)?[0m ?[0mSET SQL_AUTO_IS_NULL=0?[0m

<-[4;36;1mStream Columns (15.0ms)?[0m ?[0;1mSHOW FIELDS FROM streams?[0m

<-[4;35;1mStream Load (0.0ms)?[0m ?[0mSELECT * FROM streams WHERE
(streams.id = 6596) ?[0m

<-[4;36;1mCACHE (0.0ms)?[0m ?[0;1mSELECT * FROM streams WHERE
(streams.id = 596) ? ?[0m

Sending X-Sendfile header
d:/dm/predator_720x480_5mbps_30fps_17minclip.264.filepart Completed in
57513ms (View: 0, DB: 15) | 200 OK [http://src/streams/download/6596]

very first times its okk i getting full 1.1 GB of file getting
downloaded,but when i am trying to download this 1.1 gb of file again,
am getting failed to allocate memroy what could be soultion?? and can i
use cache.clear before send_file function??
and more thing to again download 1.1 gb of file i have to restart my
servers..

thanks??
-- 
Posted via http://www.ruby-forum.com/.

On Wed, Sep 22, 2010 at 10:01 AM, Amit Tomar <lists at ruby-forum.com>
wrote:
> hii all,
> am using apache as frontend to mongrel and loaded xsendfile
> module in apache ,am looking for 1 GB of download .this is download
> routine
>
Replied your previous message to the list also. Please provide the
answers to my questions.
-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

Luis Lavena wrote:
> On Wed, Sep 22, 2010 at 10:01 AM, Amit Tomar <lists at
ruby-forum.com>
> wrote:
>> hii all,
>> am using apache as frontend to mongrel and loaded xsendfile
>> module in apache ,am looking for 1 GB of download .this is download
>> routine
>>
> 
> Replied your previous message to the list also. Please provide the
> answers to my questions.
> --
> Luis Lavena
> AREA 17
> -
> Perfection in design is achieved not when there is nothing more to add,
> but rather when there is nothing more to take away.
> Antoine de Saint-Exup?

just now i seen your message...
i am running apache 2.2.11 that comes with XAMPP 1.7.1 and mongrel 
version is
1.1.2,
ruby version is ruby 1.8.6

luis you are saying



Rails is serving the request, the 1.1GB, not Apache
but how do i know rails is serving the request..or Apache


2. d:/dm  is not a network drive..its folder on d drive
how do i know whther it available  to apache ??

i am sending you access logs of apache after few mintues
-- 
Posted via http://www.ruby-forum.com/.

when i am trying to downnload following happen in logs of apache

127.0.0.1 - - [22/Sep/2010:19:11:12 +0530] "GET /streams/download/6598 
HTTP/1.1" 500 1024 "http://src/streams/6598" "Mozilla/5.0
(Windows; U;
Windows NT 5.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) 
Chrome/6.0.472.33 Safari/534.3"

where 6598 is id of file to be downloaded
-- 
Posted via http://www.ruby-forum.com/.

On Wed, Sep 22, 2010 at 10:50 AM, Amit Tomar <lists at ruby-forum.com>
wrote:
> just now i seen your message...
> i am running apache 2.2.11 that comes with XAMPP 1.7.1 and mongrel
> version is
> 1.1.2,
> ruby version is ruby 1.8.6
>
>
> Rails is serving the request, the 1.1GB, not Apache
> but how do i know rails is serving the request..or Apache
>
Please answer all the questions I put, analyze the process memory usage.

If you don''t see in apache serving the X-Sendfile request, then is not.


Try setting the headers on your own and render nothing:

Implement the first code example of this page:

http://john.guen.in/past/2007/4/17/send_files_faster_with_xsendfile/

Not the plugin.
> 2. d:/dm ?is not a network drive..its folder on d drive
> how do i know whther it available ?to apache ??
>
Try to setup a virtual server configuration that access D: and see if it works.


-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

soory but how do i setup s virtual server configuration??
and i have to install that plugin??
-- 
Posted via http://www.ruby-forum.com/.

On Wed, Sep 22, 2010 at 11:22 AM, Amit Tomar <lists at ruby-forum.com>
wrote:
>
> soory but how do i setup s virtual server configuration??
Read apache documentation.
> and i have to install that plugin??
No, in your controller action set the headers, *do not* install the plugin.
-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

Luis Lavena wrote:
> On Wed, Sep 22, 2010 at 11:22 AM, Amit Tomar <lists at
ruby-forum.com>
> wrote:
>>
>> soory but how do i setup s virtual server configuration??
> 
> Read apache documentation.
> 
>> and i have to install that plugin??
> 
> No, in your controller action set the headers, *do not* install the 
> plugin.
> --
> Luis Lavena
> AREA 17
> -
> Perfection in design is achieved not when there is nothing more to add,
> but rather when there is nothing more to take away.
> Antoine de Saint-Exup?
luis i change my code as you said

filename =  "#{@stream.location}"
response.headers[''Content-Type''] =
"application/force-download"
 response.headers[''Content-Disposition''] = "attachment; 
filename=\"#{File.basename(filename)}\""
response.headers["X-Sendfile"] = filename
response.headers[''Content-length''] = File.size(filename)
render :nothing => true


this is how it is processed...


-- 
Posted via http://www.ruby-forum.com/.

Amit Tomar wrote:
> Luis Lavena wrote:
>> On Wed, Sep 22, 2010 at 11:22 AM, Amit Tomar <lists at
ruby-forum.com>
>> wrote:
>>>
>>> soory but how do i setup s virtual server configuration??
>> 
>> Read apache documentation.
>> 
>>> and i have to install that plugin??
>> 
>> No, in your controller action set the headers, *do not* install the 
>> plugin.
>> --
>> Luis Lavena
>> AREA 17
>> -
>> Perfection in design is achieved not when there is nothing more to add,
>> but rather when there is nothing more to take away.
>> Antoine de Saint-Exup?
> 
> luis i change my code as you said
> 
> filename =  "#{@stream.location}"
> response.headers[''Content-Type''] =
"application/force-download"
>  response.headers[''Content-Disposition''] =
"attachment;
> filename=\"#{File.basename(filename)}\""
> response.headers["X-Sendfile"] = filename
> response.headers[''Content-length''] = File.size(filename)
> render :nothing => true
> 
> 
> this is how it is processed...
Processing StreamsController#download (for 127.0.0.1 at 2010-09-22 
20:22:06) [GET]
  Parameters: {"id"=>"6600"}
  ?[4;36;1mSQL (0.0ms)?[0m   ?[0;1mSET NAMES ''utf8''?[0m
  ?[4;35;1mSQL (0.0ms)?[0m   ?[0mSET SQL_AUTO_IS_NULL=0?[0m
  ?[4;36;1mStream Columns (0.0ms)?[0m   ?[0;1mSHOW FIELDS FROM 
`streams`?[0m
  ?[4;35;1mStream Load (0.0ms)?[0m   ?[0mSELECT * FROM `streams` WHERE 
(`streams`.`id` = 6600) ?[0m
  ?[4;36;1mCACHE (0.0ms)?[0m   ?[0;1mSELECT * FROM `streams` WHERE 
(`streams`.`id` = 6600) ?[0m
0) ?[0m
Completed in 65347ms (View: 0, DB: 0) | 200 OK 
[http://src/streams/download/6600]
-- 
Posted via http://www.ruby-forum.com/.

On Wed, Sep 22, 2010 at 12:00 PM, Amit Tomar <lists at ruby-forum.com>
wrote:
>
> luis i change my code as you said
>
> filename = ?"#{@stream.location}"
> response.headers[''Content-Type''] =
"application/force-download"
> ?response.headers[''Content-Disposition''] =
"attachment;
> filename=\"#{File.basename(filename)}\""
> response.headers["X-Sendfile"] = filename
> response.headers[''Content-length''] = File.size(filename)
> render :nothing => true
>
> this is how it is processed...
>
This? How? can you provide answers to my previous questions about
process memory usage? Please?

-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

Amit Tomar wrote:
> Amit Tomar wrote:
>> Luis Lavena wrote:
>>> On Wed, Sep 22, 2010 at 11:22 AM, Amit Tomar <lists at
ruby-forum.com>
>>> wrote:
>>>>
>>>> soory but how do i setup s virtual server configuration??
>>> 
>>> Read apache documentation.
>>> 
>>>> and i have to install that plugin??
>>> 
>>> No, in your controller action set the headers, *do not* install the
>>> plugin.
>>> --
>>> Luis Lavena
>>> AREA 17
>>> -
>>> Perfection in design is achieved not when there is nothing more to
add,
>>> but rather when there is nothing more to take away.
>>> Antoine de Saint-Exup?
>> 
>> luis i change my code as you said
>> 
>> filename =  "#{@stream.location}"
>> response.headers[''Content-Type''] =
"application/force-download"
>>  response.headers[''Content-Disposition''] =
"attachment;
>> filename=\"#{File.basename(filename)}\""
>> response.headers["X-Sendfile"] = filename
>> response.headers[''Content-length''] =
File.size(filename)
>> render :nothing => true
>> 
>> 
>> this is how it is processed...
> 
> Processing StreamsController#download (for 127.0.0.1 at 2010-09-22 
> 20:22:06) [GET]
>   Parameters: {"id"=>"6600"}
>   ?[4;36;1mSQL (0.0ms)?[0m   ?[0;1mSET NAMES ''utf8''?[0m
>   ?[4;35;1mSQL (0.0ms)?[0m   ?[0mSET SQL_AUTO_IS_NULL=0?[0m
>   ?[4;36;1mStream Columns (0.0ms)?[0m   ?[0;1mSHOW FIELDS FROM 
> `streams`?[0m
>   ?[4;35;1mStream Load (0.0ms)?[0m   ?[0mSELECT * FROM `streams` WHERE 
> (`streams`.`id` = 6600) ?[0m
>   ?[4;36;1mCACHE (0.0ms)?[0m   ?[0;1mSELECT * FROM `streams` WHERE 
> (`streams`.`id` = 6600) ?[0m
> 0) ?[0m
> Completed in 65347ms (View: 0, DB: 0) | 200 OK 
> [http://src/streams/download/6600]
and these are the lines i added in httpd.conf
<Directory "d:/dm">
Options Indexes FollowSymLinks
AllowOverride FileInfo
Order allow,deny
Allow from all
</Directory>


for virtual server configuration to  access D:
what should i do now
-- 
Posted via http://www.ruby-forum.com/.

yaa am looking to provide answer for that also memory usage
-- 
Posted via http://www.ruby-forum.com/.

Amit Tomar wrote:
> 
> yaa am looking to provide answer for that also memory usage
Luis when i download file
httpd.exe  is varied from at 49,892k to 50,264k
while ruby.exe                934800 k to 1,208688k
-- 
Posted via http://www.ruby-forum.com/.

On Wed, Sep 22, 2010 at 12:29 PM, Amit Tomar <lists at ruby-forum.com>
wrote:
> Amit Tomar wrote:
>>
>> yaa am looking to provide answer for that also memory usage
>
> Luis when i download file
> httpd.exe ?is varied from at 49,892k to 50,264k
> while ruby.exe ? ? ? ? ? ? ? ?934800 k to 1,208688k
That makes then clear that Ruby is trying to read the file, not apache.

Sorry, I can''t help you more if you don''t keep an structure
conversation and you investigate also.
-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

On Wed, Sep 22, 2010 at 12:01 PM, Amit Tomar <lists at ruby-forum.com>
wrote:
>
> Processing StreamsController#download (for 127.0.0.1 at 2010-09-22
> 20:22:06) [GET]
> ?Parameters: {"id"=>"6600"}
> ??[4;36;1mSQL (0.0ms)?[0m ? ?[0;1mSET NAMES ''utf8''?[0m
> ??[4;35;1mSQL (0.0ms)?[0m ? ?[0mSET SQL_AUTO_IS_NULL=0?[0m
> ??[4;36;1mStream Columns (0.0ms)?[0m ? ?[0;1mSHOW FIELDS FROM
> `streams`?[0m
> ??[4;35;1mStream Load (0.0ms)?[0m ? ?[0mSELECT * FROM `streams` WHERE
> (`streams`.`id` = 6600) ?[0m
> ??[4;36;1mCACHE (0.0ms)?[0m ? ?[0;1mSELECT * FROM `streams` WHERE
> (`streams`.`id` = 6600) ?[0m
> 0) ?[0m
> Completed in 65347ms (View: 0, DB: 0) | 200 OK
> [http://src/streams/download/6600]
Rails log do not help, we need to know Apache headers, responses and
also mongrel headers sent to apache via the proxy.

Please use curl --head on mongrel to request the URL and provide the
complete output
Also do the same from apache.

Beyond that, I cannot help you more because I don''t have time to
attempt replicate your environment and scenario.
-- 
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exup?ry

Amit Tomar wrote:
> Amit Tomar wrote:
>> 
>> yaa am looking to provide answer for that also memory usage
> 
> Luis when i download file
> httpd.exe  is varied from at 49,892k to 50,264k
> while ruby.exe                934800 k to 1,208688k
and louis as soon while getting downloded memoryuse of httd.exe is 
getting decrerase to 768 k and the when the file completely downloded 
memory use get down to 324 k
-- 
Posted via http://www.ruby-forum.com/.

Amit Tomar wrote:
> Amit Tomar wrote:
>> Amit Tomar wrote:
>>> 
>>> yaa am looking to provide answer for that also memory usage
>> 
>> Luis when i download file
>> httpd.exe  is varied from at 49,892k to 50,264k
>> while ruby.exe                934800 k to 1,208688k
> 
> and louis as soon while getting downloded memoryuse of httd.exe is 
> getting decrerase to 768 k and the when the file completely downloded 
> memory use get down to 324 k
yaa louis tahnks for your help.....
you must be very good person...
-- 
Posted via http://www.ruby-forum.com/.

On 22 Sep 2010, at 12:00, Amit Tomar wrote:
> luis i change my code as you said
> 
> filename =  "#{@stream.location}"
> response.headers[''Content-Type''] =
"application/force-download"
> response.headers[''Content-Disposition''] =
"attachment;
> filename=\"#{File.basename(filename)}\""
The problem is here ^^^^
> response.headers["X-Sendfile"] = filename
> response.headers[''Content-length''] = File.size(filename)
File.size should fail, as rails will be in Dir.pwd == Rails.root, and as such,
File.size(filename) for filename with no path should fail. This should be
raising Errno::ENOENT.

but louis,how do i force apache to read the file and this problem can
occur with Apache also??
-- 
Posted via http://www.ruby-forum.com/.

James Tucker wrote:
> On 22 Sep 2010, at 12:00, Amit Tomar wrote:
>> luis i change my code as you said
>> 
>> filename =  "#{@stream.location}"
>> response.headers[''Content-Type''] =
"application/force-download"
>> response.headers[''Content-Disposition''] =
"attachment;
>> filename=\"#{File.basename(filename)}\""
> 
> The problem is here ^^^^
> 
>> response.headers["X-Sendfile"] = filename
>> response.headers[''Content-length''] =
File.size(filename)
> 
> File.size should fail, as rails will be in Dir.pwd == Rails.root, and as 
> such, File.size(filename) for filename with no path should fail. This 
> should be raising Errno::ENOENT.
but tucker, i am getting failed to allocate memory and that is also of 
second time....
-- 
Posted via http://www.ruby-forum.com/.

On 23 Sep 2010, at 02:42, Amit Tomar wrote:
> James Tucker wrote:
>> On 22 Sep 2010, at 12:00, Amit Tomar wrote:
>>> luis i change my code as you said
>>> 
>>> filename =  "#{@stream.location}"
>>> response.headers[''Content-Type''] =
"application/force-download"
>>> response.headers[''Content-Disposition''] =
"attachment;
>>> filename=\"#{File.basename(filename)}\""
>> 
>> The problem is here ^^^^
>> 
>>> response.headers["X-Sendfile"] = filename
>>> response.headers[''Content-length''] =
File.size(filename)
>> 
>> File.size should fail, as rails will be in Dir.pwd == Rails.root, and
as
>> such, File.size(filename) for filename with no path should fail. This 
>> should be raising Errno::ENOENT.
> 
> but tucker, i am getting failed to allocate memory and that is also of 
> second time....
You confuse me for someone that doesn''t read. You also clearly
don''t understand "should", or what I''ve just told
you.