thr3ads.net - Rails - Uploading Large Files [Mar 2008]

If this information is useful, please help other people find it:
Share via:

Jimmy Palmer

2008-Mar-31 19:24 UTC

Uploading Large Files

I built an app that allows users to purchase video files to download,
but unfortunately media temple''s grid server isn''t letting me
upload
large files.  Everything goes great unless the files are over 100 mb.
I''m using attachement_fu.

Is there a service or technique that can allow me to put large files on
my server.
-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

andrew.ohnstad-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

2008-Mar-31 20:31 UTC

head link

Re: Uploading Large Files

Sounds like a case for Amazon S3.

On Mar 31, 3:24 pm, Jimmy Palmer
<rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org>
wrote:> I built an app that allows users to purchase video files to download,
> but unfortunately media temple''s grid server isn''t
letting me upload
> large files.  Everything goes great unless the files are over 100 mb.
> I''m using attachement_fu.
>
> Is there a service or technique that can allow me to put large files on
> my server.
> --
> Posted viahttp://www.ruby-forum.com/.--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Peter De Berdt

2008-Mar-31 20:33 UTC

head link

Re: Uploading Large Files

On 31 Mar 2008, at 21:24, Jimmy Palmer wrote:
> I built an app that allows users to purchase video files to download,
> but unfortunately media temple''s grid server isn''t
letting me upload
> large files.  Everything goes great unless the files are over 100 mb.
> I''m using attachement_fu.
>
> Is there a service or technique that can allow me to put large files  
> on
> my server.
FTP them, then use the Ruby File classes to read them into your  
attachment_fu models. But my worry is how you''re going to serve those  
files if mediatemple won''t install the xsendfile module. If
you''re
going to stream those using send_file or send_data, you''re going to  
get into some serious problems, no matter how they serve their rails  
applications. I''d advise you to look into Amazon S3 since you seem to  
be charging for your videos anyway?


Best regards

Peter De Berdt


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Jimmy Palmer

2008-Apr-01 04:12 UTC

head link

uploading large files

I have built a rails web application that requires me to upload several
large files (200+mb). Unfortunately, I have not been successful in doing
so with rails so I tried it through ftp, but was still not successful.
It says that the server dropped the connection after about 80 mb had
been loaded.  I''m using media temple''s grid server.

Does anyone know what the problem might be?  I''m not getting much
response from media temple.

 I have over 50 files ranging anywhere from 30 mb to 300 to upload. What
is the best way to do this? Is there certain server settings that would
make this possible? Do I need to switch servers?  Is there a third party
that would be advantageous to use?

Please advise.
thank you.
-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Peter De Berdt

2008-Apr-01 07:40 UTC

head link

Re: uploading large files

On 01 Apr 2008, at 06:12, Jimmy Palmer wrote:
> I have built a rails web application that requires me to upload  
> several
> large files (200+mb). Unfortunately, I have not been successful in  
> doing
> so with rails so I tried it through ftp, but was still not successful.
> It says that the server dropped the connection after about 80 mb had
> been loaded.  I''m using media temple''s grid server.
FTP is resumable. Just resume uploading your files. If they won''t  
resume, then mt has measures in place to prevent you from doing so and  
it is time to find a new host.
> Does anyone know what the problem might be?  I''m not getting much
> response from media temple.
Shared cheap hosting, always a joy.
> I have over 50 files ranging anywhere from 30 mb to 300 to upload.  
> What
> is the best way to do this? Is there certain server settings that  
> would
> make this possible? Do I need to switch servers?  Is there a third  
> party
> that would be advantageous to use?
As has been told twice (by another poster and myself): if you want to  
serve files fast, use Amazon S3 and you''ll pay for bandwidth used.  
This means, especially with video, you pay for what your customers  
download and you can easily work out how to make a decent margin on  
that. If you don''t feel like using S3, you''d better look to
shell out
some decent money and get a decent VPS or even a dedicated server.  
There are a few hosting companies out there that are recommended time  
after time: slicehost, rimuhosting, engineyard, …


Best regards

Peter De Berdt


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Jimmy Palmer

2008-Jun-09 23:59 UTC

head link

Re: Uploading Large Files

> FTP them, then use the Ruby File classes to read them into your
> attachment_fu models. But my worry is how you''re going to serve
those
> files if mediatemple won''t install the xsendfile module. If
you''re
> going to stream those using send_file or send_data, you''re going
to
> get into some serious problems, no matter how they serve their rails
> applications. I''d advise you to look into Amazon S3 since you seem
to
> be charging for your videos anyway?
> 
> 
> Best regards
> 
> Peter De Berdt
Hey Peter - this was posted a long time ago...hopefully you are still 
following this thread, but I was wondering what you mean by
"you''re
going to get into some serious problems"?  I''m actually using
send_file
and my Ruby on Rails Container keeps terminating because it has "run out 
of Ram".  This happens when the system is using the send_file function 
for large files.  This may have been what you previously warned me 
about.  Do you have any advise or techniques to overcome this issue?

I ended up doing the FTP thing as you advised, but failed to take into 
consideration your send_file warning.  Let me know if you have any 
ideas.

thanks...
-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Peter De Berdt

2008-Jun-10 13:00 UTC

head link

Re: Uploading Large Files

On 10 Jun 2008, at 01:59, Jimmy Palmer wrote:
>> FTP them, then use the Ruby File classes to read them into your
>> attachment_fu models. But my worry is how you''re going to
serve those
>> files if mediatemple won''t install the xsendfile module. If
you''re
>> going to stream those using send_file or send_data, you''re
going to
>> get into some serious problems, no matter how they serve their rails
>> applications. I''d advise you to look into Amazon S3 since you
seem to
>> be charging for your videos anyway?
>
> Hey Peter - this was posted a long time ago...hopefully you are still
> following this thread, but I was wondering what you mean by
"you''re
> going to get into some serious problems"?  I''m actually using
> send_file
> and my Ruby on Rails Container keeps terminating because it has "run  
> out
> of Ram".  This happens when the system is using the send_file function
> for large files.  This may have been what you previously warned me
> about.  Do you have any advise or techniques to overcome this issue?
>
> I ended up doing the FTP thing as you advised, but failed to take into
> consideration your send_file warning.  Let me know if you have any
> ideas.
send_file and send_data will load the entire data stream into memory  
before serving it, that''s why your app goes down due to excessive  
memory use.

Now, as I said, there are a couple of options you might consider:

1. Use Apache''s or nginx'' XSendFile module to serve the files.
If
you''re using some kind of shared hosting, where you have no control  
over the actual web, then this method is gonna fall flat on its face,  
because they probably won''t want to install that module for you.  
Apache will then serve a file with a temporary url.

2. Use an external hosting service like Amazon''s S3 to serve the  
files, where you basically offload the file serving to a service  
that''s made just for doing so. Attachment_fu should be able to help  
you out there.


Best regards

Peter De Berdt


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Jimmy Palmer

2008-Jun-12 20:58 UTC

head link

Re: Uploading Large Files

thanks paul...

one last question - I''ve got over 18 gigs of video files on my media 
temple server that I will need to store on the amazon s3 storage. 
What''s the best way to transfer these over?


-- 
Posted via http://www.ruby-forum.com/.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Peter De Berdt

2008-Jun-13 08:43 UTC

head link

Re: Uploading Large Files

On 12 Jun 2008, at 22:58, Jimmy Palmer wrote:
> one last question - I''ve got over 18 gigs of video files on my
media
> temple server that I will need to store on the amazon s3 storage.
> What''s the best way to transfer these over?
If you have shell access to you media temple account and can run ruby  
on it, using s3sync could be an option:
http://s3sync.net/wiki

Haven''t used it, but seems to fit your needs.


Best regards

Peter De Berdt


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Rails - Mar 2008 - Uploading Large Files

Uploading Large Files

Re: Uploading Large Files

Re: Uploading Large Files

uploading large files

Re: uploading large files

Re: Uploading Large Files

Re: Uploading Large Files

Re: Uploading Large Files

Re: Uploading Large Files