Hi, I''d like to allow users to upload images and store those images in a Postgres database. I''ve looked through the example in "aguile development with rails" which suggests using the MySQL blob field. When I wrote a similar app in Java, we Base64 encoded the file and then stored it as a text field. Is this easy to do in RoR too? Any advice you can give me would be greatly appreciated :) -- Posted via http://www.ruby-forum.com/.
Am Freitag, den 24.03.2006, 11:04 +0100 schrieb Pete:> Hi, > > I''d like to allow users to upload images and store those images in a > Postgres database. I''ve looked through the example in "aguile > development with rails" which suggests using the MySQL blob field. > > When I wrote a similar app in Java, we Base64 encoded the file and then > stored it as a text field. Is this easy to do in RoR too? Any advice you > can give me would be greatly appreciated :)Why not store a file in the filesystem. That''s easy. If you want, store it in the database as an base64 encoded text. That would be only a few lines of code. I think the procedure is the same as in java, only the language differs. Personally i never caught the sense of storing a file in a database. -- Norman Timmler http://blog.inlet-media.de
file_column plugin is awesome for storing files on the file system. In my opinion, that''s the best route to go if you''re going to be serving these images via a web server. That way, the web server can deal with them instead of having to invoke your Rails application just to pull a file out of the database. The only reason I can see a need to store the images in the database is, like sessions, you need to scale across many servers. If that''s the case, I believe there are some wonderful entries on storing binary files into the database on the wiki. On 3/24/06, Norman Timmler <lists@inlet-media.de> wrote:> > Am Freitag, den 24.03.2006, 11:04 +0100 schrieb Pete: > > Hi, > > > > I''d like to allow users to upload images and store those images in a > > Postgres database. I''ve looked through the example in "aguile > > development with rails" which suggests using the MySQL blob field. > > > > When I wrote a similar app in Java, we Base64 encoded the file and then > > stored it as a text field. Is this easy to do in RoR too? Any advice you > > can give me would be greatly appreciated :) > > Why not store a file in the filesystem. That''s easy. If you want, store > it in the database as an base64 encoded text. That would be only a few > lines of code. I think the procedure is the same as in java, only the > language differs. Personally i never caught the sense of storing a file > in a database. > > -- > Norman Timmler > > http://blog.inlet-media.de > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060324/a2317602/attachment-0001.html
Another reason for storing in the database is that you get a backup of your whole system just by backing up the database. No need to include a list of other directories. Brian Hogan wrote:> file_column plugin is awesome for storing files on the file system. In my > opinion, that''s the best route to go if you''re going to be serving these > images via a web server. That way, the web server can deal with them instead > of having to invoke your Rails application just to pull a file out of the > database. > > The only reason I can see a need to store the images in the database is, > like sessions, you need to scale across many servers. If that''s the case, I > believe there are some wonderful entries on storing binary files into the > database on the wiki. > > > On 3/24/06, Norman Timmler <lists@inlet-media.de> wrote: >> Am Freitag, den 24.03.2006, 11:04 +0100 schrieb Pete: >>> Hi, >>> >>> I''d like to allow users to upload images and store those images in a >>> Postgres database. I''ve looked through the example in "aguile >>> development with rails" which suggests using the MySQL blob field. >>> >>> When I wrote a similar app in Java, we Base64 encoded the file and then >>> stored it as a text field. Is this easy to do in RoR too? Any advice you >>> can give me would be greatly appreciated :) >> Why not store a file in the filesystem. That''s easy. If you want, store >> it in the database as an base64 encoded text. That would be only a few >> lines of code. I think the procedure is the same as in java, only the >> language differs. Personally i never caught the sense of storing a file >> in a database. >> >> -- >> Norman Timmler >> >> http://blog.inlet-media.de
I have a similar need, how can we store a file on the file system with rails? Saiho --- Brian Hogan <bphogan@gmail.com> wrote:> file_column plugin is awesome for storing files on > the file system. In my > opinion, that''s the best route to go if you''re going > to be serving these > images via a web server. That way, the web server > can deal with them instead > of having to invoke your Rails application just to > pull a file out of the > database. > > The only reason I can see a need to store the images > in the database is, > like sessions, you need to scale across many > servers. If that''s the case, I > believe there are some wonderful entries on storing > binary files into the > database on the wiki. > > > On 3/24/06, Norman Timmler <lists@inlet-media.de> > wrote: > > > > Am Freitag, den 24.03.2006, 11:04 +0100 schrieb > Pete: > > > Hi, > > > > > > I''d like to allow users to upload images and > store those images in a > > > Postgres database. I''ve looked through the > example in "aguile > > > development with rails" which suggests using the > MySQL blob field. > > > > > > When I wrote a similar app in Java, we Base64 > encoded the file and then > > > stored it as a text field. Is this easy to do in > RoR too? Any advice you > > > can give me would be greatly appreciated :) > > > > Why not store a file in the filesystem. That''s > easy. If you want, store > > it in the database as an base64 encoded text. That > would be only a few > > lines of code. I think the procedure is the same > as in java, only the > > language differs. Personally i never caught the > sense of storing a file > > in a database. > > > > -- > > Norman Timmler > > > > http://blog.inlet-media.de > > > > _______________________________________________ > > Rails mailing list > > Rails@lists.rubyonrails.org > > > http://lists.rubyonrails.org/mailman/listinfo/rails > > > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >The mind is its own place, and in itself. Can make a Heaven of Hell, a Hell of Heaven. http://www.geocities.com/sayoyo/ __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
The first hit when you search for upload in the RubyonRails Wiki is this: http://wiki.rubyonrails.com/rails/pages/HowtoUploadFiles I have been using the info from this article for several apps. --Nis. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060324/4eff2a33/attachment.html
> > file_column plugin is awesome for storing files on the file system. In my > > opinion, that''s the best route to go if you''re going to be serving these > > images via a web server. That way, the web server can deal with them instead > > of having to invoke your Rails application just to pull a file out of the > > database. > > > > The only reason I can see a need to store the images in the database is, > > like sessions, you need to scale across many servers. If that''s the case, I > > believe there are some wonderful entries on storing binary files into the > > database on the wiki.Share the filesystem via nfs or some other means. Storing images in your db doesn''t scale. Make a reference in your db to your file instead. regards Claus
I think storing images on the filesystem is definitely the way to go. And if you need to scale at a later date use something like MogileFS (http://www.danga.com/mogilefs/) to handle replication across machines. Steve On 24 Mar 2006, at 10:04, Pete wrote:> Hi, > > I''d like to allow users to upload images and store those images in a > Postgres database. I''ve looked through the example in "aguile > development with rails" which suggests using the MySQL blob field. > > When I wrote a similar app in Java, we Base64 encoded the file and > then > stored it as a text field. Is this easy to do in RoR too? Any > advice you > can give me would be greatly appreciated :) > > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails
Am Freitag, den 24.03.2006, 17:44 +0100 schrieb Claus Guttesen:> > > file_column plugin is awesome for storing files on the file system. In my > > > opinion, that''s the best route to go if you''re going to be serving these > > > images via a web server. That way, the web server can deal with them instead > > > of having to invoke your Rails application just to pull a file out of the > > > database. > > > > > > The only reason I can see a need to store the images in the database is, > > > like sessions, you need to scale across many servers. If that''s the case, I > > > believe there are some wonderful entries on storing binary files into the > > > database on the wiki. > > Share the filesystem via nfs or some other means. Storing images in > your db doesn''t scale. Make a reference in your db to your file > instead.+1. Would you store 100,000 1MB files in a DB? A filesystem is made for handling files. It''s fast and reliable. DB''s are made for organizing and finding data, not for storing large binaries. -- Norman Timmler http://blog.inlet-media.de
Oh trust me, I totally agree. I''ve written quite a few apps that do this kind of thing and have *never* stored files in my database. I don''t think "it''s easier to backup" is really that big of an issue.... that sounds like lazy dba talk to me :) (I''m an Oracle dba, btw) Like I said... the "only reason" I can think of is the scaling issue... and even that''s not a good one. On 3/24/06, Norman Timmler <lists@inlet-media.de> wrote:> > Am Freitag, den 24.03.2006, 17:44 +0100 schrieb Claus Guttesen: > > > > file_column plugin is awesome for storing files on the file system. > In my > > > > opinion, that''s the best route to go if you''re going to be serving > these > > > > images via a web server. That way, the web server can deal with them > instead > > > > of having to invoke your Rails application just to pull a file out > of the > > > > database. > > > > > > > > The only reason I can see a need to store the images in the database > is, > > > > like sessions, you need to scale across many servers. If that''s the > case, I > > > > believe there are some wonderful entries on storing binary files > into the > > > > database on the wiki. > > > > Share the filesystem via nfs or some other means. Storing images in > > your db doesn''t scale. Make a reference in your db to your file > > instead. > > +1. Would you store 100,000 1MB files in a DB? A filesystem is made for > handling files. It''s fast and reliable. DB''s are made for organizing and > finding data, not for storing large binaries. > > -- > Norman Timmler > > http://blog.inlet-media.de > > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060324/cdacbf44/attachment-0001.html
> > Share the filesystem via nfs or some other means. Storing images in > > your db doesn''t scale. Make a reference in your db to your file > > instead. > > +1. Would you store 100,000 1MB files in a DB?No I wouldn''t do that.> A filesystem is made for > handling files. It''s fast and reliable. DB''s are made for organizing and > finding data, not for storing large binaries.When I wrote "share the filesystem" I meant (maby not so obvious :-) that one keeps the images in files, which is then shared by nfs or some other means. *Only* keep a pointer in the db which then references the file (located on a filesystem). Storing images in the db does *not* scale, so if the site does have ambitions several hundreds GB''s reaching into the TB''s in the db, is *not* the way to do it. regards Claus
Am Freitag, den 24.03.2006, 18:07 +0100 schrieb Claus Guttesen:> > > Share the filesystem via nfs or some other means. Storing images in > > > your db doesn''t scale. Make a reference in your db to your file > > > instead. > > > > +1. Would you store 100,000 1MB files in a DB? > > No I wouldn''t do that. > > > A filesystem is made for > > handling files. It''s fast and reliable. DB''s are made for organizing and > > finding data, not for storing large binaries. > > When I wrote "share the filesystem" I meant (maby not so obvious :-) > that one keeps the images in files, which is then shared by nfs or > some other means. *Only* keep a pointer in the db which then > references the file (located on a filesystem). > > Storing images in the db does *not* scale, so if the site does have > ambitions several hundreds GB''s reaching into the TB''s in the db, is > *not* the way to do it.Sorry, you misunderstood me! I totally agree with you! The question was directed to Chris Schumann. NFS is also my prefered way. -- Norman Timmler http://blog.inlet-media.de
On Mar 24, 2006, at 8:51 AM, Norman Timmler wrote:> Am Freitag, den 24.03.2006, 17:44 +0100 schrieb Claus Guttesen: >>>> file_column plugin is awesome for storing files on the file >>>> system. In my >>>> opinion, that''s the best route to go if you''re going to be >>>> serving these >>>> images via a web server. That way, the web server can deal with >>>> them instead >>>> of having to invoke your Rails application just to pull a file >>>> out of the >>>> database. >>>> >>>> The only reason I can see a need to store the images in the >>>> database is, >>>> like sessions, you need to scale across many servers. If that''s >>>> the case, I >>>> believe there are some wonderful entries on storing binary files >>>> into the >>>> database on the wiki. >> >> Share the filesystem via nfs or some other means. Storing images in >> your db doesn''t scale. Make a reference in your db to your file >> instead. > > +1. Would you store 100,000 1MB files in a DB? A filesystem is made > for > handling files. It''s fast and reliable. DB''s are made for > organizing and > finding data, not for storing large binaries.What precludes large binaries from being data? And, as often as it''s said, people should wonder why every database has a BLOB type column! I guess someone should alert the database coders and vendors that they should discontinue BLOB support in the future, since that''s not what databases should be used for. :-) I don''t entirely disagree with the position, but I don''t entirely agree with it either. -- -- Tom Mornini
Norman Timmler wrote:> Am Freitag, den 24.03.2006, 18:07 +0100 schrieb Claus Guttesen: >>>> Share the filesystem via nfs or some other means. Storing images in >>>> your db doesn''t scale. Make a reference in your db to your file >>>> instead. >>> +1. Would you store 100,000 1MB files in a DB? >> No I wouldn''t do that. >> >>> A filesystem is made for >>> handling files. It''s fast and reliable. DB''s are made for organizing and >>> finding data, not for storing large binaries. >> When I wrote "share the filesystem" I meant (maby not so obvious :-) >> that one keeps the images in files, which is then shared by nfs or >> some other means. *Only* keep a pointer in the db which then >> references the file (located on a filesystem). >> >> Storing images in the db does *not* scale, so if the site does have >> ambitions several hundreds GB''s reaching into the TB''s in the db, is >> *not* the way to do it. > > Sorry, you misunderstood me! I totally agree with you! The question was > directed to Chris Schumann. NFS is also my prefered way. >What question? Nothing quoted above was from me. I wrote that one could use a single backup point as a reason to put images in a database. Someone else wrote it could scale. I have no idea, and wouldn''t do it either. I just wrote it as a response to someone who wrote that he couldn''t think of another reason to do it. Chris Schumann
On Mar 24, 2006, at 8:45 AM, Steven Mohapi-Banks wrote:> On 24 Mar 2006, at 10:04, Pete wrote: > >> I''d like to allow users to upload images and store those images in a >> Postgres database. I''ve looked through the example in "aguile >> development with rails" which suggests using the MySQL blob field. >> >> When I wrote a similar app in Java, we Base64 encoded the file and >> then >> stored it as a text field. Is this easy to do in RoR too? Any >> advice you >> can give me would be greatly appreciated :) > > I think storing images on the filesystem is definitely the way to > go. And if you need to scale at a later date use something like > MogileFS (http://www.danga.com/mogilefs/) to handle replication > across machines.This is what we did for 43 Things. Stored images on an NFS-mounted directory until the disk got nearly full, then switched to MogileFS. So now there''s a Ruby MogileFS client: http://blog.segment7.net/articles/2006/03/22/mogilefs-for-ruby -- Eric Hodel - drbrain@segment7.net - http://blog.segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com
Am Freitag, den 24.03.2006, 11:47 -0600 schrieb Chris Schumann:> What question? Nothing quoted above was from me. > > I wrote that one could use a single backup point as a reason to put > images in a database. Someone else wrote it could scale. I have no > idea, > and wouldn''t do it either. I just wrote it as a response to someone > who > wrote that he couldn''t think of another reason to do it.Sorry for the confusion. I missed to quote you. "Another reason for storing in the database is that you get a backup of your whole system just by backing up the database. No need to include a list of other directories." I thought you would suggest this way as an answer to the initial question of Pete, what was: "I''d like to allow users to upload images and store those images in a Postgres database. I''ve looked through the example in "aguile development with rails" which suggests using the MySQL blob field. When I wrote a similar app in Java, we Base64 encoded the file and then stored it as a text field. Is this easy to do in RoR too? Any advice you can give me would be greatly appreciated :)" I did not realize the diferent context. -- Norman Timmler http://blog.inlet-media.de
You may also want to check out this very interesting discussion we had at Sheeri''s blog about the same topic, i.e. storing images (36 million) in a database. http://sheeri.com/archives/39 Includes comments from James Day of Wikipedia. I second Brian in that you shouldn''t do this just because "it''s easier to backup". Frank Brian Hogan <bphogan@gmail.com> wrote:Oh trust me, I totally agree. I''ve written quite a few apps that do this kind of thing and have *never* stored files in my database. I don''t think "it''s easier to backup" is really that big of an issue.... that sounds like lazy dba talk to me :) (I''m an Oracle dba, btw) Like I said... the "only reason" I can think of is the scaling issue... and even that''s not a good one. On 3/24/06, Norman Timmler <lists@inlet-media.de> wrote: Want to read my Pro Rails book or Pro Server Management book? Become a beta reader now. Write me to learn more. Rails Blog: http://railsruby.blogspot.com MySQL Blog: http://mysqldatabaseadministration.blogspot.com Linux / Security Blog: http://frankmash.blogspot.com Programming One Liners: http://programming-oneliners.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060324/23d7f9a0/attachment.html
> You may also want to check out this very interesting discussion we had at > Sheeri''s blog about the same topic, i.e. storing images (36 million) in a > database. > > http://sheeri.com/archives/39Quoting from the blog: "My company''s site has over 2 million images to store/retrieve, and our current setup is at its limit and doesn''t scale well ? 2 NFS servers. We''d rather avoid adding another NFS server, ..." You don''t add another nfs-server, you implement a solution where you upgrade your current nfs-server with more disk-storage and add that storage to some sort of storage-pool. Storing and retrieving 2 mill. images is also about planning your layout, either by date or incrementing a counter or by other means dividing your images into sub-dirs. You may have two nfs-server, but that''s usually to provide some sort of fail-over. But this is becoming off-topic and ought to be continued on a db- or serverperformance-related list instead. regards Claus
What I don''t get is how quickly discussions get labeled as off-topic. Using Rails does involves server / db issues. Why not let discussions run their course to accumulate the specific practices being used by the Rails community. Just my two cents. Frank Claus Guttesen <kometen@gmail.com> wrote: But this is becoming off-topic and ought to be continued on a db- or serverperformance-related list instead. regards Claus Want to read my Pro Rails book or Pro Server Management book? Become a beta reader now. Write me to learn more. Rails Blog: http://railsruby.blogspot.com MySQL Blog: http://mysqldatabaseadministration.blogspot.com Linux / Security Blog: http://frankmash.blogspot.com Programming One Liners: http://programming-oneliners.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060325/6c9a2fd6/attachment.html
I just solved this problem myself. I used the file_column plugin and let it upload to its default / RAILS_ROOT/public/upload directory. I think the folder was named after my model. While in development, i had subversion ignore this directory so that when i deployed, it wouldn''t be there. I then created a symlink to a directory under my "shared" directory outside of my app. This way when i redeploy, users aren''t going to lose their uploads. /current -> /releases/200629030912 /releases /releases/200629030912 /releases/200629030912/public/upload -> /shared/upload /shared /shared/upload I hope that wasn''t too confusing. On Mar 24, 2006, at 11:16 AM, Nis Sarup wrote:> The first hit when you search for upload in the RubyonRails Wiki is > this: http://wiki.rubyonrails.com/rails/pages/HowtoUploadFiles > > I have been using the info from this article for several apps. > > --Nis. > _______________________________________________ > Rails mailing list > Rails@lists.rubyonrails.org > http://lists.rubyonrails.org/mailman/listinfo/rails-------------- next part -------------- An HTML attachment was scrubbed... URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060329/06f99a49/attachment.html