I''ve a method within an Event model..
def batch_import(folder)
require ''find''
Find::find("#{SOURCE_PATH}/#{folder}") do |path|
if path[-3..-1].downcase == ''jpg''
@photo = Picture.new()
@photo.data = File.new(path)
@photo.filename = File.basename(path)
@photo.caption = "none"
self.pictures << @photo
self.save
end
end
end
Where Picture is ..
class Picture < FlexImage::Model
file_store ''pictures''
belongs_to :event, :counter_cache => true
acts_as_list :scope => :event
acts_as_solr :fields => [:caption]
end
and Event is..
class Event < ActiveRecord::Base
has_many :pictures, :order =>
''position'',:dependent=>:destroy
When I run the code it uses a massive 1.5GB of memory to process 28
pictures. Ruby eventually returns the memory once you''ve browsed to
another view, but still?!?
Anyone have any ideas how/why?
should I be forcing garbage collection on @photo after each image?!?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---
On 8/13/07, kevin evans <kwevans-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > I''ve a method within an Event model.. > > def batch_import(folder) > require ''find'' > > Find::find("#{SOURCE_PATH}/#{folder}") do |path| > if path[-3..-1].downcase == ''jpg'' > @photo = Picture.new() > @photo.data = File.new(path) > @photo.filename = File.basename(path) > @photo.caption = "none" > self.pictures << @photo > self.save > end > end > end > > Where Picture is .. > > class Picture < FlexImage::Model > file_store ''pictures'' > > belongs_to :event, :counter_cache => true > acts_as_list :scope => :event > > acts_as_solr :fields => [:caption] > > end > and Event is.. > > class Event < ActiveRecord::Base > has_many :pictures, :order => ''position'',:dependent=>:destroy > > > When I run the code it uses a massive 1.5GB of memory to process 28 > pictures. Ruby eventually returns the memory once you''ve browsed to > another view, but still?!? > > Anyone have any ideas how/why? > should I be forcing garbage collection on @photo after each image?!?I don''t know if that''ll matter. It looks like FlexImage is reading the files into memory, and it still might be referenced in Event''s self.pictures array. -- Rick Olson http://lighthouseapp.com http://weblog.techno-weenie.net http://mephistoblog.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Have you tried just doing the Find::find stuff without a body and seeing if that accumulates a ton of memory? If it doesn''t, then you might want to explicitly nil out the @photo after it''s been saved. Likewise, it look like you never close all the Files you are opening. You might want to watch out for that. Finally, how big are these jpgs? Are they super-compressed, so that they actually do take up a ton of memory when loaded? -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Thanks for the replies. Bryan, Soory I''m a bit of a ruby newbie (still!!!), do you mean use some form of syntax for the Find that returns all images as an array, for processing? also @photo = nil (is that what you mean? On Aug 13, 10:17 pm, Bryan Duxbury <rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> Have you tried just doing the Find::find stuff without a body and seeing > if that accumulates a ton of memory? If it doesn''t, then you might want > to explicitly nil out the @photo after it''s been saved. Likewise, it > look like you never close all the Files you are opening. You might want > to watch out for that. > > Finally, how big are these jpgs? Are they super-compressed, so that they > actually do take up a ton of memory when loaded? > -- > Posted viahttp://www.ruby-forum.com/.--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Actually, on further testing it does look like it the ''self.pictures << @photo '' that eats up all the memory. Yes, each image is very large. So time to look for a new solution. Shame. On Aug 13, 10:32 pm, kevin evans <kwev...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Thanks for the replies. > > Bryan, Soory I''m a bit of a ruby newbie (still!!!), do you mean use > some form of syntax for the Find that returns all images as an array, > for processing? > > also @photo = nil (is that what you mean? > > On Aug 13, 10:17 pm, Bryan Duxbury <rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> > wrote: > > > Have you tried just doing the Find::find stuff without a body and seeing > > if that accumulates a ton of memory? If it doesn''t, then you might want > > to explicitly nil out the @photo after it''s been saved. Likewise, it > > look like you never close all the Files you are opening. You might want > > to watch out for that. > > > Finally, how big are these jpgs? Are they super-compressed, so that they > > actually do take up a ton of memory when loaded? > > -- > > Posted viahttp://www.ruby-forum.com/.--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
kevin evans wrote:> Thanks for the replies. > > Bryan, Soory I''m a bit of a ruby newbie (still!!!), do you mean use > some form of syntax for the Find that returns all images as an array, > for processing? > > also @photo = nil (is that what you mean? > > On Aug 13, 10:17 pm, Bryan Duxbury <rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org>In my first suggestion, I''m saying that you should try Find::find("#{SOURCE_PATH}/#{folder}") do |path| end and watch the memory usage of just that alone and see if it''s poor. I''m not familiar with that library personally, so I''d want to verify that it''s not the memory hog. You have it right on the other one, @photo = nil will make sure that reference goes away which could lead to quicker garbage collection. Actually, it looks like @photo should really be a local variable, since it''s lifetime should only be within the Find::find block, so drop the @ from @photo. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---