Hi all, A short while ago some of us were wondering on rails-contrib/here about the feasibility of Base.find returning a proxy object and cool things that we could do with that. I''ve been toying with a small plugin that adds some of that stuff. It''s now in a state where it might be fun for people to play with (Rails'' unit tests basically past with this plugin loaded. There are a few failures, but nothing super interesting) What there''s right now: posts = Post.find :all posts.load :comments => :author #preloads the comments association and the author from comments. Same syntax as your normal :include/ :joins conditions posts.comments.first #=> triggers an eager load of comments on all posts posts.comments.first.author #=> triggers an eager load of the author of all comments Of the quirks I am aware of/anticipate the biggest cause is the fact that in some cases eager loading a collection behaves slightly differently to loading it normally. (for example :order on has many through is a bit off). With this plugin installed basically every association load is an eager load so no hiding from any of those issues. There are interesting questions around garbage collection: because each record references the collection it was part of, keeping one of them around keeps them all around. Playing with WeakRef is a possibility (but right now that would actually break some of the functionality). Detaching all objects with each request cycle is another as is figuring what patterns indicate that the user probably does want to detach the record and do that automatically. I feel that picking one of the options right now would be premature. Caveats/Gotchas/etc: - Requires Rails 2.1 - Anything before http://github.com/rails/rails/commit/b7a37b742c0abd1df8ea48cc82f76385cc0c41ea (sometime last week) will die horribly if you try and load a has_one through (because preload of has_one :through is borked before that). - has_many associations with a :limit option are excluded because they can''t be eager loaded nicely. - has_many :through with options/conditions on the join table will probably be screwed up: has_many :foos, :conditions => ... has_many :bars, :through => :foos, :conditions => something only on bars should be ok has_many :foos has_many :bars, :through => :foos, :conditions => something on foos probably won''t work Code: http://github.com/fcheung/ar_result_set/tree I''m at railsconf so grab me/ smack me on the face if you feel so inclined Fred --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
Just to follow up on this (not that I''ve had time to touch this since last week), what do people think we could do with this that would be useful ? Fred --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On Sep 9, 7:53 pm, Frederick Cheung <frederick.che...@gmail.com> wrote:> Just to follow up on this (not that I''ve had time to touch this since > last week), what do people think we could do with this that would be > useful ? > > FredIt might be useful for lazy loading from the database inside cached view fragments. One would normally only want to hit the database if the data inside a view fragment isn''t in the cache. Have you had any feedback on RailsConf? --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On Sep 10, 12:57 pm, Hongli Lai <hongli...@gmail.com> wrote:> On Sep 9, 7:53 pm, Frederick Cheung <frederick.che...@gmail.com> > wrote: > > > Just to follow up on this (not that I''ve had time to touch this since > > last week), what do people think we could do with this that would be > > useful ? > > > Fred > > It might be useful for lazy loading from the database inside cached > view fragments. One would normally only want to hit the database if > the data inside a view fragment isn''t in the cache. > > Have you had any feedback on RailsConf?Ah wait, I just realized that such a proxy could be useful in very large result sets. The proxy could only fetch a small amount into memory at a time, e.g. by using cursors. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 10 Sep 2008, at 11:58, Hongli Lai wrote:> > On Sep 10, 12:57 pm, Hongli Lai <hongli...@gmail.com> wrote: >> On Sep 9, 7:53 pm, Frederick Cheung <frederick.che...@gmail.com> >> wrote: >> >>> Just to follow up on this (not that I''ve had time to touch this >>> since >>> last week), what do people think we could do with this that would be >>> useful ? >> >>> Fred >> >> It might be useful for lazy loading from the database inside cached >> view fragments. One would normally only want to hit the database if >> the data inside a view fragment isn''t in the cache. >> >> Have you had any feedback on RailsConf?Yeah, I chatted to Koz about it. I''m planning to play around with identity maps and stuff like that in conjunction with that (not that I have very definite plans - more of a lets toss all these things in the air and see what happens :-) )>> > > Ah wait, I just realized that such a proxy could be useful in very > large result sets. The proxy could only fetch a small amount into > memory at a time, e.g. by using cursors.That''s certainly a thought (but I have to confess ignorance as far as cursors go, beyond the dictionary definition) Fred> > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On Sep 10, 1:14 pm, Frederick Cheung <frederick.che...@gmail.com> wrote:> Yeah, I chatted to Koz about it. I''m planning to play around with > identity maps and stuff like that in conjunction with that (not that I > have very definite plans - more of a lets toss all these things in the > air and see what happens :-) ) > > > > > Ah wait, I just realized that such a proxy could be useful in very > > large result sets. The proxy could only fetch a small amount into > > memory at a time, e.g. by using cursors. > > That''s certainly a thought (but I have to confess ignorance as far as > cursors go, beyond the dictionary definition)http://www.postgresql.org/docs/8.1/static/plpgsql-cursors.html On the other hand, I don''t know what identity maps are. :) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 11 Sep 2008, at 10:07, Hongli Lai wrote:> > On Sep 10, 1:14 pm, Frederick Cheung <frederick.che...@gmail.com> > wrote: >> Yeah, I chatted to Koz about it. I''m planning to play around with >> identity maps and stuff like that in conjunction with that (not >> that I >> have very definite plans - more of a lets toss all these things in >> the >> air and see what happens :-) ) >> >> >> >>> Ah wait, I just realized that such a proxy could be useful in very >>> large result sets. The proxy could only fetch a small amount into >>> memory at a time, e.g. by using cursors. >> >> That''s certainly a thought (but I have to confess ignorance as far as >> cursors go, beyond the dictionary definition) > > http://www.postgresql.org/docs/8.1/static/plpgsql-cursors.html >thanks> On the other hand, I don''t know what identity maps are. :)Quite simple in theory - don''t have more than one instance of an activerecord object per row in the database. Plenty of edge cases in practise :-) Fred --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
Hongli Lai wrote:> On Sep 10, 1:14 pm, Frederick Cheung <frederick.che...@gmail.com> > wrote: >> Yeah, I chatted to Koz about it. I''m planning to play around with >> identity maps and stuff like that in conjunction with that (not that I >> have very definite plans - more of a lets toss all these things in the >> air and see what happens :-) ) >> >>> Ah wait, I just realized that such a proxy could be useful in very >>> large result sets. The proxy could only fetch a small amount into >>> memory at a time, e.g. by using cursors. >> That''s certainly a thought (but I have to confess ignorance as far as >> cursors go, beyond the dictionary definition) > > http://www.postgresql.org/docs/8.1/static/plpgsql-cursors.html > > On the other hand, I don''t know what identity maps are. :)http://www.martinfowler.com/eaaCatalog/identityMap.html --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On Sep 10, 2008, at 3:57 AM, Hongli Lai wrote:> On Sep 9, 7:53 pm, Frederick Cheung <frederick.che...@gmail.com> > wrote: >> Just to follow up on this (not that I''ve had time to touch this since >> last week), what do people think we could do with this that would be >> useful ? > > It might be useful for lazy loading from the database inside cached > view fragments. One would normally only want to hit the database if > the data inside a view fragment isn''t in the cache.I was wanting a result set proxy just recently to hang stuff like sorters off off. Might be useful in conjunction with named_scopes. -- Josh Susser http://blog.hasmanythrough.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---