what''s the rational behind not letting the find method receive a block
as in
@students = []
Student.find(:all) do |student|
@students << student if student.age < rand(42)
end
are there development plans for the omitted block to these functions? to me
it seems natural to use it for a cursor/iterator function that would allow
very complex post filtering of results and, most notably, early break in cases
where a big query must be returned but not results would actually be used.
sorry if this has come up before.
kind regards.
-a
--
==============================================================================|
ara [dot] t [dot] howard [at] noaa [dot] gov
| all happiness comes from the desire for others to be happy. all misery
| comes from the desire for oneself to be happy.
| -- bodhicaryavatara
===============================================================================
You can do Student.find(:all).each do |student| Also instead of pulling all the student records and then generating that list, you could do an SQL query instead. Unfortunately I don''t know the exact syntax off the top of my head, but I know it''s been covered a lot on the list. Pat On 12/4/05, Ara.T.Howard <ara.t.howard-32lpuo7BZBA@public.gmane.org> wrote:> > what''s the rational behind not letting the find method receive a block as in > > @students = [] > > Student.find(:all) do |student| > @students << student if student.age < rand(42) > end > > are there development plans for the omitted block to these functions? to me > it seems natural to use it for a cursor/iterator function that would allow > very complex post filtering of results and, most notably, early break in cases > where a big query must be returned but not results would actually be used. > > sorry if this has come up before. > > kind regards. > > -a > -- > ==============================================================================> | ara [dot] t [dot] howard [at] noaa [dot] gov > | all happiness comes from the desire for others to be happy. all misery > | comes from the desire for oneself to be happy. > | -- bodhicaryavatara > ==============================================================================> > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
Also, I''m sure that the rationale is probably similar to why you can''t do @students = [] @students do |student| :) On 12/4/05, Ara.T.Howard <ara.t.howard-32lpuo7BZBA@public.gmane.org> wrote:> > what''s the rational behind not letting the find method receive a block as in > > @students = [] > > Student.find(:all) do |student| > @students << student if student.age < rand(42) > end > > are there development plans for the omitted block to these functions? to me > it seems natural to use it for a cursor/iterator function that would allow > very complex post filtering of results and, most notably, early break in cases > where a big query must be returned but not results would actually be used. > > sorry if this has come up before. > > kind regards. > > -a > -- > ==============================================================================> | ara [dot] t [dot] howard [at] noaa [dot] gov > | all happiness comes from the desire for others to be happy. all misery > | comes from the desire for oneself to be happy. > | -- bodhicaryavatara > ==============================================================================> > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
On Sun, 4 Dec 2005, Pat Maddox wrote:> You can do > Student.find(:all).each do |student| > > Also instead of pulling all the student records and then generating that > list, you could do an SQL query instead. Unfortunately I don''t know the > exact syntax off the top of my head, but I know it''s been covered a lot on > the list. > > Patthis is very different. for example Student.find(:all).each do |student| end this says find all 1,000,000,000,000 students as an array. then run each(&block) over that entire result set. it''s __exactly__ equiv to this: students = Student::find :all students.each do |student| end and saves you nothing at all in terms of speed or space. allowing #find to take a block solves a different problem than selecting by sql/condition. for example, say you''re writing a wilderness 911 website that will iterate over a list of volunteer responders and attempt to contact each one via a text message. as soon as a single responder is found and contaceted he will be recorded as being on the case. so you might say r = nil Responder::find(:all) do |responder| if can_contact(responder.contact_info) r = responder and break end end # do something with r now imagine the responder table has a million records. in this case you only keep __one__ record in memory at a time and can break as soon as your condition is met. you cannot do this kind of logic with sql so a block makes filtering your results after sql/select possible and has the potential to reduce memory usage in many cases. regards. -a -- ==============================================================================| ara [dot] t [dot] howard [at] noaa [dot] gov | all happiness comes from the desire for others to be happy. all misery | comes from the desire for oneself to be happy. | -- bodhicaryavatara ===============================================================================
On Dec 4, 2005, at 11:30 AM, Ara.T.Howard wrote:> > what''s the rational behind not letting the find method receive a > block as in > > @students = [] > > Student.find(:all) do |student| > @students << student if student.age < rand(42) > end > > are there development plans for the omitted block to these > functions? to me > it seems natural to use it for a cursor/iterator function that > would allow > very complex post filtering of results and, most notably, early > break in cases > where a big query must be returned but not results would actually > be used. > > sorry if this has come up before. > > kind regards. > > -a > --I like this idea Ara. I would use this as a patch if you made one. I don''t know if there is some reason to not allow this though. Cheers- -Ezra Zygmuntowicz WebMaster Yakima Herald-Republic Newspaper ezra-gdxLOakOTQ9oetBuM9ipNAC/G2K4zDHf@public.gmane.org 509-577-7732
> and saves you nothing at all in terms of speed or space. allowing #find to > take a block solves a different problem than selecting by sql/condition. for > example, say you''re writing a wilderness 911 website that will iterate over a > list of volunteer responders and attempt to contact each one via a text > message. as soon as a single responder is found and contaceted he will be > recorded as being on the case. so you might say > > r = nil > > Responder::find(:all) do |responder| > if can_contact(responder.contact_info) > r = responder and break > end > end > > # do something with r > > now imagine the responder table has a million records. in this case you only > keep __one__ record in memory at a time and can break as soon as your > condition is met. you cannot do this kind of logic with sql so a block makes > filtering your results after sql/select possible and has the potential to > reduce memory usage in many cases.I think you''re missing the point on how SQL databases work. Are you suggesting that we make n queries to pull n items out of the table? Regardless of memory usage, this is probably a bad idea, since SQL communications are expensive compared to, say, garbage collection. A good solution within the current framework would be to use the limit and offset specifiers to grab, say 100 entries at a time and iterate over them the "old" way. There''s no meaningful way to abort an in-progress query over SQL, nor, as I understand the DBI, incrementally unpack a query (though a lazy database interface would be pretty neat, I don''t know that it is well defined with regard to ALL supported database backends). Brian
On Mon, 5 Dec 2005, Brian L. wrote:> I think you''re missing the point on how SQL databases work. Are you > suggesting that we make n queries to pull n items out of the table?no. in fact most ruby database apis work to support this already. for example: harp:~ > cat a.rb require "sqlite" db = SQLite::Database::new "db", 0 p db.execute("select * from foo").size db.execute("select * from foo") do |tuple| p tuple break end harp:~ > ruby a.rb 3 {0=>"1", "x"=>"1"} this is one query and saves __massively__ on memory for large queries. try something like the above in sqlite or postgres with a result set of 1,000,000 rows and check the memory usage on your machine. by using an iterator (block) you will see zero memory usage. if you merely say result = db.execute sql memory will vanish.> Regardless of memory usage, this is probably a bad idea, since SQL > communications are expensive compared to, say, garbage collection. A good > solution within the current framework would be to use the limit and offset > specifiers to grab, say 100 entries at a time and iterate over them the > "old" way.this is very dangerous unless done within a transaction since the queries will not be isolated.> There''s no meaningful way to abort an in-progress query over SQL, > nor, as I understand the DBI, incrementally unpack a query (though a lazy > database interface would be pretty neat, I don''t know that it is well > defined with regard to ALL supported database backends).see example above. for databases which don''t support it can easily be made to support the same interface: def execute sql if block_given? conn.execute(sql).each{|tuple| yield tuple} else conn.execute(sql) end end regards. -a -- ==============================================================================| ara [dot] t [dot] howard [at] noaa [dot] gov | all happiness comes from the desire for others to be happy. all misery | comes from the desire for oneself to be happy. | -- bodhicaryavatara ===============================================================================