I didn''t know how to title this question sorry :P I wrote this application using a MySQL DB and Solr Server (updated via acts_as_solr plugin) I need to load tons of documents to test it out, the app is a newspaper article collection repository that I upload via SGML corpus. Now the strange is..that after loading about 30K articles..the app started to act weird: it does not load anymore or it does after a lot of minutes..yesterday I''ve kept loading..it reached the 56K articles..but now is stuck again... In testing this on a different machine that I use from remote.. any clue? (the solr server is perfectly reachable...) thx, Andrea -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Jeffrey L. Taylor
2010-Feb-07 19:40 UTC
Re: How much "records" can development env stands?!
Quoting adedip <adedip-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:> I didn''t know how to title this question sorry :P > > I wrote this application using a MySQL DB and Solr Server (updated via > acts_as_solr plugin) > > I need to load tons of documents to test it out, the app is a > newspaper article collection repository that I upload via SGML corpus. > > Now the strange is..that after loading about 30K articles..the app > started to act weird: it does not load anymore or it does after a lot > of minutes..yesterday I''ve kept loading..it reached the 56K > articles..but now is stuck again... > > In testing this on a different machine that I use from remote.. any > clue? > > (the solr server is perfectly reachable...) >I have a development system with over 5 million rows in one table. The string field is typically 5-50 characters, the rest is a half dozen integers of different sizes. The production system has over 6 million rows. That said, ActiveRecord is convenient and memory greedy. For that afore mentioned table, I just use straight SQL (inserts and selects can run to thousands of rows). For the other tables, I continue to use ActiveRecord. HTH, Jeffrey -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Marnen Laibow-Koser
2010-Feb-07 19:47 UTC
Re: How much "records" can development env stands?!
Jeffrey L. Taylor wrote:> Quoting adedip <adedip-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>: >> of minutes..yesterday I''ve kept loading..it reached the 56K >> articles..but now is stuck again... >> >> In testing this on a different machine that I use from remote.. any >> clue? >> >> (the solr server is perfectly reachable...) >> > I have a development system with over 5 million rows in one table. The > string > field is typically 5-50 characters, the rest is a half dozen integers of > different sizes. The production system has over 6 million rows. > > That said, ActiveRecord is convenient and memory greedy. For that afore > mentioned table, I just use straight SQL (inserts and selects can run to > thousands of rows).Have you tried ar-extensions? This should let you do bulk queries properly.> For the other tables, I continue to use > ActiveRecord. > > HTH, > JeffreyBest, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.