Hello, I have ten CVS file with about 20000 rows in each. If I want to search in these files for example get every row that has the word "apple" i a column, which is faster: If I read all these ten files into the database and then making search in the database or If I search directly in the files for this inforamtion? Thank you! Please motivate your answer and include a code to make the search. Best regards, Mark -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Frederick Cheung
2007-Dec-26 16:34 UTC
Re: Which is faster: searching in a file or in a database
On 26 Dec 2007, at 16:04, Mark Toth wrote:> > Hello, > > I have ten CVS file with about 20000 rows in each. > If I want to search in these files for example get every row that has > the word "apple" i a column, which is faster: > > If I read all these ten files into the database and then making search > in the database > > or > > If I search directly in the files for this inforamtion? > > Thank you! Please motivate your answer and include a code to make the > search. >Assuming you aren''t processing a fresh set of csv files, I''d be very surprised if parsing the csv files in ruby each time wasn''t slower than sticking it in the database. It''s what databases are supposed to do. That''s just my hunch though, and I''m afraid I''ve got enough of my own work to do before doing yours :-) Fred> Best regards, > > Mark > -- > Posted via http://www.ruby-forum.com/. > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Frederick Cheung wrote:> On 26 Dec 2007, at 16:04, Mark Toth wrote: > >> or >> >> If I search directly in the files for this inforamtion? >> >> Thank you! Please motivate your answer and include a code to make the >> search. >> > Assuming you aren''t processing a fresh set of csv files, I''d be very > surprised if parsing the csv files in ruby each time wasn''t slower > than sticking it in the database. It''s what databases are supposed to > do. That''s just my hunch though, and I''m afraid I''ve got enough of my > own work to do before doing yours :-) > > FredThank you for your answer. Just another quick question. I am using fasterCSV to read the file and a column looks like: This is the first line This is the "second" line This is the third line I get an error on the second line from fasterCSV since it includes " char. Can you help me how I can avvoid this problem? Thanks, Mark -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Exactly I get this error: "Unclosed quoted field on line 21" -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Bala Paranj
2007-Dec-26 20:48 UTC
Re: Which is faster: searching in a file or in a database
You have to do the validation and have a valid CSV file to parse it correctly. Correct the file and try again. On Dec 26, 2007 11:20 AM, Mark Toth <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> > Exactly I get this error: > > "Unclosed quoted field on line 21" > >-- http://www.rubyplus.org/ Free Ruby and Rails Screencasts --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Bala Paranj wrote:> You have to do the validation and have a valid CSV file to parse it > correctly. Correct the file and try again. > On Dec 26, 2007 11:20 AM, Mark Toth <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> > wrote: > >> >> Exactly I get this error: >> >> "Unclosed quoted field on line 21" >> >> > -- > http://www.rubyplus.org/ > Free Ruby and Rails ScreencastsWell, I have a correct CSV file, but with some small changes, for example I didn´t use " for the strings. I have this column: This is the first line This is the "second" line This is the third line How can I read this information to my database? Is there any command to replace the " char in the whole file with nothing? -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Greg Donald
2007-Dec-27 15:50 UTC
Re: Which is faster: searching in a file or in a database
On 12/27/07, Mark Toth <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> How can I read this information to my database? Is there any command to > replace the " char in the whole file with nothing?In bash, using a temp file, it''d be something like: cp foo.csv foo.csv.tmp sed -e ''s/"//g'' foo.csv.tmp >foo.csv rm foo.csv.tmp -- Greg Donald http://destiney.com/ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Mark Wilden
2007-Dec-28 03:04 UTC
Re: Which is faster: searching in a file or in a database
On Dec 27, 5:30 am, Mark Toth <rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> How can I read this information to my database? Is there any command to > replace the " char in the whole file with nothing?There''s always a text editor. :) Seriously, using the IO class to 1) read each line of the input file, 2) use gsub to remove the quotes, and 3) write each line of the file, sounds to me like a useful exercise. ///ark --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---