search for: max_buffer_memory

...erret, so this might be a silly question. I was wondering, where can I find details about search memory usage? I read the O''Reilly booklet + googled but couldn''t find much info. There is a good explanation of how memory is used at indexing time [bound by, amongst other things, :max_buffer_memory and :max_buffered_docs]. But how does it work at search time - do the same options apply? Will parts of the index be cached as they are accessed? How about search results, do they get cached until :max_buffer_memory and/or :max_buffered_docs is reached? I understand that the OS will perform page...

...9;s a problem in result memory allocation, i don''t know what i could add to help you, i''m trying to reproduce it in a little code... Ah, just another detail, this is how our index is created : INDEX_OPTIONS = { :path => path, :auto_flush => false, :max_buffer_memory => 0x4000000, :max_buffered_docs => 100000, :use_compound_file => false} INDEX = Ferret::Index::Index.new(INDEX_OPTIONS) but i tried without any max_* and it''s the same... Tell me what i can do to help if i don''t manage to reproduce it in a pastable cod...

I have got my database in Mysql. I used ferret to index a table with 10 million rows. On limiting the selection of data to 1000 initial retrieval, it takes 200 seconds but for the whole table it took more than four hours and after which i had to close my indexing application. I used the StandardAnalyser for it. There is no problem from the database side as retrieval of all the data in the table