Hello everyone,
I was stress-testing my application (running on Rails via FastCGI) by
letting two concurrent users (not human .. an app called
''siege'')
a) save an Article and b) search for all Articles.
I am searching via
Article.ferret_index.search_each( ..) do |doc_id,score|
doc = index[doc_id]
..
end
and writing via
Article.ferret_index << self.to_doc
where Article.ferret_index is implemented as in
''act_as_ferret'':
@@ferret_index = nil
def Article::ferret_index
@@ferret_index ||= Ferret::Index::Index.new( :path => ferret_path,
:auto_flush => true, :create_if_missing => false )
end
The 2 errors I got (when I do "doc = index[doc_id]") were :
ArgumentError (:12250 is out of range [0..12243] for IndexWriter#[]):
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in
`[]''
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in
`[]''
/usr/lib/ruby/1.8/monitor.rb:229:in `synchronize''
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in
`[]''
...
Ferret::StateError (State Error occured at <except.c>:79 in xraise
Error occured in index.c:3404 - sr_get_lazy_doc
Document 0 has already been deleted
):
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in
`[]''
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in
`[]''
/usr/lib/ruby/1.8/monitor.rb:229:in `synchronize''
/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in
`[]''
...
So, obviously the index changed after #search_each and before "doc =
index[doc_id]".
Is this expected behaviour? How to avoid this?
Then I did the same thing without a webserver, just 2 consoles. One for
saving, one for searching.
The one searching now just ends with
"/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:363:
[BUG] Segmentation fault"
As for now, to avoid the problem, I use an IndexReader that I newly
create for every search.
But I guess that is not the best approach?
I am using ferret 0.10.8 and ruby 1.8.4 on Debian Sarge.
Best regards
josh
--
Posted via http://www.ruby-forum.com/.
David Balmain
2006-Sep-27 13:13 UTC
[Ferret-talk] concurrency / #search_each problem / segfault
On 9/27/06, Josh D. <josh.nug at gmail.com> wrote:> Hello everyone, > > I was stress-testing my application (running on Rails via FastCGI) by > letting two concurrent users (not human .. an app called ''siege'') > a) save an Article and b) search for all Articles. > > I am searching via > Article.ferret_index.search_each( ..) do |doc_id,score| > doc = index[doc_id] > .. > end > > and writing via > Article.ferret_index << self.to_doc > > where Article.ferret_index is implemented as in ''act_as_ferret'': > @@ferret_index = nil > def Article::ferret_index > @@ferret_index ||= Ferret::Index::Index.new( :path => ferret_path, > :auto_flush => true, :create_if_missing => false ) > end > > The 2 errors I got (when I do "doc = index[doc_id]") were : > ArgumentError (:12250 is out of range [0..12243] for IndexWriter#[]): > /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in > `[]'' > /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in > `[]'' > /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize'' > /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in > `[]'' > ... > > Ferret::StateError (State Error occured at <except.c>:79 in xraise > Error occured in index.c:3404 - sr_get_lazy_doc > Document 0 has already been deleted > > ): > /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in > `[]'' > /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in > `[]'' > /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize'' > /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in > `[]'' > ... > > So, obviously the index changed after #search_each and before "doc > index[doc_id]". > Is this expected behaviour? How to avoid this? > > Then I did the same thing without a webserver, just 2 consoles. One for > saving, one for searching. > The one searching now just ends with > "/usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:363: > [BUG] Segmentation fault" > > As for now, to avoid the problem, I use an IndexReader that I newly > create for every search. > But I guess that is not the best approach? > > I am using ferret 0.10.8 and ruby 1.8.4 on Debian Sarge. > > > Best regards > > josh >Hi Josh, Just thought I''d let you know that I''m working on fixing this. Expect a solution in the next release. Cheers, Dave
David Balmain
2006-Sep-27 13:21 UTC
[Ferret-talk] concurrency / #search_each problem / segfault
On 9/27/06, Josh D. <josh.nug at gmail.com> wrote:> Hello everyone, > > I was stress-testing my application (running on Rails via FastCGI) by > letting two concurrent users (not human .. an app called ''siege'') > a) save an Article and b) search for all Articles. > > I am searching via > Article.ferret_index.search_each( ..) do |doc_id,score| > doc = index[doc_id] > .. > endHi Josh, Just to you this time. What is the rest of the code in this loop (above). ie, what is the "..". It should help me sort out the problem. By the way, you should upgrade to Ferret-0.10.9. Cheers, Dave
Hi Dave,> Just to you this time. What is the rest of the code in this loop > (above). ie, what is the "..". It should help me sort out the problem.here are the complete stack traces: ArgumentError (:12250 is out of range [0..12243] for IndexWriter#[]): /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in `[]'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in `[]'' /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in `[]'' /app/models/article.rb:150:in `fulltext_search'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:364:in `search_each'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:363:in `search_each'' /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:359:in `search_each'' /app/models/article.rb:149:in `fulltext_search'' /app/controllers/public/article_controller.rb:91:in `search'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:941:in `perform_action_without_filters'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:368:in `perform_action_without_benchmark'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue'' /usr/local/lib/site_ruby/benchmark.rb:300:in `measure'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/rescue.rb:82:in `perform_action'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:408:in `process_without_filters'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:377:in . `process_without_session_management_support'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/session_management.rb:117:in `process'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispatcher.rb:38:in `dispatch'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:150:in `process_request'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:54:in `process!'' /usr/lib/ruby/1.8/fcgi.rb:600:in `each_cgi'' /usr/lib/ruby/1.8/fcgi.rb:597:in `each_cgi'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:53:in `process!'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:23:in `process!'' Ferret::StateError (State Error occured at <except.c>:79 in xraise Error occured in index.c:3404 - sr_get_lazy_doc Document 0 has already been deleted ): /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in `[]'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:382:in `[]'' /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:375:in `[]'' /app/models/article.rb:150:in `fulltext_search'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:364:in `search_each'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:363:in `search_each'' /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize'' /usr/lib/ruby/gems/1.8/gems/ferret-0.10.8/lib/ferret/index.rb:359:in `search_each'' /app/models/article.rb:149:in `fulltext_search'' /app/controllers/public/article_controller.rb:91:in `search'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:941:in `perform_action_without_filters'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:368:in `perform_action_without_benchmark'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue'' /usr/local/lib/site_ruby/benchmark.rb:300:in `measure'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/rescue.rb:82:in `perform_action'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:408:in `process_without_filters'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:377:in . `process_without_session_management_support'' /usr/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/session_management.rb:117:in `process'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispatcher.rb:38:in `dispatch'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:150:in `process_request'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:54:in `process!'' /usr/lib/ruby/1.8/fcgi.rb:600:in `each_cgi'' /usr/lib/ruby/1.8/fcgi.rb:597:in `each_cgi'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:53:in `process!'' /usr/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/fcgi_handler.rb:23:in `process!'' Hope that helps.> By the way, you should upgrade to Ferret-0.10.9.Ok, will do that. Thank you josh -- Posted via http://www.ruby-forum.com/.