Hi-
I asked this question over on Ruby Forum, but it never made it to this
list, so I''m posting it directly now-
==
I''ve got an index with over 11M entries. Both server and clients are
running Ferret gem 0.11.4.
If I do "UserMessage.find_id_by_contents(''kenn*'', :limit
=> 49)", it
works fine. If I do "
UserMessage.find_id_by_contents(''kenn*'', :limit
=> 50)", it throws an exception, like this:
UserMessage.find_id_by_contents ''kenn*'', :limit => 50
IOError: IO Error occured at <except.c>:93 in xraise
Error occured in fs_store.c:293 - fsi_seek_i
seeking pos -1473943740: <Invalid argument>
from (druby://ferret.lingr.com:9009) /usr/lib/ruby/gems/1.8/
gems/ferret-0.11.4/lib/ferret/index.rb:411:in `[]''
from (druby://ferret.lingr.com:9009) /usr/lib/ruby/gems/1.8/
gems/ferret-0.11.4/lib/ferret/index.rb:411:in `[]''
from (druby://ferret.lingr.com:9009) /usr/lib/ruby/1.8/
monitor.rb:229:in `synchronize''
from (druby://ferret.lingr.com:9009) /usr/lib/ruby/gems/1.8/
gems/ferret-0.11.4/lib/ferret/index.rb:403:in `[]''
... yada yada yada ...
Any idea what I can do to recover from this? Rebuilding this index
would take a LOOOOOOOOOOOOONG time :-)
- Danny Burkes
http://www.lingr.com/help/about#danny
Hi, I''m not sure but the negative seeking pos could be caused by an overflow of some sort. Maybe a large file problem? What does your index look like with ''ls -l''? Could you try reproducing this on a copy of your index with a small script with plain Ferret, and then retry with 0.11.6? Cheers, Jens On Wed, Feb 06, 2008 at 09:33:00AM -0800, Danny Burkes wrote:> Hi- > > I asked this question over on Ruby Forum, but it never made it to this > list, so I''m posting it directly now- > > ==> > I''ve got an index with over 11M entries. Both server and clients are > running Ferret gem 0.11.4. > > If I do "UserMessage.find_id_by_contents(''kenn*'', :limit => 49)", it > works fine. If I do " UserMessage.find_id_by_contents(''kenn*'', :limit > => 50)", it throws an exception, like this: > > UserMessage.find_id_by_contents ''kenn*'', :limit => 50 > IOError: IO Error occured at <except.c>:93 in xraise > Error occured in fs_store.c:293 - fsi_seek_i > seeking pos -1473943740: <Invalid argument> > from (druby://ferret.lingr.com:9009) /usr/lib/ruby/gems/1.8/ > gems/ferret-0.11.4/lib/ferret/index.rb:411:in `[]'' > from (druby://ferret.lingr.com:9009) /usr/lib/ruby/gems/1.8/ > gems/ferret-0.11.4/lib/ferret/index.rb:411:in `[]'' > from (druby://ferret.lingr.com:9009) /usr/lib/ruby/1.8/ > monitor.rb:229:in `synchronize'' > from (druby://ferret.lingr.com:9009) /usr/lib/ruby/gems/1.8/ > gems/ferret-0.11.4/lib/ferret/index.rb:403:in `[]'' > ... yada yada yada ... > > Any idea what I can do to recover from this? Rebuilding this index > would take a LOOOOOOOOOOOOONG time :-) > > - Danny Burkes > http://www.lingr.com/help/about#danny > _______________________________________________ > Ferret-talk mailing list > Ferret-talk at rubyforge.org > http://rubyforge.org/mailman/listinfo/ferret-talk >-- Jens Kr?mer Finkenlust 14, 06449 Aschersleben, Germany VAT Id DE251962952 http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database
> I''m not sure but the negative seeking pos could be caused by an > overflow > of some sort. Maybe a large file problem? What does your index look > like > with ''ls -l''? >I''ve attached the listing below...> Could you try reproducing this on a copy of your index with a small > script with plain Ferret, and then retry with 0.11.6? >Will do- I''ll post the results here soon. Best Regards, Danny == dburkes at ferret.lingr.com:/var/ferret/production/user_message$ ls -l total 4041936 -rw------- 1 root root 3160 2007-08-10 07:01 _2ti3z.fdt.saved -rw------- 1 root root 8 2007-08-10 07:01 _2ti3z.fdx.saved -rw------- 1 root root 1326528 2008-02-07 01:17 _7bgxh_5d3.del -rw------- 1 root root 3861431389 2008-01-27 08:36 _7bgxh.cfs -rw------- 1 root root 10067 2008-02-03 11:17 _7dutf_5.del -rw------- 1 root root 37559822 2008-01-28 22:19 _7dutf.cfs -rw------- 1 root root 37273935 2008-01-30 06:41 _7g8n1.cfs -rw------- 1 root root 11371 2008-01-31 17:30 _7g8n1_ox.del -rw------- 1 root root 11559 2008-02-02 20:57 _7imep_2.del -rw------- 1 root root 35903751 2008-01-31 14:44 _7imep.cfs -rw------- 1 root root 36105008 2008-02-02 04:11 _7l096.cfs -rw------- 1 root root 10511 2008-02-03 18:26 _7l096_g.del -rw------- 1 root root 11659 2008-02-03 20:12 _7ne15_0.del -rw------- 1 root root 36128119 2008-02-03 13:04 _7ne15.cfs -rw------- 1 root root 34444491 2008-02-05 03:40 _7prs3.cfs -rw------- 1 root root 36773717 2008-02-06 13:41 _7s5pu.cfs -rw------- 1 root root 3380507 2008-02-06 18:49 _7seal.cfs -rw------- 1 root root 3260942 2008-02-06 23:37 _7smva.cfs -rw------- 1 root root 54 2008-02-07 04:55 _7svfx_0.del -rw------- 1 root root 3590095 2008-02-07 04:01 _7svfx.cfs -rw------- 1 root root 4030326 2008-02-07 06:32 _7t40k.cfs -rw------- 1 root root 347713 2008-02-07 06:48 _7t4vf.cfs -rw------- 1 root root 348108 2008-02-07 07:03 _7t5qa.cfs -rw------- 1 root root 359057 2008-02-07 07:16 _7t6l5.cfs -rw------- 1 root root 353181 2008-02-07 07:28 _7t7g0.cfs -rw------- 1 root root 397144 2008-02-07 07:41 _7t8av.cfs -rw------- 1 root root 343822 2008-02-07 07:52 _7t95q.cfs -rw------- 1 root root 348366 2008-02-07 08:04 _7ta0l.cfs -rw------- 1 root root 355600 2008-02-07 08:14 _7tavg.cfs -rw------- 1 root root 365283 2008-02-07 08:26 _7tbqb.cfs -rw------- 1 root root 34225 2008-02-07 08:28 _7tbte.cfs -rw------- 1 root root 36869 2008-02-07 08:29 _7tbwh.cfs -rw------- 1 root root 37095 2008-02-07 08:30 _7tbzk.cfs -rw------- 1 root root 36708 2008-02-07 08:31 _7tc2n.cfs -rw------- 1 root root 36468 2008-02-07 08:32 _7tc5q.cfs -rw------- 1 root root 38334 2008-02-07 08:33 _7tc8t.cfs -rw------- 1 root root 4941 2008-02-07 08:33 _7tc94.cfs -rw------- 1 root root 5270 2008-02-07 08:34 _7tc9f.cfs -rw------- 1 root root 5070 2008-02-07 08:34 _7tc9q.cfs -rw------- 1 root root 6055 2008-02-07 08:34 _7tca1.cfs -rw------- 1 root root 5695 2008-02-07 08:34 _7tcac.cfs -rw------- 1 root root 0 2008-02-07 07:27 ferret- write.lck -rw------- 1 dburkes dburkes 16 2008-02-07 08:34 segments -rw------- 1 root root 733 2008-02-07 08:34 segments_832jn
On Thu, Feb 07, 2008 at 08:44:44AM -0800, Danny Burkes wrote: [..]> dburkes at ferret.lingr.com:/var/ferret/production/user_message$ ls -l > total 4041936 > -rw------- 1 root root 3160 2007-08-10 07:01 > _2ti3z.fdt.saved > -rw------- 1 root root 8 2007-08-10 07:01 > _2ti3z.fdx.saved > -rw------- 1 root root 1326528 2008-02-07 01:17 _7bgxh_5d3.del > -rw------- 1 root root 3861431389 2008-01-27 08:36 _7bgxh.cfs[..] Ok, might be a large file issue. There has been at least one patch [1] after 0.11.4 was released regarding LFS, so upgrading to 0.11.6 might indeed help. 0.11.4 and 0.11.6 are index compatible, so no rebuild should be necessary. Btw, from your directory listing it looks like you''re running the DRb server as root - please *don''t* do that. Given proper write access to the index and log directories it''s not necessary at all. I usually run it as the same (non-root) user that''s running the Mongrels. Cheers, Jens [1] http://ferret.davebalmain.com/trac/ticket/215 -- Jens Kr?mer webit! Gesellschaft f?r neue Medien mbH Schnorrstra?e 76 | 01069 Dresden Telefon +49 351 46766-0 | Telefax +49 351 46766-66 kraemer at webit.de | www.webit.de Amtsgericht Dresden | HRB 15422 GF Sven Haubold
> Ok, might be a large file issue. There has been at least one patch [1] > after 0.11.4 was released regarding LFS, so upgrading to 0.11.6 might > indeed help. > > 0.11.4 and 0.11.6 are index compatible, so no rebuild should be > necessary. >Thanks, Jens! I upgraded to the 0.11.6 gem, and the problem went away. Best Regards, Danny