thr3ads.net - Ferret talk - [Ferret-talk] Error on optimize leads to corrupt index? [Apr 2007]

If this information is useful, please help other people find it:
Share via:

Erik Morton

2007-Apr-14 12:46 UTC

[Ferret-talk] Error on optimize leads to corrupt index?

The following exception occurred while trying optimize a large index:

vendor/gems/rdig-0.3.4/lib/rdig/index.rb:46:in `optimize'': End-of- 
File Error occured at <except.c>:93 in xraise (EOFError)
Error occured in store.c:216 - is_refill
         current pos = 0, file length = 0

Now, I get the following error any time I try to create a new index  
on the directory that I was trying to optimize.

Ferret::Index::IndexReader.new("indexes/my_index_dir")
IOError: IO Error occured at <except.c>:93 in xraise
Error occured in fs_store.c:127 - fs_each
         doing ''each'' in : <No such file or directory>

Any thoughts? The directory is definitely there. I''d like to be able  
to recover from this error someone. Your help is very much appreciated.

Erik

David Balmain

2007-Apr-14 22:35 UTC

head link

[Ferret-talk] Error on optimize leads to corrupt index?

On 4/14/07, Erik Morton <eimorton at gmail.com>
wrote:> The following exception occurred while trying optimize a large index:
>
> vendor/gems/rdig-0.3.4/lib/rdig/index.rb:46:in `optimize'': End-of-
> File Error occured at <except.c>:93 in xraise (EOFError)
> Error occured in store.c:216 - is_refill
>          current pos = 0, file length = 0
>
> Now, I get the following error any time I try to create a new index
> on the directory that I was trying to optimize.
>
> Ferret::Index::IndexReader.new("indexes/my_index_dir")
> IOError: IO Error occured at <except.c>:93 in xraise
> Error occured in fs_store.c:127 - fs_each
>          doing ''each'' in : <No such file or
directory>
>
> Any thoughts? The directory is definitely there. I''d like to be
able
> to recover from this error someone. Your help is very much appreciated.
Hi Erik,

Can you give us a listing of what is in the directory? Use `ls -l` so
I can see the size of the files as well.

Cheers,
Dave

-- 
Dave Balmain
http://www.davebalmain.com/

Erik Morton

2007-Apr-15 00:11 UTC

head link

[Ferret-talk] Error on optimize leads to corrupt index?

Dave,
Thanks for getting back to me. Unfortunately I killed the directory.  
There were about 12 files in the directory with a total size of about  
4GB. I have a 40 server index process running tonight. If the problem  
crops up again I''ll keep the output this time! Thanks again.

Erik
On Apr 14, 2007, at 6:35 PM, David Balmain wrote:
> On 4/14/07, Erik Morton <eimorton at gmail.com> wrote:
>> The following exception occurred while trying optimize a large index:
>>
>> vendor/gems/rdig-0.3.4/lib/rdig/index.rb:46:in `optimize'':
End-of-
>> File Error occured at <except.c>:93 in xraise (EOFError)
>> Error occured in store.c:216 - is_refill
>>          current pos = 0, file length = 0
>>
>> Now, I get the following error any time I try to create a new index
>> on the directory that I was trying to optimize.
>>
>> Ferret::Index::IndexReader.new("indexes/my_index_dir")
>> IOError: IO Error occured at <except.c>:93 in xraise
>> Error occured in fs_store.c:127 - fs_each
>>          doing ''each'' in : <No such file or
directory>
>>
>> Any thoughts? The directory is definitely there. I''d like to
be able
>> to recover from this error someone. Your help is very much  
>> appreciated.
>
> Hi Erik,
>
> Can you give us a listing of what is in the directory? Use `ls -l` so
> I can see the size of the files as well.
>
> Cheers,
> Dave
>
> -- 
> Dave Balmain
> http://www.davebalmain.com/
> _______________________________________________
> Ferret-talk mailing list
> Ferret-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/ferret-talk

Erik Morton

2007-Apr-15 04:53 UTC

head link

[Ferret-talk] Error on optimize leads to corrupt index?

Ok, tonight two out of 38 nodes in my index cluster had the same  
error from last night. Here is the error and the contents of the  
indexing directory for each node. Any thoughts are greatly appreciated:

/var/www/apps/search/releases/20070414203450/script/../config/../ 
vendor/gems/rdig-0.3.4/lib/rdig/index.rb:47:in `optimize'': End-of- 
File Error occured at <except.c>:93 in xraise (EOFError)
Error occured in store.c:216 - is_refill
         current pos = 0, file length = 0

[root at domU-12-31-34-00-02-B5 current]# ls -l /mnt/domU-12-31-34-00-02- 
B5/
total 55456
-rw-------  1 initiate initiate 18327540 Apr 14 18:58 _0.cfs
-rw-------  1 initiate initiate 21500604 Apr 14 20:51 _1.cfs
-rw-------  1 initiate initiate   814722 Apr 14 20:51 _1.frq
-rw-------  1 initiate initiate  1080838 Apr 14 20:51 _1.prx
-rw-------  1 initiate initiate       78 Apr 14 20:51 _1.tfx
-rw-------  1 initiate initiate   568278 Apr 14 20:51 _1.tis
-rw-------  1 initiate initiate     8151 Apr 14 20:51 _1.tix
-rw-------  1 initiate initiate     5195 Apr 14 20:51 _1_0.f0
-rw-------  1 initiate initiate     5195 Apr 14 20:51 _1_0.f1
-rw-------  1 initiate initiate     5195 Apr 14 20:51 _1_0.f2
-rw-------  1 initiate initiate     5195 Apr 14 20:51 _1_0.f3
-rw-------  1 initiate initiate     5195 Apr 14 20:51 _1_0.f4
-rw-------  1 initiate initiate     5195 Apr 14 20:51 _1_0.f5
-rw-------  1 initiate initiate     5195 Apr 14 20:51 _1_0.f6
-rw-------  1 initiate initiate     5195 Apr 14 20:51 _1_0.f7
-rw-------  1 initiate initiate  6965872 Apr 14 21:53 _2.cfs
-rw-------  1 initiate initiate   225414 Apr 14 21:53 _2.frq
-rw-------  1 initiate initiate   235601 Apr 14 21:53 _2.prx
-rw-------  1 initiate initiate       73 Apr 14 21:53 _2.tfx
-rw-------  1 initiate initiate   211232 Apr 14 21:53 _2.tis
-rw-------  1 initiate initiate     3129 Apr 14 21:53 _2.tix
-rw-------  1 initiate initiate     2104 Apr 14 21:53 _2_0.f0
-rw-------  1 initiate initiate     2104 Apr 14 21:53 _2_0.f1
-rw-------  1 initiate initiate     2104 Apr 14 21:53 _2_0.f2
-rw-------  1 initiate initiate     2104 Apr 14 21:53 _2_0.f3
-rw-------  1 initiate initiate     2104 Apr 14 21:53 _2_0.f4
-rw-------  1 initiate initiate     2104 Apr 14 21:53 _2_0.f5
-rw-------  1 initiate initiate     2104 Apr 14 21:53 _2_0.f6
-rw-------  1 initiate initiate     2104 Apr 14 21:53 _2_0.f7
-rw-------  1 initiate initiate  6579407 Apr 14 21:52 _3.fdt
-rw-------  1 initiate initiate    27756 Apr 14 21:52 _3.fdx
-rw-------  1 initiate initiate        0 Apr 14 21:52 _3.frq
-rw-------  1 initiate initiate        0 Apr 14 21:52 _3.prx
-rw-------  1 initiate initiate        0 Apr 14 21:52 _3.tfx
-rw-------  1 initiate initiate        0 Apr 14 21:52 _3.tis
-rw-------  1 initiate initiate        0 Apr 14 21:52 _3.tix
-rw-------  1 initiate initiate       16 Apr 14 21:52 segments
-rw-------  1 initiate initiate      212 Apr 14 21:52 segments_2


/var/www/apps/search/releases/20070414203712/script/../config/../ 
vendor/gems/rdig-0.3.4/lib/rdig/index.rb:47:in `optimize'': End-of- 
File Error occured at <except.c>:93 in xraise (EOFError)
Error occured in store.c:216 - is_refill
         current pos = 0, file length = 0
[root at domU-12-31-33-00-03-BC current]# ls -l /mnt/domU-12-31-33-00-03- 
BC/
total 183640
-rw-------  1 initiate initiate 19169762 Apr 14 18:47 _0.cfs
-rw-------  1 initiate initiate 34424829 Apr 14 20:28 _1.cfs
-rw-------  1 initiate initiate 33267878 Apr 14 21:41 _2.cfs
-rw-------  1 initiate initiate   518883 Apr 14 21:41 _2.frq
-rw-------  1 initiate initiate   618308 Apr 14 21:41 _2.prx
-rw-------  1 initiate initiate       77 Apr 14 21:41 _2.tfx
-rw-------  1 initiate initiate   402762 Apr 14 21:41 _2.tis
-rw-------  1 initiate initiate     5929 Apr 14 21:41 _2.tix
-rw-------  1 initiate initiate     4984 Apr 14 21:41 _2_0.f0
-rw-------  1 initiate initiate     4984 Apr 14 21:41 _2_0.f1
-rw-------  1 initiate initiate     4984 Apr 14 21:41 _2_0.f2
-rw-------  1 initiate initiate     4984 Apr 14 21:41 _2_0.f3
-rw-------  1 initiate initiate     4984 Apr 14 21:41 _2_0.f4
-rw-------  1 initiate initiate     4984 Apr 14 21:41 _2_0.f5
-rw-------  1 initiate initiate     4984 Apr 14 21:41 _2_0.f6
-rw-------  1 initiate initiate     4984 Apr 14 21:41 _2_0.f7
-rw-------  1 initiate initiate 33713079 Apr 14 21:15 _3.cfs
-rw-------  1 initiate initiate 65463409 Apr 14 21:15 _4.fdt
-rw-------  1 initiate initiate   127272 Apr 14 21:15 _4.fdx
-rw-------  1 initiate initiate        0 Apr 14 21:15 _4.frq
-rw-------  1 initiate initiate        0 Apr 14 21:15 _4.prx
-rw-------  1 initiate initiate        0 Apr 14 21:15 _4.tfx
-rw-------  1 initiate initiate        0 Apr 14 21:15 _4.tis
-rw-------  1 initiate initiate        0 Apr 14 21:15 _4.tix
-rw-------  1 initiate initiate       16 Apr 14 21:15 segments
-rw-------  1 initiate initiate      212 Apr 14 21:15 segments_1
-rw-------  1 initiate initiate      223 Apr 14 21:15 segments_3


On Apr 14, 2007, at 6:35 PM, David Balmain wrote:
> On 4/14/07, Erik Morton <eimorton at gmail.com> wrote:
>> The following exception occurred while trying optimize a large index:
>>
>> vendor/gems/rdig-0.3.4/lib/rdig/index.rb:46:in `optimize'':
End-of-
>> File Error occured at <except.c>:93 in xraise (EOFError)
>> Error occured in store.c:216 - is_refill
>>          current pos = 0, file length = 0
>>
>> Now, I get the following error any time I try to create a new index
>> on the directory that I was trying to optimize.
>>
>> Ferret::Index::IndexReader.new("indexes/my_index_dir")
>> IOError: IO Error occured at <except.c>:93 in xraise
>> Error occured in fs_store.c:127 - fs_each
>>          doing ''each'' in : <No such file or
directory>
>>
>> Any thoughts? The directory is definitely there. I''d like to
be able
>> to recover from this error someone. Your help is very much  
>> appreciated.
>
> Hi Erik,
>
> Can you give us a listing of what is in the directory? Use `ls -l` so
> I can see the size of the files as well.
>
> Cheers,
> Dave
>
> -- 
> Dave Balmain
> http://www.davebalmain.com/
> _______________________________________________
> Ferret-talk mailing list
> Ferret-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/ferret-talk

Maybe Matching Threads

Search for more apparently analagous threads

Ferret talk - Apr 2007 - Error on optimize leads to corrupt index?

[Ferret-talk] Error on optimize leads to corrupt index?

[Ferret-talk] Error on optimize leads to corrupt index?

[Ferret-talk] Error on optimize leads to corrupt index?

[Ferret-talk] Error on optimize leads to corrupt index?

Maybe Matching Threads