Ferret talk - Mar 2007 - Too many open files error

Hi Dave,

i just stumbled across a new error i haven''t seen before :)

caught error inside loop: IO Error occured at <except.c>:93 in xraise
Error occured in fs_store.c:264 - fs_new_output
         couldn''t create OutStream /var/www/localhost/rails/current/ 
script/backgroundrb/../../config/../db/ferret.index.production/ 
_jei_0.f0: <Too many open files>

my ulimit is set to 1024 files, the error occurs regularly.. any idea?

Ben

On Mar 12, 2007, at 18:54, Benjamin Krause wrote:
> Hi Dave,
>
> i just stumbled across a new error i haven''t seen before :)
>
> caught error inside loop: IO Error occured at <except.c>:93 in xraise
> Error occured in fs_store.c:264 - fs_new_output
>          couldn''t create OutStream
/var/www/localhost/rails/current/
> script/backgroundrb/../../config/../db/ferret.index.production/
> _jei_0.f0: <Too many open files>
>
> my ulimit is set to 1024 files, the error occurs regularly.. any idea?
ah, and i should add, this is from our backgroundrb indexing process
there is basically nothing else than opening and closing the index, all
of the time. this isn''t necessarily a ferret problem, but the process
isn''t doing much else.

ben

On 3/13/07, Benjamin Krause <bk at benjaminkrause.com>
wrote:
>
> On Mar 12, 2007, at 18:54, Benjamin Krause wrote:
>
> > Hi Dave,
> >
> > i just stumbled across a new error i haven''t seen before :)
> >
> > caught error inside loop: IO Error occured at <except.c>:93 in
xraise
> > Error occured in fs_store.c:264 - fs_new_output
> >          couldn''t create OutStream
/var/www/localhost/rails/current/
> > script/backgroundrb/../../config/../db/ferret.index.production/
> > _jei_0.f0: <Too many open files>
> >
> > my ulimit is set to 1024 files, the error occurs regularly.. any idea?
>
> ah, and i should add, this is from our backgroundrb indexing process
> there is basically nothing else than opening and closing the index, all
> of the time. this isn''t necessarily a ferret problem, but the
process
> isn''t doing much else.
This is not a bug but rather a limitation of the operating system.
There are a few solutions. If you are getting this problem you should
definitely be sure to set :use_compound_file to true (which is the
default setting). You might also like to lower the merge_factor which
defaults to 10. Having a lower merge_factor will slow indexing a
little but it will actually make search faster. Try setting it to 4.

Alternatively, in your situation, I would probably just increase the
file handle limit. I''m sure you''d have enough memory to do
that.

Cheers,
Dave

-- 
Dave Balmain
http://www.davebalmain.com/