On Tue, 2005-09-06 at 16:20 +0200, Manuel BERTRAND wrote:
> Hi list,
> this problem is already known and I'm sorry to bother if an acceptable
> workaround was already debated on the list.
> I was getting trouble with a 'grep something /var/log*' which
caused the
> "Memory exhausted" message. With some deeper search I found the
lastlog
> file in /var/log/ to be 1.2T sized. This seems to come from the
> nfsnobody's uid to be 4294967294 on x86_64 system (and -1 on i386) and
> the pre-allocation space for every uid (so from 0 to
> ...4294967294...hum..) from lastlog.
> Since I do not use NFS at all, can I just erase the nfsnobody from
> /etc/passwd and start with a blank lastlog file (this looks like a great
> way to nuke my system)? Or do I need to explicitly exclude this file
> from all my search? How do you -x86_64 users- deal with this? It seems
> the mainstream people don't see this as an issue.
According to https://bugzilla.redhat.com/bugzilla/show_bug.cgi?
id=149407#c22 there is a fix for this in nfs-utils-1.0.6-60.EL4, but I
don't think it's the correct fix (meaning that, while it may solve the
symptom, it doens't solve the root problem).
Although the file is listed as being 1.2T, it's a sparse file, meaning
that it doesn't really take up too much space on disk. The problem
we're having is that grep and many other utilities don't handle sparse
files well.
There are a few ways to deal with this. I'm presenting them in order,
from "most correct" to "easiest to implement".
"Correct" in this case
means "most directly solving the root problem," and not necessarily
"most practical."
1) Redesign the kernel's vfs layer to handle sparse files automatically
even when accessed by programs that don't know how to handle sparse
files. I'm not sure how this might be done. At the moment, several
programs (including grep and tar) attempt to read the entire 1.2TB file,
instead of just the non-zero bits. I'm not sure if this is a problem
with grep & tar, with open(), or with the vfs.
2) Change the format of lastlog so it doesn't use a sparse file. This
isn't trivial, but I think it's probably the best solution in the long
run (unless #1 happens).
3) Make grep, tar, rpm, rsync, scp, cp, and so on handle sparse files
better. This probably needs to be done anyway.
4) Remove the nfs-utils package, remove /var/log/lastlog, and recreate
it. This should make lastlog much smaller, unless you have a lot of
users on your system.
5) Exclude lastlog from any greps you do in /var/log.
For an end user, #5 is the easiest, followed by #4.
References:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=64891
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=138676
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=144538
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=145305
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=146214
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=149407
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=156809
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=163273
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=164614
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=165058