Hi, I have been noticing some serious load averages occurring when users are browsing or using directories with a large number of files. A case in point is our Sage users. Sage is a nasty program and spits out three small files into a spool directory every time something is printed and does not clear them out. This means that the spool directories rapidly fill with tiny files and by the end of the month there might be upwards of 5,000 files in each spool directory. Since we have around eight people using Sage and, via the program, accessing these spool directories then the server goes berserk. It got to the point where we were looking at a load average of +8.0! Needless to say the whole server started creaking. Once we had got the Sage users to clear out the spool directories (this is a unfortunately a manual process) everything went back to the normal 0.01 - 0.20 load average. The only way around this is to kick the Sage people each month to clear their stuff out. We have a similar experience if the web developers start poking around in directories with many small HTML files using Explorer. Anyone else noticed this ? Perhaps it occurs on all servers but I cannot say we ever had an inkling of this behaviour using Novell or NT which is what we have replaced. Is it more a problem with the file system ? The block sizing on ext2 ? Or is it Samba in the middle working overtime ? Noel
Noel Kelly wrote:> > Hi, > > I have been noticing some serious load averages occurring when users are > browsing or using directories with a large number of files. > > A case in point is our Sage users. Sage is a nasty program and spits out > three small files into a spool directory every time something is printed and > does not clear them out. This means that the spool directories rapidly fill > with tiny files and by the end of the month there might be upwards of 5,000 > files in each spool directory.I can't tell you how to solve the problem, but I can tell you how to fix the symptoms. We have a similar problem with other software. It likes to leaves work files around that quickly gobble up disk space. I just run a find -mtime +3 -exec rm -f {}; . That finds all files that have not been touched in three days and deletes them. That line above will do exactly that, so please man find if you have not used find. It's a bit tricky, and my script is set up to use three business days (it ignores Sat/Sun). --Yan -- Future fighter pilots: Me: Akari, WHAT are you DOING? Akari, age 3: Pushing the envelope. 6:28am up 9 days, 10:37, 7 users
Thanks Yan - but the trouble with these accounts people is that the spool files cannot be deleted until they are happy they have everything on hard copy so unfortuantely that is why it must remain a manual process. But I appreciate your help - thanks again. Noel -----Original Message----- From: Yan Seiner [mailto:yan@cardinalengineering.com] Sent: 14 February 2002 11:34 To: Noel Kelly Cc: 'samba@lists.samba.org' Subject: Re: [Samba] Small Files/Load Average Information Noel Kelly wrote:> > Hi, > > I have been noticing some serious load averages occurring when users are > browsing or using directories with a large number of files. > > A case in point is our Sage users. Sage is a nasty program and spits out > three small files into a spool directory every time something is printedand> does not clear them out. This means that the spool directories rapidlyfill> with tiny files and by the end of the month there might be upwards of5,000> files in each spool directory.I can't tell you how to solve the problem, but I can tell you how to fix the symptoms. We have a similar problem with other software. It likes to leaves work files around that quickly gobble up disk space. I just run a find -mtime +3 -exec rm -f {}; . That finds all files that have not been touched in three days and deletes them. That line above will do exactly that, so please man find if you have not used find. It's a bit tricky, and my script is set up to use three business days (it ignores Sat/Sun). --Yan -- Future fighter pilots: Me: Akari, WHAT are you DOING? Akari, age 3: Pushing the envelope. 6:28am up 9 days, 10:37, 7 users