freebsd stable - Jun 2008 - Areca Raid 6 ARC-1231 Raid 6 Slow LS Listing Performance on large directory

Hello,

I have a RAID-6 Partition with the Areca ARC-1231 card on a S5000PAL 
Intel system with  6 disks as part of the raid volume. The system has 
been set up as Write-back cache and the raid card has a 2 GIG memory 
cache on it. It is installed on Freebsd 7.0 STABLE with SCHED_ULE enabled.

I have a folder with a lot of small and big files in it that total 
3009 files. In the user system we have 2200 users in the password file.

1) When I do a ls -lh on the raid 6 array with 6 disks in the array 
it takes aver 16 seconds before it starts to display anything on the screen.
2) While running a tar command on another shell, the time goes to 28 
seconds for the same list to start showing.
3) When I do a ls (with no other options) it starts to list right away.
4) When I do a ls -ln it displays right away as well  pointing to the 
slowdown being the mapping of the users in the db lookup.

I have the same directory with the same number of files on a Raid 5 
SCSI partition on Freebsd 4.X and it only takes 2 seconds to start 
displaying the list with the command ls -lh.

Any ideas why it takes so long for this on Freebsd 7.0 stable?

The partition this folder is on it /dev/da0s1f  with a total size of 
1.7T  and a usage of  63G

Any suggestions or help on this would be greatly appreciated.

Thanks,

Paul

At 01:32 PM 6/11/2008, Paul wrote:
>Any ideas why it takes so long for this on Freebsd 7.0 stable?
>
>The partition this folder is on it /dev/da0s1f  with a total size of 
>1.7T  and a usage of  63G
>
>Any suggestions or help on this would be greatly appreciated.
Couple of things to check

In /etc/nsswitch.conf try changing to

group: files
passwd: files

and check to make sure
vfs.ufs.dirhash_maxmem is > than vfs.ufs.dirhash_mem.  If 
vfs.ufs.dirhash_mem is hit the max, increase the maxmem value

         ---Mike

At 02:11 PM 11/06/2008, Mike Tancsa wrote:
>>Any ideas why it takes so long for this on Freebsd 7.0 stable?
>>
>>The partition this folder is on it /dev/da0s1f  with a total size 
>>of 1.7T  and a usage of  63G
>>
>>Any suggestions or help on this would be greatly appreciated.
>
>Couple of things to check
>
>In /etc/nsswitch.conf try changing to
>
>group: files
>passwd: files
>
>and check to make sure
>vfs.ufs.dirhash_maxmem is > than vfs.ufs.dirhash_mem.  If 
>vfs.ufs.dirhash_mem is hit the max, increase the maxmem value
>
>         ---Mike

Thanks a lot for the quick reply.

Changing from compat to files fixes the problem.

It is now superfast when running ls -lh.

Thanks again!

Cheers

Paul

At 04:04 PM 6/11/2008, Paul wrote:
>Changing from compat to files fixes the problem.
>
>It is now superfast when running ls -lh.
Hi,
         Not sure, but a more "proper" fix might be to look at the 
nscd caching daemon. Take a look at nscd and nscd.conf. I havent used 
it myself, but I seem to recall others suggesting this as a "fix" as
well.

         ---Mike

2008/6/12 Paul <bsdlist@cogeco.ca>:
> 1) When I do a ls -lh on the raid 6 array with 6 disks in the array it
takes
> aver 16 seconds before it starts to display anything on the screen.
> 2) While running a tar command on another shell, the time goes to 28
seconds
> for the same list to start showing.
> 3) When I do a ls (with no other options) it starts to list right away.
> 4) When I do a ls -ln it displays right away as well  pointing to the
> slowdown being the mapping of the users in the db lookup.
>
> I have the same directory with the same number of files on a Raid 5 SCSI
> partition on Freebsd 4.X and it only takes 2 seconds to start displaying
the
> list with the command ls -lh.
>
> Any ideas why it takes so long for this on Freebsd 7.0 stable?
>
> The partition this folder is on it /dev/da0s1f  with a total size of 1.7T
>  and a usage of  63G
>
> Any suggestions or help on this would be greatly appreciated.
Could you please do a couple of other tests, if you're able to? I've
got a PR to look into this issue.

Could you see if using a smaller password file makes the ls start/run quicker?

Could you possibly run ls inside "truss" on both FreeBSD-4 and
FreeBSD-7 and email me a snippet of the output (say, a few hundred
lines) ? Something like:

truss ls <flags> >foo 2>&1

Thanks,



Adrian

-- 
Adrian Chadd - adrian@freebsd.org