The ls -lR command will only access directory entries and inodes. These
are cached in slabs (see /proc/slabinfo). Not sure what happens to the
disk pages that they live in, but those disk pages can be discarded
immediately after the slab caches have been populated. So it's probably
not a good test of filesystem data caching. You may want to explain
what you hope to achieve with this: more cache hits on directory and
inode entries, or on file data? Are you seeing more disk reads in this
configuration than in the one you're comparing with?
Thanks,
Herbert.
Brian Kroth wrote:> I've got a question about tuning mem usage which may or may not be
ocfs2
> related. I have some VMs that share an iSCSI device formatted with
> ocfs2. They're all running a debian based 2.6.26 kernel. We basically
> just dialed the kernel down to 100HZ rather than the default 1000HZ.
> Everything else is the same. All the machines have 2 CPUs and 2GB of
> RAM.
>
> Over time I would expect that the amount of free mem decreases towards 0
> and the amount of (fs) cached mem increases. I think one can simulate
> this by doing the following:
> echo 1 >> /proc/sys/vm/drop_caches
> ls -lR /ocfs2/ > /dev/null
>
> When I do this on a physical machine with a large ext3 volume, the
> cached field steadily increases as I expected. However, on the ocfs2
> volume what I actually see is that the free mem and cache mem remains
> fairly constant.
>
> # free
> total used free shared buffers cached
> Mem: 2076376 989256 1087120 0 525356 33892
> -/+ buffers/cache: 430008 1646368
> Swap: 1052216 0 1052216
>
> # top -n1 | head -n5
> top - 11:11:19 up 2 days, 20:26, 2 users, load average: 1.45, 1.39, 1.26
> Tasks: 140 total, 1 running, 139 sleeping, 0 stopped, 0 zombie
> Cpu(s): 0.8%us, 3.7%sy, 0.0%ni, 79.5%id, 15.2%wa, 0.1%hi, 0.6%si,
0.0%st
> Mem: 2076376k total, 989772k used, 1086604k free, 525556k buffers
> Swap: 1052216k total, 0k used, 1052216k free, 33920k cached
>
> I've tried to trick the machines into allowing for more cached inodes
by
> decreasing vfs_cache_pressure, but it doesn't seem to have had much
> affect. I also get the same results if only one machine has the ocfs2
> fs mounted. I have also tried mounting it with the localalloc=16 option
> that I found in a previous mailing list post.
>
> The ocfs2 filesystem is 2TB and has about 600GB of maildirs on it (many
> small files). The ext3 machine is about 200GB and has a couple of
> workstation images on it (a mix of file sizes).
>
> I haven't yet been able to narrow down whether this is vm vs. physical,
> ocfs2 vs. ext3, iSCSI vs. local, or something else.
>
> Has anyone else seen similar results or have some advice as to how to
> improve the situation?
>
> Thanks,
> Brian
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
>