Ocfs2 users - Aug 2009 - Quantifying OCFS2 network traffic...

Hi all,

I'm trying to understand and quantify network traffic for a 32 node high
performance OCFS2 installation.   Part of what I'm doing is attempting
to size the network hardware requirements.  To do this I need some help
understanding the kinds and amounts of traffic generated during
"normal"
use.  

Once populated, the system will have a very high READ to WRITE op ratio.
I need to optimize OCFS2 (both hard/software-wise) for this use.

Conceptually, it would seem that DLM would be responsible for most of
the network traffic associated with ocfs2.  Correct?  Is DLM traffic the
same for a READ as for a WRITE?

Does ofcs2 itself send messages for reads and/or writes in normal use?  

Any metrics known relating IO size to network traffic?  Guesses? 

If I'm using a 10GigE network, would I have a need for adding another
NIC and bonding?   

Thanks,
-PWM

With the o2cb stack, the network traffic will mostly be dlm related.
Lock co-ordination. No ios.

With the pacemaker stack, you will have some traffic required for ordered
messaging too. I don't have the details on it. Again, no ios.

For o2cb, the packets should be small. Tiny infact. But there can be a
lot of them.

The amount will depend on the number of lockres masteries and the degree
of lock contention.

Number of masteries will depend on the number of files and whether
the node has enough memory to cache all the inodes. This is the kernel
remember. It does not take much for the kernel to flush its inode/dentry
cache. An userspace app only has to ask for more memory for it to start
flushing the caches.

Probably the worst case scenario for lock contention is if multiple
nodes are reading and writing the same file. Say all nodes are appending
to the same file as well as "tail -f" it. In this case, each write on
a node
will trigger a downconvert on the rest of them followed by an upconvert.

Sunil

Peter W. Morreale wrote:
> Hi all,
>
> I'm trying to understand and quantify network traffic for a 32 node
high
> performance OCFS2 installation.   Part of what I'm doing is attempting
> to size the network hardware requirements.  To do this I need some help
> understanding the kinds and amounts of traffic generated during
"normal"
> use.  
>
> Once populated, the system will have a very high READ to WRITE op ratio.
> I need to optimize OCFS2 (both hard/software-wise) for this use.
>
> Conceptually, it would seem that DLM would be responsible for most of
> the network traffic associated with ocfs2.  Correct?  Is DLM traffic the
> same for a READ as for a WRITE?
>
> Does ofcs2 itself send messages for reads and/or writes in normal use?  
>
> Any metrics known relating IO size to network traffic?  Guesses? 
>
> If I'm using a 10GigE network, would I have a need for adding another
> NIC and bonding?   
>
> Thanks,
> -PWM
>
>
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
>