Ocfs2 users - Aug 2009 - ocfs2 configuration/performance questions...

Hi all, 

I'm trying to determine the performance implications of various
configurations for ocfs2.   (I'm new to ocfs2, but have read through all
the docs for both 1.2 and 1.4, so please be gentle :)  This would be a
1.4 installation.

I searched through www.mail-archive.com and didn't see anything
appropriate.  If there are other links, please let me know.  

In my case, I want to plan for expansion of a cluster on a SMP blade
system.  Runtime performance in this system will be critical.  The
system will have a dedicated network for cluster traffic.  The app is a
distributed database system. (ie: Not a general purpose/access system)

For example purposes, assume we're starting with 8 blades and will
expand to 32 at some future point.  The questions are: 

1) What is/are the impacts of preconfiguring all 16 IP addrs in the
cluster config?  Impacts on mount? umount?  runtime?  DLM? 

2) assume we create the fs instance with all 32 slots.  I would assume
that HB is impacted since it would scan all 32 slots each iteration, is
that correct?  

3) Prioritization...  Has anybody performed any testing wrt to changing
priorities of the various OCFS2 tasks?  I might presume that increasing
the priorities of the DLM (dlm_wq, dlm_thread) tasks might be a Good
Thing(tm).  OTOH, could it lead to an unbalanced system wrt the other
ocfs2 tasks?  Perhaps increasing all ocfs2 tasks? (wrt to the app
referencing the fs?) I'm a little worried about creating priority
inversion issues as this will be on a non-RT Linux OS instance. Should I
be?

4) Localization...  What about the creation of CPU sets and/or affinity
for the various tasks?  Anybody tried doing same any have some results
to share?  

I have done quite a bit of performance tuning in the past and I fully
realize I'm not giving adequate details about the installation.  To be
completely honest, I don't have all those details at the moment. I'd be
grateful for any generalizations based on experience/educated guesses
right now.  

I'd also be grateful for any links to ocfs2 performance tuning... 

 Thanks in advance for any information.

Best,
-PWM

Peter W. Morreale wrote:
> Hi all, 
>
> I'm trying to determine the performance implications of various
> configurations for ocfs2.   (I'm new to ocfs2, but have read through
all
> the docs for both 1.2 and 1.4, so please be gentle :)  This would be a
> 1.4 installation.
>
> I searched through www.mail-archive.com and didn't see anything
> appropriate.  If there are other links, please let me know.  
>
> In my case, I want to plan for expansion of a cluster on a SMP blade
> system.  Runtime performance in this system will be critical.  The
> system will have a dedicated network for cluster traffic.  The app is a
> distributed database system. (ie: Not a general purpose/access system)
>
> For example purposes, assume we're starting with 8 blades and will
> expand to 32 at some future point.  The questions are: 
>
> 1) What is/are the impacts of preconfiguring all 16 IP addrs in the
> cluster config?  Impacts on mount? umount?  runtime?  DLM? 
No impact. The network connections between nodes are only made when
nodes mount one or more volumes. Quorum is computed based on nodes
that are part of the active cluster.
> 2) assume we create the fs instance with all 32 slots.  I would assume
> that HB is impacted since it would scan all 32 slots each iteration, is
> that correct?
No. heartbeat i/o depends on the number of nodes configured in
cluster.conf. Take the largest node number configured - 1 and
multiply with 512 bytes. That's the amount read every 2 secs.
Either way, that size is negligible.
> 3) Prioritization...  Has anybody performed any testing wrt to changing
> priorities of the various OCFS2 tasks?  I might presume that increasing
> the priorities of the DLM (dlm_wq, dlm_thread) tasks might be a Good
> Thing(tm).  OTOH, could it lead to an unbalanced system wrt the other
> ocfs2 tasks?  Perhaps increasing all ocfs2 tasks? (wrt to the app
> referencing the fs?) I'm a little worried about creating priority
> inversion issues as this will be on a non-RT Linux OS instance. Should I
> be?
We have not meddled with the priorities.
> 4) Localization...  What about the creation of CPU sets and/or affinity
> for the various tasks?  Anybody tried doing same any have some results
> to share?  
>
> I have done quite a bit of performance tuning in the past and I fully
> realize I'm not giving adequate details about the installation.  To be
> completely honest, I don't have all those details at the moment.
I'd be
> grateful for any generalizations based on experience/educated guesses
> right now.  
>
> I'd also be grateful for any links to ocfs2 performance tuning...
>