Re: Slow NFS Performance

Andrew Suffield
Mon, 31 Mar 2008 19:18:17 -0000

On Mon, Mar 31, 2008 at 11:47:54AM -0700, Tom Eastep
wrote:
>> After restarting Shorewall (firewall), speeds immediately returned to
>> normal. Steps I took to restart the server were
"/etc/init.d/shorewall
>> stop &&
>> shorewall clear && /etc/init.d/shorewall start".  (I was
trying to see if the
>> configuration files were directly hindering performance.  Upon
restarting
>> shorewall, no further speed issues were noted.)
>>
>> This leads me to believe, Shorewall is borking someplace -- or more
correctly
>> put, one of the kernel modules concerning Netfilter is failing??
>
> Given that Shorewall isn''t something that runs continuously in
your
> system, it''s hard to imagine that this problem is
Shorewall-related.

My bet is on a kernel, hardware, or network fabric issue that was
shaken loose by the brief traffic interruption that occurs when
shorewall starts. It could also be a traffic shaping corner case (some
flawed configurations have cascade failure modes, where everything
seizes up for certain traffic patterns but not others, and the effect
is self-continuing).

>> Here''s the requested debug info for this case scenario:
>
> Better problem documentation would include:
>
> a) Output of "shorewall dump" at the time that the problem was
encountered.
> b) Output of "shorewall dump" after recovery measures were taken
and
> performance restored.

Other significant data points:

 - what''s on the network when the problem occurs? Steady-but-slow NFS
 traffic, fast-but-rare bursts, collisions, corrupted packets,
 something else entirely? Whenever I see something like this, I hit
 tcpdump -w first and think about it later; a packet dump explains
 most issues when you can study it at leisure.

 - relevant /proc/mounts entries. Do these change when the problem occurs?

 - physical network configuration

 - vmstat output when the problem occurs

But I doubt it''s a shorewall problem.