samba - Nov 2001 - Samba swamps the server's CPU

Hi.  I'm running Samba 2.2.2 and Linux 2.4.7 on a dual PPro-200/256KB
server.  The relevant NIC is an Intel EtherExpress PRO/100+.  The
problem I'm having is that certain programs will swamp the server; the
load will spike immediately to 50% (i.e., 100% on one CPU) and
performance will, naturally, bite.  Having another active connection
to the server at the same time causes _monster_ slowdowns.  I didn't
think much of this until I ran vmstat and noticed that when I ran one
of these programs, the context switches per second would jump from
about 300 to over 10000.  I pulled out strace and this is what I got:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 41.82   12.716340         638     19934           select
 19.31    5.870802         294     19935           send
 14.33    4.357302          91     47760           read
  5.15    1.567179          51     30435           fcntl64
  5.03    1.530087        9162       167           write
  4.03    1.226924         201      6110           getdents64
  2.85    0.867619          40     21692           gettimeofday
  2.84    0.864095          72     12044      2812 stat64
  1.55    0.470409          44     10642           _llseek
  1.12    0.339316          80      4263       192 open
  0.91    0.276100          45      6190           fstat64
  0.74    0.226241          55      4084         3 close
  0.12    0.036709         243       151           ftruncate64
  0.03    0.009962          39       255           geteuid32
  0.03    0.009618          38       253           getegid32
[snipped for brevity]
------ ----------- ----------- --------- --------- ----------------
100.00   30.407128                184547      3021 total

I have no idea where to go from here, really.  There's nothing in the
SMB log.  The client on the other end is a Windows 2000 box (Server,
but the same thing happens with Pro).  Samba is running as a daemon
rather than from inetd.

The odd thing is that 98% of programs work just fine.  (Winamp, most
noticeably, but I can also copy files and do regular things without
this problem.)  I'm certain that this problem exists in both 2.2.1 and
2.2.2, but I believe that, in some previous version, it didn't exist;
at least, Alpha Centauri (one of the offending programs) now runs
unacceptably slowly and it used to work just fine.

Can anyone point me in the right direction or provide a solution?

-- 
Eric McCoy <ctr2sprt@yahoo.com>

"I woke up this morning and realized what the game needed: pirates,
pimps, and gay furries."  - Rich "Lowtax" Kyanka

On Sat, Nov 03, 2001 at 04:17:36AM -0500, Eric P. McCoy
wrote:
> Hi.  I'm running Samba 2.2.2 and Linux 2.4.7 on a dual PPro-200/256KB
> server.  The relevant NIC is an Intel EtherExpress PRO/100+.  The
> problem I'm having is that certain programs will swamp the server; the
> load will spike immediately to 50% (i.e., 100% on one CPU) and
> performance will, naturally, bite.  Having another active connection
> to the server at the same time causes _monster_ slowdowns.  I didn't
> think much of this until I ran vmstat and noticed that when I ran one
> of these programs, the context switches per second would jump from
> about 300 to over 10000.  I pulled out strace and this is what I got:
> 
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  41.82   12.716340         638     19934           select
>  19.31    5.870802         294     19935           send
>  14.33    4.357302          91     47760           read
>   5.15    1.567179          51     30435           fcntl64
>   5.03    1.530087        9162       167           write
>   4.03    1.226924         201      6110           getdents64
>   2.85    0.867619          40     21692           gettimeofday
>   2.84    0.864095          72     12044      2812 stat64
>   1.55    0.470409          44     10642           _llseek
>   1.12    0.339316          80      4263       192 open
>   0.91    0.276100          45      6190           fstat64
>   0.74    0.226241          55      4084         3 close
>   0.12    0.036709         243       151           ftruncate64
>   0.03    0.009962          39       255           geteuid32
>   0.03    0.009618          38       253           getegid32
> [snipped for brevity]
> ------ ----------- ----------- --------- --------- ----------------
> 100.00   30.407128                184547      3021 total
> 
> I have no idea where to go from here, really.  There's nothing in the
> SMB log.  The client on the other end is a Windows 2000 box (Server,
> but the same thing happens with Pro).  Samba is running as a daemon
> rather than from inetd.
> 
> The odd thing is that 98% of programs work just fine.  (Winamp, most
> noticeably, but I can also copy files and do regular things without
> this problem.)  I'm certain that this problem exists in both 2.2.1 and
> 2.2.2, but I believe that, in some previous version, it didn't exist;
> at least, Alpha Centauri (one of the offending programs) now runs
> unacceptably slowly and it used to work just fine.
> 
> Can anyone point me in the right direction or provide a solution?
Can you test something for me - add the line "kernel oplocks = no"
to the [global] section of the smb.conf. 

Is the slowdown 100% reproducible with certain apps ? If so, a
debug level 10 trace would alsio help.

Thanks,

	Jeremy.