samba - Nov 2002 - How is the Samba people stress testing Samba.

Over the years I have seen a number of strange integrity problems when using
Samba servers, and
I sometime wonder how Samba is stress tested.

As long as a Samba server is used as a normal file server, we very seldom se any
problems, but if
heavily used, flat file databases are used, some versions of Samba are more
stable then others.
Even with oplocks disabled and strict locking enabled, we could not always stop
the corruption of files.


A number of years ago I wrote a DOS program designed to run on multiple client
hosts, stress and integrity testing
a shared file on a Windows NT 3.51 file servers. In principal the program
continuously locks a random part of the testfile
(on the server), reads the data and test the data for errors, then writes a new
testpattern and removes the lock.

Lately I have tried it on different Samba servers with very different results.

When I tried to test a Samba 2.2.5 server (SuSE 8.0 Kernel 2.4.19, oplocks
disabled, AMD XP1900+, 256MB) with two Win98
clients, the clients could never run more then maybe 6 hours before one of the
clients completely lost contact
with the Samba server and had to be rebooted. Two bytes in the 250MB testfile
was corrupted. If only one client was run, the
client still lost contact to the server and had to be rebooted.

When the same test was run on a Samba 2.2.6 server (SuSE 8.1 Kernel 2.4.19,
oplocks disabled, AMD XP1900+, 256MB, and
the same Win98 clients), the clients never lost contact to the server, but I
have not yet been able to run the test more the maybe 48
hours without a corruption of the testfile.


The reason I'm looking into this problem is that I have 3 users (clients
here are two W2K, and one Win98) that run an old DOS
shared flatfile database that we have moved from a Win98 machine to Linux, and
it has really been a nightmare. First we
moved it to the Samba 2.2.5 server, but it did not take more then a couple of
hours before the database was corrupted. Then
we moved to the 2.2.6 server and things started to look a lot better, but the
corruption did not stop, it just took maybe one or two
days before we saw corruption. The application has run for more the 4 years on
the Win98 machine, and data has never been
corrupted.
I'm not yet sure, but errors happens even if only one user using the
database. I think the database is based on Paradox.


I'm thinking of reviewing my old DOS program (if I can find the source code)
and see what happens if it's recompiled with a
new compiler. I Think the program was compiled with Borland C 3.0.


Any suggestions (besides moving back to Win98).


 
Thanks in advance
Bo Jacobsen

-------------- next part --------------
HTML attachment scrubbed and removed

Bo:

I'm in the "testing and proof" stage of converting from Windows NT
4 to
Linux/SAMBA and I'm very interested in any "stress testing" that
you (or
anyone else) could point me to.  If you get your DOS Program working,
would you mind sharing it?

The other thing that interests me in your post is the corruption of a
Paradox database.  I'm also trying to convert of a buddy of mine to the
SAMBA scene at his wife's Veterinarian clinic.  They use a Paradox
database for just about ALL of their record keeping and billing.  Last
week we setup a test server with Red Hat 8 and Samba 2.2.5 and got nearly
4 times the performance over Windows 2000 running the database on the same
hardware.  The problem is that they *pound* on the database 8-12 hours a
day 6 days a week.  I I'm looking at corruption 4-48 into production,
I'm
not going to have a buddy for very long....  Have been able to learn
anything more on this front?

I'm going back out the clinic tonight to do some more testing and I'll
report anything that happens while I'm there.

Kevin



-----Original Message-----
From: Bo Jacobsen [mailto:subs@systemhouse.dk]
Sent: Thursday, November 14, 2002 7:03 AM
To: samba@lists.samba.org
Subject: [Samba] How is the Samba people stress testing Samba.


Over the years I have seen a number of strange integrity problems when
using Samba servers, and
I sometime wonder how Samba is stress tested.

As long as a Samba server is used as a normal file server, we very seldom
se any problems, but if
heavily used, flat file databases are used, some versions of Samba are
more stable then others.
Even with oplocks disabled and strict locking enabled, we could not always
stop the corruption of files.


A number of years ago I wrote a DOS program designed to run on multiple
client hosts, stress and integrity testing
a shared file on a Windows NT 3.51 file servers. In principal the program
continuously locks a random part of the testfile
(on the server), reads the data and test the data for errors, then writes
a new testpattern and removes the lock.

Lately I have tried it on different Samba servers with very different
results.

When I tried to test a Samba 2.2.5 server (SuSE 8.0 Kernel 2.4.19, oplocks
disabled, AMD XP1900+, 256MB) with two Win98
clients, the clients could never run more then maybe 6 hours before one of
the clients completely lost contact
with the Samba server and had to be rebooted. Two bytes in the 250MB
testfile was corrupted. If only one client was run, the
client still lost contact to the server and had to be rebooted.

When the same test was run on a Samba 2.2.6 server (SuSE 8.1 Kernel
2.4.19, oplocks disabled, AMD XP1900+, 256MB, and
the same Win98 clients), the clients never lost contact to the server, but
I have not yet been able to run the test more the maybe 48
hours without a corruption of the testfile.


The reason I'm looking into this problem is that I have 3 users (clients
here are two W2K, and one Win98) that run an old DOS
shared flatfile database that we have moved from a Win98 machine to Linux,
and it has really been a nightmare. First we
moved it to the Samba 2.2.5 server, but it did not take more then a couple
of hours before the database was corrupted. Then
we moved to the 2.2.6 server and things started to look a lot better, but
the corruption did not stop, it just took maybe one or two
days before we saw corruption. The application has run for more the 4
years on the Win98 machine, and data has never been
corrupted.
I'm not yet sure, but errors happens even if only one user using the
database. I think the database is based on Paradox.


I'm thinking of reviewing my old DOS program (if I can find the source
code) and see what happens if it's recompiled with a
new compiler. I Think the program was compiled with Borland C 3.0.


Any suggestions (besides moving back to Win98).



Thanks in advance
Bo Jacobsen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 2270 bytes
Desc: not available
Url :
http://lists.samba.org/archive/samba/attachments/20021114/a0d190ab/smime.bin

> Bo:
> 
> I'm in the "testing and proof" stage of converting from
Windows NT 4 to
> Linux/SAMBA and I'm very interested in any "stress testing"
that you (or
> anyone else) could point me to.  If you get your DOS Program working,
> would you mind sharing it?
> 
> The other thing that interests me in your post is the corruption of a
> Paradox database.  I'm also trying to convert of a buddy of mine to the
> SAMBA scene at his wife's Veterinarian clinic.  They use a Paradox
> database for just about ALL of their record keeping and billing.  Last
> week we setup a test server with Red Hat 8 and Samba 2.2.5 and got nearly
> 4 times the performance over Windows 2000 running the database on the same
> hardware.  The problem is that they *pound* on the database 8-12 hours a
> day 6 days a week.  I I'm looking at corruption 4-48 into production,
I'm
> not going to have a buddy for very long....  Have been able to learn
> anything more on this front?
> 
> I'm going back out the clinic tonight to do some more testing and
I'll
> report anything that happens while I'm there.
> 
> Kevin
> 
> 
I understand your worry. I'm right now considering going back to Win98 as we
have to figure out, where exactly the problem lies. I have tried with different
kernels
2.4.16 through 2.4.19 but have not yet been able to solve this issue.

If I were you I would be very careful about moving the database as things are
right now. I suggest that you at least run a very lengthy concurrent paradox
stress
test (from multiply workstations) before moving.


You could have a copy of my DOS testprogram, but it's in Danish and I have
never
translated it to English.
The program was written as we had problems with database corruption when moving
a
financial database from Netware to Windows NT3.5.


The reason I posted the original message is that I'm really interested in
knowing how
Samba has been stress tested. 


Bo
-------------- next part --------------
HTML attachment scrubbed and removed

On Thu, Nov 14, 2002 at 01:03:04PM +0100, Bo Jacobsen
wrote:
> 
> A number of years ago I wrote a DOS program designed to run on multiple
client hosts, stress and integrity testing
> a shared file on a Windows NT 3.51 file servers. In principal the program
continuously locks a random part of the testfile
> (on the server), reads the data and test the data for errors, then writes a
new testpattern and removes the lock.
> 
> Lately I have tried it on different Samba servers with very different
results.
> 
> When I tried to test a Samba 2.2.5 server (SuSE 8.0 Kernel 2.4.19, oplocks
disabled, AMD XP1900+, 256MB) with two Win98
> clients, the clients could never run more then maybe 6 hours before one of
the clients completely lost contact
> with the Samba server and had to be rebooted. Two bytes in the 250MB
testfile was corrupted. If only one client was run, the
> client still lost contact to the server and had to be rebooted.
> 
> When the same test was run on a Samba 2.2.6 server (SuSE 8.1 Kernel 2.4.19,
oplocks disabled, AMD XP1900+, 256MB, and
> the same Win98 clients), the clients never lost contact to the server, but
I have not yet been able to run the test more the maybe 48
> hours without a corruption of the testfile.
Can you send me this test please so I can run it against Samba ?
amba is stress tested in many different ways. Whenever we discover a
new locking problem we add it to the smbtorture test suite.

Jeremy.