freebsd stable - Feb 2007 - Filesystem corruption and bad perfomance with SRCS16 and PAE (raid 5 2TB)

Hello!

I've got an Intel  2400A  server to install FBSD. It has 4GB RAM,
SRCS16 raid controller, configured 2TB (actually a little less, 
about 1.9TB) RAID-5 array, Xeon 3.2Ghz

Recently i have installed 6.2-R, cvsupped it to 6.2-STABLE
(today, thanks to included csup utility, very convinient) and
decided to turn on PAE so i can utilize all 4GB of RAM (w/o PAE
i am loosing 512MB, too bad).

I did that and then tried to run perl. Got  "segmantaion fault".
Ignored  that and tried to copy port tree and then i found
out that some files are damanged and connot be read. Also,
directoried could not be deleted because it is not empty, but when
i do ls in that dir it IS empty. weird.
Booted single user, did fsck - found  a buch of errors, many
softupdate inconsistencies.
Fixed them. Rebooted (still in PAE). Run buildworld and
copy ports tree at the same time. AFter 10 minutes again the same
errors.
Built a kernel without PAE, installed, rebooted in single user,
fixed all filesystems, booted multiuser, run buildworld and copy of
ports tree. Everything is okay. Did it many times. Still everything is ok.

Is it something bad about amr driver working under PAE?

Also, with and after running in PAE (i mean, run PAE kernel,
then install normal kernel and run it) the kernel compains like this
_vfs_done(): amrd0 [WRITE(offset=8192, length=1536)] error = 5
(no other message before  or aftrer this, disks are fully okay becuase
dd works like a charm)

Also, i did a simple thing
dd if=/dev/amrd0 of=/dev/null bs=1M (tried 2m, 5m, 512k)
The throughoutput is about 29MB/sec w/o PAE and 27MB/sec with PAE

Did the same on a very old DUal Pentium III with 3Ware 7xxxx with RAID-5
array and older harddrives. Got over 50Mb/sec.

unixbench show terrible number for file perfomance.

Why the heck amr is so slow? 

--
Regards,
Artem Kuchin

Hi,

Can you try the AMD64 version on that server?

Regards,
Andras


Artem Kuchin wrote:
> Hello!
>
> I've got an Intel  2400A  server to install FBSD. It has 4GB RAM,
> SRCS16 raid controller, configured 2TB (actually a little less, about 
> 1.9TB) RAID-5 array, Xeon 3.2Ghz
>
> Recently i have installed 6.2-R, cvsupped it to 6.2-STABLE
> (today, thanks to included csup utility, very convinient) and
> decided to turn on PAE so i can utilize all 4GB of RAM (w/o PAE
> i am loosing 512MB, too bad).
>
> I did that and then tried to run perl. Got  "segmantaion fault".
> Ignored  that and tried to copy port tree and then i found
> out that some files are damanged and connot be read. Also,
> directoried could not be deleted because it is not empty, but when
> i do ls in that dir it IS empty. weird.
> Booted single user, did fsck - found  a buch of errors, many
> softupdate inconsistencies.
> Fixed them. Rebooted (still in PAE). Run buildworld and
> copy ports tree at the same time. AFter 10 minutes again the same
> errors.
> Built a kernel without PAE, installed, rebooted in single user,
> fixed all filesystems, booted multiuser, run buildworld and copy of
> ports tree. Everything is okay. Did it many times. Still everything is 
> ok.
>
> Is it something bad about amr driver working under PAE?
>
> Also, with and after running in PAE (i mean, run PAE kernel,
> then install normal kernel and run it) the kernel compains like this
> _vfs_done(): amrd0 [WRITE(offset=8192, length=1536)] error = 5
> (no other message before  or aftrer this, disks are fully okay becuase
> dd works like a charm)
>
> Also, i did a simple thing
> dd if=/dev/amrd0 of=/dev/null bs=1M (tried 2m, 5m, 512k)
> The throughoutput is about 29MB/sec w/o PAE and 27MB/sec with PAE
>
> Did the same on a very old DUal Pentium III with 3Ware 7xxxx with RAID-5
> array and older harddrives. Got over 50Mb/sec.
>
> unixbench show terrible number for file perfomance.
>
> Why the heck amr is so slow?
> -- 
> Regards,
> Artem Kuchin
>
> _______________________________________________
> freebsd-stable@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to
"freebsd-stable-unsubscribe@freebsd.org"
>

Hi Artem,

Yes amd64 is for both Intel and AMD platforms.

I have been having troubles with the SRCS16 on the amd64 platform 6.2 
stable to give you the heads up if you are going this route.

I was able to get around the problem you mentioned (if it is the same 
problem) by creating two arrays. One array was 100GIGs and the other 
was the balance of the drive (about 1.7 TB). This seemed to get rid 
of the corruption issues. Perhaps give this a try? going too close to 
2TB was what I presumed was causing the corruption and read errors I 
was going as it went away when I installed the two arrays. I found a 
coupe of posts on another list that talk about this.

I am using a S5000PAL with 5 * 500GIG SATA drives with 6.2 stable amd64.

I did notice it was really SLOW when installing many parts of the new 
system and I am still very unhappy with the setup. Let me know your 
experience if you find it. I have done a bunch of benchmarking on 
this with the port tools but I am stuck at the freezing issue as that 
makes the system not usable.

I wonder if you try the amd64 if you have the same problem I am 
having (slow performance, it locks for like 10-15 minutes when trying 
to mount root on bootup, and there are freezes that seem to happen 
from time to time when writing to the file system). I am still trying 
to debug these problems but have not found anyone with the same 
mother board yet with a working 6.2 amd64 install.

I hope this helps.

Cheers,

Paul


At 02:43 PM 06/02/2007, you wrote:
>Hello!
>
>I've got an Intel  2400A  server to install FBSD. It has 4GB RAM,
>SRCS16 raid controller, configured 2TB (actually a little less, 
>about 1.9TB) RAID-5 array, Xeon 3.2Ghz
>
>Recently i have installed 6.2-R, cvsupped it to 6.2-STABLE
>(today, thanks to included csup utility, very convinient) and
>decided to turn on PAE so i can utilize all 4GB of RAM (w/o PAE
>i am loosing 512MB, too bad).