freebsd stable - Jun 2005 - Show stopper for large disks with 5.4-RELEASE

Hi,

we are setting up a mail server for ~50 000 users, with around 1.8TB
on a DAS storage (HP MSA 500).

We were planning to use FreeBSD (5.4-RELEASE), as with all other
servers in our machine room.

However, we are encountering a show stopper: after an unclean
shutdown, the snapshot that fsck creates is taking too much time
(more than 20 minutes). During the most part of this time, all I/O
are frozen on this large disk, so the server cannot serve our
clients. Our SLA constraints do not allow us to have these recovery
times.

The options used to create the file system were standard sysinstall
options. During normal operations, performances are very good.

Our tests showed that Linux doesn't present the same problem. With
ReiserFS, the reboot after crash takes only 20 seconds to read the
journal and recover the file system.

Is the snapshot time for large volumes a known problem? We don't
see such a long time with smaller file systems.

Do you have a clue to help us use FreeBSD and not switch on Linux
for this service?


Philippe Pegon & Pierre David

Hi, Pierre,

On Wed, Jun 08, 2005 at 02:31:37PM +0200, Pierre DAVID
wrote:
> Hi,
> 
> we are setting up a mail server for ~50 000 users, with around 1.8TB
> on a DAS storage (HP MSA 500).
> 
> We were planning to use FreeBSD (5.4-RELEASE), as with all other
> servers in our machine room.
> 
> However, we are encountering a show stopper: after an unclean
> shutdown, the snapshot that fsck creates is taking too much time
> (more than 20 minutes). During the most part of this time, all I/O
> are frozen on this large disk, so the server cannot serve our
> clients. Our SLA constraints do not allow us to have these recovery
> times.
Would you please provide a bit more of information so we can investigate
what was happening, like:

	- first few lines dumpfs(8) output from your storage filesystem
	- df -i on your storag filesystem
	- dmesg.boot from your /var/run

I think the delay is too long.  Taking a snapshot on a large volume
is slow, but should not be that slow :-)  We run similiar mail server
at company, but with ~10x of users and 2x storage (divided into two
RAID groups).  Hope I would be able to provide some help.

Cheers,
-- 
Xin LI <delphij frontfree net>	http://www.delphij.net/
See complete headers for GPG key and other information.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20050608/67c85559/attachment.bin

Hi Pierre!

On Wed, 08 Jun 2005, Pierre DAVID wrote:
> Do you have a clue to help us use FreeBSD and not switch on Linux
> for this service?
Bad workaround.
You can create many small partitions and mount_unionfs.

By.
Dmitriy