zfs discuss - Aug 2007 - ZFS on X4500 and Legato considerations (?)

Hi,
Are there any Legato backup system considerations that I should take in account
while building zpools and file systems on a Sun X4500 machine?  I was thinking
of such consideration such as the zpool size, file system properties such as
compressions,  record size etc.
As far as I understand EMC Legato NetWorker 7.3.2 supports Sun solaris ZFS.

thanks,

Itay.
 
 
This message posted from opensolaris.org

Are you considering using the X4500 as a disk-cache? Or are you simply 
trying to back the X4500 up?

I''ve been doing some testing with the X4500 as a disk-cache storage
node
and have found the following

* configure 1 big zpool (RAIDZ/RAIDZ2/Mirror, depending on your 
space/perf/reliability business needs)
* use adv_file as your device type
* configure multiple ZFS filesystems, Legato spawns 1 nsrmmd writer 
process per "device" or adv_file type. If you only configure 1
adv_file
device, you only get a single writer, no matter how much data your 
clients throw at it. Limits you to ~200-250MB/sec. Set reasonable limits 
on the number of concurrent streams each device will handle to load 
balance across all available devices.

ie. 8 clients sending 2 streams each. Configure 4 ZFS filesystems and 
let each filesystem handle 4 concurrent streams.

Also, configuring multiple adv_file devices gives you the ability to do 
multiple clone sessions if that is part of your process. So if you have 
4 tape drives waiting, 4 adv_file devices will be able to run 
simultaneous clones.

I haven''t found any advantage in changing the recordsize. I have not 
tested compression yet. You can tune the NSR BLOCK SIZE, but I''ve found
the default to be robust. Anything much over 256k has shown to only 
cause higher service times and wait queues on the disks, with no 
increase in streaming performance (this was tested more on NBU though).

Hope that gives you some starting points

-ryan




Itay Menahem wrote:
>Hi,
>Are there any Legato backup system considerations that I should take in
account while building zpools and file systems on a Sun X4500 machine?  I was
thinking of such consideration such as the zpool size, file system properties
such as compressions,  record size etc.
>As far as I understand EMC Legato NetWorker 7.3.2 supports Sun solaris ZFS.
>
>thanks,
>
>Itay.
> 
> 
>This message posted from opensolaris.org
>_______________________________________________
>zfs-discuss mailing list
>zfs-discuss at opensolaris.org
>http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>  
>

On 8/16/07, Itay Menahem <itaymen at gmail.com>
wrote:
> Hi,
> Are there any Legato backup system considerations that I should take in
account while building zpools and file systems on a Sun X4500 machine?  I was
thinking of such consideration such as the zpool size, file system properties
such as compressions,  record size etc.
> As far as I understand EMC Legato NetWorker 7.3.2 supports Sun solaris ZFS.
Works fine for me.

It may be worth thinking about how long it takes to back up
a filesystem. For example, I have a filesystem that takes
almost a day to back up - anything much larger is going to
make a daily backup schedule difficult. So, for a single pool
using raidz2 and everything at the default (we''re interested
in safety and space), it takes me about 22 hours to backup
a terabyte filesystem with 10 million files. Therefore I would
like individual filesystems to come in smaller than that to
be comfortable.

The pool size is essentially irrelevant. For other parameters,
I would expect that if it helped general performance then
it''s going to help backup performance too.

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/

The X4500 will be used as a file server for a Linux high performance cluster.

I''m planning to have an X4500 with 4 or 5 RAIDZ2 zpools (according to
the MTTDL / U_MTBSI recommendations) and multiple ZFS File Systems (with various
sizes ranging from 0.3TB to 1.2TB) .

Does this change the picture, or should I still stick with your recommendations?
 
 
This message posted from opensolaris.org

thanks for your detailed reply.
 
 
This message posted from opensolaris.org

On 8/19/07, Itay Menahem <itaymen at gmail.com>
wrote:
> The X4500 will be used as a file server for a Linux high performance
cluster.
>
> I''m planning to have an X4500 with 4 or 5 RAIDZ2 zpools (according
to the MTTDL / U_MTBSI recommendations) and multiple ZFS File Systems (with
various sizes ranging from 0.3TB to 1.2TB) .
Why multiple pools rather than a single large pool?

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/