zfs discuss - Jan 2008 - ZFS versus VxFS as file system inside Netbackup 6.0 DSSU

Hello experts,
  
  
We have a large implementation of Symantec Netbackup 6.0 with disk staging.
Today, the customer is using VxFS as file system inside Netbackup 6.0 DSSU (disk
staging).

The customer would like to know if it is best to use ZFS or VxFS as file system
inside Netbackup disk staging in order to get the best performance possible.

Could you provide some information regarding this topic? 


Thanks in advance for your help

Regards
  
Patrick

Veritas products tend to work best with... well... other Veritas products.

On 1/11/08, Patrick Herman <Patrick.Herman at sun.com>
wrote:
>
> Hello experts,
>
>
> We have a large implementation of Symantec Netbackup 6.0 with disk staging.
Today, the customer is using VxFS as file system inside Netbackup 6.0 DSSU (disk
staging).
>
> The customer would like to know if it is best to use ZFS or VxFS as file
system inside Netbackup disk staging in order to get the best performance
possible.
>
> Could you provide some information regarding this topic?
>
>
> Thanks in advance for your help
>
> Regards
>
> Patrick
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

-- 
_________________________________/ sengork.blogspot.com /????

with zfs you can compress data on disk ...that is a grat advantage
when doing backup to disk
also, for DSSU you need to multiply number of filesystem (1 fs per
stu), the advantage of zfs is that you don''t need to fix the size
of the fs upfront  (the space is shared among all the fs)
s-

On Jan 10, 2008 2:12 PM, Patrick Herman <Patrick.Herman at sun.com>
wrote:
>
> Hello experts,
>
>
> We have a large implementation of Symantec Netbackup 6.0 with disk staging.
Today, the customer is using VxFS as file system inside Netbackup 6.0 DSSU (disk
staging).
>
> The customer would like to know if it is best to use ZFS or VxFS as file
system inside Netbackup disk staging in order to get the best performance
possible.
>
> Could you provide some information regarding this topic?
>
>
> Thanks in advance for your help
>
> Regards
>
> Patrick
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>


-- 
------------------------------------------------------
Blog: http://fakoli.blogspot.com/

On 1/15/08, Selim Daoud <selim.daoud at gmail.com> wrote:
> with zfs you can compress data on disk ...that is a grat advantage
> when doing backup to disk
> also, for DSSU you need to multiply number of filesystem (1 fs per
> stu), the advantage of zfs is that you don''t need to fix the size
> of the fs upfront  (the space is shared among all the fs)
        But ... NBU (at least version 6.0) attempts to estimate the
size of the backup and make suer there is enough room on the DSSU to
handle it. What happens when the free space reported by ZFS isn''t
really the free space ?

        We are using NBU DSSU against both UFS and ZFS (but not
against VxFS) and have not noticed any FS related performance
limitations. The clients and the network are all slower.

-- 
{--------1---------2---------3---------4---------5---------6---------7---------}
Paul Kraus
-> Sound Designer, Noel Coward''s Hay Fever
    @ Albany Civic Theatre, Feb./Mar. 2008
-> Facilities Coordinator, Albacon 2008

Regarding the question asked below namely "What happens when the free 
space reported by ZFS isn''t really the free space ?", is there an
open
bug for this ?

Thanks,

Sri
Paul Kraus wrote:
> On 1/15/08, Selim Daoud <selim.daoud at gmail.com> wrote:
>
>   
>> with zfs you can compress data on disk ...that is a grat advantage
>> when doing backup to disk
>> also, for DSSU you need to multiply number of filesystem (1 fs per
>> stu), the advantage of zfs is that you don''t need to fix the
size
>> of the fs upfront  (the space is shared among all the fs)
>>     
>
>         But ... NBU (at least version 6.0) attempts to estimate the
> size of the backup and make suer there is enough room on the DSSU to
> handle it. What happens when the free space reported by ZFS isn''t
> really the free space ?
>
>         We are using NBU DSSU against both UFS and ZFS (but not
> against VxFS) and have not noticed any FS related performance
> limitations. The clients and the network are all slower.
>
>

Sri Sudarsan wrote:
> Regarding the question asked below namely "What happens when the free 
> space reported by ZFS isn''t really the free space ?", is
there an open
> bug for this ?
>   
Not a bug.  It is a result of the dynamic nature of ZFS.  For example,
when compression is enabled, we cannot tell in advance how well
the data will compress, so how could we say how much space is
available?  Other items to consider: dynamically allocated, redundant,
and compressed metadata; snapshots; multiple file systems in a pool,
each with potentially different features including compression
algorithms and data redundancy; clones; failed media; failed devices;
etc. Kinda reminds me of the old question: how much stuff can you
put into a hole in your pocket?
 -- richard
> Thanks,
>
> Sri
> Paul Kraus wrote:
>   
>> On 1/15/08, Selim Daoud <selim.daoud at gmail.com> wrote:
>>
>>   
>>     
>>> with zfs you can compress data on disk ...that is a grat advantage
>>> when doing backup to disk
>>> also, for DSSU you need to multiply number of filesystem (1 fs per
>>> stu), the advantage of zfs is that you don''t need to fix
the size
>>> of the fs upfront  (the space is shared among all the fs)
>>>     
>>>       
>>         But ... NBU (at least version 6.0) attempts to estimate the
>> size of the backup and make suer there is enough room on the DSSU to
>> handle it. What happens when the free space reported by ZFS
isn''t
>> really the free space ?
>>
>>         We are using NBU DSSU against both UFS and ZFS (but not
>> against VxFS) and have not noticed any FS related performance
>> limitations. The clients and the network are all slower.
>>
>>   
>>     
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

zfs-discuss-bounces at opensolaris.org wrote on 01/15/2008 03:04:15
PM:
>
> Sri
> Paul Kraus wrote:
> > On 1/15/08, Selim Daoud <selim.daoud at gmail.com> wrote:
> >
> >
> >> with zfs you can compress data on disk ...that is a grat advantage
> >> when doing backup to disk
> >> also, for DSSU you need to multiply number of filesystem (1 fs per
> >> stu), the advantage of zfs is that you don''t need to fix
the size
> >> of the fs upfront  (the space is shared among all the fs)
> >>
> >
> >         But ... NBU (at least version 6.0) attempts to estimate the
> > size of the backup and make suer there is enough room on the DSSU to
> > handle it. What happens when the free space reported by ZFS
isn''t
> > really the free space ?
> >
> >         We are using NBU DSSU against both UFS and ZFS (but not
> > against VxFS) and have not noticed any FS related performance
> > limitations. The clients and the network are all slower.
> >
> >
> Regarding the question asked below namely "What happens when the free
> space reported by ZFS isn''t really the free space ?", is
there an open
> bug for this ?
>
I do not believe it is a ZFS bug.  Consider:

The NetBackup server scans a backup client system,
It determines it will need 600gb of disk space on the disk store.
It stats the zfs volume and sees there is 700 gb free (enough for the
backup)
Starts writing 600gb over multiple hours.
in the meantime, 500gb is used elsewhere in the pool.
NetBackup Fails differently that on vmfs+vxvm in this case?

Isn''t it NetBackups issue to make sure that it has reserved diskspace
or at
least checks for space _as_ it writes?

-Wade

AFAIK , nbu does of estimated the size of backup prior starting the job.
as the backup job is split in fixed-size segments , if a segment does
not fit, it will try to backup into
another disk or will wait for more space

On Jan 15, 2008 8:42 PM, Paul Kraus <pk1048 at gmail.com>
wrote:
> On 1/15/08, Selim Daoud <selim.daoud at gmail.com> wrote:
>
> > with zfs you can compress data on disk ...that is a grat advantage
> > when doing backup to disk
> > also, for DSSU you need to multiply number of filesystem (1 fs per
> > stu), the advantage of zfs is that you don''t need to fix the
size
> > of the fs upfront  (the space is shared among all the fs)
>
>         But ... NBU (at least version 6.0) attempts to estimate the
> size of the backup and make suer there is enough room on the DSSU to
> handle it. What happens when the free space reported by ZFS isn''t
> really the free space ?
>
>         We are using NBU DSSU against both UFS and ZFS (but not
> against VxFS) and have not noticed any FS related performance
> limitations. The clients and the network are all slower.
>
> --
>
{--------1---------2---------3---------4---------5---------6---------7---------}
> Paul Kraus
> -> Sound Designer, Noel Coward''s Hay Fever
>     @ Albany Civic Theatre, Feb./Mar. 2008
> -> Facilities Coordinator, Albacon 2008
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>


-- 
------------------------------------------------------
Blog: http://fakoli.blogspot.com/

Previous posts from various people:
> > >         But ... NBU (at least version 6.0) attempts to estimate
the
> > > size of the backup and make suer there is enough room on the DSSU
to
> > > handle it. What happens when the free space reported by ZFS
isn''t
> > > really the free space ?
> > Regarding the question asked below namely "What happens when the
free
> > space reported by ZFS isn''t really the free space ?", is
there an open
> > bug for this ?
        As others have said, not a ZFS bug, but a feature :-) Of
course, this behavior can be eliminated using ZFS reservations. My
comment was regarding the utility of using one ZFS pool to contain
MULTIPLE NBU Disk Stage Storage Units ... I just don''t see the utility
there and I do see a downside.
> The NetBackup server scans a backup client system,
> It determines it will need 600gb of disk space on the disk store.
> It stats the zfs volume and sees there is 700 gb free (enough for the
> backup)
> Starts writing 600gb over multiple hours.
> in the meantime, 500gb is used elsewhere in the pool.
> NetBackup Fails differently that on vmfs+vxvm in this case?
>
> Isn''t it NetBackups issue to make sure that it has reserved
diskspace or at
> least checks for space _as_ it writes?
        If a disk stage fills during a backup (and there is nothing to
prevent another application from filling it either) it first triggers
DSSU garbage collection to remove the oldest backup images that have
already been duplicated to other storage, if that does not succeed in
freeing up enough space (and I have seen it trigger GC multiple
times), then I have observed two different behaviors (probably related
to differing patch versions):

1. backup fails with a 129 error

2. backup is continued on tape media

        This latter ''solution'' ends up creating a unusable
backup
image as NBU now doesn''t know how to deal with an image that crosses
storage units, but that is very off topic for this list :-)

       My question was more toward how NBU will deal with the apparent
SIZE of the DSSU on ZFS changing on a frequent basis.

-- 
{--------1---------2---------3---------4---------5---------6---------7---------}
Paul Kraus
-> Sound Designer, Noel Coward''s Hay Fever
    @ Albany Civic Theatre, Feb./Mar. 2008
-> Facilities Coordinator, Albacon 2008

unfortunately in this area, Symantec is not helping anyone. they even
take their time to officially include zfs in their compatibility lists
s-

On Jan 16, 2008 1:26 PM, Paul Kraus <pk1048 at gmail.com>
wrote:
> Previous posts from various people:
>
> > > >         But ... NBU (at least version 6.0) attempts to
estimate the
> > > > size of the backup and make suer there is enough room on the
DSSU to
> > > > handle it. What happens when the free space reported by ZFS
isn''t
> > > > really the free space ?
>
> > > Regarding the question asked below namely "What happens when
the free
> > > space reported by ZFS isn''t really the free space
?", is there an open
> > > bug for this ?
>
>         As others have said, not a ZFS bug, but a feature :-) Of
> course, this behavior can be eliminated using ZFS reservations. My
> comment was regarding the utility of using one ZFS pool to contain
> MULTIPLE NBU Disk Stage Storage Units ... I just don''t see the
utility
> there and I do see a downside.
>
> > The NetBackup server scans a backup client system,
> > It determines it will need 600gb of disk space on the disk store.
> > It stats the zfs volume and sees there is 700 gb free (enough for the
> > backup)
> > Starts writing 600gb over multiple hours.
> > in the meantime, 500gb is used elsewhere in the pool.
> > NetBackup Fails differently that on vmfs+vxvm in this case?
> >
> > Isn''t it NetBackups issue to make sure that it has reserved
diskspace or at
> > least checks for space _as_ it writes?
>
>         If a disk stage fills during a backup (and there is nothing to
> prevent another application from filling it either) it first triggers
> DSSU garbage collection to remove the oldest backup images that have
> already been duplicated to other storage, if that does not succeed in
> freeing up enough space (and I have seen it trigger GC multiple
> times), then I have observed two different behaviors (probably related
> to differing patch versions):
>
> 1. backup fails with a 129 error
>
> 2. backup is continued on tape media
>
>         This latter ''solution'' ends up creating a
unusable backup
> image as NBU now doesn''t know how to deal with an image that
crosses
> storage units, but that is very off topic for this list :-)
>
>        My question was more toward how NBU will deal with the apparent
> SIZE of the DSSU on ZFS changing on a frequent basis.
>
> --
>
{--------1---------2---------3---------4---------5---------6---------7---------}
> Paul Kraus
> -> Sound Designer, Noel Coward''s Hay Fever
>     @ Albany Civic Theatre, Feb./Mar. 2008
> -> Facilities Coordinator, Albacon 2008
> _______________________________________________
>
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>


-- 
------------------------------------------------------
Blog: http://fakoli.blogspot.com/

Selim,

Symantec does support ZFS as DSSU targets.  I''ve also seen a SUN white 
paper outlining the use of Thumper (Sun X4500) as a NB 6.5 media server, 
where the best practice was to to configure multiple NB disk storage 
units to use a distinct ZFS file system. In this case, all the ZFS file 
systems serving as DSSUs utilized one zpool.

Hope this helps,

Sri

Selim Daoud wrote:
> unfortunately in this area, Symantec is not helping anyone. they even
> take their time to officially include zfs in their compatibility lists
> s-
>
> On Jan 16, 2008 1:26 PM, Paul Kraus <pk1048 at gmail.com> wrote:
>   
>> Previous posts from various people:
>>
>>     
>>>>>         But ... NBU (at least version 6.0) attempts to
estimate the
>>>>> size of the backup and make suer there is enough room on
the DSSU to
>>>>> handle it. What happens when the free space reported by ZFS
isn''t
>>>>> really the free space ?
>>>>>           
>>>> Regarding the question asked below namely "What happens
when the free
>>>> space reported by ZFS isn''t really the free space
?", is there an open
>>>> bug for this ?
>>>>         
>>         As others have said, not a ZFS bug, but a feature :-) Of
>> course, this behavior can be eliminated using ZFS reservations. My
>> comment was regarding the utility of using one ZFS pool to contain
>> MULTIPLE NBU Disk Stage Storage Units ... I just don''t see the
utility
>> there and I do see a downside.
>>
>>     
>>> The NetBackup server scans a backup client system,
>>> It determines it will need 600gb of disk space on the disk store.
>>> It stats the zfs volume and sees there is 700 gb free (enough for
the
>>> backup)
>>> Starts writing 600gb over multiple hours.
>>> in the meantime, 500gb is used elsewhere in the pool.
>>> NetBackup Fails differently that on vmfs+vxvm in this case?
>>>
>>> Isn''t it NetBackups issue to make sure that it has
reserved diskspace or at
>>> least checks for space _as_ it writes?
>>>       
>>         If a disk stage fills during a backup (and there is nothing to
>> prevent another application from filling it either) it first triggers
>> DSSU garbage collection to remove the oldest backup images that have
>> already been duplicated to other storage, if that does not succeed in
>> freeing up enough space (and I have seen it trigger GC multiple
>> times), then I have observed two different behaviors (probably related
>> to differing patch versions):
>>
>> 1. backup fails with a 129 error
>>
>> 2. backup is continued on tape media
>>
>>         This latter ''solution'' ends up creating a
unusable backup
>> image as NBU now doesn''t know how to deal with an image that
crosses
>> storage units, but that is very off topic for this list :-)
>>
>>        My question was more toward how NBU will deal with the apparent
>> SIZE of the DSSU on ZFS changing on a frequent basis.
>>
>> --
>>
{--------1---------2---------3---------4---------5---------6---------7---------}
>> Paul Kraus
>> -> Sound Designer, Noel Coward''s Hay Fever
>>     @ Albany Civic Theatre, Feb./Mar. 2008
>> -> Facilities Coordinator, Albacon 2008
>> _______________________________________________
>>
>> zfs-discuss mailing list
>> zfs-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>>
>>     
>
>
>
>