zfs discuss - Oct 2008 - questions about replacing a raidz2 vdev disk with a larger one

I''d like to replace/upgrade two 500GB disks in RaidZ2 vdev with 1TB
disks, but I have some preliminary questions/concerns before trying
''zfs replace dpool ?''

Will ZFS permit this replacement?

Will ZFS use the extra space in a heterogeneous RaidZ2 vdev, or is the size
limited by the smallest disk in the vdev?

Thanks in advance,
Vizzini

The system is currently running snv_98, zfs on disk version of 10, and eight
500GB disks in the following configuration:

NAME        STATE     READ WRITE CKSUM
dpool       ONLINE       0     0     0
  raidz2    ONLINE       0     0     0
    c7t0d0  ONLINE       0     0     0
    c7t1d0  ONLINE       0     0     0
    c7t2d0  ONLINE       0     0     0
    c7t3d0  ONLINE       0     0     0
    c7t4d0  ONLINE       0     0     0
    c7t5d0  ONLINE       0     0     0
    c7t6d0  ONLINE       0     0     0
    c7t7d0  ONLINE       0     0     0
--
This message posted from opensolaris.org

ZFS will allow the replacement.  The available size is, however,
be determined by the smallest of the lot.  Once you''ve replaced
*all* 500GB disks with 1TB disks, the available space will double.

One suggestion: replace as many disks as you intend to at the same time,
so that ZFS only has to do one resilver operation.  It''s faster that
way.

Jeff

On 11 October, 2008 - Vizzini Sampere sent me these 1,4K bytes:
> I''d like to replace/upgrade two 500GB disks in RaidZ2 vdev with
1TB
> disks, but I have some preliminary questions/concerns before trying
> ''zfs replace dpool ???''
> 
> Will ZFS permit this replacement?
Yes.
> Will ZFS use the extra space in a heterogeneous RaidZ2 vdev, or is the
> size limited by the smallest disk in the vdev?
No, until you''ve replaced all disks, it will still be 500G*N.

You can chop up the disk in two 500G chunks and put one into the raidz
and use the other for something else if 500G=>1000G isn''t in the
near
future..

/Tomas
-- 
Tomas ?gren, stric at acc.umu.se, http://www.acc.umu.se/~stric/
|- Student at Computing Science, University of Ume?
`- Sysadmin at {cs,acc}.umu.se

> No, until you''ve replaced all disks, it will still be 500G*N.
ah, thank you very much!

-v
--
This message posted from opensolaris.org

Jeff Bonwick wrote:
> One suggestion: replace as many disks as you intend to at the same time,
> so that ZFS only has to do one resilver operation.  It''s faster
that way.
>
> Jeff
>   
Just to be more clear on this: 

Assuming you have data you care about on the current raidz2 zpool, you 
should replace UP TO [2] drives at once.  That way, you minimize 
re-silver times, while keeping all your data intact.  If you replace 
more than 2 at ones, you''ll destroy the array''s redundancy,
and have to
restore the data from backup.  If you replace one at a time, you''ll
have
to wait for each to resilver before replacing anymore.

If you don''t care about the data, then, just destroy the zpool, replace
the drives, and recreate the zpool from scratch. It''s faster and easier
than waiting for the resilvers.


-- 
Erik Trimble
Java System Support
Mailstop:  usca22-123
Phone:  x17195
Santa Clara, CA

Actually, you can replace them all at once, as long as you don''t unplug
the old ones first.  Let''s say you have a raidz2 setup like this:

mypool
	raidz2
		a
		b
		c
		d

and you say this:

# zpool replace mypool a A
# zpool replace mypool b B
# zpool replace mypool c C
# zpool replace mypool d D

Your pool configuration will then become:

mypool
	raidz2
		replacing
			a
			A
		replacing
			b
			B
		replacing
			c
			C
		replacing
			d
			D

The original drives (a, b, c, d) will remain in the pool until the
new drives (A, B, C, D) have all the data, at which point the old
drives will be detached and the final pool configuration will be:

mypool
	raidz2
		A
		B
		C
		D

This assumes, of course, that you have enough slots to plug them all in.
If you''re slot-limited -- i.e. you can''t add a new drive
without pulling
and old one -- then Eric is right, and in fact I''d go further: in that
case, replace only one at a time so you maintain the ability to survive
a disk failing while you''re going all this.

Jeff

On Sat, Oct 11, 2008 at 06:37:17PM -0700, Erik Trimble
wrote:
> Jeff Bonwick wrote:
> >One suggestion: replace as many disks as you intend to at the same
time,
> >so that ZFS only has to do one resilver operation.  It''s
faster that way.
> >
> >Jeff
> >  
> Just to be more clear on this: 
> 
> Assuming you have data you care about on the current raidz2 zpool, you 
> should replace UP TO [2] drives at once.  That way, you minimize 
> re-silver times, while keeping all your data intact.  If you replace 
> more than 2 at ones, you''ll destroy the array''s
redundancy, and have to
> restore the data from backup.  If you replace one at a time,
you''ll have
> to wait for each to resilver before replacing anymore.
> 
> If you don''t care about the data, then, just destroy the zpool,
replace
> the drives, and recreate the zpool from scratch. It''s faster and
easier
> than waiting for the resilvers.
> 
> 
> -- 
> Erik Trimble
> Java System Support
> Mailstop:  usca22-123
> Phone:  x17195
> Santa Clara, CA
>