Hi, I had a plan to set up a zfs pool with different raid levels but I ran into an issue based on some testing I''ve done in a VM. I have 3x 750 GB hard drives and 2x 320 GB hard drives available, and I want to set up a RAIDZ for the 750 GB and mirror for the 320 GB and add it all to the same pool. I tested detaching a drive and it seems to seriously mess up the entire pool and I can''t figure out a way to recover. # zpool create mypool raidz c1t1d0 c1t2d0 c1t3d0 # zpool add -f mypool mirror c1t4d0 c1t5d0 # zpool status pool: mypool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM mypool ONLINE 0 0 0 raidz1 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 mirror ONLINE 0 0 0 c1t4d0 ONLINE 0 0 0 c1t5d0 ONLINE 0 0 0 errors: No known data errors So far so good...now try to detach a drive. # zpool detach mypool c1t4d0 # zpool status pool: mypool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM mypool ONLINE 0 0 0 raidz1 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 c1t5d0 ONLINE 0 0 0 c1t5d0 was part of a mirror but with c1t4d0 removed it now appears as a single drive. Is there a way to recover from this by recreating the mirror with c1t4d0? I''ve also heard that you can upgrade disks in a raidz one at a time to a higher capacity but I can''t detach or remove any of the disks in the raidz. I''m guessing that is because there''s no spare drive and the only way to do it is to remove the drive physically and stick a new one in. It would be degraded and a zfs replace could be done. Is that right? Thanks.
On 07 June, 2008 - Fu Leow sent me these 2,0K bytes:> Hi, > > I had a plan to set up a zfs pool with different raid levels but I ran > into an issue based on some testing I''ve done in a VM. I have 3x 750 > GB hard drives and 2x 320 GB hard drives available, and I want to set > up a RAIDZ for the 750 GB and mirror for the 320 GB and add it all to > the same pool. > > # zpool detach mypool c1t4d0 > # zpool status > pool: mypool > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > mypool ONLINE 0 0 0 > raidz1 ONLINE 0 0 0 > c1t1d0 ONLINE 0 0 0 > c1t2d0 ONLINE 0 0 0 > c1t3d0 ONLINE 0 0 0 > c1t5d0 ONLINE 0 0 0 > > c1t5d0 was part of a mirror but with c1t4d0 removed it now appears as > a single drive. Is there a way to recover from this by recreating the > mirror with c1t4d0?zpool attach mypool c1t5d0 c1t4d0> I''ve also heard that you can upgrade disks in a raidz one at a time to > a higher capacity but I can''t detach or remove any of the disks in the > raidz. I''m guessing that is because there''s no spare drive and the > only way to do it is to remove the drive physically and stick a new > one in. It would be degraded and a zfs replace could be done. Is that > right?zpool replace mypool c1t1d0 c1t6d0 should work.. .. or just yank a drive out and put a different one in, and then run zpool scrub mypool repeat for t1..t3 /Tomas -- Tomas ?gren, stric at acc.umu.se, http://www.acc.umu.se/~stric/ |- Student at Computing Science, University of Ume? `- Sysadmin at {cs,acc}.umu.se
> c1t5d0 was part of a mirror but with c1t4d0 removed it now appears as > a single drive. Is there a way to recover from this by recreating the > mirror with c1t4d0?Detaching a drive from a two-way mirror effectively breaks it up and turns it into a single drive. That''s normal. Just attach it back to c1t5d0 and it''ll become a mirror again. Retry your experiment by detaching a drive from the RAID-Z array and you''ll see what you were expecting. -mg
On Sat, Jun 7, 2008 at 4:13 AM, Mario Goebbels <me at tomservo.cc> wrote:>> c1t5d0 was part of a mirror but with c1t4d0 removed it now appears as >> a single drive. Is there a way to recover from this by recreating the >> mirror with c1t4d0? > > Detaching a drive from a two-way mirror effectively breaks it up and > turns it into a single drive. That''s normal. Just attach it back to > c1t5d0 and it''ll become a mirror again. > > Retry your experiment by detaching a drive from the RAID-Z array and > you''ll see what you were expecting. > > -mg >Thank you Tomas and Mario. I was able to recreate the mirror by reattaching. It seems that once a vdev has been added to a pool it is no longer possible to remove it. Is that right? For example if I created a pool that consisted of 3 single drives there would not be a way to remove a drive and reduce the number of devices in a pool from 3 to 2. Even if there is enough space in those 2 remaining drives to hold all the data.
On 07 June, 2008 - Fu Leow sent me these 1,1K bytes:> On Sat, Jun 7, 2008 at 4:13 AM, Mario Goebbels <me at tomservo.cc> wrote: > >> c1t5d0 was part of a mirror but with c1t4d0 removed it now appears as > >> a single drive. Is there a way to recover from this by recreating the > >> mirror with c1t4d0? > > > > Detaching a drive from a two-way mirror effectively breaks it up and > > turns it into a single drive. That''s normal. Just attach it back to > > c1t5d0 and it''ll become a mirror again. > > > > Retry your experiment by detaching a drive from the RAID-Z array and > > you''ll see what you were expecting. > > > > -mg > > > > Thank you Tomas and Mario. I was able to recreate the mirror by reattaching. > > It seems that once a vdev has been added to a pool it is no longer > possible to remove it. Is that right? For example if I created a pool > that consisted of 3 single drives there would not be a way to remove a > drive and reduce the number of devices in a pool from 3 to 2. Even if > there is enough space in those 2 remaining drives to hold all the > data.Currently, yes. It''s being worked on as far as I know. /Tomas -- Tomas ?gren, stric at acc.umu.se, http://www.acc.umu.se/~stric/ |- Student at Computing Science, University of Ume? `- Sysadmin at {cs,acc}.umu.se