xen discuss - Dec 2009 - OpenSolaris + ZFS in DomU - upgrade rpool drive

Good day,

does someone have any luck upgrading rpool drive of OpenSolaris DomU?
I''ve tried in several ways, after my manipulations OS boots but pygrub
can''t find a correct $ZFS-BOOT device, the kernel can''t mount
root and crashes.

Can someone provide a step-by-step instructions? I would be greatly appreciated.
-- 
This message posted from opensolaris.org

Just curious - are you trying to swap out the drive? I haven''t done it
myself but from what I''ve seen people recommend creating a mirror and
then removing the original
-- 
This message posted from opensolaris.org

On Sun, Dec 20, 2009 at 7:33 PM, Michael Disserman <disserman@gmail.com>
wrote:
> Good day,
>
> does someone have any luck upgrading rpool drive of OpenSolaris DomU?
I''ve tried in several ways, after my manipulations OS boots but pygrub
can''t find a correct $ZFS-BOOT device, the kernel can''t mount
root and crashes.
>
> Can someone provide a step-by-step instructions? I would be greatly
appreciated.
I believe the problem is related to pygrub''s zfs and pool version
support. Are you using the same version of opensolaris as dom0 and
domU? Or is it some other OS as dom0?

-- 
Fajar

>from what I''ve seen people recommend creating a mirror and then
removing the original
I have also tried to do this as well as a few other methods. After I replace the
drive pygrub can''t find a valid root device. As well I can''t
boot from mirror. As well you can''t replace a drive in PV DomU (bug
6852962), you must temporary connect a drive to the HVM DomU running
OpenSolaris.
-- 
This message posted from opensolaris.org

>Are you using the same version of opensolaris as dom0 and
domU?

I use 2009.6 for both.
-- 
This message posted from opensolaris.org

On Sat, Dec 26, 2009 at 9:46 PM, Michael Disserman <disserman@gmail.com>
wrote:
>>Are you using the same version of opensolaris as dom0 and
> domU?
>
> I use 2009.6 for both.
I might''ve misunderstood your point. By "upgrading rpool
drive", did
you mean upgrading rpool version? If yes, this doesn''t make sense
since you use the same version for dom0 and domU, and by default
you''ll get newest version available during install time.

If you mean "increase the size of rpool", the steps on a normal
machine should work. Something like
http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide#ZFS_Root_Pool_Recovery
The biggest difference is getting to boot from CD, where you probably
need to have a special domU config file (which is what I did) or just
repeat the steps you did when installing domU (I assume it can get you
a root console booting from CD/ISO image).

Now since you said you tried attaching a mirror and removing the
original, you could probably get it to work by assigning the mirror to
domU as the original drive (xvda), boot domU from CD, import the pool,
export it, shutdown, and then boot domU normally.

Since it''s domU, if all you need is increase the rpool disk size,
there''s probably a much simpler method. Extend the disk backing store
(like by resising the zvol). The hard part then is adjusting the
partition and slice (which I haven''t personally tried) so that zfs
will recognize the new disk size.

-- 
Fajar

>By "upgrading rpool drive", did you mean upgrading rpool version?
no, of course I mean upgrading a drive which contains rpool to the higher
capacity
>Since it''s domU, if all you need is increase the rpool disk size,
there''s probably a much simpler method. Extend the disk backing store
(like by resising the zvol). The hard part then is adjusting the
partition and slice (which I haven''t personally tried) so that zfs
will recognize the new disk size.

that''s one of the ways however I don''t know how correctly
extend the partition and how to extend ZFS when the new partition table is set.
There''s no any resizezfs like i.e. resize2fs in Linux.
-- 
This message posted from opensolaris.org

On Sun, Dec 27, 2009 at 12:03 AM, Michael Disserman <disserman@gmail.com>
wrote:
>>Since it''s domU, if all you need is increase the rpool disk
size,
> there''s probably a much simpler method. Extend the disk backing
store
> (like by resising the zvol). The hard part then is adjusting the
> partition and slice (which I haven''t personally tried) so that zfs
> will recognize the new disk size.
>
> that''s one of the ways however I don''t know how correctly
extend the partition and how to extend ZFS when the new partition table is set.
There''s no any resizezfs like i.e. resize2fs in Linux.
Ah, OK, now we''re getting somewhere :D

What happens is that zfs (until now, I think) will simply use all
available space. So it automatcially extends itself to occupy the
available space after a boot or import. There''s no need for a
resize2fs-equivalent. The term "available space" can differ in meaning
though.

When used directly on top of a block device (like on top of LVM with
zfs-fuse on Linux), if you extend that LV and do export-import of the
pool, zfs will recognize the new size (this is what I usually do).
Same thing goes with partition, if you put zfs on partiton, extend the
partition (i.e. with parted or similar) and zfs will recognize the new
size after the next export-import.

In rpool, zfs uses slice. I don''t now of any program that can safely
extend a solaris slice, so you MIGHT be able to achieve a similar
effect by writing down the old partition table and solaris vtoc, then
creating a new one that reflects the new size. Note that I haven''t
tried this personally (never had the need to), but this was mentioned
some time ago in this list.

As I recall there were also plans to make zfs be able to recoginze the
new size without export/import, and that the "automatic extend"
feature should be tunable via a zpool property, but I haven''t heard
about it for some time, so most likely they still behave like what I
mentioned above.

-- 
Fajar

thanks for the info. I can''t resize a slices until I resize a
partition. But I don''t know how to resize it. If I just delete and
recreate it with fdisk, it changes a number of blocks per cylinder and the old
layout becomes completely inaccessible.
-- 
This message posted from opensolaris.org

Fajar A. Nugraha wrote:
> On Sun, Dec 27, 2009 at 12:03 AM, Michael Disserman
<disserman@gmail.com> wrote:
>   
>>> Since it''s domU, if all you need is increase the rpool
disk size,
>>>       
>> there''s probably a much simpler method. Extend the disk
backing store
>> (like by resising the zvol). The hard part then is adjusting the
>> partition and slice (which I haven''t personally tried) so that
zfs
>> will recognize the new disk size.
>>
>> that''s one of the ways however I don''t know how
correctly extend the partition and how to extend ZFS when the new partition
table is set. There''s no any resizezfs like i.e. resize2fs in Linux.
>>     
>
> Ah, OK, now we''re getting somewhere :D
>
> What happens is that zfs (until now, I think) will simply use all
> available space. So it automatcially extends itself to occupy the
> available space after a boot or import. There''s no need for a
> resize2fs-equivalent. The term "available space" can differ in
meaning
> though.
>
> When used directly on top of a block device (like on top of LVM with
> zfs-fuse on Linux), if you extend that LV and do export-import of the
> pool, zfs will recognize the new size (this is what I usually do).
> Same thing goes with partition, if you put zfs on partiton, extend the
> partition (i.e. with parted or similar) and zfs will recognize the new
> size after the next export-import.
>
> In rpool, zfs uses slice. I don''t now of any program that can
safely
> extend a solaris slice, so you MIGHT be able to achieve a similar
> effect by writing down the old partition table and solaris vtoc, then
> creating a new one that reflects the new size. Note that I haven''t
> tried this personally (never had the need to), but this was mentioned
> some time ago in this list.
>
> As I recall there were also plans to make zfs be able to recoginze the
> new size without export/import, and that the "automatic extend"
> feature should be tunable via a zpool property, but I haven''t
heard
> about it for some time, so most likely they still behave like what I
> mentioned above
The new Pool property is "autoexpand" ;  set it to "on" to
have it
automatically resize the zpool when the underlying device changes 
size.   No import/export needed.  With it set to "off", it will NOT 
change the zpool size, even after an import/export.

It was implemented awhile ago (B117, I think).




-- 
Erik Trimble
Java System Support
Mailstop:  usca22-123
Phone:  x17195
Santa Clara, CA
Timezone: US/Pacific (GMT-0800)

On Sun, Dec 27, 2009 at 2:15 AM, Michael Disserman <disserman@gmail.com>
wrote:
> thanks for the info. I can''t resize a slices until I resize a
partition. But I don''t know how to resize it. If I just delete and
recreate it with fdisk, it changes a number of blocks per cylinder and the old
layout becomes completely inaccessible.
Yeah, I just tried it on Linux dom0, resizing the LVM from 1G to 10G,
and got what you describe :D
There''s a workardound though. Use parted with sectors as unit.
Here''s
what I did (the pool name is test, on opensolaris snv_125)

#=======================================================#parted
/dev/dsk/c7t5d0p0
(parted) unit s
unit s
(parted) print
print
Model: Generic Ide (ide)
Disk /dev/dsk/c7t5d0p0: 20971520s
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start  End       Size      Type     File system  Flags
 1      2048s  2097151s  2095104s  primary               boot

(parted) rm 1
rm 1
(parted) mkpart primary solaris 2048s 100%
mkpart primary solaris 2048s 100%
(parted) toggle 1 boot
toggle 1 boot
(parted) print
print
Model: Generic Ide (ide)
Disk /dev/rdsk/c7t5d0p0: 20971520s
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start  End        Size       Type     File system  Flags
 1      2048s  20964824s  20962777s  primary               boot

(parted) quit
quit
#=======================================================

then I adjust the slices to be like the original one with "format",
plus
#zpool export test
#zpool import test
#zpool set autoexpand=on test

By specifing sectors as unit to parted, modifying partition table and
slices to start on the same place as the original, and setting
autoexpand property to on (as Erik mentioned), the pool was extended
correctly. I haven''t tested this on rpool yet (there''s no
"safe"
snapshot - extend feature on Linux LVM, and I like to keep my rpool
size just the way it is), but it should work.

-- 
Fajar

parted did a trick. thank you very much for the idea!
-- 
This message posted from opensolaris.org