zfs discuss - Jul 2008 - ZFS boot - upgrade from UFS & swap slices

I''m upgrading my B92 UFS-boot system to ZFS root using Live Upgrade. 
It
appears to work fine so far, but I''m wondering why it allocates a ZFS 
filesystem for swap when I already have a dedicated swap slice. 
Shouldn''t it just use any existing swap slice rather than creating a
ZFS
one?

-- 
Alan Burlison
--

Hi Alan,

ZFS doesn''t swap to a slice in build 92. In this build, a ZFS root
environment requires separate ZFS volumes for swap and dump devices.

The ZFS boot/install project and information trail starts here:

http://opensolaris.org/os/community/zfs/boot/

Cindy





Alan Burlison wrote:
> I''m upgrading my B92 UFS-boot system to ZFS root using Live
Upgrade.  It
> appears to work fine so far, but I''m wondering why it allocates a
ZFS
> filesystem for swap when I already have a dedicated swap slice. 
> Shouldn''t it just use any existing swap slice rather than creating
a ZFS
> one?
>

Cindy.Swearingen at Sun.COM wrote:
> ZFS doesn''t swap to a slice in build 92. In this build, a ZFS root
> environment requires separate ZFS volumes for swap and dump devices.
> 
> The ZFS boot/install project and information trail starts here:
> 
> http://opensolaris.org/os/community/zfs/boot/
Is this going to be supported in a later build?

I got it to use the existing swap slice by manually reconfiguring the 
ZFS-root BE post-install to use the swap slice as swap & dump - the 
resulting BE seems to work just fine, so I''m not sure why LU insists on
creating ZFS swap & dump.

Basically I want to migrate my root filesystem from UFS to ZFS and leave 
everything else as it it, there doesn''t seem to be a way to do this.

-- 
Alan Burlison
--

Alan,

Just make sure you use dumpadm to point to valid dump device and
this setup should work fine. Please let us know if it doesn''t.

The ZFS strategy behind automatically creating separate swap and
dump devices including the following:

o Eliminates the need to create separate slices
o Enables underlying ZFS architecture for swap and dump devices
o Enables you to set characteristics like compression on swap
and dump devices, and eventually, encryption

Cindy

Alan Burlison wrote:
> Cindy.Swearingen at Sun.COM wrote:
> 
>> ZFS doesn''t swap to a slice in build 92. In this build, a ZFS
root
>> environment requires separate ZFS volumes for swap and dump devices.
>>
>> The ZFS boot/install project and information trail starts here:
>>
>> http://opensolaris.org/os/community/zfs/boot/
> 
> 
> Is this going to be supported in a later build?
> 
> I got it to use the existing swap slice by manually reconfiguring the 
> ZFS-root BE post-install to use the swap slice as swap & dump - the 
> resulting BE seems to work just fine, so I''m not sure why LU
insists on
> creating ZFS swap & dump.
> 
> Basically I want to migrate my root filesystem from UFS to ZFS and leave 
> everything else as it it, there doesn''t seem to be a way to do
this.
>

Alan Burlison wrote:
>Cindy.Swearingen at Sun.COM wrote:
>
>  
>
>>ZFS doesn''t swap to a slice in build 92. In this build, a ZFS
root
>>environment requires separate ZFS volumes for swap and dump devices.
>>
>>The ZFS boot/install project and information trail starts here:
>>
>>http://opensolaris.org/os/community/zfs/boot/
>>    
>>
>
>Is this going to be supported in a later build?
>
>I got it to use the existing swap slice by manually reconfiguring the 
>ZFS-root BE post-install to use the swap slice as swap & dump - the 
>resulting BE seems to work just fine, so I''m not sure why LU
insists on
>creating ZFS swap & dump.
>
>Basically I want to migrate my root filesystem from UFS to ZFS and leave 
>everything else as it it, there doesn''t seem to be a way to do
this.
>  
>
It''s hard to know what the "right" thing to do is from within
the installation software.  Does the user want to preserve
as much of their current environment as possible?  Or does
the user want to move toward the new "standard" configuration
(which is pretty much zfs-everything)?  Or something in between?

In designing the changes to the install software, we had to
decide whether to be all things to all people or make some
default choices.  Being all things to all people makes the
interface a lot more complicated and takes a lot more
engineering effort (we''d still be developing it and zfs boot
would not be available if we''d taken that path).  We
erred on the "make default choices" side (although with
some opportunities for customization), and leaned toward
the "move the system toward zfs" side in our choices for
those defaults.  We leaned a little too far in that direction in
our selection of default choices for swap/dump space in
the interactive install and so we''re fixing that.

In this case, LU does move the system toward using
swap and dump zvols within the root pool.  If you really
don''t want that, you can still use your existing swap and
dump slice and delete the swap/dump zvol.   I know it''s
not ideal because it requires some manual steps, and maybe
you''ll have to repeat those manual actions with subsequent
lucreates (or maybe not, I''m actually not sure how that works).
But is there any really good reason NOT to move to the
use of swap/dump zvols?  If your existing swap/dump slice
is contiguous with your root pool, you can grow the root
pool into that space (using format to merge the slices.
A reboot or re-import of the pool will cause it to grow into
the newly-available space).

Keep these comments coming!  We''ve tried to make the
best choices, balancing all the many considerations, but
as in the case or swap, I''m sure we made some choices
that were wrong or at least non-optimal and we want to
continue to refine how zfs works as a root file system. 

Lori

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080724/7abefde9/attachment.html>

Cindy.Swearingen at Sun.COM wrote:
> Alan,
> 
> Just make sure you use dumpadm to point to valid dump device and
> this setup should work fine. Please let us know if it doesn''t.
> 
> The ZFS strategy behind automatically creating separate swap and
> dump devices including the following:
> 
> o Eliminates the need to create separate slices
> o Enables underlying ZFS architecture for swap and dump devices
> o Enables you to set characteristics like compression on swap
> and dump devices, and eventually, encryption
Hi
also makes resizing easy to do as well.
ie
zfs set volsize=8G lupool/dump


Enda
> 
> Cindy
> 
> Alan Burlison wrote:
>> Cindy.Swearingen at Sun.COM wrote:
>>
>>> ZFS doesn''t swap to a slice in build 92. In this build, a
ZFS root
>>> environment requires separate ZFS volumes for swap and dump
devices.
>>>
>>> The ZFS boot/install project and information trail starts here:
>>>
>>> http://opensolaris.org/os/community/zfs/boot/
>>
>> Is this going to be supported in a later build?
>>
>> I got it to use the existing swap slice by manually reconfiguring the 
>> ZFS-root BE post-install to use the swap slice as swap & dump - the
>> resulting BE seems to work just fine, so I''m not sure why LU
insists on
>> creating ZFS swap & dump.
>>
>> Basically I want to migrate my root filesystem from UFS to ZFS and
leave
>> everything else as it it, there doesn''t seem to be a way to do
this.
>>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Cindy.Swearingen at Sun.COM wrote:
> Just make sure you use dumpadm to point to valid dump device and
> this setup should work fine. Please let us know if it doesn''t.
Yep, works fine.
> The ZFS strategy behind automatically creating separate swap and
> dump devices including the following:
> 
> o Eliminates the need to create separate slices
> o Enables underlying ZFS architecture for swap and dump devices
> o Enables you to set characteristics like compression on swap
> and dump devices, and eventually, encryption
Makes sense.  I assume that unless you are actually using swap or dump, 
no space is actually used (other than the zfs filesystem overhead) - is 
that correct?

I''m now coming up with a mad scheme involving ZFS boot, a USB disk, 
string and prayer to enable me to get rid of the old UFS root & swap 
slices on my root disk and switch over to UFS :-)

-- 
Alan Burlison
--

Lori Alt wrote:
> In designing the changes to the install software, we had to
> decide whether to be all things to all people or make some
> default choices.  Being all things to all people makes the
> interface a lot more complicated and takes a lot more
> engineering effort (we''d still be developing it and zfs boot
> would not be available if we''d taken that path).  We
> erred on the "make default choices" side (although with
> some opportunities for customization), and leaned toward
> the "move the system toward zfs" side in our choices for
> those defaults.  We leaned a little too far in that direction in
> our selection of default choices for swap/dump space in
> the interactive install and so we''re fixing that.
Great, thanks :-)  Is this just a case of letting people use the -m flag 
with the ZFS filesystem type, and allowing the dump device to be 
specified with -m, or is there more to it than that?

I also notice that there doesn''t appear to be any way to specify the 
size of the swap & dump areas when migrating - I thought I saw somewhere 
in the documentation that swap is sized to 1/2 physmem.  That might be 
problematic on machines with large amounts of memory.
> In this case, LU does move the system toward using
> swap and dump zvols within the root pool.  If you really
> don''t want that, you can still use your existing swap and
> dump slice and delete the swap/dump zvol.   I know it''s
> not ideal because it requires some manual steps, and maybe
> you''ll have to repeat those manual actions with subsequent
> lucreates (or maybe not, I''m actually not sure how that works).
Seems to work, at least if you create the new BE in the same pool as the 
source BE.  I can''t get it to work if I try to use a different pool
though.
> But is there any really good reason NOT to move to the
> use of swap/dump zvols?  If your existing swap/dump slice
> is contiguous with your root pool, you can grow the root
> pool into that space (using format to merge the slices.
> A reboot or re-import of the pool will cause it to grow into
> the newly-available space).
For some reason, when I initially installed I ended up with this:

Part      Tag    Flag     Cylinders         Size            Blocks
   0       root    wm    1046 -  2351       10.00GB    (1306/0/0)   20980890
   1       swap    wu       1 -  1045        8.01GB    (1045/0/0)   16787925

So physically swap comes before root, so I can''t do the trick you 
suggested.  I also have a second root slice that comes just after the 
first one, for my second current LU environment.  Really I want to 
collapse  those 3 into 1.  What I''m planning to do is evacuate 
everything else off my first disk onto a USB disk, then re-layout the 
disk.  Everything else on the machine bar the boot slices is already 
ZFS, so I can create a ZFS BE in the pool on my 2nd disk, boot into that 
then re-layout the first disk.
> Keep these comments coming!  We''ve tried to make the
> best choices, balancing all the many considerations, but
> as in the case or swap, I''m sure we made some choices
> that were wrong or at least non-optimal and we want to
> continue to refine how zfs works as a root file system.
I''m really liking what I see so far, it''s just a question of
getting my
head around the best way of setting things up, and figuring out the 
easiest way of migrating.

-- 
Alan Burlison
--

Lori Alt <Lori.Alt at Sun.COM> writes:
> use of swap/dump zvols?  If your existing swap/dump slice
> is contiguous with your root pool, you can grow the root
> pool into that space (using format to merge the slices.
> A reboot or re-import of the pool will cause it to grow into
> the newly-available space).
That had been my plan, and that''s how I laid out my slices for zpools
and
UFS BEs before ZFS boot came along.  Unfortunately, at least once this
resizing exercise went wrong, fatally, it seems, but so far nobody cared to
comment:

http://mail.opensolaris.org/pipermail/zfs-discuss/2008-July/049180.html

And on SPARC, the hopefully safe method from a failsafe environment is
hampered by

http://mail.opensolaris.org/pipermail/install-discuss/2008-July/006754.html

I think at least the second issue needs to be resolved before ZFS root is
appropriate for general use.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Faculty of Technology, Bielefeld University

Alan Burlison wrote:
>Lori Alt wrote:
>
>  
>
>>In designing the changes to the install software, we had to
>>decide whether to be all things to all people or make some
>>default choices.  Being all things to all people makes the
>>interface a lot more complicated and takes a lot more
>>engineering effort (we''d still be developing it and zfs boot
>>would not be available if we''d taken that path).  We
>>erred on the "make default choices" side (although with
>>some opportunities for customization), and leaned toward
>>the "move the system toward zfs" side in our choices for
>>those defaults.  We leaned a little too far in that direction in
>>our selection of default choices for swap/dump space in
>>the interactive install and so we''re fixing that.
>>    
>>
>
>Great, thanks :-)  Is this just a case of letting people use the -m flag 
>with the ZFS filesystem type, and allowing the dump device to be 
>specified with -m, or is there more to it than that?
>
The changes to which I was referring are to the interactive initial 
install program,
not LU.

I''m checking into how swap and dump zvols are sized in LU.

The two changes we made to the interactive initial install are:

1) Change the default size of the swap and dump zvols to 1/2 of
   physmem, but no more than 2 GB, and no less than 512 MB.
   Previously, we were allowing swap and dump to go as high
   as 32 GB, but that''s just way too much for some systems.  Not
   only were we setting the size too high, we didn''t give the user the
   opportunity to change it, so we made this change:
2)  Allow the user to change the swap and dump size to anything
   from 0 (i.e. no swap or dump zvol at all) to the maximum size
   that will fit in the pool).   We don''t recommend the smaller sizes
   (especially for dump.  A system that can''t take a crash dump is
   a system with a serviceability problem.), but we won''t prevent
   users from setting it up that way.

>I also notice that there doesn''t appear to be any way to specify
the
>size of the swap & dump areas when migrating - I thought I saw somewhere
>in the documentation that swap is sized to 1/2 physmem.  That might be 
>problematic on machines with large amounts of memory.
>
>  
>
>>In this case, LU does move the system toward using
>>swap and dump zvols within the root pool.  If you really
>>don''t want that, you can still use your existing swap and
>>dump slice and delete the swap/dump zvol.   I know it''s
>>not ideal because it requires some manual steps, and maybe
>>you''ll have to repeat those manual actions with subsequent
>>lucreates (or maybe not, I''m actually not sure how that works).
>>    
>>
>
>Seems to work, at least if you create the new BE in the same pool as the 
>source BE.  I can''t get it to work if I try to use a different pool
though.
>
>  
>
>>But is there any really good reason NOT to move to the
>>use of swap/dump zvols?  If your existing swap/dump slice
>>is contiguous with your root pool, you can grow the root
>>pool into that space (using format to merge the slices.
>>A reboot or re-import of the pool will cause it to grow into
>>the newly-available space).
>>    
>>
>
>For some reason, when I initially installed I ended up with this:
>
>Part      Tag    Flag     Cylinders         Size            Blocks
>   0       root    wm    1046 -  2351       10.00GB    (1306/0/0)   20980890
>   1       swap    wu       1 -  1045        8.01GB    (1045/0/0)   16787925
>
>So physically swap comes before root, so I can''t do the trick you 
>suggested.  I also have a second root slice that comes just after the 
>first one, for my second current LU environment.  Really I want to 
>collapse  those 3 into 1.  What I''m planning to do is evacuate 
>everything else off my first disk onto a USB disk, then re-layout the 
>disk.  Everything else on the machine bar the boot slices is already 
>ZFS, so I can create a ZFS BE in the pool on my 2nd disk, boot into that 
>then re-layout the first disk.
>  
>
What if you turned slice 1 into a pool (a new one), migrated your BE 
into it,
then grow that pool to soak up the space in the slices that follow it?  
You might
still need to save some stuff elsewhere while you''re doing the
transition.

Just a suggestion.  It sounds like you''re working out a plan.

Lori
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080724/5a163111/attachment.html>

Lori Alt wrote:
> What if you turned slice 1 into a pool (a new one), migrated your BE 
> into it,
> then grow that pool to soak up the space in the slices that follow it?  
> You might
> still need to save some stuff elsewhere while you''re doing the
transition.
Doesn''t work, because LU wants to create both swap & dump ZFS 
filesystems in there too, my machine has 16Gb of memory and the slice is 
8Gb - so there isn''t enough space & LU throws a cog.  Which is why
I
wanted to get it to use the old swap partition in the first place...

-- 
Alan Burlison
--

Alan Burlison wrote:
> Lori Alt wrote:
>
>> What if you turned slice 1 into a pool (a new one), migrated your BE 
>> into it,
>> then grow that pool to soak up the space in the slices that follow 
>> it?  You might
>> still need to save some stuff elsewhere while you''re doing the
>> transition.
>
>
> Doesn''t work, because LU wants to create both swap & dump ZFS 
> filesystems in there too, my machine has 16Gb of memory and the slice 
> is 8Gb - so there isn''t enough space & LU throws a cog.  Which
is why
> I wanted to get it to use the old swap partition in the first place...

Sounds like LU needs some of the same swap/dump flexibility
that we just gave initial install.  I''ll bring this up within the team.

Lori

I will look into this.  I don''t know why it would have failed.

Lori

Rainer Orth wrote:
>Lori Alt <Lori.Alt at Sun.COM> writes:
>
>  
>
>>use of swap/dump zvols?  If your existing swap/dump slice
>>is contiguous with your root pool, you can grow the root
>>pool into that space (using format to merge the slices.
>>A reboot or re-import of the pool will cause it to grow into
>>the newly-available space).
>>    
>>
>
>That had been my plan, and that''s how I laid out my slices for
zpools and
>UFS BEs before ZFS boot came along.  Unfortunately, at least once this
>resizing exercise went wrong, fatally, it seems, but so far nobody cared to
>comment:
>
>http://mail.opensolaris.org/pipermail/zfs-discuss/2008-July/049180.html
>
>And on SPARC, the hopefully safe method from a failsafe environment is
>hampered by
>
>http://mail.opensolaris.org/pipermail/install-discuss/2008-July/006754.html
>
>I think at least the second issue needs to be resolved before ZFS root is
>appropriate for general use.
>
>	Rainer
>
>  
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080724/5e7139e3/attachment.html>

Lori Alt wrote:
> Sounds like LU needs some of the same swap/dump flexibility
> that we just gave initial install.  I''ll bring this up within the
team.
The (partial) workaround I tried was:

1. create a ZFS BE in an existing pool that has enough space
2  lumount the BE, edit the vfstab to use the old swap slice
3. activate the new BE & boot it
4. use dumpadm to switch to using swap
5. delete the ZFS swap & dump filesystems created in step 1
6. delete the original UFS BE so we can reuse the slice
7. create a new ZFS pool in the old UFS slice

It then falls apart at the next step:

8. copy the ZFS BE into the new pool made from the UFS BE

because I can''t get LU to create the BE in a different ZFS pool.

-- 
Alan Burlison
--

Lori Alt wrote:
> It''s hard to know what the "right" thing to do is from
within
> the installation software.  Does the user want to preserve
> as much of their current environment as possible?  Or does
> the user want to move toward the new "standard" configuration
> (which is pretty much zfs-everything)?  Or something in between?
It''s all a bit academic now anyway, as LU has for some reason decided
to
stop installing entries in menu.lst, no matter what I do.  No error 
messages, no warnings, just doesn''t work.  Bizzare - this did work at 
one point.  I''ve tried blitzing and reinstalling LU entirely - still no
joy.

-- 
Alan Burlison
--

Alan Burlison wrote:
> Lori Alt wrote:
> 
>> It''s hard to know what the "right" thing to do is
from within
>> the installation software.  Does the user want to preserve
>> as much of their current environment as possible?  Or does
>> the user want to move toward the new "standard" configuration
>> (which is pretty much zfs-everything)?  Or something in between?
> 
> It''s all a bit academic now anyway, as LU has for some reason
decided to
> stop installing entries in menu.lst, no matter what I do.  No error 
> messages, no warnings, just doesn''t work.  Bizzare - this did work
at
> one point.  I''ve tried blitzing and reinstalling LU entirely -
still no joy.
> 
probably
6722767 lucreate did not add new BE to menu.lst ( or grub )

On Thu, 24 Jul 2008, Alan Burlison wrote:
>> What if you turned slice 1 into a pool (a new one), migrated your BE 
>> into it, then grow that pool to soak up the space in the slices that 
>> follow it? You might still need to save some stuff elsewhere while 
>> you''re doing the transition.
>
> Doesn''t work, because LU wants to create both swap & dump ZFS 
> filesystems in there too, my machine has 16Gb of memory and the slice is 
> 8Gb - so there isn''t enough space & LU throws a cog.  Which is
why I
> wanted to get it to use the old swap partition in the first place...
As a workaround, you can pre-create the swap & dump zvols.  E.g.:

 	zfs create -V 512m {pool}/swap
 	zfs create -V 2g {pool}/dump

If LU sees that the zvols already exist, it assumes they are correctly 
sized.


Regards,
markm

Enda O''Connor wrote:
> probably
> 6722767 lucreate did not add new BE to menu.lst ( or grub )
Yeah, I found that bug, added A CR & bumped the priority.  Unfortunatrly 
there''s no analysis or workaround in the bug, so I''ve no idea
what the
real problem is.

-- 
Alan Burlison
--

Mark J Musante wrote:
> As a workaround, you can pre-create the swap & dump zvols.  E.g.:
> 
>     zfs create -V 512m {pool}/swap
>     zfs create -V 2g {pool}/dump
> 
> If LU sees that the zvols already exist, it assumes they are correctly 
> sized.
Nice tip, thanks :-)

-- 
Alan Burlison
--

On Fri, 25 Jul 2008, Alan Burlison wrote:
> Enda O''Connor wrote:
>
>> probably
>> 6722767 lucreate did not add new BE to menu.lst ( or grub )
>
> Yeah, I found that bug, added A CR & bumped the priority. 
> Unfortunatrly there''s no analysis or workaround in the bug, so
I''ve no
> idea what the real problem is.
Alan, can you point me at your machine (if it''s on SWAN)?  I''d
like to see
what''s going on in there.


Regards,
markm

Mark J Musante wrote:
> Alan, can you point me at your machine (if it''s on SWAN)? 
I''d like to
> see what''s going on in there.
Might be easiest to use sun shared shell to get you access...

-- 
Alan Burlison
--

Mark J Musante wrote:
> Alan, can you point me at your machine (if it''s on SWAN)? 
I''d like to
> see what''s going on in there.
Many thanks to Mark for his help, I eventually figured out the problem:

6730154 Grub: findroot fails to find ZFS BE

Precis: USB disks, LU & ZFS boot don''t mix.

-- 
Alan Burlison
--