zfs discuss - Feb 2007 - Downsides to zone roots on ZFS?

Hi all,

Last time I checked, having one''s zone roots (zonepaths) on
ZFS file systems was not a recommended practice, despite the
fact that this works.  IIRC, the problem was that the upgrade
code didn''t grok zfs and would therefore get terribly confused
should the zone roots reside on ZFS.

However, given that S10 11/06 doesn''t support upgrading of zones
anyway (even if they reside on UFS file systems), is this point
moot?  (Or is that applicable to live upgrade only?)

More succinctly: apart from not being able to upgrade (presumably
an initial reinstall will be required), what are the reasons for
NOT hosting zone roots on ZFS?

Background: I have a Nevada build 33 server that I will be putting
into production very soon, by nuking it and doing a fresh install
of S10 11/06.  The server''s zones currently live on ZFS, and I was
hoping to duplicate this arrangement on the production installation.

TIA,

-- 
Rich Teer, SCSA, SCNA, SCSECA, OpenSolaris CAB member

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

Rich,

Rich Teer wrote:
> Hi all,
> 
> Last time I checked, having one''s zone roots (zonepaths) on
> ZFS file systems was not a recommended practice, despite the
> fact that this works.  IIRC, the problem was that the upgrade
> code didn''t grok zfs and would therefore get terribly confused
> should the zone roots reside on ZFS.
> 
> However, given that S10 11/06 doesn''t support upgrading of zones
> anyway (even if they reside on UFS file systems), is this point
> moot?  (Or is that applicable to live upgrade only?)
This is incorrect.  All S10 updates have supported upgrading systems
with zones.  I believe what you are thinking of is that live-upgrade
does not support upgrading systems with zones.  This is being
fixed in the next S10 update.  It is already fixed in nevada.
> More succinctly: apart from not being able to upgrade (presumably
> an initial reinstall will be required), what are the reasons for
> NOT hosting zone roots on ZFS?
That is the only real reason.  The only other reason I know of
is fairly obscure.  The patch tools don''t know about zfs so they
can miscalculate space when you have a set of zones, each on
their own zfs dataset, but in the same zpool.  If you were really
tight on space, the patch process might fail partway through as a
result.  This is probably not an issue for most people but is
the only other one I know of.

Jerry

John Clingan wrote:
>> This is incorrect.  All S10 updates have supported upgrading systems
>> with zones.  I believe what you are thinking of is that live-upgrade
>> does not support upgrading systems with zones.  This is being
>> fixed in the next S10 update.  It is already fixed in nevada.
>>
> Which Nevada build?
The install code changes look like they were integrated into b53.  Some
of the zones support that was needed had been integrated earlier.
The bug is 6264796.
> When will zone roots on ZFS be supported by live upgrade?
I don''t know.  Asking the install team would be your best bet.

Jerry

On Wed, 7 Feb 2007, Jerry Jelinek wrote:
> This is incorrect.  All S10 updates have supported upgrading systems
> with zones.  I believe what you are thinking of is that live-upgrade
> does not support upgrading systems with zones.  This is being
> fixed in the next S10 update.  It is already fixed in nevada.
Gotcha; thanks for the clarification.
> That is the only real reason.  The only other reason I know of
> is fairly obscure.  The patch tools don''t know about zfs so they
> can miscalculate space when you have a set of zones, each on
> their own zfs dataset, but in the same zpool.  If you were really
> tight on space, the patch process might fail partway through as a
> result.  This is probably not an issue for most people but is
> the only other one I know of.
Excellent; disk space won''t be an issue for me, nor will the
non-live-upgradability, so I''ll be putting my zone roots on
ZFS.

Much obliged!

-- 
Rich Teer, SCSA, SCNA, SCSECA, OpenSolaris CAB member

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

Jerry Jelinek wrote:
> John Clingan wrote:
>>> This is incorrect.  All S10 updates have supported upgrading
systems
>>> with zones.  I believe what you are thinking of is that
live-upgrade
>>> does not support upgrading systems with zones.  This is being
>>> fixed in the next S10 update.  It is already fixed in nevada.
>>>
>> Which Nevada build?
>
> The install code changes look like they were integrated into b53.  Some
> of the zones support that was needed had been integrated earlier.
> The bug is 6264796.
>
>> When will zone roots on ZFS be supported by live upgrade?
>
> I don''t know.  Asking the install team would be your best bet.
Zone roots on ZFS will definitely be supported by liveupgrade
at the time that zfs as a root file system is supported, currently
planned for Update 5.  There is a possibility that liveupgrade for
zone roots on ZFS will be supported earlier than Update 5, if it
can be released as a bug fix. 

Lori

Rich Teer wrote:
> Excellent; disk space won''t be an issue for me, nor will the
> non-live-upgradability, so I''ll be putting my zone roots on
> ZFS.
Rich,

Just to be clear, both live-upgrade and the mini-root upgrade
do not yet know about zfs so if you place your zones on zfs,
you won''t be able to do either style of upgrade until that is fixed.

Jerry

On Wed, 7 Feb 2007, Jerry Jelinek wrote:
> Just to be clear, both live-upgrade and the mini-root upgrade
> do not yet know about zfs so if you place your zones on zfs,
> you won''t be able to do either style of upgrade until that is
fixed.
Understood; that won''t be an issue for me.

Many thanks for answering my question.  Hopefully my noisy X4200
will be installed in the data centre tomorrow (Thursday); I had
a set back today while fighting with the Remote Console feature
of ILOM 1.1.1 (i.e., it doesn''t work).  :-(

-- 
Rich Teer, SCSA, SCNA, SCSECA, OpenSolaris CAB member

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

>Many thanks for answering my question.  Hopefully my noisy X4200
>will be installed in the data centre tomorrow (Thursday); I had
>a set back today while fighting with the Remote Console feature
>of ILOM 1.1.1 (i.e., it doesn''t work).  :-(
Just ssh into it and use the serial console from within SSH.

Casper

Ivan Buetler wrote:
> Is this true for OpenSolaris? My experience: 
> 
> I was trying to upgrade from "SunOS 5.11 snv_28" to "SunOS
5.11 snv_54" where
> my NGZ zone roots were set to a zfs mount point like below:
> 
> NAME                 USED  AVAIL  REFER  MOUNTPOINT
> zpool               93.8G  40.1G    26K  /zpool
> zpool/zones         3.50G  40.1G  1.68G  /zpool/zones
> 
> Upgrading to SNV_54 did not work for me (CD|DVD|Live-Upgrade). The install 
> procedure was cancelled after it came to the NGZ ZFS setup part. However -
I
> was enforced to to a full re-install of the whole OS. By this time, I
decided
> to have an OS independent application setup: I decided to leave all my 
> Non-Solaris apps within the following structure:
> 
> NAME                 USED  AVAIL  REFER  MOUNTPOINT
> zpool               93.8G  40.1G    26K  /zpool
> zpool/applic        2.40G  40.1G  2.40G  /zpool/applic
> zpool/bin            108M  40.1G   108M  /zpool/bin
> zpool/data           644M  40.1G   644M  /zpool/data
> zpool/logs          1.03G  40.1G  1.03G  /zpool/logs
> 
> This means, Apache, Tomcat, Bind DNS, Postfix, MySQL, Berkeley-DB, ... was 
> installed using a prefix (e.g. ./configure --prefix=/zpool/applic/named)
> 
> This gives me some independencies to the core OS located 
> in /sbin; /usr/bin, ...
> 
> After I moved all my apps into my own prefix path (ZFS mount poing), I did 
> another full reinstall of the OS, where I found out that I should have
backed
> up some files from the core OS before. Especially I should have backed up
the
> following files from the GZ and all NGZ. 
> 
> a) /etc/hosts, /etc/passwd, /etc/shadow, /etc/nsswitch.conf,
/etc/resolv.conf
> b) /etc/hostname.XX, 
> c) /etc/init.d/startup-scripts (my own releases)
> 
> After I did another full setup (not upgrading), I created the zones using
the
> famous zonemgr script and brought back all applications by just mounting 
> the /zpool/applic/path into the NGZ path. 
> 
> This way, I was pretty fast in upgrading the whole system to a new Nevada 
> build, even upgrading would be the preferred solution to me. 
> 
> I do not know if I with SNV_54, another upgrade from SNV_54 to SNV_55 is 
> supported by OpenSolaris. That is why this thread is of interest to me. 
Ivan,

I am not sure if I completely understand your configuration, but you
can upgrade a system with zones that have delegated zfs datasets or
where you just used lofs mounts to mount the zfs filesystems into the
zone.  This would apply when all you have is data or non-Solaris pkgs
installed in the zfs filesystems.  Since the upgrade code does not
have to discover and mount the zfs filesystems to perform the upgrade
of the OS, this type of configuration works fine.  We would have to
see your actual zonecfg info to be sure that you haven''t set things
up in a way that would prevent the upgrade though.

Jerry

Ivan Buetler wrote:
> Jerry, Thank you for your response. See my zonecfg of the named NGZ here:
> 
> root at treo[global] ~ # zonecfg -z named export
> create -b
> set zonepath=/zpool/zones/named
> set autoboot=true
> add inherit-pkg-dir
> set dir=/lib
> end
> add inherit-pkg-dir
> set dir=/platform
> end
> add inherit-pkg-dir
> set dir=/sbin
> end
> add inherit-pkg-dir
> set dir=/usr
> end
> add fs
> set dir=/zpool/applic/bind-9.3.2-P1
> set special=/zpool/applic/bind-9.3.2-P1/
> set type=lofs
> add options ro
> add options nodevices
> end
> add fs
> set dir=/zpool/data/named
> set special=/zpool/data/named
> set type=lofs
> add options rw
> add options nodevices
> end
> add net
> set address=1.2.3.4/27
> set physical=qfe3
> end
> add net
> set address=21.2.3.5/27
> set physical=qfe3
> end
> add net
> set address=10.10.10.10/24
> set physical=qfe4
> end
> add attr
> set name=comment
> set type=string
> set value="Zone named"
> end
It looks like you won''t be able to upgrade.  Assuming /zpool
is the mount of your zfs zpool, then your zonepath is on a
zfs dataset so this is the exact issue that upgrade cannot
handle yet.  If you were to place your zones on a UFS filesystem
then the other fs entries you have to mount zfs datasets
within the zone would be fine.

Jerry

On Thu, 8 Feb 2007, Casper.Dik at Sun.COM wrote:
> >Many thanks for answering my question.  Hopefully my noisy X4200
> >will be installed in the data centre tomorrow (Thursday); I had
> >a set back today while fighting with the Remote Console feature
> >of ILOM 1.1.1 (i.e., it doesn''t work).  :-(
> 
> Just ssh into it and use the serial console from within SSH.
That''s how I usually use the console on the X4200.  However, that
arrangement doesn''t work when one wants to (re)install Solaris.
Unless there''s a way of telling the installer to use the serial
console while booting from DVD, rather than using the GUI?

-- 
Rich Teer, SCSA, SCNA, SCSECA, OpenSolaris CAB member

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

>That''s how I usually use the console on the X4200.  However, that
>arrangement doesn''t work when one wants to (re)install Solaris.
>Unless there''s a way of telling the installer to use the serial
>console while booting from DVD, rather than using the GUI?

I thought there were a grub "use ttya" and "use ttyb" line
on the DVD?

Casper

On Thu, 8 Feb 2007, Casper.Dik at Sun.COM wrote:
> I thought there were a grub "use ttya" and "use ttyb"
line on the DVD?
Yes but one needs to be able to see that menu in order to select
the correct item first.  A chicken-and-egg situation!

Not that it matters so much for this case now, as I''ve hooked up
a spare monitor and keyboard to it.  But connecting a monitor and
keyboard directly to a server just feels ... wrong.  But then I''m
an old-school SPARC guy, so I guess that''s not too surprising!  :-)

-- 
Rich Teer, SCSA, SCNA, SCSECA, OpenSolaris CAB member

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

>Yes but one needs to be able to see that menu in order to select
>the correct item first.  A chicken-and-egg situation!
But the console redirection setting in the BIOS should address
that, right?

Casper