zfs discuss - Aug 2008 - OpenSolaris installer can''t be run, if target ZFS pool exists.

Hi ZFS team,

I am currently working on fixes for couple of bugs in OpenSolaris Caiman
installer and since they are related to the ZFS, I would like to kindly
ask you, if you could please help me to understand, if the issues
encountered and mentioned below are known (somebody works on them or
if they are already fixed), or if some workarounds might be used.

Also, please let us know if there is possibility that other approach
(like other/new API, command, subcommand) might be used in order to
solve the problem.

Any help/suggestions/comments are much appreciated.
Thank you very much,
Jan


Let me describe the main problem we are trying to address by the fix
which is tracked by following bug:

1771 Install will fail if rpool ZFS pool already exists

When OpenSolaris starts process of installation, it creates ZFS root pool
"rpool" on the target, which is then populated with appropriate ZFS
datasets
and ZFS volumes (used for swap and dump) - then Solaris is installed there.

If installer exits either due to some kind of failure or user intervention,
we would like to make sure, it can be restarted again. Because of this,
we need to destroy all allocated resources, which in ZFS case means we
need to release "rpool", so that installer could use it again.

The problem is that we don''t know, if "rpool" was just
created by the
installer or imported by user (for example other Solaris instance might 
exist
on other disk). If latter is the case, we don''t want to destroy the
pool.

We were trying to address this problem in following ways, but none of them
worked for us due to the issues encountered:

[1] Installing into "temporary" pool
===================================Installer created "rpool_tmp" for
installation and when installation
successfully finished, it was "renamed" to "rpool".
If installer failed during installation, we knew that we could safely
remove "rpool_tmp", since this was only temporary pool and thus
couldn''t
contain valid Solaris instance.

We were "renaming" the pool in following way:

# zpool create -f rpool_tmp <device>
[...installation...]
# pool_id=`zpool list -H -o guid rpol_tmp`
# zpool export -f rpool_tmp
# zpool import -f $pool_id rpool

However, we encountered problem that sometimes root "/rpool" dataset
was not mounted due to some reason and we failed to access it. Please
see more details in bug report

1350 Install without menu.lst

[2] Exporting "rpool" if present on the system
=============================================When installer started, it checked
if "rpool" is present and
in that case it exported it.

# zpool export -f rpool

The problem is that this operation made "rpool" unbootable:

1365 zfs import/export will modify the boot up information of ZFS

So if user had some Solaris instance on that pool, it was not able
to boot it anymore (without importing the pool).

Hi John,

I like this idea - it would be clear solution for the problem.
Is it possible to manage custom parameters with standard CLI
commands (these are used by the installer for manipulating ZFS
entities) - zpool(1M), zfs(1M), ... ?

I am asking, since I haven''t found information about setting
custom parameters neither in man pages nor in
"Solaris ZFS Administration Guide" available on opensolaris.org,
I have probably missed it.

Thank you,
Jan


John Langley wrote:
> What about setting a custom parameter on rpool when you create it and 
> then changing the value after a successful install.  Then you can test 
> for the value of the parameter and know if it was a failed install.
>
> John
>
> On Mon, Aug 18, 2008 at 11:04 AM, Jan Damborsky <Jan.Damborsky at
sun.com
> <mailto:Jan.Damborsky at sun.com>> wrote:
>
>     Hi ZFS team,
>
>     I am currently working on fixes for couple of bugs in OpenSolaris
>     Caiman
>     installer and since they are related to the ZFS, I would like to
>     kindly
>     ask you, if you could please help me to understand, if the issues
>     encountered and mentioned below are known (somebody works on them or
>     if they are already fixed), or if some workarounds might be used.
>
>     Also, please let us know if there is possibility that other approach
>     (like other/new API, command, subcommand) might be used in order to
>     solve the problem.
>
>     Any help/suggestions/comments are much appreciated.
>     Thank you very much,
>     Jan
>
>
>     Let me describe the main problem we are trying to address by the fix
>     which is tracked by following bug:
>
>     1771 Install will fail if rpool ZFS pool already exists
>
>     When OpenSolaris starts process of installation, it creates ZFS
>     root pool
>     "rpool" on the target, which is then populated with
appropriate
>     ZFS datasets
>     and ZFS volumes (used for swap and dump) - then Solaris is
>     installed there.
>
>     If installer exits either due to some kind of failure or user
>     intervention,
>     we would like to make sure, it can be restarted again. Because of
>     this,
>     we need to destroy all allocated resources, which in ZFS case means we
>     need to release "rpool", so that installer could use it
again.
>
>     The problem is that we don''t know, if "rpool" was
just created by the
>     installer or imported by user (for example other Solaris instance
>     might
>     exist
>     on other disk). If latter is the case, we don''t want to
destroy
>     the pool.
>
>     We were trying to address this problem in following ways, but none
>     of them
>     worked for us due to the issues encountered:
>
>     [1] Installing into "temporary" pool
>     ===================================>     Installer created
"rpool_tmp" for installation and when installation
>     successfully finished, it was "renamed" to "rpool".
>     If installer failed during installation, we knew that we could safely
>     remove "rpool_tmp", since this was only temporary pool and
thus
>     couldn''t
>     contain valid Solaris instance.
>
>     We were "renaming" the pool in following way:
>
>     # zpool create -f rpool_tmp <device>
>     [...installation...]
>     # pool_id=`zpool list -H -o guid rpol_tmp`
>     # zpool export -f rpool_tmp
>     # zpool import -f $pool_id rpool
>
>     However, we encountered problem that sometimes root "/rpool"
dataset
>     was not mounted due to some reason and we failed to access it. Please
>     see more details in bug report
>
>     1350 Install without menu.lst
>
>     [2] Exporting "rpool" if present on the system
>     =============================================>     When installer
started, it checked if "rpool" is present and
>     in that case it exported it.
>
>     # zpool export -f rpool
>
>     The problem is that this operation made "rpool" unbootable:
>
>     1365 zfs import/export will modify the boot up information of ZFS
>
>     So if user had some Solaris instance on that pool, it was not able
>     to boot it anymore (without importing the pool).
>
>     _______________________________________________
>     zfs-discuss mailing list
>     zfs-discuss at opensolaris.org <mailto:zfs-discuss at
opensolaris.org>
>     http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
>

jan damborsky wrote:
> Hi John,
> 
> I like this idea - it would be clear solution for the problem.
> Is it possible to manage custom parameters with standard CLI
> commands (these are used by the installer for manipulating ZFS
> entities) - zpool(1M), zfs(1M), ... ?
> 
> I am asking, since I haven''t found information about setting
> custom parameters neither in man pages nor in
> "Solaris ZFS Administration Guide" available on opensolaris.org,
> I have probably missed it.
As of snv_96 pool''s don''t have user properties otherwise you
would
be able to do this:

	zpool set caiman:install=preparing rpool
....
	zpool set caiman:install=ready rpool
...
	zpool set caiman:install=complete rpool

You can set user properties on a dataset so you could do this:

	zfs set caiman:install=preparing rpool/ROOT

And log an RFE for having user defined properties at the pool (if one 
doesn''t already exist).

-- 
Darren J Moffat

Perhaps user properties on pools would be useful here? At present only ZFS
filesystems can have user properties - not pools. Not really an immediate
solution to your problem though.

Cheers

Andrew.
 
 
This message posted from opensolaris.org

Hi Darren,

thank you very much for your help.
Please see my comments below.

Jan


Darren J Moffat wrote:
> jan damborsky wrote:
>> Hi John,
>>
>> I like this idea - it would be clear solution for the problem.
>> Is it possible to manage custom parameters with standard CLI
>> commands (these are used by the installer for manipulating ZFS
>> entities) - zpool(1M), zfs(1M), ... ?
>>
>> I am asking, since I haven''t found information about setting
>> custom parameters neither in man pages nor in
>> "Solaris ZFS Administration Guide" available on
opensolaris.org,
>> I have probably missed it.
>
> As of snv_96 pool''s don''t have user properties otherwise
you would
> be able to do this:
>
>     zpool set caiman:install=preparing rpool
> ....
>     zpool set caiman:install=ready rpool
> ...
>     zpool set caiman:install=complete rpool
>
> You can set user properties on a dataset so you could do this:
>
>     zfs set caiman:install=preparing rpool/ROOT
That sounds reasonable. It is not atomic operation from installer
point of view, but the time window is really short (installer can
set ZFS user property almost immediately after pool is created).

As far as pools imported manually by user are concerned, is there
anything installer could do about this before 6733239 is fixed ?

I mean, if user imports "rpool" (e.g. from another disk) which
contains
valid installation of Solaris, installer can''t proceed with
installation,
since it can''t export the pool in order to release the
"rpool" name
for its own purposes (when exported, pool can''t be booted anymore).
>
> And log an RFE for having user defined properties at the pool (if one 
> doesn''t already exist).
I will go through the existing issues and will file new RFE for this if 
one doesn''t exists.

Hi Andrew,

this is what I am thinking about based on John''s
and Darren''s responses.

I will file RFE for having possibility to set user
properties for pools (if it doesn''t already exist).

Thank you,
Jan


andrew wrote:
> Perhaps user properties on pools would be useful here? At present only ZFS
filesystems can have user properties - not pools. Not really an immediate
solution to your problem though.
>
> Cheers
>
> Andrew.
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

jan damborsky wrote:
>>     zfs set caiman:install=preparing rpool/ROOT
> 
> That sounds reasonable. It is not atomic operation from installer
> point of view, but the time window is really short (installer can
> set ZFS user property almost immediately after pool is created).
Not it isn''t but this would be and should set the property on the rpool
dataset rather than the rpool/ROOT dataset.

zpool create -O caiman:install=preparing rpool c1d0

However that doesn''t work, it seems we can''t set user
properties with -O
only system properties.  I''ve logged 6738716 for this part of the
issue.

-- 
Darren J Moffat

Darren J Moffat wrote:
> jan damborsky wrote:
>>>     zfs set caiman:install=preparing rpool/ROOT
>> That sounds reasonable. It is not atomic operation from installer
>> point of view, but the time window is really short (installer can
>> set ZFS user property almost immediately after pool is created).
>
> Not it isn''t but this would be and should set the property on the
rpool
> dataset rather than the rpool/ROOT dataset.
>
> zpool create -O caiman:install=preparing rpool c1d0
>
> However that doesn''t work, it seems we can''t set user
properties with -O
> only system properties.  I''ve logged 6738716 for this part of the
issue.
>
Thanks a lot !
Jan

Darren J Moffat wrote:
> jan damborsky wrote:
>> Hi John,
>>
>> I like this idea - it would be clear solution for the problem.
>> Is it possible to manage custom parameters with standard CLI
>> commands (these are used by the installer for manipulating ZFS
>> entities) - zpool(1M), zfs(1M), ... ?
>>
>> I am asking, since I haven''t found information about setting
>> custom parameters neither in man pages nor in
>> "Solaris ZFS Administration Guide" available on
opensolaris.org,
>> I have probably missed it.
> 
> As of snv_96 pool''s don''t have user properties otherwise
you would
> be able to do this:
> 
> 	zpool set caiman:install=preparing rpool
> ....
> 	zpool set caiman:install=ready rpool
> ...
> 	zpool set caiman:install=complete rpool
> 
> You can set user properties on a dataset so you could do this:
> 
> 	zfs set caiman:install=preparing rpool/ROOT
> 
> And log an RFE for having user defined properties at the pool (if one 
> doesn''t already exist).
> 
Allow me to parrot the suggestion in the zfs man page to use reverse DNS 
names in user properties:

     zfs set org.opensolaris.caiman:install=preparing rpool/ROOT

This will allow other people to use the name ''caiman'' for
their own user
properties.

Menno
-- 
Menno Lageman - Sun Microsystems - http://blogs.sun.com/menno

>> And log an RFE for having user defined properties at the pool (if one 
>> doesn''t already exist).
>>     
6739057 was filed to track this.

Thank you,
Jan