zfs discuss - Sep 2009 - Yet another "where did the space go" question

An attempt to pkg image-update from snv111b to snv122 failed
miserably for a number of reasons which are probably out of
scope here. Suffice it to say that it ran out of disk space
after the third attempt.

Before starting, I was careful to make a baseline snapshot,
but rolling back to that snapshot has not freed up all the
space - this on a small disk dedicated to experimenting with
ZFS booting on SPARC. The disk is nominally 20GB.

After "zfs rollback -rR rpool/ROOT/opensolaris at baseline" from
a different BE  (snv103 booted from UFS)

# zpool list
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
rpool  17.5G  10.1G  7.39G    57%  ONLINE  -
space  1.36T   314G  1.05T    22%  ONLINE  -

# zfs list -r -o space rpool
NAME                        AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV 
USEDCHILD
rpool                       7.11G  10.1G         0     20K              0     
10.1G
rpool/ROOT                  7.11G  10.1G         0     18K              0     
10.1G
rpool/ROOT/opensolaris      7.11G  10.1G      942K   10.0G              0     
68.6M
rpool/ROOT/opensolaris/opt  7.11G  68.6M         0   68.6M              0       
0

Before the aborted pkg image-updates, the rpool took around 6GB,
so 4GB has vanished somewhere. Even if pkg put it''s updates
in a well hidden place (there are no hidden directories in / ),
surely the rollback should have deleted them.

# zfs list -t snapshot
NAME                                             USED  AVAIL  REFER  MOUNTPOINT
rpool at baseline                                      0      -    20K  -
rpool/ROOT at baseline                                 0      -    18K  -
rpool/ROOT/opensolaris at baseline                  718K      -  10.0G  -
rpool/ROOT/opensolaris/opt at baseline                 0      -  68.6M  -

The rollback obviously worked because afterwards even the
pkg set-publisher changes were gone, and other post-snapshot
files were deleted. If the worst come to the worst I could
obviously save the snapshot to a file and then restore it,
but it sure would be nice to know where the 4GB went.

BTW one image-update failure occurred because there was an X86
rpool mounted to an alternate root, and pkg somehow found it
and seemed to get confused about X86 vs. SPARC, insisting on
trying to create a menu.lst in /rpool/boot, which, of course,
doesn''t exist on SPARC. I suppose this should be a bug...

Thanks -- Frank

Correction

On 09/06/09 12:00 PM, I wrote:
> (there are no hidden directories in / ),
Well, there is .zfs, of course, but it is normally hidden,
apparently by default on SPARC rpool, but not on X86 rpool
or non-rpool pools on either. Hmmm. I don''t recollect setting
the snapdir property on any pools, ever.
-----
Arrg! It just failed again!

# pkg image-update --be-name=snv122
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                1486/1486 73091/73091 1520.59/1520.59

WARNING: menu.lst file /rpool/boot/menu.lst does not exist,
          generating a new menu.lst file
pkg: Unable to clone the current boot environment.
----
# BE_PRINT_ERR=true beadm create newbe
be_get_uuid: failed to get uuid property from BE root dataset user properties.
be_get_uuid: failed to get uuid property from BE root dataset user properties.
# zfs list -t snapshot | grep newbe
rpool/ROOT/opensolaris at newbe                      30K      -  11.9G  -
rpool/ROOT/opensolaris/opt at newbe                    0      -  68.6M  -

So it can create a new BE. So what happened this time?
I guess I''ll try again with
BE_PRINT_ERR=true...........................

Is the get uuid property failure fatal to pkg but not to beadm? Has
anyone managed to go from snv111b  to snv122 on SPARC?

Thanks -- Frank

Near Success! After 5 (yes, five) attempts, managed to do
an update of snv111b to snv122, until it ran out of space
again. Looks like I need to get a bigger disk...

Sorry about the monolog, but there might be someone on this
list trying to use pkg on SPARC who, like me, has been
unable to subscribe to the indiana list, so an update
might be useful to any such person... Perhaps someone
who can might forward this to the appropriate list --
the issues are known CR''s, but don''t seem to be mentioned
in the release notes.

On 09/06/09 04:55 PM, I wrote:
> WARNING: menu.lst file /rpool/boot/menu.lst does not exist,
> generating a new menu.lst file
> pkg: Unable to clone the current boot environment.
1) If there isn''t a directory /rpool/boot, pkg will fail
2) If you try again after mkdir /rpool/boot, it will
    create menu.1st. If it fails for any reason and
    you have to restart  then:
3) If there is a menu.lst containing  opensolaris-1
    it will fail again even if you had used be-name=.
4) If you delete menu.lst it will fail - touch it after
    deleting it (the CRs are ambiguous about this).

So to do this upgrade, you must do mkdir /rpool/boot
and touch /rpool/boot/menu.lst before you start. It
might just work if you do this, but only if you have
at least 11GB of space to spare (Google says 8GB).

BTW pkg always says "/rpool/boot/menu.lst does not exist"
even if it does.

http://defect.opensolaris.org/bz/show_bug.cgi?id=6744
says "Fixed in source"

http://defect.opensolaris.org/bz/show_bug.cgi?id=7880
says "accepted". But the fix for 6744 messes up 7880.
This is making a SPARC upgrade really painful, especially
annoying since SPARC doesn''t even use grub (or menu.lst).

Cheers -- Frank

PS My hat''s off to the ZFS and pkg teams! An amazing
accomplishment and a few glitches are to be expected. I''m
sure there are fixes in the works, but it would seem upgrading
to snv122 isn''t in the cards unless I get a bigger 3rd boot
disk...