zfs discuss - Jan 2009 - Questions about OS 2008.11 partitioning scheme

Hi all,

I did an install of OpenSolaris in which I specified that the whole disk should
be used for the installation. Here is what "format> verify"
produces for that disk:

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       1 - 60797      465.73GB    (60797/0/0) 976703805
  1 unassigned    wm       0                0         (0/0/0)             0
  2     backup    wu       0 - 60797      465.74GB    (60798/0/0) 976719870
  3 unassigned    wm       0                0         (0/0/0)             0
  4 unassigned    wm       0                0         (0/0/0)             0
  5 unassigned    wm       0                0         (0/0/0)             0
  6 unassigned    wm       0                0         (0/0/0)             0
  7 unassigned    wm       0                0         (0/0/0)             0
  8       boot    wu       0 -     0        7.84MB    (1/0/0)         16065
  9 unassigned    wm       0                0         (0/0/0)             0

I have several questions. First, what is the purpose of partitions 2 and 8 here?
Why not simply have partition 0, the "root" partition, be the only
partition, and start at cylinder 0 as opposed to 1?

My second question concerns the disk I have used to mirror the first root zpool
disk. After I set up the second disk to mirror the first one with "zpool
attach -f rpool c3t0d0s0 c4t0d0s0", I got the response

Please be sure to invoke installgrub(1M) to make ''c4t0d0s0''
bootable.

Is that correct? Or do I want to make c4t0d0s8 bootable, given that the label of
that partition is "boot"? I cannot help finding this a little
confusing. As far as i can tell, c4t0d0s8 (as well as c3t0d0s8 from the original
disk which I mirrored), cylinder 0, is not used for anything.

Finally, is the correct command to make the disk I have added to mirror the
first disk bootable

"installgrub -m /boot/grub/stage1 /boot/grub/stage2
/dev/rdsk/c4t0d0s0" ?
-- 
This message posted from opensolaris.org

Alex,

I think the root cause of your confusion is that the format utility and
disk labels are very unfriendly and confusing.

Partition 2 identifies the whole disk and on x86 systems, space is
needed for boot-related information and is currently stored in
partition 8. Neither of these partitions require any administration and 
should not be used for anything. You can read more here:

http://docs.sun.com/app/docs/doc/817-5093/disksconcepts-20068?a=view

(To add more confusion, partitions are also referred to as slices.)

However, the system actually boots from the root file system, in 
partition 0 on your system, which is why you need to run the installgrub 
command on c4t0d0s0. Your installgrub syntax looks correct to me.

My wish for this year is to boot from EFI-labeled disks so examining
disk labels is mostly unnecessary because ZFS pool components could be
constructed as whole disks, and the unpleasant disk
format/label/partitioning experience is just a dim memory...

Cindy



Alex Viskovatoff wrote:
> Hi all,
> 
> I did an install of OpenSolaris in which I specified that the whole disk
should be used for the installation. Here is what "format> verify"
produces for that disk:
> 
> Part      Tag    Flag     Cylinders         Size            Blocks
>   0       root    wm       1 - 60797      465.73GB    (60797/0/0) 976703805
>   1 unassigned    wm       0                0         (0/0/0)             0
>   2     backup    wu       0 - 60797      465.74GB    (60798/0/0) 976719870
>   3 unassigned    wm       0                0         (0/0/0)             0
>   4 unassigned    wm       0                0         (0/0/0)             0
>   5 unassigned    wm       0                0         (0/0/0)             0
>   6 unassigned    wm       0                0         (0/0/0)             0
>   7 unassigned    wm       0                0         (0/0/0)             0
>   8       boot    wu       0 -     0        7.84MB    (1/0/0)         16065
>   9 unassigned    wm       0                0         (0/0/0)             0
> 
> I have several questions. First, what is the purpose of partitions 2 and 8
here? Why not simply have partition 0, the "root" partition, be the
only partition, and start at cylinder 0 as opposed to 1?
> 
> My second question concerns the disk I have used to mirror the first root
zpool disk. After I set up the second disk to mirror the first one with
"zpool attach -f rpool c3t0d0s0 c4t0d0s0", I got the response
> 
> Please be sure to invoke installgrub(1M) to make
''c4t0d0s0'' bootable.
> 
> Is that correct? Or do I want to make c4t0d0s8 bootable, given that the
label of that partition is "boot"? I cannot help finding this a little
confusing. As far as i can tell, c4t0d0s8 (as well as c3t0d0s8 from the original
disk which I mirrored), cylinder 0, is not used for anything.
> 
> Finally, is the correct command to make the disk I have added to mirror the
first disk bootable
> 
> "installgrub -m /boot/grub/stage1 /boot/grub/stage2
/dev/rdsk/c4t0d0s0" ?

On Tue, Jan 06, 2009 at 10:22:20AM -0800, Alex Viskovatoff
wrote:
> I did an install of OpenSolaris in which I specified that the whole disk
should be used for the installation. Here is what "format> verify"
produces for that disk:
> 
> Part      Tag    Flag     Cylinders         Size            Blocks
>   0       root    wm       1 - 60797      465.73GB    (60797/0/0) 976703805
>   1 unassigned    wm       0                0         (0/0/0)             0
>   2     backup    wu       0 - 60797      465.74GB    (60798/0/0) 976719870
>   3 unassigned    wm       0                0         (0/0/0)             0
>   4 unassigned    wm       0                0         (0/0/0)             0
>   5 unassigned    wm       0                0         (0/0/0)             0
>   6 unassigned    wm       0                0         (0/0/0)             0
>   7 unassigned    wm       0                0         (0/0/0)             0
>   8       boot    wu       0 -     0        7.84MB    (1/0/0)         16065
>   9 unassigned    wm       0                0         (0/0/0)             0
> 
> I have several questions. First, what is the purpose of partitions 2 and 8
here? Why not simply have partition 0, the "root" partition, be the
only partition, and start at cylinder 0 as opposed to 1?
It''s traditional in the VTOC label to have slice 2 encompass all
cylinders.  You don''t have to use it.

In SPARC, the boot blocks fit into the 15 "free" blocks before the
filesystem actually starts writing data.  On x86, the boot code requires
more data.  So putting a UFS filesystem on cylinder 0 would not leave
sufficient room for boot code.  The traditional solution is that data
slices on x86 begin at cylinder 1, leaving cylinder 0 for boot data.

-- 
Darren

> http://docs.sun.com/app/docs/doc/817-5093/disksconcepts-20068?a=view
>
> (To add more confusion, partitions are also referred to as slices.)
Nope, at least not on x86 systems.  A partition holds the Solaris part
of the disk, and that part is subdivided into slices.  Partitions
are visible to other OSes on the box; slices aren''t.  Whereever the
wrong term appears in Sun docs, it should be treated as a doc bug.

For Sparc systems, some people intermix the two terms, but it''s not
really correct there either.


Regards -- Volker
-- 
------------------------------------------------------------------------
Volker A. Brandt                  Consulting and Support for Sun Solaris
Brandt & Brandt Computer GmbH                   WWW: http://www.bb-c.de/
Am Wiesenpfad 6, 53340 Meckenheim                     Email: vab at bb-c.de
Handelsregister: Amtsgericht Bonn, HRB 10513              Schuhgr??e: 45
Gesch?ftsf?hrer: Rainer J. H. Brandt und Volker A. Brandt

On Tue, Jan 06, 2009 at 11:49:27AM -0700, Cindy.Swearingen at Sun.COM
wrote:
> My wish for this year is to boot from EFI-labeled disks so examining
> disk labels is mostly unnecessary because ZFS pool components could be
> constructed as whole disks, and the unpleasant disk
> format/label/partitioning experience is just a dim memory...
Is there any non-EFI hardware that would support EFI boot?

-- 
Darren

Hello Darren,
This one, ok, was a validate thought/question --

On Solaris, root pools cannot have EFI labels (the boot firmware
doesn''t
support booting from them).
http://blog.yucas.info/2008/11/26/zfs-boot-solaris/

But again, this is a ZFS discussion, and obvously EFI is not a ZFS, or even 
Sun thing.
http://en.wikipedia.org/wiki/Extensible_Firmware_Interface

Hence, on ZFS turf, I would offer the following comment, in the notion of 
innovation, not trashing EFI.
The ZFS design point is to "make IT simple", so EFI or not EFI can be 
debate-able.
http://kerneltrap.org/node/6884

;-)
best,
z

----- Original Message ----- 
From: "A Darren Dunham" <ddunham at taos.com>
To: <zfs-discuss at opensolaris.org>
Sent: Tuesday, January 06, 2009 3:38 PM
Subject: Re: [zfs-discuss] Questions about OS 2008.11 partitioning scheme

> On Tue, Jan 06, 2009 at 11:49:27AM -0700, Cindy.Swearingen at Sun.COM
wrote:
>> My wish for this year is to boot from EFI-labeled disks so examining
>> disk labels is mostly unnecessary because ZFS pool components could be
>> constructed as whole disks, and the unpleasant disk
>> format/label/partitioning experience is just a dim memory...
>
> Is there any non-EFI hardware that would support EFI boot?
>
> -- 
> Darren
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Cindy,

Well, it worked. The system can boot off c4t0d0s0 now.

But I am still a bit perplexed. Here is how the invocation of installgrub went:

av at diotiima:~# installgrub -m /boot/grub/stage1 /boot/grub/stage2
/dev/rdsk/c4t0d0s0
Updating master boot sector destroys existing boot managers (if any).
continue (y/n)?y
stage1 written to partition 0 sector 0 (abs 16065)
stage2 written to partition 0, 267 sectors starting at 50 (abs 16115)
stage1 written to master boot sector
av at diotima:~# 

So installgrub writes to partition 0. How does one know that those sectors have
not already been used by zfs, in its mirroring of the first drive by this second
drive? And why is writing to partition 0 even necessary? Since c3t0d0 must
contain stage1 and stage2 in its partition 0, wouldn''t c4t0d0 already
have stage1 and stage 2 in its partition 0 through the silvering process?

I don''t find the present disk format/label/partitioning experience
particularly unpleasant (except for grubinstall writing directly into a
partition which belongs to a zpool). I just wish I understood what it involves.

Thank you for that link to the System Administration Guide. I just looked at it
again, and it says partition 8 "Contains GRUB boot information". So
partition 8 is the master boot sector and contains GRUB stage1?

Alex
-- 
This message posted from opensolaris.org

Hi Alex,

The fact that you have to install the boot blocks manually on the
second disk that you added with zpool attach is a bug! I should have
mentioned this bug previously.

If you had used the initial installation method to create a mirrored
root pool, the boot blocks would have been applied automatically.

I don''t think a way exists to discern whether the boot blocks are
already applied. I can''t comment on why resilvering can''t do
this step.

Cindy

Alex Viskovatoff wrote:
> Cindy,
> 
> Well, it worked. The system can boot off c4t0d0s0 now.
> 
> But I am still a bit perplexed. Here is how the invocation of installgrub
went:
> 
> av at diotiima:~# installgrub -m /boot/grub/stage1 /boot/grub/stage2
/dev/rdsk/c4t0d0s0
> Updating master boot sector destroys existing boot managers (if any).
> continue (y/n)?y
> stage1 written to partition 0 sector 0 (abs 16065)
> stage2 written to partition 0, 267 sectors starting at 50 (abs 16115)
> stage1 written to master boot sector
> av at diotima:~# 
> 
> So installgrub writes to partition 0. How does one know that those sectors
have not already been used by zfs, in its mirroring of the first drive by this
second drive? And why is writing to partition 0 even necessary? Since c3t0d0
must contain stage1 and stage2 in its partition 0, wouldn''t c4t0d0
already have stage1 and stage 2 in its partition 0 through the silvering
process?
> 
> I don''t find the present disk format/label/partitioning experience
particularly unpleasant (except for grubinstall writing directly into a
partition which belongs to a zpool). I just wish I understood what it involves.
> 
> Thank you for that link to the System Administration Guide. I just looked
at it again, and it says partition 8 "Contains GRUB boot information".
So partition 8 is the master boot sector and contains GRUB stage1?
> 
> Alex

On Tue, Jan 06, 2009 at 04:10:10PM -0500, JZ wrote:
> Hello Darren,
> This one, ok, was a validate thought/question --
Darn, I was hoping...
> On Solaris, root pools cannot have EFI labels (the boot firmware
doesn''t
> support booting from them).
> http://blog.yucas.info/2008/11/26/zfs-boot-solaris/
Yup.  If that were to change, it would make this much simpler.

-- 
Darren

On Tue, Jan 06, 2009 at 01:24:17PM -0800, Alex Viskovatoff wrote:
> av at diotiima:~# installgrub -m /boot/grub/stage1 /boot/grub/stage2
/dev/rdsk/c4t0d0s0
> Updating master boot sector destroys existing boot managers (if any).
> continue (y/n)?y
> stage1 written to partition 0 sector 0 (abs 16065)
> stage2 written to partition 0, 267 sectors starting at 50 (abs 16115)
> stage1 written to master boot sector
> av at diotima:~# 
> So installgrub writes to partition 0. How does one know that those
> sectors have not already been used by zfs, in its mirroring of the
> first drive by this second drive?
Because this is a VTOC label partition (necessary for Solaris boot), I
believe ZFS lives only within slice 0 (I need to verify this).  So VTOC
cylinder 0 is free.
> And why is writing to partition 0 even necessary? Since c3t0d0 must
> contain stage1 and stage2 in its partition 0, wouldn''t c4t0d0
already
> have stage1 and stage 2 in its partition 0 through the silvering
> process?
I doubt that all of partition 0 is mirrored.  Only the data under ZFS
control (not the boot blocks) are copied.

Do not confuse the MBR partition 0 (the "Solaris partition") with the
VTOC slice 0 (the one that has all the disk cylinders other than
cylinder 0 in it and that appeared in your earlier "label" output).
> I don''t find the present disk format/label/partitioning experience
> particularly unpleasant (except for grubinstall writing directly into
> a partition which belongs to a zpool). I just wish I understood what
> it involves.
Partition 0 contains all of Solaris.  So the OS just needs to keep
things straight.  It does this with the VTOC slicing within that
partition.
> Thank you for that link to the System Administration Guide. I just
> looked at it again, and it says partition 8 "Contains GRUB boot
> information". So partition 8 is the master boot sector and contains
> GRUB stage1?
It should probably refer to "slice 8" to reduce confusion.  Boot
loaders
(GRUB since that''s what is in use here) are simultaneously in partition
0 and slice 8.

-- 
Darren

Thanks for clearing that up. That all makes sense.

I was wondering why ZFS doesn''t use the whole disk in the standard
OpenSolaris install. That explains it.
-- 
This message posted from opensolaris.org

Hi Cindy,

I now suspect that the boot blocks are located outside of the space in partition
0 that actually belongs to the zpool, in which case it is not necessarily a bug
that zpool attach does not write those blocks, IMO. Indeed, that must be the
case, since GRUB needs to get to stage2 in order to be able to read zfs file
systems. I''m just glad zpool attach warned me that I need to invoke
grubinstall manually!

Thank you for making things less mysterious.

Alex
-- 
This message posted from opensolaris.org