freebsd stable - Jul 2014 - Gmirror + gpart corruption on 9.3-PRE

At 9.3-PRERELEASE #0 r268066M I've been trying unsucessfuly to set up a 
brand shiny new gmirror + gpt style Raid 0 mirror using the following 
procedure on a disk

  gpart create -s gpt ada0
  ( shows 931G of space)
  gpart add -s 96G -t freebsd-swap -l swap0 ada0
  gpart add -t freebsd-ufs -l rw0 ada0
  gpart create -s gpt ada1
  gpart add -s 96G -t freebsd-swap -l swap1 ada1
  gpart add -t freebsd-ufs -l rw1 ada1
  gmirror label swap /dev/ada0p1 /dev/ada1p1
  gmirror label rw   /dev/ada0p2 /dev/ada1p2
  gpart create -s gpt mirror/swap
  gpart add -s95G -t freebsd-swap -l FBCDSWAP mirror/swap
  gpart create -s gpt mirror/rw
  gpart add -s833G -t freebsd-ufs -l FBCDRW mirror/rw

This consistently gives me the dreaded

  GEOM: mirror/rw: the secondary GPT table is corrupt or invalid

just as soon as I provision the mirror/rw device.

I have tried many ideas of padding these partitions so I don't get this 
message, but to noavail. I'm willing to try more, the machine isn't live
(yet) so it can sit here while it gets figured out. :)

Back at 9.2-PRERELEASE #0 r255456 I used the same procedure for setting 
up a gmirror, and this worked fine with no corruption. Here's the 
partition table:

 > gpart show
=>       34  976773101  ada0  GPT  (465G)
          34   46245840     1  freebsd-swap  (22G)
    46245874  930523232     2  freebsd-ufs  (443G)
   976769106       4029        - free -  (2M)

=>       34  976773101  ada1  GPT  (465G)
          34   46245840     1  freebsd-swap  (22G)
    46245874  930523232     2  freebsd-ufs  (443G)
   976769106       4029        - free -  (2M)

=>      34  46245772  mirror/swap0  GPT  (22G)
         34  46245772             1  freebsd-swap  (22G)

=>       34  930523164  mirror/rw0  GPT  (443G)
          34  926941184           1  freebsd-ufs  (442G)
   926941218    3581980              - free -  (1.7G)


I've since upgraded this machine to the exact same 9.3 revision and it 
still works fine, no corruption. Am I missing something or did something 
change in between these two revisions that makes it difficult or 
different to set up?

Thanks in advance for any insight you all can provide.
-- 
Dave Hayes - Consultant - Altadena CA, USA - dave at jetcafe.org
 >>>> *The opinions expressed above are entirely my own*
<<<<

Enjoyment is not a goal,
it is a feeling that accompanies important ongoing activity

On Thu, 24 Jul 2014, Dave Hayes wrote:
> At 9.3-PRERELEASE #0 r268066M I've been trying unsucessfuly to set up a
brand
> shiny new gmirror + gpt style Raid 0 mirror using the following procedure
on
> a disk
>
> gpart create -s gpt ada0
> ( shows 931G of space)
> gpart add -s 96G -t freebsd-swap -l swap0 ada0
> gpart add -t freebsd-ufs -l rw0 ada0
> gpart create -s gpt ada1
> gpart add -s 96G -t freebsd-swap -l swap1 ada1
> gpart add -t freebsd-ufs -l rw1 ada1
> gmirror label swap /dev/ada0p1 /dev/ada1p1
> gmirror label rw   /dev/ada0p2 /dev/ada1p2
Mirrors of GPT partitions on different disks is fine.  Or rather, it's 
sub-optimal but will work.
> gpart create -s gpt mirror/swap
> gpart add -s95G -t freebsd-swap -l FBCDSWAP mirror/swap
> gpart create -s gpt mirror/rw
> gpart add -s833G -t freebsd-ufs -l FBCDRW mirror/rw
> This consistently gives me the dreaded
>
> GEOM: mirror/rw: the secondary GPT table is corrupt or invalid
>
> just as soon as I provision the mirror/rw device.
I believe that GPT tables somehow work only on the drive level.  That 
is, the GPT partitioning created inside the mirror actually overwrites 
the existing one on the drive.  As odd as it sounds, I think this is 
intentional: GPT tables are only supposed to be at the beginning and end 
of a physical drive.  But I'm also not the one to explain it.

However, the attempted GPT partitioning inside the mirrors is not needed 
and can be left out.
> I've since upgraded this machine to the exact same 9.3 revision and it
still
> works fine, no corruption. Am I missing something or did something change
in
> between these two revisions that makes it difficult or different to set up?
GPT tests got more strict at some point.  Maybe the rules for GPT tables 
did also.

For reference, here is my article on mirroring GPT disk partitions.  I 
do not recommend it.  (Consider the head contention when multiple 
mirrors on the same drive attempt to rebuild.  Or disable automatic 
rebuilding, but then it's going to be unpleasant in an emergency.) 
http://www.wonkity.com/~wblock/docs/html/gmirror.html

For drives under 2TB, use MBR and bsdlabels, as ugly as it is.  The 
recently-rewritten Handbook procedure shows the right way to do it, 
including alignment: 
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom-mirror.html

If the memory is available, ZFS might handle mirrored partitions better. 
Maybe a per-pool configurable resilvering priority, but I don't know.