freebsd stable - Jan 2012 - ZFS: i/o error - all block copies unavailable on large disk number machines

We did a minor kernel update on a large storage machine here today which runs
FreeBSD 8.2 and to our surprise it failed to boot at
the loader with "ZFS: i/o error - all block copies unavailable".

After some digging we discovered that this was likely due to the fact that the
BIOS only enumerates the first 12 disks and this
machine has more than that in the root zpool which was a striped raidz2 volume.
This in turn means that the bootcode can't
complete and hence the machine can't boot.

Our solution was to migrate the root fs off the raidz2 volume and to a mirrored
volume which was on two disks which where
accessible from the BIOS.

It would of course be nice if zfs warned or even prevented this, any thoughts?

More info available if we find it here:-
http://blog.multiplay.co.uk/2012/01/zfs-io-error-all-block-copies-unavailable-on-large-disk-number-machines/

    Regards
    Steve 


===============================================This e.mail is private and
confidential between Multiplay (UK) Ltd. and the person or entity to whom it is
addressed. In the event of misdirection, the recipient is prohibited from using,
copying, printing or otherwise disseminating it or any information contained in
it.

In the event of misdirection, illegible or incomplete transmission please
telephone +44 845 868 1337
or return the E.mail to postmaster@multiplay.co.uk.

On Jan 23, 2012, at 9:04 AM, Steven Hartland wrote:
> After some digging we discovered that this was likely due to the fact that
the BIOS only enumerates the first 12 disks and this machine has more than that
in the root zpool which was a striped raidz2 volume. This in turn means that the
bootcode
> can't complete and hence the machine can't boot.
As far as I can tell, ZFS best practices guides recommend no more than nine
drives in a group/pool.  Putting more than that into a pool, much less something
you are trying to boot from, seems like a fine experiment to make but is not
something which I would rely upon...

Regards,
-- 
-Chuck