freebsd stable - Nov 2018 - FreeBSD/powerpc64 (POWER9) with ZFS on /

ZFS is by default not available for partitioning in bsdinstall. We can, however,
use manual partitioning in shell to make ZFS partitions.

But what if we want to do ZFS on /? This is currently not supported
out-of-the-box. The reason is that Petitboot, used on POWER9 to boot kernel,
doesn't support loading modules (at least when booting FreeBSD). ZFS is by
default distributed only as a module, to overcome licensing issues.

There are two ways to boot ZFS on root with Petitboot:
1. We can use initramfs, which will load necessary modules. It's said to be
supported on FreeBSD/powerpc64, but no one knows how to create one for FreeBSD
:)
2. We can actually compile a custom kernel with ZFS in it (not as a module).
This the approach that I use. Unfortunately, I can't redistribute this
kernel, because of ZFS's license. I created a custom installation iso, which
uses my kernel with ZFS included.

Here are the steps to do it (on an installed powerpc64 system, but should be
possible on other architectures after redefining TARGET and TARGET_ARCH):
1. cd /usr/src/sys/powerpc/conf
2. cat << EOF > ZFS
include GENERIC64
options ZFS
EOF
3. cd ../../../
3. make buildworld buildkernel KERNCONF=ZFS
4. cd release
5. make release

When it's done, there will be iso files in
/usr/obj/usr/src/powerpc.powerpc64/release.

You can use it to install ZFS on root.

Note that you need a FAT32 partition to mount on /boot.
This partition needs to have /boot/etc/kboot.conf file, with a line like:
FreeBSD='/kernel/kernel kernelname=/boot/kernel/kernel
vfs.root.mountfrom=zfs:zroot/ROOT/default'

You also need the appriopriate /etc/fstab:
/dev/nda0p1     /boot           msdosfs rw      2       2

Other than that, howto at
https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot#Create_Partitions (from point 2.,
omit 1.) is totally ok to follow.

Thanks to mmacy for informing me about this little trick with including ZFS to
kernel.

-- 
 _________________________________________ 
/ It has been said that man is a rational \
| animal. All my life I have been         |
| searching for evidence which could      |
| support this.                           |
|                                         |
\ -- Bertrand Russell                     /
 ----------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 898 bytes
Desc: not available
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20181105/7a94ef1a/attachment.sig>

On Mon, Nov 5, 2018, 3:45 AM Piotr Kubaj via freebsd-ppc
<freebsd-ppc at freebsd.org wrote:
>
> ZFS is by default not available for partitioning in bsdinstall. We can,
however, use manual partitioning in shell to make ZFS partitions.
>
> But what if we want to do ZFS on /? This is currently not supported
out-of-the-box. The reason is that Petitboot, used on POWER9 to boot kernel,
doesn't support loading modules (at least when booting FreeBSD). ZFS is by
default distributed only as a module, to overcome licensing issues.
>
> There are two ways to boot ZFS on root with Petitboot:
> 1. We can use initramfs, which will load necessary modules. It's said
to be supported on FreeBSD/powerpc64, but no one knows how to create one for
FreeBSD :)
> 2. We can actually compile a custom kernel with ZFS in it (not as a
module). This the approach that I use. Unfortunately, I can't redistribute
this kernel, because of ZFS's license. I created a custom installation iso,
which uses my kernel with ZFS included.
>
> Here are the steps to do it (on an installed powerpc64 system, but should
be possible on other architectures after redefining TARGET and TARGET_ARCH):
> 1. cd /usr/src/sys/powerpc/conf
> 2. cat << EOF > ZFS
> include GENERIC64
> options ZFS
> EOF
> 3. cd ../../../
> 3. make buildworld buildkernel KERNCONF=ZFS
> 4. cd release
> 5. make release
>
> When it's done, there will be iso files in
/usr/obj/usr/src/powerpc.powerpc64/release.
>
> You can use it to install ZFS on root.
>
> Note that you need a FAT32 partition to mount on /boot.
> This partition needs to have /boot/etc/kboot.conf file, with a line like:
> FreeBSD='/kernel/kernel kernelname=/boot/kernel/kernel
vfs.root.mountfrom=zfs:zroot/ROOT/default'
>
> You also need the appriopriate /etc/fstab:
> /dev/nda0p1     /boot           msdosfs rw      2       2
>
> Other than that, howto at
https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot#Create_Partitions (from point 2.,
omit 1.) is totally ok to follow.
>
> Thanks to mmacy for informing me about this little trick with including ZFS
to kernel.
>
> --
>  _________________________________________
> / It has been said that man is a rational \
> | animal. All my life I have been         |
> | searching for evidence which could      |
> | support this.                           |
> |                                         |
> \ -- Bertrand Russell                     /
>  -----------------------------------------
>         \   ^__^
>          \  (oo)\_______
>             (__)\       )\/\
>                 ||----w |
>                 ||     ||
I don't think there are any license issues distributing a FreeBSD
kernel with ZFS statically linked like there are with Linux. For a
long time it wasn't supported and ZFS will probably never be in
GENERIC but a custom kernel with ZFS built in should be freely
redistributable. I don't claim to be an expert on the license issues
so if someone else has better information feel free correct me.

-Bill Sorenson