freebsd stable - Dec 2014 - ZFS 'mount error 5' hits production server

On 12/04/2014 01:47, Steven Hartland wrote:
> Unfortunately error 5 isn't very helpful as that's just EIO :(
>
> When booted from the old kernel try moving out the cache file e.g.
> mv /boot/zfs/zpool.cache /boot/zfs/zpool.cache.old
>
> If for some reason your cache file and your disks are out of sync that
> might help.
Ok, will try this.
> Also ensure your loading zfs in /etc/rc.conf
> zfs_enable="YES"
It was there.
> Check you don't have anything zfs custom in /boot/loader.conf
>
> You should just have the two zfs related lines such as:
> zfs_load="YES"
> vfs.root.mountfrom="zfs:tank/root"
It was there. I commented all others out.
> Your pool layout does look a little odd from your zpool get all as
> bootfs seems to be set to just "tank" is that correct as its
quite none
> standard? If not and your root is tank/root then that might also be your
> issue.
It is tank. Back in 2012 there was no installer with ZFS features,
it was done manually, with gnop trick and aligned partitions. Datasets
are used for certain purposes, but kernel and userland resides on the
root dataset 'tank'.
> Next up there should be no need for mountpoint legacy / should do just
> fine.
>
> If you do have a single partition install you might want to consider
> switching to something more standard which looks something like:
> NAME             USED  AVAIL  REFER  MOUNTPOINT
> tank             853G   292G  46.5K  none
> tank/root       21.6G   292G  21.3G  /
> tank/root/tmp   11.2M   292G  11.2M  /tmp
> tank/root/var    270M   292G   270M  /var
>
> Oh and ensure your boot loader has been updated:
> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
I have already done this, at least the gpt zfs bootcode.

So I will give the cache file a try when I am at work tomorrow and
report back.

Thank you, much appreciated!
   Michael

On 04.12.2014 02:06, Michael Schmiedgen wrote:
> On 12/04/2014 01:47, Steven Hartland wrote:
>> Unfortunately error 5 isn't very helpful as that's just EIO :(
>>
>> When booted from the old kernel try moving out the cache file e.g.
>> mv /boot/zfs/zpool.cache /boot/zfs/zpool.cache.old
>>
>> If for some reason your cache file and your disks are out of sync that
>> might help.
I created a new cache file. It is slightly larger in size. But
no luck either. I am running out of ideas. :(

As I said I had a similar 'mount error 5' problem at home some weeks
ago with CURRENT and custom kernel:
https://lists.freebsd.org/pipermail/freebsd-current/2014-October/052900.html

Solution was there to not remove the virtual machine stuff on
bottom of the kernel config. Very strange. Also the commit r271963
(callout convert) that first triggered the problem seems to be
unrelated. Strange.

And some months ago I had a 'mount error 5' problem if loading
nvidia driver in loader.conf, also on my desktop machine at home.

I found similar 'error 5' problems on the mailing list, but no solution
at all.

One thing I can try is to 4K align the start of the boot partition, but
at the moment this is a bit risky, I have no physical access to the
machine.

Please tell if I should provide some more information.

Thanks,
   Michael