freebsd stable - May 2019 - ZFS...

> On 1 May 2019, at 04:26, Michelle Sullivan <michelle at sorbs.net>
wrote:
> 
>        mfid8   ONLINE       0     0     0
Anyway I think this is a mistake (mfid). I know, HBA makers have been insisting
on having their firmware getting in the middle,
which is a bad thing.

The right way to use disks is to give ZFS access to the plain CAM devices, not
thorugh some so-called JBOD on a RAID
controller which, at least for a long time, has been a *logical* ?RAID0? volume
on a single disk. That additional layer can
completely break the semantics of transaction writes and cache flushes. 

With some older cards it can be tricky to achieve, from patching source drivers
to enabling a sysctl tunable or even
flashing the card to turn it into a plain HBA with no RAID features (or minimal
ones).

If your drives are not called /dev/daX or /dev/adaX you are likely to be in
trouble. Unless something has really changed recently
you don?t want ?mfid? or ?mfisyspd?.

I have suffered hidden data corruption due to a faulty HBA and failures of old
disks, and in all cases ZFS has survived brilianty.

And actually ZFS works on somewhat unreliable hardware. The problem is not
non-perfect hardware, but *evil* hardware with
firmware based on some assumptions that won?t work with ZFS. 

But I agree, non-ECC memory can be a problem. In my case all of the servers had
ECC.





Borja.

On 03/05/2019 08:09, Borja Marcos via freebsd-stable wrote:
> The right way to use disks is to give ZFS access to the plain CAM devices,
not thorugh some so-called JBOD on a RAID
> controller which, at least for a long time, has been a *logical* ?RAID0?
volume on a single disk. That additional layer can
> completely break the semantics of transaction writes and cache flushes.
> 
> With some older cards it can be tricky to achieve, from patching source
drivers to enabling a sysctl tunable or even
> flashing the card to turn it into a plain HBA with no RAID features (or
minimal ones).
Oddly enough I got bitten by something like this yesteray. I have a 
machine containing an HP P400 RAID controller, which is nice enough, but 
I run ZFS so I have made the drives all into RAID-0 as being as close as 
I can get to accessing the raw SAS drives.

BSD seems them as da0, da1, da2, da3 - but the RAID controller oly 
presents one of them to the BIOS, so my booting has to be all from that 
drive. This has been da0 for as long as I can remember, but yesteday it 
decided to start using what BSD sees as da1. Of course this is very hard 
to recognise as da0 and da1 are pretty much mirrors of each other. Spent 
a long time trying to work out why the fixes I was applying to da0 were 
not being used at boot time.

( Having to use Windows XP to talk to the iLo due to browsers dropping 
support for the old ciphers doesnt help either, what do other people 
with oldish HP hardware do about this ? I know its off topic, but theres 
a lot of it out there... )

-pete.