freebsd stable - Sep 2020 - Cannot find announcement that min supported i386 CPU is now i686

On 2020-09-01 03:20, Eugene Grosbein wrote:
> CPU: Geode(TM) Integrated Processor by AMD PCS (499.91-MHz 586-class CPU)
>   Origin="AuthenticAMD"  Id=0x5a2  Family=0x5  Model=0xa 
Stepping=2
>   Features=0x88a93d<FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CLFLUSH,MMX>
>   AMD Features=0xc0400000<MMX+,3DNow!+,3DNow!>
> 
> Of course, its NanoBSD image is built with CPUTYPE=i586 as well as
installed packages.
You've got CMOV so you'll be fine with 11.4-RELEASE, whereas I
don't:

CPU: Pentium/P55C (232.68-MHz 586-class CPU)
  Origin="GenuineIntel"  Id=0x543  Family=0x5  Model=0x4  Stepping=3
  Features=0x8003bf<FPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,MMX>

Getting 11.4 to work has been a *major* undertaking:

Setting CPUTYPE in `make.conf` doesn't set -march for the kernel build
tools, so while you can compile world and kernel for i386, you can't
actually link it to create a working kernel on a real i586 (I found that
out with `/usr/obj` and `/usr/src` mounted over NFS).

clang is hypersensitive to command line arguments - if you just fiddle
with CFLAGS etc you'll get a duplicate -O2 and/or -march and that
results in a kernel/world that can't build itself (ZFS wedges).

You must set NO_CPU_CFLAGS= and then each of CFLAGS/CXXFLAGS/COPTFLAGS.

Don't try -Os or -Oz (which really do help on a real i586) - ZFS will
wedge quickly.

11.4 i386 doesn't honour `vfs.zfs.arc_max` in any meaningful way
resulting in one of the `find`s in periodic security wedging ZFS.

By ZFS wedging I mean something gets stuck on zio->i having eaten all of
kmem (I presume - on the real i586 I expect ~200M in ARC would do that
and that's what `top` shows) and that's it for the machine - it
doesn't
panic, but there's no more disk IO either.
This also happens in VirtualBox with 4GB RAM, so it's not a hardware
problem.

To get 11.4 to behave itself (and it's only been 72 hours so still early
days) I've had to set:

options	KSTACK_PAGES=4

and

kern.maxvnode="1024"
vm.kmem_size="330M"
vm.kmem_size_max="330M"
vfs.zfs.arc_max="125M"
vfs.zfs.arc_meta_limit="16M"

I had to set kmem for 11.3 but not the rest.

TL;DR: I'd avoid 11.4 i386 as it doesn't appear to have been tested on
i486/i586 at all.

Unfortunately I'm not sure 12 is an option yet either - 12.0 chokes on
my SiI3124 SATA controllers, and I've yet to compile 12.1 to test
(RELEASE has the same problem as 11.4).

-C

14.09.2020 4:30, Charles Lecklider via freebsd-stable wrote:
> On 2020-09-01 03:20, Eugene Grosbein wrote:
>> CPU: Geode(TM) Integrated Processor by AMD PCS (499.91-MHz 586-class
CPU)
>>   Origin="AuthenticAMD"  Id=0x5a2  Family=0x5  Model=0xa 
Stepping=2
>>  
Features=0x88a93d<FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CLFLUSH,MMX>
>>   AMD Features=0xc0400000<MMX+,3DNow!+,3DNow!>
>>
>> Of course, its NanoBSD image is built with CPUTYPE=i586 as well as
installed packages.
> 
> You've got CMOV so you'll be fine with 11.4-RELEASE, whereas I
don't:
> 
> CPU: Pentium/P55C (232.68-MHz 586-class CPU)
>   Origin="GenuineIntel"  Id=0x543  Family=0x5  Model=0x4 
Stepping=3
>   Features=0x8003bf<FPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,MMX>
> 
> Getting 11.4 to work has been a *major* undertaking:
> 
> Setting CPUTYPE in `make.conf` doesn't set -march for the kernel build
> tools, so while you can compile world and kernel for i386, you can't
> actually link it to create a working kernel on a real i586 (I found that
> out with `/usr/obj` and `/usr/src` mounted over NFS).
Build time for modern FreeBSD version is too gross and needs way too much
memory,
so I stopped building image for my i586 hardware "in place" quite long
ago.
I use my i7-based desktop to build NanoBSD image for its upgrade, it works just
fine
by default setting ARCH/TARGET_ARCH to i386 and correct CPUTYPE.
> Don't try -Os or -Oz (which really do help on a real i586) - ZFS will
> wedge quickly.
Currently i586-class CPU needs as many speed improvements as possible
and I was told that -Os might drop performance comparing with -O2,
so why bother?
> 11.4 i386 doesn't honour `vfs.zfs.arc_max` in any meaningful way
> resulting in one of the `find`s in periodic security wedging ZFS.
Been there, seen that. The problem pre-dates 11.x series and appeared in 10.x.
It was much better in times of 9.x, though.
> By ZFS wedging I mean something gets stuck on zio->i having eaten all of
> kmem (I presume - on the real i586 I expect ~200M in ARC would do that
> and that's what `top` shows) and that's it for the machine - it
doesn't
> panic, but there's no more disk IO either.
> This also happens in VirtualBox with 4GB RAM, so it's not a hardware
> problem.
> 
> To get 11.4 to behave itself (and it's only been 72 hours so still
early
> days) I've had to set:
> 
> options	KSTACK_PAGES=4
This is default for 12.x/i386 but was not merged to stable/11.
> and
> 
> kern.maxvnode="1024"
> vm.kmem_size="330M"
> vm.kmem_size_max="330M"
> vfs.zfs.arc_max="125M"
> vfs.zfs.arc_meta_limit="16M"
> 
> I had to set kmem for 11.3 but not the rest.
You might find useful these also:

vfs.zfs.vdev.cache.size="8M"
vfs.zfs.prefetch_disable="1"

And for kernel config:

options         KVA_PAGES=512

This makes ZFS more stable giving it bigger kernel virtual area with less
utilization for it.
> TL;DR: I'd avoid 11.4 i386 as it doesn't appear to have been tested
on
> i486/i586 at all.
Finally I gave up running ZFS for vmem-contrained systems,
e.g. I moved my i386 virtual machines that benefit from ZFS compression to
amd64.

As for real i586-class hardware uncapable of Long Mode,
I do not use ZFS at all, UFS2/async+gjournal works just fine there including
WiFi, IPSec etc. but no ZFS.