freebsd stable - Sep 2016 - 11.0-RELEASE tier level for arm64/aaarch64 and the officially built arm/armv6 variants?

On Sat, Sep 24, 2016 at 11:10 PM, Warner Losh <imp at bsdimp.com>
wrote:
> On Sat, Sep 24, 2016 at 10:21 PM, Russell Haley <russ.haley at
gmail.com> wrote:
>> On Sat, Sep 24, 2016 at 5:12 PM, Mark Millard <markmi at
dsl-only.net> wrote:
>>> On 2016-Sep-24, at 2:11 PM, Warner Losh <imp at bsdimp.com>
wrote:
>>>
>>>> On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard <markmi at
dsl-only.net> wrote:
>>>>> [A resend since I forget to list free-arm in the To: the
first time.]
>>>>>
>>>>> From https://www.freebsd.org/platforms/arm.html :
>>>>>
>>>>>> 32-bit ARM is officially a Tier 2 architecture, as the
FreeBSD project does not provide official releases or pre-built packages for
this platform due to it primarily targeting the embedded arena. However,
FreeBSD/ARM is being actively developed and maintained, is well supported, and
provides an excellent framework for building ARM-based systems. FreeBSD/arm
supports ARMv4 and ARMv5 processors. FreeBSD/armv6 supports ARMv6 and ARMv7
processors, including SMP on the latter.
>>>>>
>>>>> "does not provide official releases or pre-built
packages"?
>>>>>
>>>>>> # uname -apKU
>>>>>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD 11.0-PRERELEASE #5
r304943M: Sun Aug 28 03:17:54 PDT 2016     markmi at
FreeBSDx64:/usr/obj/clang/arm.armv6/usr/src/sys/RPI2-NODBG  arm armv6 1100502
1100502
>>>>>
>>>>>> # pkg search '.*' | wc
>>>>>>  21349  155540 1596736
>>>>>
>>>>> Will 11.0-RELEASE change the tier level for any of the
specific arm-armv6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files
built, such as for RPI2?
>>>>>
>>>>> Even if all the officially built arm-armv6 variants stay
tier 2, the wording on the web page likely needs to be changed because so much
is built and available that the above quote claims is not available.
>>>>
>>>> armv6 is basically Tier 1 right now, though not as Tier 1 as
i386 or
>>>> amd64 due to the fragmented nature of the arm world. On the
platforms
>>>> we run on and create releases for, however, it's my opinion
that it is
>>>> Tier 1: it has been running in production a while, things
people
>>>> expect from a FreeBSD system are present, you can get decent
support
>>>> if you ask questions, there's no known major gotchas in
deploying this
>>>> hardware. The only remaining annoying issue is the
'u-boot' problem
>>>> where we have to have a different u-boot image for every board
and no
>>>> standardized way to convert a 'generic' image into one
that's specific
>>>> for specific boards.
>>
>> I'll point out again that barebox is an excellent alternative to
u-boot (IMHO):
>
> Doesn't matter, still has the same issues that u-boot has.
u-boot has a different sources for almost each board we support (due
to the usual FOSS issues). That is NOT the case in barebox. There is
one source and it's kept up to date by the team, not the vendors.
> But does it support the u-boot ABI?
I'd have to look into this.
>
>> - Supports most, if not all of the boards that FreeBSD supports, plus
>> many that it doesn't
>> - Single source tree for all boards. Specify build time parameters to
>> build one or all the images
>> - Well supported community with central maintainer-ship
>> - Simple, familiar shell interface  (*awesome*)
>> - Excellent documentation (u-boots is good too though)
>> - Has support for (U)EFI
>> - Supports quemu aarch64 (not *quite*sure what the means though)
>
> Right, u-boot has all these things, except maybe the shell interface
> (not sure what you mean by that).
Instead of stringing together variables and commands, it uses a
scripting language like a simplified sh. Want to change how something
boots? Update a script. Save it to disk (it has it's own persistence
mechanisms) and export it.
>> To be fair, I'm not saying the problem is the fault of denx, but
>> barebox has a lot going for it. The maintainer was very keen to see it
>> ported top FreeBSD and was willing to support the effort. I ran into
>> some build time linux api requirements, but he didn't think that
would
>> be much to overcome (and it wasn't I just kept adding the patches
he
>> sent me and the build moved forward. As always, I ran out of time for
>> the really fun stuff). While I am a hopeless dreamer and I'm sure
I've
>> over simplified the problem, I thought it would be neat to see FreeBSD
>> upstream support for zfs and ufs to the barebox boot loader and do
>> away with ubldr. We would then have a modern, easy to use, boot loader
>> that supports the standard startup toolchain.
>
> We can't easily do away with ubldr if we want to support tunables,
kernel
> modules loaded at boot time and a few other nifty features like nextboot.
Are these things not in standard loader? Should they be?
>> Either way, if installers move to a pkgng based method (so cool) then
>> installing u-boot and arm binaries from pkg-static will be the same as
>> x86 (ha ha I said that with a straight face!).
>
> Yea, not so much. You have to build the bootable image not on the
> target system, like you do on x86.
Doesn't the current ports and packages cross build everything that's
marked?
> We'd have to have something that
> installs uboot onto a generic image (perhaps with hooks for kernels
> since those aren't generic on armv6) and then put that into a bootable
> SD card.
The x86 installer (that I argue is a platform legacy) has to customize
the bootloader for each installation. If we HAVE to use an installer
on arm, what wrong prompting the user for some input (i.e. what som
are you using) while including all the u-boots  in the ports tree and
all the supported kernels, then just installing the correct ones for
the board (with input from the user)?
>>>>For x86 this is all done with the installer since
>>>> that boot environment is more standardized. Does this last
issue keep
>>>> arm from being Tier 1? That's a judgement call, but I think
the
>>>> project should promote w/o this last issue.
>>
>> How does a platform get promoted? Is that something the Core team
decides?
>
> Yes.
>
>> I see two facts about current Arm support that show platform maturity:
>> a) u-boot is in the ports tree and we have Lego-easy build scripts in
>> crochet that could be called an installation method. Building for arm
>> is not difficult anymore.
>
> Except corchet isn't in the tree, and the solution is horrible.
It's
> a script for each SoC, and those scripts are now scattered about.
> Plus that's a creation from source model, not a creation from
> RE produced bits model, which is needed for Tier 1.
Both correctable sins, especially as it's BSD licensed. My point was
that even building a SOM specific image is relatively painless with
the right scripts. Hell, I've even got a custom build script that
could be modified for generic use.
>> b)Arm requires images, not installers. Correct me if I'm wrong but,
>> installers are a tool primarily invented for x86 PC type computers.
>> FreeBSD publishes standardized ISOs for all supported Arm platforms
>> that work by simply "xzcat | dd" onto the sd card (or
wherever you
>> need it).   I'm not sure how standardized or manual that build
process
>> is, but I would think that if the Arm platform support is able to keep
>> up with the standard FreeBSD release cycle (i.e. not break every other
>> release) then there would be no reason to NOT call it tier 1?
>
> Creating the images is currently a pita. That's it.
I see. So not so mature.
>> What I don't know about is "official" documentation for
building for
>> arm and supporting cross building to Arm. Will someone need to write
>> an "Arm Handbook" to be promoted?
>
> No. While useful, most of that already exists.
Thanks for the response Warner, I always appreciate the chance to
learn more about FreeBSD.
Russ
> Warner
>
>>> Interesting and good to know. Thanks.
>>>
>>> I might have guessed that going along with the u-boot issue would
be the fanout in:
>>>
>>> 11.0/sys/arm/ . . .
>>>
>>> allwinner/a10/
>>> allwinner/a20/
>>> allwinner/a31/
>>> allwinner/a83t/
>>> allwinner/h3/
>>> . . .
>>> broadcom/bcm2835/
>>> . . .
>>>
>>> (Full list not shown.)
>>>
>>> I was thinking that this might make the tier level specific to the
status of each such directory's content so that it was the combination of
that and the sysutils/u-boot-*/ status that made the difference for assigning
the level.  I'd guess that lack of a usable directory in either place would
not be tier 2 even. Similarly until the required sys/arm/*/* and
sysutils/u-boot-*/ directory-tree content have reached a sufficiently complete
status.
>>>
>>> I'd expect that there will always be a lag for what exists in
the world vs. what has these materials worked out in FreeBSD.
>>>
>>>
>>>>> Also from https://www.freebsd.org/platforms/arm.html :
>>>>>
>>>>>> Initial support for 64-bit ARM is complete. 64-bit ARM
platforms follow a set of standard conventions, and a single FreeBSD build will
work on hardware from multiple vendors. As a result, FreeBSD will provide
official releases for FreeBSD/arm64 and packages will be available.
FreeBSD/arm64 is on the path to becoming a Tier 1 architecture.
>>>>>
>>>>> Will 11.0-RELEASE make arm64/aarch64 Tier 1?
>>>>>
>>>>> [I will note that, while there are no official builds for
the Pine64 family (A64 based) that are under the Allwinner arm activity, the
SOC's involved are Cortex-A53 64-bit arm based. They likely do not fit in
the "standard conventions" or arm64/aarch64 would be where they would
have been supported. Some rewording might be appropriate for the above quote as
well.]
>>>>
>>>> No. aarch64 isn't Tier 1 yet. There's many small bits
that are
>>>> missing. It is quite solidly Tier 2, but we don't have a
linker, we
>>>> don't have widespread hardware availability, we don't
have production
>>>> experience with the platform. Most things work, but there's
still some
>>>> gotchas. There's still the 'u-boot' problem with
many arm64 systems
>>>> because for systems that use u-boot to bootstrap UEFI, you need
a
>>>> different image for each board (some closely related board
families
>>>> can get by with one to be pedantic). All these issues are still
>>>> significant barriers to production use. It's not been
officially
>>>> promoted yet and I don't think the time is quite right yet.
>>>
>>> Intersting as well. I'd guess that conceptually this probably
would apply to both:
>>>
>>> sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/
>>> (presuming, contrary to fact, that 11.0 had sys/arm/allwinner/a64/
)
>>>
>>> and. . .
>>>
>>> sys/arm64/cavium/
>>> sys/arm64/cloudabi64/
>>>
>>> So just sys/arm/ vs. sys/arm64/ for an aarch64 would not really
make a difference yet for tier level.
>>>
>>>> Warner
>>>
>>> Thanks again for the notes.
>>>
>>> ==>>> Mark Millard
>>> markmi at dsl-only.net
>>>
>>> _______________________________________________
>>> freebsd-arm at freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
>>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe at
freebsd.org"

On Sun, 25 Sep 2016 00:13:31 -0700
Russell Haley <russ.haley at gmail.com> wrote:
> On Sat, Sep 24, 2016 at 11:10 PM, Warner Losh <imp at bsdimp.com>
wrote:
> > On Sat, Sep 24, 2016 at 10:21 PM, Russell Haley <russ.haley at
gmail.com> wrote:
> >> On Sat, Sep 24, 2016 at 5:12 PM, Mark Millard <markmi at
dsl-only.net> wrote:
> >>> On 2016-Sep-24, at 2:11 PM, Warner Losh <imp at
bsdimp.com> wrote:
> >>>
> >>>> On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard <markmi
at dsl-only.net> wrote:
> >>>>> [A resend since I forget to list free-arm in the To:
the first time.]
> >>>>>
> >>>>> From https://www.freebsd.org/platforms/arm.html :
> >>>>>
> >>>>>> 32-bit ARM is officially a Tier 2 architecture, as
the FreeBSD project does not provide official releases or pre-built packages for
this platform due to it primarily targeting the embedded arena. However,
FreeBSD/ARM is being actively developed and maintained, is well supported, and
provides an excellent framework for building ARM-based systems. FreeBSD/arm
supports ARMv4 and ARMv5 processors. FreeBSD/armv6 supports ARMv6 and ARMv7
processors, including SMP on the latter.
> >>>>>
> >>>>> "does not provide official releases or pre-built
packages"?
> >>>>>
> >>>>>> # uname -apKU
> >>>>>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD
11.0-PRERELEASE #5 r304943M: Sun Aug 28 03:17:54 PDT 2016     markmi at
FreeBSDx64:/usr/obj/clang/arm.armv6/usr/src/sys/RPI2-NODBG  arm armv6 1100502
1100502
> >>>>>
> >>>>>> # pkg search '.*' | wc
> >>>>>>  21349  155540 1596736
> >>>>>
> >>>>> Will 11.0-RELEASE change the tier level for any of the
specific arm-armv6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files
built, such as for RPI2?
> >>>>>
> >>>>> Even if all the officially built arm-armv6 variants
stay tier 2, the wording on the web page likely needs to be changed because so
much is built and available that the above quote claims is not available.
> >>>>
> >>>> armv6 is basically Tier 1 right now, though not as Tier 1
as i386 or
> >>>> amd64 due to the fragmented nature of the arm world. On
the platforms
> >>>> we run on and create releases for, however, it's my
opinion that it is
> >>>> Tier 1: it has been running in production a while, things
people
> >>>> expect from a FreeBSD system are present, you can get
decent support
> >>>> if you ask questions, there's no known major gotchas
in deploying this
> >>>> hardware. The only remaining annoying issue is the
'u-boot' problem
> >>>> where we have to have a different u-boot image for every
board and no
> >>>> standardized way to convert a 'generic' image into
one that's specific
> >>>> for specific boards.
> >>
> >> I'll point out again that barebox is an excellent alternative
to u-boot (IMHO):
> >
> > Doesn't matter, still has the same issues that u-boot has.
> u-boot has a different sources for almost each board we support (due
> to the usual FOSS issues). That is NOT the case in barebox. There is
> one source and it's kept up to date by the team, not the vendors.
 This is not true, U-Boot support all the platforms we are running on
right now in a single source tree.
 I think that the only ports that is not using the main U-Boot is the
wandboard one and I think that Warner have it working now.
> > But does it support the u-boot ABI?
> I'd have to look into this.
> >
> >> - Supports most, if not all of the boards that FreeBSD supports,
plus
> >> many that it doesn't
> >> - Single source tree for all boards. Specify build time parameters
to
> >> build one or all the images
> >> - Well supported community with central maintainer-ship
> >> - Simple, familiar shell interface  (*awesome*)
> >> - Excellent documentation (u-boots is good too though)
> >> - Has support for (U)EFI
> >> - Supports quemu aarch64 (not *quite*sure what the means though)
> >
> > Right, u-boot has all these things, except maybe the shell interface
> > (not sure what you mean by that).
> Instead of stringing together variables and commands, it uses a
> scripting language like a simplified sh. Want to change how something
> boots? Update a script. Save it to disk (it has it's own persistence
> mechanisms) and export it.
 You can do the same with U-Boot.
> >> To be fair, I'm not saying the problem is the fault of denx,
but
> >> barebox has a lot going for it. The maintainer was very keen to
see it
> >> ported top FreeBSD and was willing to support the effort. I ran
into
> >> some build time linux api requirements, but he didn't think
that would
> >> be much to overcome (and it wasn't I just kept adding the
patches he
> >> sent me and the build moved forward. As always, I ran out of time
for
> >> the really fun stuff). While I am a hopeless dreamer and I'm
sure I've
> >> over simplified the problem, I thought it would be neat to see
FreeBSD
> >> upstream support for zfs and ufs to the barebox boot loader and do
> >> away with ubldr. We would then have a modern, easy to use, boot
loader
> >> that supports the standard startup toolchain.
> >
> > We can't easily do away with ubldr if we want to support tunables,
kernel
> > modules loaded at boot time and a few other nifty features like
nextboot.
> 
> Are these things not in standard loader? Should they be?
 They are in the standard loader, using ubldr,loader or loader.efi
doesn't matter but we have to use one of them.
> >> Either way, if installers move to a pkgng based method (so cool)
then
> >> installing u-boot and arm binaries from pkg-static will be the
same as
> >> x86 (ha ha I said that with a straight face!).
> >
> > Yea, not so much. You have to build the bootable image not on the
> > target system, like you do on x86.
> 
> Doesn't the current ports and packages cross build everything
that's marked?
> 
> > We'd have to have something that
> > installs uboot onto a generic image (perhaps with hooks for kernels
> > since those aren't generic on armv6) and then put that into a
bootable
> > SD card.
> 
> The x86 installer (that I argue is a platform legacy) has to customize
> the bootloader for each installation. If we HAVE to use an installer
> on arm, what wrong prompting the user for some input (i.e. what som
> are you using) while including all the u-boots  in the ports tree and
> all the supported kernels, then just installing the correct ones for
> the board (with input from the user)?
> 
> >>>>For x86 this is all done with the installer since
> >>>> that boot environment is more standardized. Does this last
issue keep
> >>>> arm from being Tier 1? That's a judgement call, but I
think the
> >>>> project should promote w/o this last issue.
> >>
> >> How does a platform get promoted? Is that something the Core team
decides?
> >
> > Yes.
> >
> >> I see two facts about current Arm support that show platform
maturity:
> >> a) u-boot is in the ports tree and we have Lego-easy build scripts
in
> >> crochet that could be called an installation method. Building for
arm
> >> is not difficult anymore.
> >
> > Except corchet isn't in the tree, and the solution is horrible.
It's
> > a script for each SoC, and those scripts are now scattered about.
> > Plus that's a creation from source model, not a creation from
> > RE produced bits model, which is needed for Tier 1.
> Both correctable sins, especially as it's BSD licensed. My point was
> that even building a SOM specific image is relatively painless with
> the right scripts. Hell, I've even got a custom build script that
> could be modified for generic use.
> 
> >> b)Arm requires images, not installers. Correct me if I'm wrong
but,
> >> installers are a tool primarily invented for x86 PC type
computers.
> >> FreeBSD publishes standardized ISOs for all supported Arm
platforms
> >> that work by simply "xzcat | dd" onto the sd card (or
wherever you
> >> need it).   I'm not sure how standardized or manual that build
process
> >> is, but I would think that if the Arm platform support is able to
keep
> >> up with the standard FreeBSD release cycle (i.e. not break every
other
> >> release) then there would be no reason to NOT call it tier 1?
> >
> > Creating the images is currently a pita. That's it.
> I see. So not so mature.
> 
> >> What I don't know about is "official" documentation
for building for
> >> arm and supporting cross building to Arm. Will someone need to
write
> >> an "Arm Handbook" to be promoted?
> >
> > No. While useful, most of that already exists.
> 
> Thanks for the response Warner, I always appreciate the chance to
> learn more about FreeBSD.
> Russ
> 
> > Warner
> >
> >>> Interesting and good to know. Thanks.
> >>>
> >>> I might have guessed that going along with the u-boot issue
would be the fanout in:
> >>>
> >>> 11.0/sys/arm/ . . .
> >>>
> >>> allwinner/a10/
> >>> allwinner/a20/
> >>> allwinner/a31/
> >>> allwinner/a83t/
> >>> allwinner/h3/
> >>> . . .
> >>> broadcom/bcm2835/
> >>> . . .
> >>>
> >>> (Full list not shown.)
> >>>
> >>> I was thinking that this might make the tier level specific to
the status of each such directory's content so that it was the combination
of that and the sysutils/u-boot-*/ status that made the difference for assigning
the level.  I'd guess that lack of a usable directory in either place would
not be tier 2 even. Similarly until the required sys/arm/*/* and
sysutils/u-boot-*/ directory-tree content have reached a sufficiently complete
status.
> >>>
> >>> I'd expect that there will always be a lag for what exists
in the world vs. what has these materials worked out in FreeBSD.
> >>>
> >>>
> >>>>> Also from https://www.freebsd.org/platforms/arm.html :
> >>>>>
> >>>>>> Initial support for 64-bit ARM is complete. 64-bit
ARM platforms follow a set of standard conventions, and a single FreeBSD build
will work on hardware from multiple vendors. As a result, FreeBSD will provide
official releases for FreeBSD/arm64 and packages will be available.
FreeBSD/arm64 is on the path to becoming a Tier 1 architecture.
> >>>>>
> >>>>> Will 11.0-RELEASE make arm64/aarch64 Tier 1?
> >>>>>
> >>>>> [I will note that, while there are no official builds
for the Pine64 family (A64 based) that are under the Allwinner arm activity, the
SOC's involved are Cortex-A53 64-bit arm based. They likely do not fit in
the "standard conventions" or arm64/aarch64 would be where they would
have been supported. Some rewording might be appropriate for the above quote as
well.]
> >>>>
> >>>> No. aarch64 isn't Tier 1 yet. There's many small
bits that are
> >>>> missing. It is quite solidly Tier 2, but we don't have
a linker, we
> >>>> don't have widespread hardware availability, we
don't have production
> >>>> experience with the platform. Most things work, but
there's still some
> >>>> gotchas. There's still the 'u-boot' problem
with many arm64 systems
> >>>> because for systems that use u-boot to bootstrap UEFI, you
need a
> >>>> different image for each board (some closely related board
families
> >>>> can get by with one to be pedantic). All these issues are
still
> >>>> significant barriers to production use. It's not been
officially
> >>>> promoted yet and I don't think the time is quite right
yet.
> >>>
> >>> Intersting as well. I'd guess that conceptually this
probably would apply to both:
> >>>
> >>> sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/
> >>> (presuming, contrary to fact, that 11.0 had
sys/arm/allwinner/a64/ )
> >>>
> >>> and. . .
> >>>
> >>> sys/arm64/cavium/
> >>> sys/arm64/cloudabi64/
> >>>
> >>> So just sys/arm/ vs. sys/arm64/ for an aarch64 would not
really make a difference yet for tier level.
> >>>
> >>>> Warner
> >>>
> >>> Thanks again for the notes.
> >>>
> >>> ==> >>> Mark Millard
> >>> markmi at dsl-only.net
> >>>
> >>> _______________________________________________
> >>> freebsd-arm at freebsd.org mailing list
> >>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> >>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe
at freebsd.org"
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at
freebsd.org"

-- 
Emmanuel Vadot <manu at bidouilliste.com> <manu at freebsd.org>

> On Sep 25, 2016, at 12:13 AM, Russell Haley <russ.haley at gmail.com>
wrote:
> 
>> We can't easily do away with ubldr if we want to support tunables,
kernel
>> modules loaded at boot time and a few other nifty features like
nextboot.
> 
> Are these things not in standard loader? Should they be?
"ubldr" *is* the standard loader.  It's built from (mostly) the
same
source as loader(8) used on x86.  But, where loader(8) uses the BIOS
interface to access the disk, "ubldr" uses the U-Boot ABI to
access the disk.

FreeBSD's current boot sequence for ARM boards looks like this:
   * U-Boot loads ubldr
   * ubldr uses U-Boot to access disk and console
   * ubldr loads the kernel, kernel modules, and sets kernel tunables

To replace U-Boot, bare box would either have to duplicate
everything ubldr does (which is a lot) or would have to provide
the U-Boot ABI (or something similar) so that ubldr can provide
the final boot stage.

Tim

On Sun, Sep 25, 2016 at 1:13 AM, Russell Haley <russ.haley at gmail.com>
wrote:
> On Sat, Sep 24, 2016 at 11:10 PM, Warner Losh <imp at bsdimp.com>
wrote:
>> On Sat, Sep 24, 2016 at 10:21 PM, Russell Haley <russ.haley at
gmail.com> wrote:
>>> On Sat, Sep 24, 2016 at 5:12 PM, Mark Millard <markmi at
dsl-only.net> wrote:
>>>> On 2016-Sep-24, at 2:11 PM, Warner Losh <imp at
bsdimp.com> wrote:
>>>>
>>>>> On Fri, Sep 23, 2016 at 8:29 PM, Mark Millard <markmi at
dsl-only.net> wrote:
>>>>>> [A resend since I forget to list free-arm in the To:
the first time.]
>>>>>>
>>>>>> From https://www.freebsd.org/platforms/arm.html :
>>>>>>
>>>>>>> 32-bit ARM is officially a Tier 2 architecture, as
the FreeBSD project does not provide official releases or pre-built packages for
this platform due to it primarily targeting the embedded arena. However,
FreeBSD/ARM is being actively developed and maintained, is well supported, and
provides an excellent framework for building ARM-based systems. FreeBSD/arm
supports ARMv4 and ARMv5 processors. FreeBSD/armv6 supports ARMv6 and ARMv7
processors, including SMP on the latter.
>>>>>>
>>>>>> "does not provide official releases or pre-built
packages"?
>>>>>>
>>>>>>> # uname -apKU
>>>>>>> FreeBSD rpi2 11.0-PRERELEASE FreeBSD
11.0-PRERELEASE #5 r304943M: Sun Aug 28 03:17:54 PDT 2016     markmi at
FreeBSDx64:/usr/obj/clang/arm.armv6/usr/src/sys/RPI2-NODBG  arm armv6 1100502
1100502
>>>>>>
>>>>>>> # pkg search '.*' | wc
>>>>>>>  21349  155540 1596736
>>>>>>
>>>>>> Will 11.0-RELEASE change the tier level for any of the
specific arm-armv6 variants that have FreeBSD-11.0-*-arm-armv6-*.img* files
built, such as for RPI2?
>>>>>>
>>>>>> Even if all the officially built arm-armv6 variants
stay tier 2, the wording on the web page likely needs to be changed because so
much is built and available that the above quote claims is not available.
>>>>>
>>>>> armv6 is basically Tier 1 right now, though not as Tier 1
as i386 or
>>>>> amd64 due to the fragmented nature of the arm world. On the
platforms
>>>>> we run on and create releases for, however, it's my
opinion that it is
>>>>> Tier 1: it has been running in production a while, things
people
>>>>> expect from a FreeBSD system are present, you can get
decent support
>>>>> if you ask questions, there's no known major gotchas in
deploying this
>>>>> hardware. The only remaining annoying issue is the
'u-boot' problem
>>>>> where we have to have a different u-boot image for every
board and no
>>>>> standardized way to convert a 'generic' image into
one that's specific
>>>>> for specific boards.
>>>
>>> I'll point out again that barebox is an excellent alternative
to u-boot (IMHO):
>>
>> Doesn't matter, still has the same issues that u-boot has.
> u-boot has a different sources for almost each board we support (due
> to the usual FOSS issues). That is NOT the case in barebox. There is
> one source and it's kept up to date by the team, not the vendors.
Actually, that's no longer the case. All boards we support have all
their bits in u-boot's main repo. But you can't say no one will fork
barebox to get their support out faster, which is why we have the
current situation in ports... u-boot forked because it was so popular,
and the forks rejoined.
>> But does it support the u-boot ABI?
> I'd have to look into this.
>>
>>> - Supports most, if not all of the boards that FreeBSD supports,
plus
>>> many that it doesn't
>>> - Single source tree for all boards. Specify build time parameters
to
>>> build one or all the images
>>> - Well supported community with central maintainer-ship
>>> - Simple, familiar shell interface  (*awesome*)
>>> - Excellent documentation (u-boots is good too though)
>>> - Has support for (U)EFI
>>> - Supports quemu aarch64 (not *quite*sure what the means though)
>>
>> Right, u-boot has all these things, except maybe the shell interface
>> (not sure what you mean by that).
> Instead of stringing together variables and commands, it uses a
> scripting language like a simplified sh. Want to change how something
> boots? Update a script. Save it to disk (it has it's own persistence
> mechanisms) and export it.
Sounds cool.
>>> To be fair, I'm not saying the problem is the fault of denx,
but
>>> barebox has a lot going for it. The maintainer was very keen to see
it
>>> ported top FreeBSD and was willing to support the effort. I ran
into
>>> some build time linux api requirements, but he didn't think
that would
>>> be much to overcome (and it wasn't I just kept adding the
patches he
>>> sent me and the build moved forward. As always, I ran out of time
for
>>> the really fun stuff). While I am a hopeless dreamer and I'm
sure I've
>>> over simplified the problem, I thought it would be neat to see
FreeBSD
>>> upstream support for zfs and ufs to the barebox boot loader and do
>>> away with ubldr. We would then have a modern, easy to use, boot
loader
>>> that supports the standard startup toolchain.
>>
>> We can't easily do away with ubldr if we want to support tunables,
kernel
>> modules loaded at boot time and a few other nifty features like
nextboot.
>
> Are these things not in standard loader? Should they be?
ubldr is the standard loader, just spelled in a funky way and living
on a place u-boot can see :). We have different loaders for different
environments. even on x86 we have /boot/loader for BIOS boots and
/boot/loader.efi for EFI boots.
>>> Either way, if installers move to a pkgng based method (so cool)
then
>>> installing u-boot and arm binaries from pkg-static will be the same
as
>>> x86 (ha ha I said that with a straight face!).
>>
>> Yea, not so much. You have to build the bootable image not on the
>> target system, like you do on x86.
>
> Doesn't the current ports and packages cross build everything
that's marked?
For builds from source, yes. But that's different than installing from
FreeBSD's installer.
>> We'd have to have something that
>> installs uboot onto a generic image (perhaps with hooks for kernels
>> since those aren't generic on armv6) and then put that into a
bootable
>> SD card.
>
> The x86 installer (that I argue is a platform legacy) has to customize
> the bootloader for each installation. If we HAVE to use an installer
> on arm, what wrong prompting the user for some input (i.e. what som
> are you using) while including all the u-boots  in the ports tree and
> all the supported kernels, then just installing the correct ones for
> the board (with input from the user)?
Ummm, you can't boot anything to do that work like you can on x86.
>>>>>For x86 this is all done with the installer since
>>>>> that boot environment is more standardized. Does this last
issue keep
>>>>> arm from being Tier 1? That's a judgement call, but I
think the
>>>>> project should promote w/o this last issue.
>>>
>>> How does a platform get promoted? Is that something the Core team
decides?
>>
>> Yes.
>>
>>> I see two facts about current Arm support that show platform
maturity:
>>> a) u-boot is in the ports tree and we have Lego-easy build scripts
in
>>> crochet that could be called an installation method. Building for
arm
>>> is not difficult anymore.
>>
>> Except corchet isn't in the tree, and the solution is horrible.
It's
>> a script for each SoC, and those scripts are now scattered about.
>> Plus that's a creation from source model, not a creation from
>> RE produced bits model, which is needed for Tier 1.
> Both correctable sins, especially as it's BSD licensed. My point was
> that even building a SOM specific image is relatively painless with
> the right scripts. Hell, I've even got a custom build script that
> could be modified for generic use.
Yea, a developer can do it with little pain. However, compared to the
pain to boot Linux on these boards, the pain is quite high and you
need to know way too much to make it work.

BTW, my 'horrible' comment was given the state of things today: we
have scattered knowledge of how to make images, custom one-off that
work on some boards well, and fail on others. We have crazy, quirky
things that aren't well integrated into the tree. Crochet itself
solves an interesting problem, but does so a bit in isolation.
>>> b)Arm requires images, not installers. Correct me if I'm wrong
but,
>>> installers are a tool primarily invented for x86 PC type computers.
>>> FreeBSD publishes standardized ISOs for all supported Arm platforms
>>> that work by simply "xzcat | dd" onto the sd card (or
wherever you
>>> need it).   I'm not sure how standardized or manual that build
process
>>> is, but I would think that if the Arm platform support is able to
keep
>>> up with the standard FreeBSD release cycle (i.e. not break every
other
>>> release) then there would be no reason to NOT call it tier 1?
>>
>> Creating the images is currently a pita. That's it.
> I see. So not so mature.
Yes. It's an area I work on when I have time to make better...

Warner
>>> What I don't know about is "official" documentation
for building for
>>> arm and supporting cross building to Arm. Will someone need to
write
>>> an "Arm Handbook" to be promoted?
>>
>> No. While useful, most of that already exists.
>
> Thanks for the response Warner, I always appreciate the chance to
> learn more about FreeBSD.
> Russ
>
>> Warner
>>
>>>> Interesting and good to know. Thanks.
>>>>
>>>> I might have guessed that going along with the u-boot issue
would be the fanout in:
>>>>
>>>> 11.0/sys/arm/ . . .
>>>>
>>>> allwinner/a10/
>>>> allwinner/a20/
>>>> allwinner/a31/
>>>> allwinner/a83t/
>>>> allwinner/h3/
>>>> . . .
>>>> broadcom/bcm2835/
>>>> . . .
>>>>
>>>> (Full list not shown.)
>>>>
>>>> I was thinking that this might make the tier level specific to
the status of each such directory's content so that it was the combination
of that and the sysutils/u-boot-*/ status that made the difference for assigning
the level.  I'd guess that lack of a usable directory in either place would
not be tier 2 even. Similarly until the required sys/arm/*/* and
sysutils/u-boot-*/ directory-tree content have reached a sufficiently complete
status.
>>>>
>>>> I'd expect that there will always be a lag for what exists
in the world vs. what has these materials worked out in FreeBSD.
>>>>
>>>>
>>>>>> Also from https://www.freebsd.org/platforms/arm.html :
>>>>>>
>>>>>>> Initial support for 64-bit ARM is complete. 64-bit
ARM platforms follow a set of standard conventions, and a single FreeBSD build
will work on hardware from multiple vendors. As a result, FreeBSD will provide
official releases for FreeBSD/arm64 and packages will be available.
FreeBSD/arm64 is on the path to becoming a Tier 1 architecture.
>>>>>>
>>>>>> Will 11.0-RELEASE make arm64/aarch64 Tier 1?
>>>>>>
>>>>>> [I will note that, while there are no official builds
for the Pine64 family (A64 based) that are under the Allwinner arm activity, the
SOC's involved are Cortex-A53 64-bit arm based. They likely do not fit in
the "standard conventions" or arm64/aarch64 would be where they would
have been supported. Some rewording might be appropriate for the above quote as
well.]
>>>>>
>>>>> No. aarch64 isn't Tier 1 yet. There's many small
bits that are
>>>>> missing. It is quite solidly Tier 2, but we don't have
a linker, we
>>>>> don't have widespread hardware availability, we
don't have production
>>>>> experience with the platform. Most things work, but
there's still some
>>>>> gotchas. There's still the 'u-boot' problem
with many arm64 systems
>>>>> because for systems that use u-boot to bootstrap UEFI, you
need a
>>>>> different image for each board (some closely related board
families
>>>>> can get by with one to be pedantic). All these issues are
still
>>>>> significant barriers to production use. It's not been
officially
>>>>> promoted yet and I don't think the time is quite right
yet.
>>>>
>>>> Intersting as well. I'd guess that conceptually this
probably would apply to both:
>>>>
>>>> sys/arm/allwinner/a64/ and sysutils/u-boot/pine64/
>>>> (presuming, contrary to fact, that 11.0 had
sys/arm/allwinner/a64/ )
>>>>
>>>> and. . .
>>>>
>>>> sys/arm64/cavium/
>>>> sys/arm64/cloudabi64/
>>>>
>>>> So just sys/arm/ vs. sys/arm64/ for an aarch64 would not really
make a difference yet for tier level.
>>>>
>>>>> Warner
>>>>
>>>> Thanks again for the notes.
>>>>
>>>> ==>>>> Mark Millard
>>>> markmi at dsl-only.net
>>>>
>>>> _______________________________________________
>>>> freebsd-arm at freebsd.org mailing list
>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
>>>> To unsubscribe, send any mail to "freebsd-arm-unsubscribe
at freebsd.org"