Daniel Ebdrup Jensen
2021-May-05 23:45 UTC
FreeBSD Quarterly Status Report - First Quarter 2021
Introduction
This report covers FreeBSD related projects for the period between January and
March, and is the first of four planned reports for 2021.
The first quarter of 2021 has been very active in both FreeBSD-CURRENT and
-STABLE, with 13.0-RELEASE work starting in January and finishing up mid-April.
It provides lots of new features, and there?s even a good chance that some
workloads will experience performance improvements.
The number of entries is slightly down, and this is probably due to a
combination of factors like code slush as well as the ongoing issues with
COVID-19, but we naturally hope that things will look up next quarter. This
combined with a switch-over to AsciiDoctor and a decision to make full use of
the status report work schedule to avoid stress, means that the report can now
be expected to come out at the end of the first month after the quarter has
finished, rather than in the middle.
This report in particular includes a number of interesting entries, covering
everything from the linuxulator, various mitigation work, long-awaited work on
OpenBSM, work on kernel sanitizers, and many more things that it is hoped you
will enjoy reading about.
Yours,
Daniel Ebdrup Jensen, with a status hat on.
???????????????????????????????????????????????????????????????????????????????
Table of Contents
? FreeBSD Team Reports
? FreeBSD Foundation
? FreeBSD Release Engineering Team
? Cluster Administration Team
? Continuous Integration
? Ports Collection
? Projects
? Git Migration Working Group
? LLDB Debugger Improvements
? Linux compatibility layer update
? Vulnerability Mitigations
? OpenBSM Synchronisation
? Kernel
? ENA FreeBSD Driver Update
? Intel wireless update
? Kernel Sanitizers
? Marvell ARM64 SoCs support
? nv(9)-based audio device enumeration
? Ports
? KDE on FreeBSD
? FreeBSD Office team 2021Q1 status report
? VirtualBox FreeBSD port
? Documentation
? DOCNG on FreeBSD
? FreeBSD Translations on Weblate
? WebApps working group
? Miscellaneous
? Discord Server & Community Growth
? Third-Party Projects
? CBSD Project
? helloSystem
? PkgBase.live
? Potluck & Potman
? sysctl improvements
???????????????????????????????????????????????????????????????????????????????
FreeBSD Team Reports
Entries from the various official and semi-official teams, as found in the
Administration Page.
FreeBSD Foundation
Contact: Deb Goodkin <deb at FreeBSDFoundation.org>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to
supporting and promoting the FreeBSD Project and community worldwide. Funding
comes from individual and corporate donations and is used to fund and manage
software development projects, conferences and developer summits, and provide
travel grants to FreeBSD contributors. The Foundation purchases and supports
hardware to improve and maintain FreeBSD infrastructure and provides resources
to improve security, quality assurance, and release engineering efforts;
publishes marketing material to promote, educate, and advocate for the FreeBSD
Project; facilitates collaboration between commercial vendors and FreeBSD
developers; and finally, represents the FreeBSD Project in executing contracts,
license agreements, and other legal arrangements that require a recognized
legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
COVID-19 Impact to the Foundation
Like most organizations, our team continued to work from home. Our temporary
ban on travel for staff members remains in effect, but continues to not affect
our output too much, since most conferences are still virtual. We continued
supporting the community and Project, even though some of our work and
responses may have been delayed because of changes in some of our priorities
and the impact of limited childcare for a few of our staff members.
Partnerships and Commercial User Support
We help facilitate collaboration between commercial users and FreeBSD
developers. We also meet with companies to discuss their needs and bring that
information back to the Project. Not surprisingly, the stay at home orders,
combined with our company ban on travel during Q1 made in-person meetings
non-existent. However, the team was able to continue meeting with our partners
and commercial users virtually. These meetings help us understand some of the
applications where FreeBSD is used.
We were thrilled for the opportunity to work with AMD early on to ensure
FreeBSD worked on their recently released third generation EPYC series. You can
read more about that here: https://freebsdfoundation.org/news-and-events/
latest-news/freebsd-well-prepared-for-amd-epyc-7003-series-processors/.
Fundraising Efforts
First, we?d like to say thank you to everyone who has given us a financial
contribution this year! Last quarter we raised $88,237, which includes
donations from organizations like Facebook and Tarsnap, as well as many
individuals. We also have a few donation commitments for this quarter.
Going forward this quarter, we will be reaching out to FreeBSD commercial users
to help support our growing efforts. At the beginning of 2021, we opened two
job positions in our software development team, to increase the amount of
support we are able to provide in this area. That includes increasing the
amount of code reviews and bug fixes we do and adding some major features to
FreeBSD, to help keep FreeBSD the innovative, secure, and reliable operating
system you rely on.
You?ll find out how we used your donations for Q1 in our report, as well as
individual reports throughout this status report.
We are excited about our plans for 2021, which include more FreeBSD online
advocacy and training, operating system course content, and the software
development work mentioned above. While we are still in this pandemic, we?re
working hard to help connect folks within the community with more virtual
opportunities.
Please consider making a donation to help us continue and increase our support
for FreeBSD in 2021: https://www.freebsdfoundation.org/donate/.
We also have the Partnership Program, to provide more benefits for our larger
commercial donors. Find out more information and share with your companies!
https://www.freebsdfoundation.org/FreeBSD-foundation-partnership-program/
OS Improvements
Over the quarter a total of 264 base system commits, 63 ports commits, and 10
doc tree commits were tagged as sponsored by the FreeBSD Foundation. The
Foundation also sponsored work that was committed to third-party repositories,
including 26 commits to LLDB (the LLVM project debugger). This includes work
from staff members, interns, and grant recipients. In other quarterly report
entries you can read more about some of these sponsored projects, such as LLDB
and other kernel debugging improvements, and kernel sanitizers.
As usual, staff members committed numerous bug fixes, minor improvements, and
security patches. Focus areas in the kernel included virtual memory, x86 pmap,
uma, tmpfs, nullfs, ffs and ufs, and job control improvements.
User space work included changes to the libc, libcasper, and libthr libraries,
the run-time linker, as well as the ldd, cmp, diff, makefs, elfctl, growfs, and
bhyve utilities.
Foundation staff also participated in many Phabricator code reviews, supported
bug triage, integrated a number of submissions from third parties, and
supported the Git transition working group.
Foundation staff also supported the promotion of the AArch64 (arm64)
architecture to Tier-1 status. Work included additions to freebsd-update,
integration of various bug fixes, and test run issue triage.
Continuous Integration and Quality Assurance
The Foundation provides a full-time staff member and funds projects on
improving continuous integration, automated testing, and overall quality
assurance efforts for the FreeBSD Project.
During the first quarter of 2021, the work was focused on pre-commit tests and
building release artifacts in the CI staging environment.
The other main working item is following the VCS migration to change the src
source from Subversion to Git and doc changed to AsciiDoc format.
See the FreeBSD CI section of this report for completed work items and detailed
information.
Supporting FreeBSD Infrastructure
The Foundation provides hardware and support to improve the FreeBSD
infrastructure. Last quarter, we continued supporting FreeBSD hardware located
around the world. We coordinated efforts between the new NYI Chicago facility
and clusteradm to start working on getting the facility prepared for some of
the new FreeBSD hardware we are planning on purchasing. NYI generously provides
this for free to the Project. We also worked on connecting with the new owners
of the NYI Bridgewater site, where most of the FreeBSD infrastructure is
located.
FreeBSD Advocacy and Education
A large part of our efforts are dedicated to advocating for the Project. This
includes promoting work being done by others with FreeBSD; producing advocacy
literature to teach people about FreeBSD and help make the path to starting
using FreeBSD or contributing to the Project easier; and attending and getting
other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD
tables, and give FreeBSD presentations.
The FreeBSD Foundation sponsors many conferences, events, and summits around
the globe. These events can be BSD-related, open source, or technology events
geared towards underrepresented groups. We support the FreeBSD-focused events
to help provide a venue for sharing knowledge, to work together on projects,
and to facilitate collaboration between developers and commercial users. This
all helps provide a healthy ecosystem. We support the non-FreeBSD events to
promote and raise awareness of FreeBSD, to increase the use of FreeBSD in
different applications, and to recruit more contributors to the Project. While
we were still unable to attend in-person meetings due to covid-19, we were able
to attend virtual events and began planning for the online Spring FreeBSD
Developer Summit. In addition to attending and planning virtual events, we are
continually working on new training initiatives and updating our selection of
how-to guides to facilitate getting more folks to try out FreeBSD. https://
www.freebsdfoundation.org/freebsd/how-to-guides/
Check out some of the advocacy and education work we did last quarter:
? Presented a workshop at Apricot 2021
? Staffed a virtual stand at FOSDEM 2021 and created a what?s new in 13.0
video to accompany the stand
? Staffed a virtual booth and was a community sponsor for FOSSASIA 2021
? Participated as an Industry Partner for USENIX FAST ?21
? Committed to be an Industry Partner for USENIX Annual Tech, USENIX OSDI,
USENIX Security and USENIX LISA
? Continued to promote the FreeBSD Office Hours series Videos from the one
hour sessions can be found on the Project?s YouTube Channel: https://
www.youtube.com/c/FreeBSDProject. See the Office Hours section of this
report for more information.
? Worked with the organizing committee to begin planning the Spring FreeBSD
Developers Summit.
? Continued recruiting for the FreeBSD Fridays series. The series will return
in May.
? Participated in an interview with The Register about FreeBSD 13.0
highlights. https://www.theregister.com/2021/03/10/the_state_of_freebsd/
Keep up to date with our latest work in our newsletters: https://
freebsdfoundation.org/our-work/latest-updates/?filter=newsletter
We help educate the world about FreeBSD by publishing the professionally
produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is
now a free publication. Find out more and access the latest issues at https://
www.freebsdfoundation.org/journal/.
You can find out more about events we attended and upcoming events at https://
www.freebsdfoundation.org/news-and-events/.
Legal/FreeBSD IP
The Foundation owns the FreeBSD trademarks, and it is our responsibility to
protect them. We also provide legal support for the core team to investigate
questions that arise.
Go to http://www.freebsdfoundation.org to find out how we support FreeBSD and
how we can help you!
???????????????????????????????????????????????????????????????????????????????
FreeBSD Release Engineering Team
Links:
FreeBSD 13.0-RELEASE schedule URL: https://www.freebsd.org/releases/13.0R/
schedule.html
FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapshots/
ISO-IMAGES/
Contact: FreeBSD Release Engineering Team, <re at FreeBSD.org>
The FreeBSD Release Engineering Team is responsible for setting and publishing
release schedules for official project releases of FreeBSD, announcing code
freezes and maintaining the respective branches, among other things.
During the first quarter of 2021, the Release Engineering Team started work on
the 13.0-RELEASE cycle, the first release from the stable/13 branch. As of this
writing, the release is progressing smoothly, with one additional BETA build
and two additional RC builds added to the schedule. The schedule has been
updated on the FreeBSD Project website to reflect the updates.
Additionally throughout the quarter, several development snapshots builds were
released for the head, stable/12, and stable/11 branches. Development snapshot
builds for stable/13 will be available after the 13.0 release.
Thank you to all that have helped test the 13.0 builds up until this point and
have reported issues. As always, we strive for quality over quantity.
Sponsor: Rubicon Communications, LLC ("Netgate") Sponsor: The FreeBSD
Foundation
???????????????????????????????????????????????????????????????????????????????
Cluster Administration Team
Contact: Cluster Administration Team <clusteradm at FreeBSD.org>
Links:
Cluster Administration Team members URL: https://www.freebsd.org/administration
/#t-clusteradm
The FreeBSD Cluster Administration Team consists of the people responsible for
administering the machines that the Project relies on for its distributed work
and communications to be synchronised. In this quarter, the team has worked on
the following:
? Installed a new package builder
? Added Git support to cluster management scripts
? Put local Git mirrors on the universe machines
? Replaced disks in the UK mirror
? Replaced a disk in pointyhat (https://pkg-status.freebsd.org)
? Recycled some old dead-weight servers eating up rackspace and power at our
primary cluster site
? Upgraded developer reference platforms
? ref{11,12,13,14}-{amd64,i386}
? universe*
? Installed two new aarch64 machines
? ref12-aarch64, ref13-aarch64, ref14-aarch64
? security-officer aarch64 freebsd-update builder
? Worked with asciidoc project to update https://www.freebsd.org and https://
docs.freebsd.org
? Installed a new mirror server in Brazil, sponsored by nic.br
? gdns points everyone from South America to this mirror
? complete {download,ftp,pkg}.freebsd.org mirror
? Helped rmacklem@ participate in this year?s NFS Bakeathon interoperability
testing event by providing a cluster machine to the testing VPN
? Ongoing day to day cluster management activity
? Putting out fires
? Babysitting pkgsync
Work in progress:
? Move pkg-master.nyi to new hardware
? Fix git fallouts
? Upgrade cluster hardware
? Upgrade developer-facing machines to 14-CURRENT
? Install ref14* machines
? Improve to the package building infrastructure
? Research and test migration away from mailman2
? Work with Git migration working group for ports tree migration
? Review the service jails and service administrators operation
? Improve the web service architecture
? Improve the cluster backup plan
? Setup powerpc pkgbuilder/ref/universal machines
? Prepare for a new mirror site in Australia, to be hosted by IX Australia
? Search for more providers that can fit the requirements for a generic
mirrored layout or a tiny mirror
???????????????????????????????????????????????????????????????????????????????
Continuous Integration
Links:
FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org
FreeBSD Hardware Testing Lab URL: https://ci.FreeBSD.org/hwlab
FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org
FreeBSD CI weekly report URL: https://hackmd.io/@FreeBSD-CI
FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins
Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI
3rd Party Software CI URL: https://wiki.freebsd.org/3rdPartySoftwareCI
Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maauwg
FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci
Contact: Jenkins Admin <jenkins-admin at FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu at FreeBSD.org>
Contact: freebsd-testing Mailing List
Contact: IRC #freebsd-ci channel on EFNet
The FreeBSD CI team maintains the continuous integration system of the FreeBSD
project. The CI system firstly checks the committed changes can be successfully
built, then performs various tests and analysis over the newly built results.
The artifacts from those builds are archived in the artifact server for further
testing and debugging needs. The CI team members examine the failing builds and
unstable tests and work with the experts in that area to fix the code or adjust
test infrastructure. The details of these efforts are available in the weekly
CI reports.
During the first quarter of 2021, we continued working with the contributors
and developers in the project to fulfil their testing needs and also keep
collaborating with external projects and companies to improve their products
and FreeBSD.
Important changes:
? All src jobs were changed to use git to follow VCS migration. Thanks
Brandon Bergren (bdragon@) again.
? Doc job was updated for following the AsciiDoc migration.
New jobs added:
? TCP test suite for main on amd64
? GCC 9 build for main on amd64
Work in progress and open tasks:
? Designing and implementing pre-commit CI building and testing
? Designing and implementing use of CI cluster to build release artifacts as
release engineering does
? Collecting and sorting CI tasks and ideas here
? Testing and merging pull requests in the FreeBSD-ci repo
? Reducing the procedures of CI/test environment setting up for contributors
and developers
? Setting up the CI stage environment and putting the experimental jobs on it
? Setting up public network access for the VM guest running tests
? Implementing automatic tests on bare metal hardware
? Adding drm ports building tests against -CURRENT
? Planning to run ztest and network stack tests
? Adding more external toolchain related jobs
? Improving the hardware lab to be more mature and adding more hardware
? Helping more software get FreeBSD support in their CI pipeline Wiki pages:
3rdPartySoftwareCI, HostedCI
? Working with hosted CI providers to have better FreeBSD support
? The build and test results will be sent to the dev-ci mailing list soon.
Feedback and help with analysis is very appreciated!
Please see freebsd-testing@ related tickets for more WIP information, and don?t
hesitate to join the effort!
Sponsor: The FreeBSD Foundation
???????????????????????????????????????????????????????????????????????????????
Ports Collection
Links:
About FreeBSD Ports URL: https://www.FreeBSD.org/ports/
Contributing to Ports URL: https://docs.freebsd.org/en/articles/contributing/
ports-contributing/
FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/
Ports Management Team URL: https://www.freebsd.org/portmgr/
Ports Tarball URL: http://ftp.freebsd.org/pub/FreeBSD/ports/ports/
Contact: Ren? Ladan <portmgr-secretary at FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr at FreeBSD.org>
The Ports Management Team is responsible for overseeing the overall direction
of the Ports Tree, building packages, and personnel matters. Below is what
happened in the last quarter.
As always, first the quarterly dashboard: * we currently have around 43,800
ports (including flavors). * the open PR count for ports is currently 2477, of
which 532 are unassigned. * during the last quarter, 9481 commits were made by
168 committers on the main branch, and 620 commits by 64 committers on the
2021Q1 branch. Compared to 2020Q4, the number of ports again grew by five
percent, the number of open PRs dropped a bit, and the number of commits on the
main branch grew with almost nine percent.
During the last quarter, we welcomed Neel Chauhan (nc@), Lewis Cook (lcook@),
and Nuno Teixeira (eduardo@). Adrian Chadd (adrian@) who is already a src
committer got a ports commit bit extension. Tobias Berner (tcberner@) asked if
he could join the portmgr-lurker program and was shortly added afterwards.
We sent another mail to the ports@ mailing list outlining further plans for
removing Python 2.7 from the Ports Tree. Currently all ports recursively
depending on Python 2.7 are marked to expire on 2021-06-23, which unfortunately
includes a lot of KDE ports due to the qt5-webengine port. We are evaluating
various mitigation strategies.
portmgr has been collaborating with the Git Working Group over the last year to
prepare the Ports Tree to be converted to Git. Tasks included: * converting
various scripts and tools to support Git * attending Git Working Group meetings
* updating documentation * updating various internal and public third-party
services * evaluating numerous test conversion (git-beta) results
Regarding the Ports Tree itself, two new USES were introduced: * kodi to ease
porting of Kodi add-ons * mpi for dependencies of MPICH and OpenMPI A new
default version for ImageMagick was added and the default version for Julia was
removed as no Julia port currently exists. pkg was updated to 1.16.3, Firefox
to 87.0, and Chromium to 89.0.4389.114
The Cluster Administration Team assisted with getting three new package
building machines running in the build cluster. Two are for arm64 builds and
one is a general builder.
antoine@ was again busy with exp-runs, 28 this time, to: * test various ports
updates * update the clang/LLVM version from 6 to 10 in USES=compiler * reduce
includes in /usr/include/crypto
???????????????????????????????????????????????????????????????????????????????
Projects
Projects that span multiple categories, from the kernel and userspace to the
Ports Collection or external projects.
Git Migration Working Group
Links:
Git transition wiki URL: https://wiki.freebsd.org/git
doc git repo URL: https://cgit.FreeBSD.org/doc
ports git repo URL: https://cgit.FreeBSD.org/ports
src (base system) git repo URL: https://cgit.FreeBSD.org/src
Committers guide Git primer URL: https://docs.freebsd.org/en/articles/
committers-guide/#git-primer
Handbook Using Git appendix URL: https://docs.freebsd.org/en/books/handbook/
mirrors/#git
Game of Trees URL: http://gameoftrees.org/
gitup URL: https://github.com/johnmehr/gitup
Contact: Li-Wen Hsu <lwhsu at FreeBSD.org>
Contact: Warner Losh <imp at FreeBSD.org>
Contact: Ed Maste <emaste at FreeBSD.org>
Contact: Ulrich Sp?rlein <uqs at FreeBSD.org>
Contact: FreeBSD-git mailing list
Contact IRC #gitcvt channel on EFnet
The doc and src trees were migrated from Subversion to Git at the end of 2020,
with some additional work extending into the first quarter of 2021. The Git
Working Group implemented or updated commit hooks, and prepared for FreeBSD 13
to be built from Git. We converted draft documentation from Markdown to
AsciiDoc and merged it into the committer?s guide and handbook.
The ports repository migration to Git started at the end of the quarter,
beginning with a final Subversion commit on March 31st to indicate that the
conversion started. We are working on portsnap and other ports infrastructure
and they will be finished before or soon after the migration.
The Git Working Group continues to track progress on two permissively-licensed
git compatible tools: Gitup and Game of Trees. Gitup is a small,
dependency-free tool to clone and update git repositories. It is used only to
keep a local tree up-to-date, and has no support for local commits.
Game of Trees is a version control client that is compatible with Git
repositories. It provides a user interface and workflow that is distinct from
that of Git. It is in no way intended to be a drop-in replacement for git, but
can be used to develop software maintained in a Git repository.
Gitup and Game of Trees are currently available as ports and packages. Future
work will evaluate them as candidates for the base system.
In the second quarter of 2021 we expect to complete some minor remaining
migration tasks. This will complete the initial phase of the Git migration, and
the working group will wind down. The core team will then begin a new effort to
investigate and evaluate new workflow changes.
Sponsor: The FreeBSD Foundation (in part)
???????????????????????????????????????????????????????????????????????????????
LLDB Debugger Improvements
Links:
Moritz Systems Project Description URL: https://www.moritz.systems/blog/
lldb-freebsd-cpu-target-support-and-userland-debugging-improvements/
Progress Report 1 URL: https://www.moritz.systems/blog/
freebsd-remote-process-plugin-on-non-x86-architectures/
Progress Report 2 URL: https://www.moritz.systems/blog/
freebsd-legacy-process-plugin-removed/
Progress Report 3 URL: https://www.moritz.systems/blog/
lldb-support-for-fork-and-vfork/
Git Repository URL: https://github.com/moritz-systems/llvm-project
Contact: Kamil Rytarowski <kamil at moritz.systems>
Contact: Micha? G?rny <mgorny at moritz.systems>
The LLDB project builds on libraries provided by LLVM and Clang to provide a
great modern debugger. It uses the Clang ASTs and the expression parser, LLVM
JIT, LLVM disassembler, etc so that it provides an experience that ?just
works?. It is also blazingly fast and more permissively licensed than GDB, the
GNU Debugger.
FreeBSD includes LLDB in the base system. At present, it has some limitations
in comparison with the GNU GDB debugger, and does not yet provide a complete
replacement. This project aimed to finish porting the FreeBSD platform support
in LLDB to the modern client-server model on all architectures originally
supported by LLDB on FreeBSD and removing the obsolete plugin.
After switching to the new process model, the project focused on implementing
support for tracing fork(2) and vfork(2) syscalls. The proposed model is
compatible with the follow-fork-mode setting from GDB. On fork, the debugger
can either continuing tracing the parent and detach the child, or switch to
tracing the child and detach the parent. The new code makes it possible to
debug child processes. It also prevents software breakpoints from leaking to
child processes and causing them to crash.
The introduced changes are expected to be shipped with LLDB 13.0.
The overall experience of FreeBSD/LLDB developers and advanced users on this
rock solid Operating System reached the state known from other environments.
Furthermore, the FreeBSD-focused work also resulted in generic improvements,
enhancing the LLDB support for Linux and NetBSD.
TODO: we are currently working on adding a ptrace(2) request to create a core
dump of the stopped program without crashing it. Afterwards, we are planning to
improve LLDB test coverage for core dump support and work on any issues we
might hit.
Sponsor: The FreeBSD Foundation
???????????????????????????????????????????????????????????????????????????????
Linux compatibility layer update
Contact: Edward Tomasz Napierala, <trasz at FreeBSD.org>
Linuxulator improvements have been ongoing for the last two years, with support
from the FreeBSD Foundation over a few distinct project grants as well as
contributions from the community. The goal of this project is to improve
FreeBSD?s ability to execute unmodified Linux binaries. Current support status
of specific Linux applications is being tracked at the Linux app status Wiki
page.
The work this quarter focused on making sure the 13.0-RELEASE ships with
Linuxulator in a good shape, and fixing problems reported by users. There are
some new directories provided by linsysfs(5), the lack of which, through a
curious chain of events, broke installation of make(1) in Ubuntu Focal. The
getcwd(2) syscall was fixed to no longer return the wrong error value for
certain conditions, which was breaking Mono. The getsockopt(2) syscall now
supports SO_PEERSEC and SO_PEERGROUPS, which are being used by su(8) and sudo
(8). Other fixes include flag handling for 32-bit send(2) syscall, and several
ptrace(2) problems, which were affecting Steam games. The kernel version was
bumped to 3.17.0 to unbreak Qt applications from Focal. The sysutils/
debootstrap port, and its corresponding debootstrap package, now correctly
handle Ubuntu?s GPG keys. The debootstrap utility now installs the mremap(2)
workaround for apt(8). This reduces the number of steps required to set up
Linux chroot or jail. Finally there have been some improvements to the startup
scripts.
Sponsor: The FreeBSD Foundation
???????????????????????????????????????????????????????????????????????????????
Vulnerability Mitigations
Contact: Ed Maste <emaste at FreeBSD.org>
Contact: Konstantin Belousov <kib at FreeBSD.org>
Contact: Marcin Wojtas <mw at FreeBSD.org>
Contact: Dawid G?recki <dgr at semihalf.com>
We added support for enforcing a write XOR execute mapping policy. It is
enabled by setting the kern.elf64.allow_wx and/or kern.elf32.allow_wx sysctls
to 0 (for 64-bit and 32-bit binaries, respectively). Binaries can indicate that
they requre writeable and executable mappings by setting the
NT_FREEBSD_FCTL_WXNEEDED ELF feature bit, set via elfctl.
In addition, elfctl received a few usability improvements to support use by
ports, targeting different FreeBSD base system versions. We added a -i flag to
ignore unknown flags (so that the same elfctl invocation could be used on older
FreeBSD versions) as well as the ability to specify features by value.
Flags that request opt-out of a mitigation now have a no prefix to make the
sense clearer. For example, the flag to indicate that the binary is not
compatible with ASLR is now named noaslr. Unprefixed flag names are still
supported, for backwards compatibility, but will emit a warning and will be
removed in a later version.
The next step is to introduce ports infrastructure to support tagging binaries
in ports that require special flags. Details can be found in PR252629.
Another update is that the base system binaries are now built as
position-independent executable (PIE) by default, for 64-bit architectures. PIE
executables are used in conjunction with address randomization (ASLR) as a
mitigation for certain types of security vulnerabilities. The ASLR feature
still remains opt-in, however the described change allows enabling it using
only sysctl knobs, without a need to rebuild the image. Enabling PIE results in
no material performance impact for most workloads.
It is also worth mentioning that a certain number of ports inherit the base
systems /usr/share/mk infrastructure, and some initially failed to build after
toggling the PIE setting. All issues detected by executing the exp-run were
addressed. The details can be found in PR253275.
The next step is to try enabling ASLR by default for 64-bit architectures. The
patch is under discussion.
Sponsor: The FreeBSD Foundation Sponsor: Stormshield
???????????????????????????????????????????????????????????????????????????????
OpenBSM Synchronisation
Links:
TrustedBSD / OpenBSM URL: http://www.trustedbsd.org/openbsm.html
OpenBSM Github Sources URL: https://github.com/openbsm/openbsm
Synchronisation with macOS Catalina URL: https://github.com/openbsm/openbsm/
commit/54a0c07cf8bac71554130e8f6760ca68e5f36c7f
Apple OpenSource URL: https://opensource.apple.com
Contact: Gordon Bergling <gbe at FreeBSD.org>
OpenBSM is a crucial part of FreeBSD, which provides auditing features for the
operating system. OpenBSM is incorporated into FreeBSD and macOS. Both Apple
and FreeBSD have currently made changes to the OpenBSM framework, which weren?t
upstreamed. This small project aims to consolidate these changes and upstream
them to the OpenBSM github repository, so that both development efforts can be
merged to FreeBSD later on.
There is currently a pull request pending that synchronizes the FreeBSD sources
with OpenBSM. A comparison was made to incorporate Apple?s Catalina changes. A
few weeks ago Apple has also made the source code of Big Sur available. In the
latest comparison against OpenBSM Apple has made overlapping ID changes, which
are making a simple import of the changes impossible. I am currently trying to
work around that issue by making OpenBSM a little vendor specific.
???????????????????????????????????????????????????????????????????????????????
Kernel
Updates to kernel subsystems/features, driver support, filesystems, and more.
ENA FreeBSD Driver Update
Links:
ENA README URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/
ena/README
Contact: Michal Krawczyk <mk at semihalf.com>
Contact: Artur Rojek <ar at semihalf.com>
Contact: Marcin Wojtas <mw at semihalf.com>
ENA (Elastic Network Adapter) is the smart NIC available in the virtualized
environment of Amazon Web Services (AWS). The ENA driver supports multiple
transmit and receive queues and can handle up to 100 Gb/s of network traffic,
depending on the instance type on which it is used.
Completed since the last update:
? Update ENA driver to v2.3.1
? Determine location of the MSIx vector table on the device and allocate it
dynamically - this enables driver usage on instances like c5gn.
? MFC of the ENA v2.3.1 driver to the FreeBSD 11/12/13-STABLE branches
? ENA v2.3.1 will be a part of the FreeBSD 13.0-RELEASE
Work in progress:
? Internal review ongoing:
? Introduce full kernel RSS API support
? Allow reconfiguration of the RSS indirection table and hash key
? Adjust iflib framework for the ENA requirements
? Add DMA width configuration field commit 6dd69f0064f1
? Add support for admin completion queues commit 09c3f04ff3be
? Prototype the driver port to the iflib framework
Sponsor: Amazon.com Inc
???????????????????????????????????????????????????????????????????????????????
Intel wireless update
Links:
The freebsd-wireless mailing list URL: https://lists.freebsd.org/mailman/
listinfo/freebsd-wireless
Contact: Bjoern A. Zeeb <bz at FreeBSD.org>
Newer Intel Wireless device support
The Intel Wireless driver update project aims to bring support for newer
chipsets.
During the first quarter the driver and firmware were synched from upstream so
that we will have support for all modern cards currently supported in Linux.
Some iwlwifi driver changes were also submitted back upstream.
Several conflicts with the original implementation of LinuxKPI were or are
being resolved and more LinuxKPI code was upstreamed to FreeBSD HEAD.
LinuxKPI 802.11 compat code was improved and as of the day of writing we have
data packets going over 11a.
The plan for the next weeks is to clean things up, land as much as possible in
HEAD, provide the code for testing and work on stability based on feedback
before filling gaps in the LinuxKPI 802.11 compat code to enhance support for
more standards and features.
Sponsor: The FreeBSD Foundation
???????????????????????????????????????????????????????????????????????????????
Kernel Sanitizers
Contact: Mark Johnston <markj at FreeBSD.org>
Work has been ongoing to port a pair of kernel sanitizers from NetBSD to
FreeBSD. Sanitizers are debugging tools which make use of compiler
instrumentation to validate memory accesses. They can automatically detect many
classes of C programming bugs, such as use-after-frees and loads of
uninitialized variables. When combined with syzkaller or other testing tools,
they are effective at detecting kernel bugs that may otherwise require a great
deal of manual effort to identify.
Two sanitizers are being ported, KASAN (AddressSanitizer) and KMSAN (
MemorySanitizer). KASAN checks the validity of all memory accesses within the
kernel map and triggers a panic upon an out-of-bounds access or a
use-after-free. Various kernel memory allocators annotate regions of memory to
denote whether corresponding accesses are valid. The initial port of KASAN is
complete and is planned to appear in the FreeBSD development branch in
mid-April. KMSAN detects uses of uninitialized memory and can detect bugs that
result in the contents of kernel memory being leaked to userspace. Both
sanitizers incur considerable memory and CPU overhead. They are intended to be
used mainly in conjunction with test frameworks, though it is certainly
possible to boot and run sanitizer-enabled kernels in a desktop or laptop
environment. Currently this work is amd64-only. It should be possible to port
it to arm64 and riscv with relatively little effort.
Future work in this area consists of finishing the KMSAN port, fixing bugs
found by the combination of KASAN and syzkaller, and making sure that
sanitizers can validate accesses to the direct map. This may consist of an
option to disable usage of the direct map, or introducing a shadow for the
direct map.
Sponsor: The FreeBSD Foundation
???????????????????????????????????????????????????????????????????????????????
Marvell ARM64 SoCs support
Contact: Zyta Szpak <zr at semihalf.com>
Contact: Kornel Dul?ba <mindal at semihalf.com>
Contact: Marcin Wojtas <mw at semihalf.com>
The Semihalf team is working on improving the FreeBSD support for the Marvell
Octeon TX2 CN913x and Armada 7k/8k SoC families.
Marvell Armada 7k8k and Octeon TX2 CN913x SoC families are quad-core 64-bit
ARMv8 Cortex-A72 processors with high speed peripherals including 10 Gb
Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking,
storage, security and industrial applications.
Although the mentioned SoCs are mostly supported in FreeBSD HEAD, some pieces
required improvements.
Applied changes:
? Add missing frequency modes in ap806_clock driver (commit a86b0839d7bf)
? Multiple fixes in mvebu_gpio driver - in cooperation with mmel (commit
a5dce53b75d8)
? Fix device tree data parsing in mv\_ap806\_gicp interrupt controller driver
(commit 622d17da46eb)
? Rework the ICU interrupt controller (mv\_cp110\_icu) and its parent (mv\
_ap806\_gicp), so that they no longer rely on the data provided by
firmware, which fixes booting the OS from the newer U-Boot/TF-A revisions (
D28803)
? PCIE Designware driver (pci_dw) fixes:
? Correct setting of outbound I/O ATU window.
? Allow mapping ATU windows bigger than 4GB.
? Generic improvements that enable proper user-space mapping and access of
the PCI BARs
TODO:
? Upstream PCIE improvements.
? Improve and merge ICU support rework.
Sponsor: Marvell
???????????????????????????????????????????????????????????????????????????????
nv(9)-based audio device enumeration
Links:
D26884 Implement sndstat nvlist-based enumeration ioctls. URL: https://
reviews.freebsd.org/D26884
commit c96151d33509655efb7fb26768cb56a041c176f1 URL: https://cgit.freebsd.org/
src/commit/?id=c96151d33509655efb7fb26768cb56a041c176f1
Contact: Ka Ho Ng <khng at FreeBSD.org>
This work presents a number of ioctl commands on /dev/sndstat using nv(9) to
expose all available audio device nodes. nv(9) is used to generate a serialized
binary stream representation of the information of audio device nodes presented
in the running system. The documented nvlist structure in sndstat(4) manual
page is stable for programming use.
For a long time, enumerating the audio device node interface required parsing
content of /dev/sndstat. It is tedious to write such a parser and handle
different hw.snd.verbose levels correctly. Using nv(9) eliminates the need to
write a text parser to do audio device nodes enumeration.
This work has been committed and is available in FreeBSD 14-CURRENT.
Sponsor: The FreeBSD Foundation
???????????????????????????????????????????????????????????????????????????????
Ports
Changes affecting the Ports Collection, whether sweeping changes that touch
most of the tree, or individual ports themselves.
KDE on FreeBSD
Links:
KDE FreeBSD URL: https://freebsd.kde.org/
KDE Community FreeBSD URL: https://community.kde.org/FreeBSD
Contact: Adriaan de Groot <kde at FreeBSD.org>
The KDE on FreeBSD project aims to package all of the software produced by the
KDE Community for the FreeBSD ports tree. The software includes a full desktop
environment called KDE Plasma, graphics applications, instant-messengers, a
video-editing suite, as well as a tea timer and hundreds of other applications
that can be used on any FreeBSD machine.
The KDE team (kde@) is part of desktop@ and x11@ as well, building the software
stack to make FreeBSD beautiful and usable as a daily-driver graphics-based
desktop machine.
The KDE Frameworks have a monthly release cycle; KDE Plasma and the rest of KDE
software run on a quarterly cycle plus monthly bugfixes. All of those releases
landed in ports in a timely manner. Around KDE there are several hundred other
applications with their own releases, of which notable or new ones are:
? deskutils/calindori, deskutils/kongress, net-im/kaidan, deskutils/semantik
and graphics/kgeotag
? net-im/ruqola and net-im/neochat for Rocket and Matrix instant-messaging,
respectively
? audio/amarok, the one-time favorite KDE music player
Infrastructure work improved the way Qt5 ports install- and un-install changes
to the global header qconfig-modules.h. CMake releases landed with distressing
regularity, and various low-level things like devel/libphonenumber and graphics
/poppler were updated as needed.
The big issue in the Qt stack on FreeBSD is Qt5-WebEngine, which is based on
Chromium. Like Chromium itself (upstream), it has a tangled mess of a build
system based on Python 2.7. The scheduled removal of Python 2.7 and ports that
depend on it is a sword looming over a large chunk of the Qt and KDE stack.
Some resolution may be forthcoming in the form of WebEngine-less ports, but the
real effort is in trying to get WebEngine to build with Python3.
More detailed descriptions of the updates in this quarter are available here
(part 1) and here (part 2).
???????????????????????????????????????????????????????????????????????????????
FreeBSD Office team 2021Q1 status report
Links:
The FreeBSD Office project URL: https://wiki.freebsd.org/Office
Contact: FreeBSD Office team ML <office at FreeBSD.org>
Contact: Dima Panov <fluffy at FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu at FreeBSD.org>
The FreeBSD Office team works on a number of office-related software suites and
tools such as OpenOffice and LibreOffice.
Work during this quarter was focused on providing the latest stable release of
LibreOffice suite and companion apps to all FreeBSD users.
Latest and quarterly ports branches got a new branch (7.1) of the LibreOffice
suite and updated to 7.1.1 release.
Meanwhile, our WIP repository got back a working CI instance again, thanks to
Li-Wen Hsu.
We are looking for people to help with the open tasks:
? The open bugs list contains all filed issues which need some attention
? Upstream local patches in ports
Patches, comments and objections are always welcome in the mailing list and
bugzilla.
???????????????????????????????????????????????????????????????????????????????
VirtualBox FreeBSD port
Links:
VirtualBox home page URL: https://www.virtualbox.org/
VirtualBox OSE port on FreshPorts URL: https://www.freshports.org/emulators/
virtualbox-ose
Contact: VirtualBox port team <vbox at FreeBSD.org>
The VirtualBox ports have been updated to the upstream 6.1.18 release.
This is a new major release with new features and better support, especially
for graphics output. This new release has support only for recent amd64 CPUs
providing virtualization support in hardware (VT-x, AMD-V bits).
The previous versions of the VirtualBox ports have been preserved as the
-legacy versions to allow people unable to use the new version to have a
virtualization solution.
The new additions port at present fails to build on i386 but the old additions
do provide basic functionality for that emulation.
???????????????????????????????????????????????????????????????????????????????
Documentation
Noteworthy changes in the documentation tree, in manpages, or in external books
/documents.
DOCNG on FreeBSD
Contact: Sergio Carlavilla <carlavilla at FreeBSD.org>
The Doc New Generation project is finished. The switch-over date was Saturday,
January 23rd.
But there?s a list of remaining tasks:
? Convert the Python scripts to Ruby using the AsciiDoctor API
? Convert from releases from 4.4 to 9.0 to AsciiDoctor
? Use rouge in the source sections instead of the CSS hack
? Split the news page to reduce the total size of the page
? Split the books
? Improve the FDP book
If you want to reduce the TODO list, give me a ping :)
???????????????????????????????????????????????????????????????????????????????
FreeBSD Translations on Weblate
Links:
Translate FreeBSD on Weblate wiki URL: https://wiki.freebsd.org/
DocTranslationOnWeblate
FreeBSD Weblate Instance URL: https://translate-dev.freebsd.org/
Contact: Danilo G. Baio <dbaio at FreeBSD.org>
After the doc migration to Hugo/AsciiDoctor the Weblate tool it?s opened again.
There are three projects on our Weblate:
? Documentation (Books and Articles) - open
? FreeBSD Doc (Archived) - former project
? Website - pending
Language teams that were using Weblate before the migration to Hugo/
AsciiDoctor, please see our Translation based on Automatic Suggestions Wiki
article for more details.
We?ve just started a project for converting the old method of automatically
translating strings, using the Machine Translation feature, to a new system
that will work for the new documentation. This will save time for our
translators.
There are still pending items: you can check the Status Page; any help is very
welcome.
The next step for the new quarter is to prepare and release the Website
translations through Weblate as well.
???????????????????????????????????????????????????????????????????????????????
WebApps working group
Contact: Sergio Carlavilla <carlavilla at FreeBSD.org>
The purpose of this working group is to redesign the Website and the
Documentation Portal.
The work will be divided into 4 phases.
Phase 1:
? Redesign the documentation portal: new design, responsive and global
search.
Phase 2:
? Redesign the manual pages scripts to generate the HTML using mandoc.
Phase 3:
? Redesign the ports scripts to create an applications portal.
Phase 4:
? Redesign the main website: new design, responsive and dark theme.
???????????????????????????????????????????????????????????????????????????????
Miscellaneous
Objects that defy categorization.
Discord Server & Community Growth
Links:
Discord Wiki Page URL: https://wiki.freebsd.org/Discord
Discord Invitation Link URL: https://discord.gg/RHprKbvWJN
Contact: Lewis Cook <lcook at FreeBSD.org>
Contact: Vincent Milum Jr <freebsd at darkain.com>
Contact: Kubilay Kocak <koobs at FreeBSD.org>
The FreeBSD project and community continues to grow, and in the last 2
quarters, we established an official FreeBSD server on the popular Discord
platform, as a complementary means for out-reach, support, collaboration and
social connection for and between members of the FreeBSD community, new and
old.
Discord boasts broad accessibility and unique features that the the community
can enjoy without the steeper learning curves associated with other support and
social mediums, that can often deter or overwhelm newcomers. It also provides
an opportunity to broaden FreeBSD?s reach outside traditional spaces and
audiences.
We currently have a respectable 480 members, with that number growing daily,
and have established baseline community guidelines and moderation processes
consistent with and complementary to other support channels and the FreeBSD
Code of Conduct.
While it?s early days, events have already been successfully hosted on the
platform. In January, Tom Jones (thj@) announced and ran an online Bugathon
focusing on issues related to the branching of FreeBSD 13. We?ve also created
dedicated text and voice channels to facilitate more of these kinds of events
in the future. We hope to see more events like these run as examples of how we
can utilize Discord constructively and in ways we haven?t as a community or
project before.
With the future in mind, we have plans to:
? Automatically announce news, updates and advisories in Discord.
? Verify and enable additional Discord features designed for large
communities.
? Set up bots with unique features, including moderation and interactive
features for members. We welcome ideas in this regard.
We are keen on project and community members to reach out to talk about how we
can best leverage Discord. Some ideas we?d love people to get involved with
include:
? Brainstorm/Suggest unique and creative ideas or features.
? Provide bug reports and user experience feedback and suggestions.
? Actively promote Discord in other social media spaces, particularly those
that maybe new or curious to learn more about FreeBSD.
? Contribute to the Wiki page and its content.
? Participate and support other members on Discord.
? Run a live stream on a FreeBSD-related topic.
? Hang out in our live audio and video channels if you?re comfortable doing
so.
???????????????????????????????????????????????????????????????????????????????
Third-Party Projects
Many projects build upon FreeBSD or incorporate components of FreeBSD into
their project. As these projects may be of interest to the broader FreeBSD
community, we sometimes include brief updates submitted by these projects in
our quarterly report. The FreeBSD project makes no representation as to the
accuracy or veracity of any claims in these submissions.
CBSD Project
Links:
CBSD API module URL: https://www.bsdstore.ru/en/cbsd_api_ssi.html
Contact: Oleg Ginzburg <olevole at olevole.ru>
What is CBSD?
CBSD is a management layer written for the FreeBSD jail(8) subsystem, bhyve(8)
and xen(4). CBSD allows users to manage jail/bhyve/xen environments at
different levels of abstraction by providing a varied number of unified
methods: vagrant-like CBSDfiles, CLI and via dialog(1).
CBSD 2021Q1 Status Report
A RestAPI service layer was added during last quarter, enabling creation of
programmable cloud solutions. In addition, work has been done to support
RestAPI through a CBSDfile, allowing for private cloud environments deployment.
In such cases the local CBSD layer acts as a thin client.
???????????????????????????????????????????????????????????????????????????????
helloSystem
Links:
Documentation URL: https://hellosystem.github.io/docs/
Contact: Simon Peter <probono at puredarwin.org>
Contact: #helloSystem on irc.freenode.net, mirrored to #helloSystem:matrix.org
on Matrix
What is helloSystem?
helloSystem is FreeBSD preconfigured as a desktop operating system with a focus
on simplicity, elegance, and usability. Its design follows the ?Less, but
better? philosophy.
Q1 2021 Status
? helloSystem and some of the motivations and core ideas behind it were
presented at the FOSDEM 2021 BSD Devroom in the talk "hello?? again?
Simplicity, elegance, and usability for the desktop". Video recordings
of
the talk and Q&A session are available WebM/VP9, mp4
? Version 0.4.0 of helloSystem was published. Installable Live ISO images are
available.
? Work has started towards 0.5.0. We are beginning to see contributed
features and bugfixes
? System menu reflects changes made in Applications immediately
? Filer file manager brings already-open windows to the front rather than
opening multiple windows for the same folder
? Initial spatial mode option (each folder opens in its own window in the
file manager that remembers its on-screen location)
Experimental and release builds of the Live ISO are available at https://
github.com/helloSystem/ISO/releases.
Contributing
Help is wanted in a number of areas, especially in the areas of the FreeBSD
core OS and kernel, and Qt/C++.
???????????????????????????????????????????????????????????????????????????????
PkgBase.live
Links:
Website URL: https://alpha.pkgbase.live/
Contact: Mina Gali? <me at igalic.co>
PkgBase.live is an unofficial repository for the FreeBSD PkgBase project.
PkgBase packages the FreeBSD base system as ca 330 packages.
The PkgBase project gives users the choice of which parts of the system to
install. Users can choose which parts of the base system to install, without
building their system from source and optionally choose to install -dbg
packages when they need them. PkgBase is built with default options. There?s no
need to build WITHOUT_SENDMAIL, when users can just chose not to install it! In
addition, PkgBase.live builds every usable kernel! This is especially important
for architectures like armv7.
As a service, PkgBase.live was inspired by up.bsd.live, which provides
freebsd-update(8) for STABLE and CURRENT branches. Despite this inspiration,
freebsd-update has been a constant point of frustrations for me, so I was
looking for alternatives.
PkgBase is not ready for prime time yet, or else the FreeBSD project would be
providing this service. With the call for testing open since 2016, I thought it
was time to offer a public service, so a broader part of the community can take
part in testing, without having to do all the work for themselves.
A lot of things already work fine, but more work needs to be done, as can be
seen from the TODO list, as well as the "Pending Changes" on the
website.
Perhaps the most important thing would be to provide ISOs which lets people
setup a fresh system with PkgBase from the get-go.
Hardware for PkgBase is kindly sponsored by a member of the FreeBSD community.
???????????????????????????????????????????????????????????????????????????????
Potluck & Potman
Links:
Potluck Repository & Project URL: https://potluck.honeyguide.net/
Potluck on github URL: https://github.com/hny-gd/potluck
Potman on github URL: https://github.com/grembo/potman
Pot Project URL https://pot.pizzamig.dev
Contact: Stephan Lichtenauer (Potluck) <sl at honeyguide.eu>
Contact: Michael Gmelin (Potman) <grembo at FreeBSD.org>
Pot is a jail management tool that also supports orchestration through Nomad.
Potluck aims to be to FreeBSD and Pot what Dockerhub is to Linux and Docker: A
repository of Pot flavours and complete images for usage with Pot and in many
cases Nomad.
The new Potman project aims to simplify building Pot images with Vagrant and
VirtualBox based on the Potluck approach, e.g. as part of a DevOps workflow for
software development and testing. That way, Pot images can more easily be
created as part of a Jenkins workflow to be deployed with Nomad and Consul.
In the last two quarters, FreeBSD 12.2 Potluck images have been built and the
11.4 images have been upgraded with the new packages. Furthermore, new images
like Wordpress and an improved flavour script (which squashes a nasty network
problem) have been created.
Future plans include further Potman workflow features, new and improved Potluck
images and publishing FreeBSD 13-based images.
As always, feedback and patches are welcome.
???????????????????????????????????????????????????????????????????????????????
sysctl improvements
Links:
sysctlinfo URL: https://gitlab.com/alfix/sysctlinfo
sysctlbyname-improved URL: https://gitlab.com/alfix/sysctlbyname-improved
BSDCan 2020 - sysctlinfo questions URL: https://git.io/Jm9x7
sysctl-libnv URL:https://gitlab.com/alfix/sysctl-libnv[https://gitlab.com/alfix
/sysctl-libnv]
sysctlmibinfo2 URL:https://gitlab.com/alfix/sysctlmibinfo2[https://gitlab.com/
alfix/sysctlmibinfo2]
sysctlview URL: https://gitlab.com/alfix/sysctlview
nsysctl URL: https://gitlab.com/alfix/nsysctl
Contact: Alfonso Sabato Siciliano <alfonso.siciliano at email.com>
The sysctl system call and the wrapper sysctl utility can get and set the
system state at runtime; the kernel exposes the available parameters as objects
of a Management Information Base. After a recent system update with a
CURRENT-GENERIC configuration, the MIB has exceeded ten thousand objects (both
internal nodes and leaves, most in the vm.uma subtree) on my laptop with a
Desktop environment without ZFS. Furthermore I received tips, ideas, PRs and
issues about sysctl so some improvement has been fulfilled; finally the
suggestions received during BSDCan 2020 have been accomplished.
Kernel space
sysutils/sysctlinfo has been updated to 20210222 and is an interface to visit
the MIB and to retrieve info about an object (description, type, format, flags,
and so on). It has been refactored so the new version is almost 100% more
efficient to explore the MIB and to pass all info about an object to userland.
Moreover new features have been implemented: to get more info about an object,
to avoid extra computation in userland, and to improve the compatibilty with
the current undocumented interface.
sysutils/sysctlbyname-improved has been updated to 20210223 and is an extension
of sysctlinfo to handle an object name with some empty string level or extended
to pass an input to the handler of a CTLTYPE_NODE; it has been updated to take
advantage of the improvements (mainly efficiency) of sysctlinfo.
sysctl-libnv is a project that provides an implementation and an example of how
to build a sysctl object with an nvlist value - to learn more about nvlist, see
the libnv(9) manual page. Properly a new sysctl handler has been defined: it is
enough to create a nvlist and to pass it to a macro; then the system call uses
the new handler to pass the nvlist to the userland and the nsysctl utility can
manage the object value.
The following tools are been updated to give advantages from new kernel
features and improvements.
Library
devel/libsysctlmibinfo2 has been updated to 2.0.1; primarily the sysctlmibinfo2
library wraps the low-level interfaces described above; moreover it defines a
struct sysctlmif_object with the properties of an object and provides a
convenient API to build data structures of sysctlmif_object (for example: a
subtree, a list of a list of a Depth First Traversal, and so on); therefore it
is useful for handling an object correctly and/or for building a sysctl-like
utility.
Obviously sysctlmibinfo2 benefits from the features of sysctlinfo: handles OIDs
up to CTL_MAXNAME levels, supports the Capability Mode, can seek an object
matching its name (avoiding having to explore the MIB just to find the
corresponding OID), gets all info about an object at a time, and manages a
special name via sysctlbyname-improved.
Version 2.0.1 takes advantage from the kernel improvements: improved efficiency
to build a sysctlmif_object and new features to get info about an object:
"handler" and "nextbyname". The new functions are:
sysctlmif_hashandler() and
sysctlmif_hashandlerbyname() to know if an object has a defined kernel handler,
sysctlmif_nextnodebyname() and sysctlmif_nextleafbyname() to explore the MIB,
sysctlmif_leaves() and sysctlmif_leavesbyname() to build only-leaf data
structures.
Documentation
The APIs described above (both kernel and userspace) are really easy:
"sysctl
-aN", "sysctl -d kern.ostype", etc., can be implemented in a few
lines of code.
Nevertheless each project provides a README with Introduction, Getting started,
Features, API, Real-world use cases, FAQ, and examples in the Public Domain to
build new projects. Of course the manuals and examples have recently been
updated.
Utilities
deskutils/sysctlview has been updated to 2.1; the first version of sysctlview
was just a graphical representation of the MIB, now it could be considered a
GUI version of sysctl. This utility exploits the object serialization of
sysctlinfo; indeed it is not feasible to have the kernel to find the same
object many times to retrieve all its properties, considering the current MIB
size. Thanks to user feedbacks the new version provides a better UI, for
example clicking a column title to sort the entries, moreover the
"Handler"
entry is been added in the "Object" window, it is useful to know if an
object
has a value or if the OID of a CTLTYPE_NODE can be extended.
sysutils/nsysctl has been updated to 2.0 and is the CLI version of sysctlview;
the output is explicitly indicated by the options and is printed via libxo in
human- and machine-readable formats; moreover some string value is parsed to
display structured output. The options are not mutually exclusive and allow
showing the properties of a parameter so nsysctl is useful to know the info to
handle an object without finding its implementation, for example: Is
Multiprocessor safe? Is Capability mode available? Is the OID extensible? Does
the integer represent a kelvin? Does it have a value? What is the label? And so
on. The new version supports libnv; it is useful to manage a non-primitive data
type and could avoid hardcoding a generic opaque type in the future. Finally
the new features of sysctlinfo allow using nsysctl to debug the MIB without a
kernel recompilation with SYSCTL_DEBUG. Note: the project provides a tutorial
to describe every feature.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 618 bytes
Desc: not available
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20210506/545e8687/attachment-0001.sig>