FreeBSD Quarterly Status Report
Introduction
This report covers FreeBSD-related projects between October and
December 2010. It is the last of the four reports planned for 2010. The
work on the new minor versions of FreeBSD, 7.4 and 8.2, has been
progressing well and they should be released around the end of this
month.
Thanks to all the reporters for the excellent work! This report
contains 37 entries and we hope you enjoy reading it.
Please note that the deadline for submissions covering the period
between January and March 2011 is April 15th, 2011.
__________________________________________________________________
Projects
* BSDInstall
* Non-executable Stacks
* Webcamd
* xz Compression for Packages and Log Files
* ZFS pool version 28
FreeBSD Team Reports
* FreeBSD Bugbusting Team Status Report
* Release Engineering Team Status Report
* The FreeBSD Foundation Status Report
Network Infrastructure
* DIstributed Firewall and Flow-shaper Using Statistical Evidence
(DIFFUSE)
* Ethernet Switch Framework
* Five New TCP Congestion Control Algorithms for FreeBSD
* FreeBSD 802.11n
* FreeBSD VirtIO Network Driver
* Generic IEEE 802.3 annex 31B full duplex flow control support for
Ethernet in mii(4)
* IPv6 and VIMAGE
* TCP SMP scalability project
Kernel
* Resource Containers
* SYSCTL Type Safety
* TRIM support for UFS
Documentation
* mdocml Replacing groff For manpage Rendering
* The FreeBSD German Documentation Project Status Report
* The FreeBSD Japanese Documentation Project
Userland Programs
* FreeBSD Services Control (fsc)
* GEOM-based ataraid(4) Replacement -- geom_raid
* gpart Improvements
Architectures
* Bringing up OMAP3
* FreeBSD on the Playstation 3
* FreeBSD/EC2
* FreeBSD/sparc64
Ports
* Chromium
* FreeBSD as Home Theater PC
* Port-Sandbox
* Portmaster
* Ports Additions
* Ports Collection
* Robot Operating System
Miscellaneous
* FOSDEM 2011
__________________________________________________________________
Bringing up OMAP3
URL: http://people.FreeBSD.org/~raj/patches/arm/dove_v6.diff
Contact: Warner Losh <imp@bsdimp.com>
Contact: Mohammed Farrag <mfarrag@FreeBSD.org>
The attached file is an old patch for ARM. We are developing new patch
and then we are going toward Porting OMAP3.
__________________________________________________________________
BSDInstall
URL: http://wiki.FreeBSD.org/BSDInstall
Contact: Nathan Whitehorn <nwhitehorn@FreeBSD.org>
BSDInstall is a replacement for the venerable sysinstall installer. It
is designed to be modular and easily extensible, while being fully
scriptable and streamlining the installation process. It is mostly
complete, and installs working systems on i386, amd64, sparc64,
powerpc, and powerpc64, with untested PC98 support.
New Features:
* Allows installation onto GPT disks on x86 systems
* Can do installations spanning multiple disks
* Allows installation into jails
* Eases PXE installation
* Virtualization friendly: can install from a live system onto disk
images
* Works on PowerPC
* Streamlined system installation
* More flexible scripting
* Easily tweakable
* All install CDs are live CDs
Open tasks:
1. Wireless networking configuration wizard.
2. ZFS installation support.
3. Itanium disk setup.
__________________________________________________________________
Chromium
URL: http://www.chromium.org/Home
URL: http://people.FreeBSD.org/~bapt/chrome9-fbsd.png
Contact: Ren? Ladan <freebsd-chromium@FreeBSD.org>
We are working on updating the Chromium web browser in our ports to
stay up to date with the latest supported release. We currently have
the Chromium 9 beta running, but not all features are fully implemented
and the port still needs some polish before it can be committed to the
Ports Collection. We have also been making arrangements with Google to
merge our work with their upstream, which should ease the number of
features and fixes we have to maintain for ourselves in the future. Our
first release should be in a few weeks and coincide with the official
release of Chromium 9.
__________________________________________________________________
DIstributed Firewall and Flow-shaper Using Statistical Evidence (DIFFUSE)
URL: http://caia.swin.edu.au/urp/diffuse/
URL: http://caia.swin.edu.au/urp/diffuse/downloads.html
Contact: Sebastian Zander <szander@swin.edu.au>
Contact: Grenville Armitage <garmitage@swin.edu.au>
DIFFUSE is a system enabling FreeBSD's IPFW firewall subsystem to
classify IP traffic based on statistical traffic properties.
With DIFFUSE, IPFW computes statistics (such as packet lengths or
inter-packet time intervals) for observed flows, and uses ML (machine
learning) techniques to assign flows into classes. In addition to
traditional packet inspection rules, IPFW rules may now also be
expressed in terms of traffic statistics or classes identified by ML
classification. This can be helpful when direct packet inspection is
problematic (perhaps for administrative reasons, or because port
numbers do not reliably identify applications).
DIFFUSE also enables one instance of IPFW to send flow information and
classes to other IPFW instances, which then can act on such traffic
(e.g. prioritise, accept, deny, etc) according to its class. This
allows for distributed architectures, where classification at one
location in your network is used to control fire-walling or
rate-shaping actions at other locations.
In December 2010 we released DIFFUSE v0.1, a set of patches for
FreeBSD-CURRENT. It can be downloaded from the project's web site. The
web site also contains a more comprehensive introduction, including
application examples, links to related work and documentation
describing the software design.
We hope to release DIFFUSE v0.2 soon. Keep an eye on the freebsd-ipfw
and freebsd-net mailing lists for project-related announcements.
__________________________________________________________________
Ethernet Switch Framework
URL: http://loos.no-ip.org/rspro/switch-1.diff
Contact: Luiz Otavio O. Souza <loos.br@gmail.com>
Implementation of a framework for ethernet switch control (directly
connected to the ethernet MAC controller) usually found on embedded
systems. Currently based on ifconfig keywords, adds the vlan control
(filter/pass) on each switch port and adds the possibility for the
management of media state on interfaces with multiple PHYs.
Currently, the code supports the IP175D (from some mikrotik
routerboards) and AR8316 (from Ubiquiti RSPRO) switches.
Open tasks:
1. Finish the IP175C driver (and maybe IP178x).
2. Better integration with miibus (rewrite of switchbus).
3. Fix (some) ifconfig keywords (better keywords, better usage
compatibility).
4. Export the ports statistics through SNMP (if available on switch
chip).
5. Add a swctl tool (?) for global settings management.
6. Write usage examples and the man page information about the new
ifconfig(8) keywords.
__________________________________________________________________
Five New TCP Congestion Control Algorithms for FreeBSD
URL: http://caia.swin.edu.au/freebsd/5cc/
URL: http://caia.swin.edu.au/urp/newtcp/
URL: http://www.FreeBSDFoundation.org/projects.shtml
URL: http://people.FreeBSD.org/~lstewart/patches/5cc/
Contact: David Hayes <dahayes@swin.edu.au>
Contact: Lawrence Stewart <lastewart@swin.edu.au>
Contact: Grenville Armitage <garmitage@swin.edu.au>
Contact: Rui Paulo <rpaulo@FreeBSD.org>
Contact: Bjoern Zeeb <bz@FreeBSD.org>
The project is nearing completion, with the following code already
available in the svn head branch:
* Modular congestion control framework.
* Modularised implementations of NewReno, CUBIC and HTCP congestion
control algorithms.
* Khelp (Kernel Helper) and Hhook (Helper Hook) frameworks.
* Basic Khelp/Hhook integration with the TCP stack.
The ERTT (Enhanced Round Trip Time) Khelp module is days away from
being imported, which will then pave the way for the delay based
congestion control algorithms to follow. Finally, a large documentation
dump will be committed in the form of new and updated man pages.
We anticipate the project will conclude around the end of January 2011.
Open tasks:
1. Import the ERTT Khelp module.
2. Import the VEGAS, HD and CHD delay based congestion control
algorithm modules.
3. Import the documentation dump for all the code
contributed/developed as part of the project.
__________________________________________________________________
FOSDEM 2011
URL: http://www.FOSDEM.org
Contact: Marius Nuennerich <marius@nuenneri.ch>
Contact: Daniel Seuffert <ds@FreeBSD.org>
FOSDEM 2011 will be held from Saturday, February 5th to Sunday February
6th in Brussels, Belgium. We will have a FreeBSD booth and a developers
room. At the booth there will be friendly supporters and a FreeBSD
Foundation member answering questions. The devroom will have 6 1-hour
long talks about different topics, technical and social. FOSDEM is one
of the biggest open-source events in Europe. It is completly free and
no registration is required.
Open tasks:
1. Get more people involved as helpers for the booth and the devroom
are still needed. Please contact Daniel or Marius if you want to
help out.
__________________________________________________________________
FreeBSD 802.11n
URL: http://wiki.FreeBSD.org/AdrianChadd/AtherosStuff
Contact: Adrian Chadd <adrian@FreeBSD.org>
* Net80211 station mode works in 2.4ghz HT/20 mode. HT/40 and 5ghz do
not currently work.
* Basic 802.11 TX and RX on the AR9160 works, from MCS0 to MCS15
* TX A-MPDU and A-MSDU do not currently implemented - so no aggregate
TX will happen
* RX A-MPDU and A-MSDU is implemented and is supposed to work but
does not -- this needs to be debugged
* 802.11n RTS/CTS protection for legacy packets does not currently
work. There is some magic required to fix the TX packet length.
This is in progress.
* WPA2 now works - a commit which enabled the hardware multicast
broke AES-CCMP encryption on at least the AR9160. Further
investigation is needed to fix this (and any other hardware
encryption bugs that are lurking.
__________________________________________________________________
FreeBSD as Home Theater PC
URL: http://wiki.FreeBSD.org/HTPC
Contact: Bernhard Froehlich <decke@FreeBSD.org>
Contact: Juergen Lock <nox@FreeBSD.org>
FreeBSD could be a much better platform for a Home Theater PC than it
currently is. We are focusing on improving support for media center
applications. Extending the major ports (MythTV, VDR, XBMC) and create
some documentation to guide interested people.
Open tasks:
1. Improve remote control support in webcamd and with lirc.
2. Port more Media Center applications (Enna, me-tv, ...).
3. Create a small guide on how to build a great FreeBSD Home Theater
PC.
__________________________________________________________________
FreeBSD Bugbusting Team Status Report
URL: http://www.FreeBSD.org/support.html#gnats
URL: http://wiki.FreeBSD.org/BugBusting
URL: http://people.FreeBSD.org/~linimon/studies/prs/
Contact: Gavin Atkinson <gavin@FreeBSD.org>
Contact: Mark Linimon <linimon@FreeBSD.org>
Contact: Remko Lodder <remko@FreeBSD.org>
Contact: Volker Werth <vwe@FreeBSD.org>
The number of non-ports PRs has held relatively steady over the last
three months, with a slightly improved resolution rate being offset by
a slightly increased rate of new arrivals. Ports PRs have increased
slightly in numbers, due in part to the ports freeze in the lead up to
the release of FreeBSD 7.4 and FreeBSD 8.2. The numbers traditionally
drop quickly again once the freeze is lifted.
In October, Gavin Atkinson and Mark Linimon held a session at the
FreeBSD Developers' Summit at EuroBSDCon, which led to some productive
discussions, and a number of people expressing interest in becoming
more involved with PR triaging and resolution.
The bugbusting team continue work on trying to make the contents of the
GNATS PR database cleaner, more accessible and easier for committers to
find and resolve PRs, by tagging PRs to indicate the areas involved,
and by ensuring that there is sufficient info within each PR to resolve
each issue.
Reports continue to be produced from the PR database, all of which can
be found from the links above. Committers interested in custom reports
are encouraged to discuss requirements with bugmeister@ - we are happy
to create new reports where needs are identified.
As always, anybody interested in helping out with the PR queue is
encouraged to do so, the easiest way being to join us on IRC in
#freebsd-bugbusters on EFnet. We are always looking for additional
help, whether your interests lie in triaging incoming PRs, generating
patches to resolve existing problems, or simply helping with the
database housekeeping (identifying duplicate PRs, ones that have
already been resolved, etc). This is a great way of getting more
involved with FreeBSD!
Open tasks:
1. Try to find ways to get more committers helping us with closing PRs
that the team has already analyzed.
2. Try to get more non-committers involved with the triaging of PRs as
they come in, and generating patches to fix reported problems.
__________________________________________________________________
FreeBSD on the Playstation 3
Contact: Nathan Whitehorn <nwhitehorn@FreeBSD.org>
On January 5, support for the Playstation 3 was imported into FreeBSD
9.0-CURRENT. This port is still somewhat raw (only netbooting is
supported, no access to the SPUs, etc.), but hardware support should be
more fleshed out by the time FreeBSD 9.0 is released. The port uses the
OtherOS mechanism, and so requires a "fat" console with firmware
earlier than 3.21.
Open tasks:
1. SATA driver.
2. Sound support.
3. SPU driver.
__________________________________________________________________
FreeBSD Services Control (fsc)
Contact: Tom Rhodes <trhodes@FreeBSD.org>
FreeBSD Services Control is a mix of binaries which integrate into the
rc.d system and provide for service (daemon) monitoring. It knows about
signals, pidfiles, and uses very little resources.
The fscd utilities will be set up as a port and, hopefully, dropped
into the ports collection in the coming weeks. This will allow easier
testing by everyone and it should make migration into -CURRENT much
easier.
__________________________________________________________________
FreeBSD VirtIO Network Driver
URL: http://www.linux-kvm.org/page/Virtio
URL:
http://lists.FreeBSD.org/pipermail/freebsd-current/2011-January/022036.
html
Contact: Bryan V. <deboomerang@gmail.com>
VirtIO is a device framework offered by KVM/Qemu and Virtualbox to
allow guests to achieve better I/O performance. A beta network driver
was made available earlier this month, and work continues on completing
the block device and refinements the existing network driver.
__________________________________________________________________
FreeBSD/EC2
URL: http://www.daemonology.net/freebsd-on-ec2/
Contact: Colin Percival <cperciva@FreeBSD.org>
FreeBSD is now able to run on t1.micro instances in the Amazon EC2
cloud. FreeBSD 9.0 is not very stable, but it seems likely that FreeBSD
8.2-RELEASE will approach the stability normally expected of FreeBSD.
A list of available FreeBSD AMIs (EC2 machine images) appears on the
FreeBSD/EC2 status page.
Open tasks:
1. Bring FreeBSD to a wider range of EC2 instance types.
2. Completely rework the locking in head/sys/i386/xen/pmap.c to
eliminate races and make 9.0-CURRENT stable under
paravirtualization.
3. Track down several possibly-related problems with scheduling and
timekeeping.
4. Fix other issues shown on the FreeBSD/EC2 status page.
__________________________________________________________________
FreeBSD/sparc64
Contact: Marius Strobl <marius@FreeBSD.org>
CPUTYPE support for sparc64 has been added to CURRENT in r216820. The
three flavors currently supported are "ultrasparc",
"ultrasparc3" and
"v9". So it is now possible to let the compiler produce code
optimize
for the family of UltraSPARC-III CPUs by setting CPUTYPE to
"ultrasparc3". Setting it to "ultrasparc" as well as
omitting it
completely optimizes for UltraSPARC-I/II family CPUs as before. Support
for generating generic 64-bit V9 code was mainly added for reference
purposes. As it turned out, at least for SPARC64-V CPUs running code
optimized for UltraSPARC-III CPUs does not perform measurably better
than UltraSPARC-I/II one though so the default is just fine for these.
This change was merged into 7-STABLE in r217005 and into 8-STABLE in
r217004 respectively, neither 7.4-RELEASE nor 8.2-RELEASE will include
it though.
Support for a certain feature available with UltraSPARC-III+ and
greater, i.e. with all sun4u CPUs following the original
UltraSPARC-III, has been added to CURRENT in r216803. The net effect of
this change is that we now can use a kernel TSB and thus a kernel
address space of virtually any size up to the full 64-bit address space
on machines equipped with these CPUs, apart from the fact that 1GB of
address space still takes up 4MB worth of data structures. Before, the
theoretical limit was 16GB due to the fact that the MMUs of these
UltraSPARC CPUs only have 16 lockable TLB slots (UltraSPARC-I/II have
64 and SPARC64 CPUs again have at least 32), with the actual limit
being several GB below that because we need some of these slots also
for mapping the PROM, the kernel itself and in MP-systems the per-CPU
page. Currently, the kernel TSB and thus the kernel virtual address
space is now always sized one time the physical memory present in these
machines with the plan being to actually allow to it extend beyond the
size of the RAM as this helps especially ZFS. Most of this is
implemented by patching the instructions used to access the kernel TSB
based on the CPU present, so the run-time overhead of this change is
rather low. Once it is also enabled and successfully tested with
SPARC64 CPUs this change will be merged back into the supported stable
branch(es).
Theoretically it should be also possible to use the same approach for
the user TSB, which already is not locked into the TLB but can cause
nested traps. However, for reasons I do not understand yet, OpenSolaris
only does this with SPARC64 CPUs. On the other hand I think that also
using it for the user TSB and thus avoiding nested traps would get us
closer to running the FreeBSD/sparc64 code on machines equipped with
sun4v CPUs, which only supports trap level 0 and 1, too, so eventually
we could have a single kernel which runs on both sun4u and sun4v
machines (as does Linux and OpenBSD).
Work on adding support for Sun Fire 3800 and similar models has begun
but still is in its early stages.
__________________________________________________________________
Generic IEEE 802.3 annex 31B full duplex flow control support for Ethernet in
mii(4)
URL: http://en.wikipedia.org/wiki/Ethernet_flow_control
Contact: Marius Strobl <marius@FreeBSD.org>
In r213878 a NetBSD-compatible mii_attach() was added to mii(4) as an
replacement for mii_phy_probe() and subsequently all Ethernet device
drivers in the tree which use this framework were converted to take
advantage of the former. This allowed to considerably clean up mii(4)
as well as the converted MAC and PHY drivers and get rid of quite a few
hacks, amongst others the infamous "EVIL HACK". However, the main
motivation of this change was to allow the addition of generic IEEE
802.3 annex 31B full duplex flow control support to mii(4), which was
ported from NetBSD but also enhanced and fixed quite a bit and
committed in r215297. Along with this bge(4), bce(4), msk(4), nfe(4)
and stge(4) as well as brgphy(4), e1000phy(4) and ip1000phy(4), which
previously all implemented their own flow control support based on
mostly undocumented special media flags separately, were converted to
take advantage of the generic support. At least for CURRENT this means
that these drivers now no longer unconditionally advertise support for
flow control but only do so if flow control was selected as media
option. The reason for implementing the generic flow control support
that way was to allow it to be switched on and off via ifconfig(8) with
the PHY specific default to typically being off in order to protect
from unwanted effects. Subsequently support for flow control based on
the generic support was added to alc(4), fxp(4), cas(4), gem(4),
jme(4), re(4) and xl(4) as well as atphy(4), bmtphy(4), gentbi(4),
inphy(4), jmphy(4), nsgphy(4), nsphyter(4) and rgephy(4). For several
of the remaining Ethernet drivers it also would only require minor
changes to enable flow control support if supported by the respective
MAC. Due to the fact that each implementation should be thoroughly
tested and tuned this was only done for drivers were hardware was
available though.
An example for identifying support for flow control based on the
generic implementation in the dmesg-output for a certain
MAC-PHY-combination would be:
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
0x002003> mem 0 xfe010000-0xfe01ffff,0xfe000000-0xfe00ffff irq 25 at
device 2.0 on pci2
bge0: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI-X
miibus0: <MII bus> on bge0
brgphy0: <BCM5704 10/100/1000baseTX PHY> PHY 1 on miibus0
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto,
auto-flow
or in the output of ifconfig -m for a given device:
supported media:
media autoselect mediaopt flowcontrol
The latter also is what one would use to enable flow control for such a
device, i.e.:
ifconfig bge0 media autoselect mediaopt flowcontrol
or in order to turn it off again:
ifconfig bge0 media autoselect -mediaopt flowcontrol
Note that some PHY drivers, currently only rgephy(4) though, also
support enabling flow control support when using manual media
configuration like in the following example:
ifconfig re0 media autoselect mediaopt full-fuplex,flowcontrol
In CURRENT this can also be further abbreviated (support for this will
eventually be merged back into the supported stable branch(es) but not
be present in 7.4-RELEASE or 8.2-RELEASE) as:
ifconfig re0 media auto mediaopt fdx,flow
For a device which has successfully negotiated flow control support
with its link partner will report it in the output of ifconfig along
with the available directions like in the following example:
media: Ethernet autoselect <flowcontrol> (100baseTX <full-duplex,
flowcontrol,rxpause,txpause>)
Another thing that was introduced with r215297 was generic support for
setting 1000baseT master mode via a media option when using manual
media configuration. Consequently, brgphy(4), ciphy(4), e1000phy(4) as
well as ip1000phy(4) have been converted to take advantage of this
generic support. At least for CURRENT this means that these drivers now
no longer take the link0 parameter for selecting master mode but the
master media option has to be used instead like in the following
example:
ifconfig bge0 media 1000baseT mediaopt full-duplex,master
Selection of master mode now is also available with all other PHY
drivers supporting 1000baseT.
With the exception of the media option abbreviations all of the above
mentioned changes were merged into 7-STABLE in r215879 and into
8-STABLE in r215881 respectively. This means that they will be part of
7.4-RELEASE and 8.2-RELEASE. In order to no break POLA, unlike as in
CURRENT bge(4), bce(4), msk(4), nfe(4) and stge(4) were changed to
continue to always advertise support of flow control to their link
partners in these stable branches with no way to turn that off as they
also did before with their custom implementations. Additionally,
brgphy(4), ciphy(4), e1000phy(4) as well as ip1000phy(4) were changed
to still also accept the link0 parameter in addition to the master
media option for setting master mode.
Open tasks:
1. We actually miserably fail to properly document the available media
types and options in manual pages. For example several of the media
lists in manual pages of MAC drivers like bge(4) already were
outdated and with the addition of generic flow control and
1000baseT master mode support these are now even more outdated. Yet
worse is the fact that for MAC drivers which use the mii(4)
framework it is technically just plain wrong to include these lists
in their manual page as the PHY drivers actually are responsible
for handling the media types and options. However, given that the
PHY drivers determine the available media types and options mostly
dynamically at run-time it generally makes no sense to have static
documentation of these in their manual pages (apart from the fact
that we currently have no manual pages for PHY drivers). One good
way out of this should be to replace the media lists in MAC drivers
using mii(4) with just a note to check the output of ifconfig -m to
get a list of the media types and options actually supported by a
given device and to add a generic ifmedia(4) manual page which
provides some general background information about media types and
options similar to what NetBSD and OpenBSD also have.
__________________________________________________________________
GEOM-based ataraid(4) Replacement -- geom_raid
URL: http://people.FreeBSD.org/~mav/graid_design.h
URL: http://svn.FreeBSD.org/viewvc/base/projects/graid/
Contact: Alexander Motin <mav@FreeBSD.org>
Contact: M. Warner Losh <imp@FreeBSD.org>
New project started to create GEOM-based replacement for ataraid(4) --
software RAID, that will be obsoleted by migration to the new CAM-based
ATA implementation.
This implementation planned with accent to modular design, that
includes common core and two sets of modules, handling data
transformations (RAID levels) and on-disk metadata formats specifics.
Such design should make further extension easier.
At this moment work focused around RAID0/RAID1 transformations and
Intel metadata format. Module is now able to read, write and create
Intel volumes. Error recovery and rebuild work is now in progress.
Support for other RAID levels and metadata formats, supported by
ataraid(4), planned later.
This project is sponsored by Cisco Systems, Inc.
Open tasks:
1. Complete error recovery/rebuild work and stabilize modules API.
2. Implement metadata modules for other formats.
3. Implement transformation modules for other RAID levels.
__________________________________________________________________
gpart Improvements
Contact: Andrey V. Elsukov <ae@FreeBSD.org>
GEOM class PART is the default disk partitioning class since FreeBSD
8.0. Compared to 8.1 now it does have several new features: Partition
resizing. New "gpart resize" subcommand was implemented for all
partitioning schemes but EBR. GPT recovering. Guid Partition Table does
have redundant metadata and it can be recovered when some of them is
damaged. New "gpart recover" subcommand was implemented for that
purpose. Ability to backup/restore of partition table. New "gpart
backup" and "gpart restore" subcommands were implemented.
__________________________________________________________________
IPv6 and VIMAGE
URL: http://ecdysis.viagenie.ca/
Contact: Bjoern A. Zeeb <bz@FreeBSD.org>
During the last quarter a lot of work was spent on quality time hunting
down and fixing open bugs and races in the network stack, mostly IPv6,
as well as testing and getting virtualized network stack parts more
stable. Tests for the pf(4) firewall update were started with VIMAGE.
In addition Viagenie's NAT64 patch was ported over.
__________________________________________________________________
mdocml Replacing groff For manpage Rendering
URL: http://mdocml.bsd.lv/
URL: https://www.spoerlein.net/cgit/cgit.cgi/freebsd.work/log/?h=mdocml
Contact: Ulrich Sp?rlein <uqs@FreeBSD.org>
Kristaps' groff-replacement (only for rendering manual pages) is
already available in NetBSD and OpenBSD, and used to render the base
system manpages for the latter. This project aims to do similar things
for FreeBSD. Since the last status report, mdocml has grown rudimentary
tbl(1) support and a whole lot of bugfixes have gone in. A groff port
has been created and needs some more testing before it can be committed
to the tree. Also the WITHOUT_GROFF support in base has been fleshed
out and is awaiting review before commit.
Open tasks:
1. Get ru@ to review WITHOUT_GROFF changes.
2. Get textproc/groff tested and committed.
3. Push more mdoc fixes into the tree.
4. Import mandoc(1), switch to catpages for base. Discuss future of
groff in base wrt. share/doc.
5. Supply necessary ports infrastructure to opt-in to mandoc(1).
__________________________________________________________________
Non-executable Stacks
Contact: Konstantin Belousov <kib@FreeBSD.org>
The support for non-executable stacks, using the approach identical to
one used by GNU toolchain and Linux'es, is implemented for amd64 and
PowerPC. The support is already committed to HEAD. For now,
non-executable stacks are turned off by default.
I plan to provide a detailed information to ports@ and switch the knob
after port tree is unfrozed for 7.4/8.2 releases.
__________________________________________________________________
Port-Sandbox
URL:
http://www.arjmobile.com/Marcelo_Araujo/Blog/Entries/2010/11/22_Port-sa
ndbox.html
URL: http://gitorious.org/port-sandbox/port-sandbox/trees/master
URL:
http://www.arjmobile.com/Marcelo_Araujo/My_Albums/Pages/Port-Sandbox.ht
ml
Contact: Marcelo Araujo <araujo@FreeBSD.org>
Port-Sandbox now works properly and it is able to run by itself through
an embedded web server and bring a lot of information about the port
build process and all dependencies related. Currently Port-Sandbox is
in the final stage and needs only only a few code changes, more tests
and should also be included in the ports tree.
Open tasks:
1. Change the way how it connects to database, fix it to maintain a
persistent connection.
2. Remove any kind of internal configuration from source code to an
external file configuration.
3. Create a Port-Sandbox port with all dependencies related to it and
test it in a clean system.
4. Create some documentation to let other people to keep helping
Port-Sandbox to grow up.
5. Finally, release it.
__________________________________________________________________
Portmaster
URL: http://dougbarton.us/portmaster-proposal.html
Contact: Doug Barton <dougb@FreeBSD.org>
Portmaster version 3.6.1 is now in the ports tree, and the emphasis in
the last year has been on improving the stability and performance of
existing features, with a few new features sprinkled in. A lot of work
has gone into error handling, both for unexpected states in the ports
system and for user input. For example, all prompts are now wrapped in
code to verify that what was entered was one of the valid options.
Perhaps the most interesting new element is that for the features -e,
-s, --clean-distfiles, --clean-packages, --check-depends and
--check-port-dbdir you can now specify either -y or -n to automatically
provide the corresponding answer to the yes/no questions. The -o, -r,
and --index-only options have received major overhauls, and now either
work better or at least as advertised.
There has also been a lot of work put into reducing the memory
footprint, especially in the environment variables that are shared
between the parent and child processes. And for those operating without
a local ports tree (--index-only/--packages-only) all of the features
that can work without the ports tree now do.
Significant support for the upgrading of operating without a ports tree
was provided by GridFury, LLC. Their support, as well as the support
received from other members of the community continues to be greatly
appreciated.
Open tasks:
1. There are still interesting features that have been suggested by
users listed on the page above that I have not been able to work
on, but would like to be able to.
__________________________________________________________________
Ports Additions
URL: http://bigbluebutton.org
URL: http://smb4k.berlios.de/
URL: http://www.freeswitch.org/
Contact: Josh Paetzel <jpaetzel@FreeBSD.org>
Bigbluebutton has joined the list of ready to run applications in the
ports tree. Dru Lavigne has been instrumental on getting it to run, as
well as offering suggestions for improvements to the port.
smb4k was updated to the latest release version, which requires kde4.
This was enough of a change that a new port was created,
net/smb4k-kde4. the initial port went through a number of quick
changes, including a patch to the source code to fix a FreeBSD source
code submitted by PC-BSD's Kris Moore. This application greatly eases
the task of working with samba shares in a FreeBSD environment.
Freeswitch is the result of 3 Asterisk developers working on a VoIP
package that fulfills their goals. They have switched away from a
release model to a "just run latest SVN checkout" model. With the
help
of Richard Neese and Eric Crist, static snapshots of their SVN repo
have been taken, the port has been modified to use the newer version,
and extensive build and run testing has been done.
__________________________________________________________________
Ports Collection
URL: http://www.FreeBSD.org/ports/
URL:
http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributing-ports/
URL: http://portsmon.FreeBSD.org/index.html
URL: http://www.FreeBSD.org/portmgr/index.html
URL: http://blogs.FreeBSDish.org/portmgr/
URL: http://www.twitter.com/freebsd_portmgr/
URL: http://www.facebook.com/group.php?gid=135441496471197
URL: http://tinderbox.marcuscom.com/
Contact: Thomas Abthorpe <portmgr-secretary@FreeBSD.org>
Contact: Port Management Team <portmgr@FreeBSD.org>
The ports tree slowly moves up closer to 23,000. The PR count still
remains at about 1000.
In Q4 we added 2 new committers, took in 2 commit bit for safe keeping,
and welcomed back 4 returning committers.
The Ports Management team bid farewell to Kris Kennaway in November
2010. Kris was the root of krismail, the mail we all got from time to
time when ports broke on pointyhat. Kris did a lot of work benchmarking
and testing FreeBSD for stability, scalability and usability.
Mark Linimon has put a lot of effort into refactoring and refining the
code that runs the 'pointyhat' package build dispatch system. In
2010,
the FreeBSD Foundation purchased for portmgr a pair of new machines,
pointyhat-west and pointyhat-east, to take over from the existing
machine. (The new machines have much greater RAM, CPU, and disk
capacity.) However, to properly utilize them, the existing code needed
to be generalized.
Persistent bugs, and some hardware troubles, have delayed the rollout
far beyond what was originally planned, but there appears to be light
at the end of the tunnel. (And, this time, it does not appear to be an
oncoming train.)
A document entitled "Mentoring Guidelines" as been circulated among
ports developers, and has been greeted with a lot of positive feedback,
and updates have been included. In the short term, updated copies will
be maintained at
http://people.FreeBSD.org/~portmgr/mentor_guidelines.txt.asc.
The Ports Management team have been running -exp runs on an ongoing
basis, verifying how base system updates may affect the ports tree, as
well as providing QA runs for major ports updates. Of note, -exp runs
were done for:
* ade: multiple runs for autotools refactoring
* ed: test to replace libgcc.a with libcompiler_rt.a
* jiles: test sh(1) against r212508
* kde: Qt 4.7.0 update
* kde: KDE 4.5.4 updte
* kwm: Gnome 2.32 update
* ports/144164: ensure package-noinstall target include rc.d scripts
* ports/145598: include etc/devd in mtree
* ports/145955: silence make fetch-required-list
* ports/147701: perform DESKTOP_ENTRIES sanity check
* ports/149657: removal of MD5 checksums
* ports/149670: remove checks in _OPTIONSFILE
* ports/150303: for INSTALL_LIBS
* ports/150337: for PLIST_DIRSTRY
* ports/151047: pass CPP to CONFIGURE/MAKE_ENV
* ports/151799: fix PLIST_DIRSTRY
* ports/151806: remove 2004 legacy hack
* ports/152055 and ports/152059: for pear infrastructure
* ports/152558: boost update
* ports/152626: fix pkg-message display if installed from package
* ports/152964: embed LICENSE name for STDOUT
* ports/153018: implement variables in Mozilla dependencies
* ports/153033: fix un-escaped shell metacharacters
* ports/153041: clean up ruby plists
* ports/153132: autotools cleanup
* ports/153318: set PGSQL default to 8.4
Open tasks:
1. Looking for help fixing ports broken on CURRENT.
2. Looking for help with Tier-2 architectures.
3. Most ports PRs are assigned, we now need to focus on testing,
committing and closing.
__________________________________________________________________
Release Engineering Team Status Report
URL: http://www.FreeBSD.org/releng/
Contact: Release Engineering Team <re@FreeBSD.org>
The Release Engineering Team reports the joint release of FreeBSD 7.4
and 8.2 has been delayed slightly but should be completed within a week
or two of the original schedule:
http://www.FreeBSD.org/releases/7.4R/schedule.html
http://www.FreeBSD.org/releases/8.2R/schedule.html
__________________________________________________________________
Resource Containers
Contact: Edward Tomasz Napierala <trasz@FreeBSD.org>
The goal of this project is to implement resource containers and a
per-jail resource limits mechanism, so that system administrators can
partition resources like memory or CPU between jails and prevent users
from DoS-ing the whole system. Project is close to completion. One big
item that needs to be fixed before releasing a patch for people to test
is %CPU accounting; initial idea of just using %CPU calculated by the
scheduler turned out to be useless. Implementing it cleanly will also
make it easier to support other similar resources (e.g.
writes-per-second) in the future.
__________________________________________________________________
Robot Operating System
URL: http://www.ros.org/wiki/
URL: ftp://rene-ladan.nl/pub/ros-freebsd.pdf
Contact: Ren? Ladan <rene@FreeBSD.org>
Porting ROS to FreeBSD started in March 2010. In May 2010, it was
possible to build devel/ros without needing to apply patches, but some
more changes were necessary to be able to write a port for it.
Currently this and several other ports related to ROS are available,
most notably devel/ros-tutorials to get up and running with ROS and
devel/ros-nxt to use LEGO Mindstorms NXT robots with ROS and FreeBSD.
Open tasks:
1. Port the software required for nxt-rviz-plugin, which is part of
devel/ros-nxt but currently excluded from the build.
__________________________________________________________________
SYSCTL Type Safety
Contact: Matthew Fleming <mdf@FreeBSD.org>
I started upstreaming a patch from Isilon that adds type-checking to
the various SYSCTL_FOO and SYSCTL_ADD_FOO macros for various scalar
types, which has turned into quite the discussion on the src mailing
list. The type-checking macros are committed to sys/sysctl.h but under
#if 0.
Open tasks:
1. As of right now, it looks like I will be rolling a new sysctl macro
for the kernel that detects they type at compile time and does the
Right Thing. Existing uses of the legacy SYSCTL_FOO and
SYSCTL_ADD_FOO for scalar types can be replaced, and will probably
turn into invocations of the new interface via preprocessor macro.
__________________________________________________________________
TCP SMP scalability project
Contact: Robert Watson <rwatson@FreeBSD.org>
A long-running TCP SMP scalability project is beginning to wrap up,
with the goal of committing a large outstanding patch to the FreeBSD
9.x tree in the next month. This work implements a derivative of
Willman, Rixner, and Cox's TCP connection group model, blended with
support for hardware load distribution features in contemporary NICs
(including RSS). Additional software distribution support can do work
redistribution based on new notions of CPU affinity for individual TCP
connections.
On-going work is refining performance on non-RSS supporting
configurations, and adding APIs to allow socket affinity to be queried
(and where supported) set by applications. These changes significantly
improve network scalability by reducing global lock contention,
encouraging CPU affinity for connections, and avoiding cache line
contention. The goal is to allow steady-state TCP connections to use
only CPU-local cache lines, with work distributed to all CPUs. Current
performance results are extremely promising.
This project has been sponsored by Juniper Networks.
Open tasks:
1. Allow the hash model to be selected at boot-time or run-time rather
than compile-time; currently "options RSS" enables RSS support
unconditionally -- for systems without RSS NICs, this leads to a
small one-time performance penalty at the creation of each call to
bind() or connect().
2. Add missing socket options to query (and override) default CPU
affinity for connections, which is derived from the active software
or hardware hash model.
3. Teach the network stack and appropriate NIC drivers to propagate
software-overridden connection affinity to hardware using new
device driver ioctls for managing TCAMs and hardware hash tables.
4. Refine software redistribution of work in the event that there are
fewer hardware queues than available CPU threads in which to
process packets; the current prototype is able to do this with
significant performance benefits, but the model requires refining.
5. Experiment with (and measure) software work redistribution at
run-time based on RSS bucket rearrangement. This will require a new
event notification to device drivers so that they can update
hardware caches of the network stack's authoritative table.
6. Commit.
__________________________________________________________________
The FreeBSD Foundation Status Report
URL: http://www.FreeBSDFoundation.org
Contact: Deb Goodkin <deb@FreeBSDFoundation.org>
We raised $325,000 towards our goal of $350,000 for 2010! This will
allow us to increase our project development and equipment spending for
2011.
We were proud to be a sponsor for EuroBSDCon 2010, BSDDay Argentina
2010, MeetBSD California 2010, and NYBSDCon 2010.
Completed the Foundation funded projects: DAHDI Project by Max Khon and
BSNMP Improvements by Shteryana Sotirova.
We kicked off a new project by the University of Melbourne called
Feed-Forward Clock Synchronization Algorithms Project. The Five New TCP
Congestion Control Algorithms for FreeBSD Project by Swinburne
University also officially started.
We continued our work on infrastructure projects to beef up hardware
for package-building, network-testing, etc. This includes purchasing
equipment as well as managing equipment donations.
Stop by and visit with us at FOSDEM (Feb 5-6), SCALE (Feb 26),
AsiaBSDCon (March 17-20), and Indiana Linuxfest (March 26).
Read more about how we supported the project and community by reading
our end-of-year newsletter at:
http://www.FreeBSDFoundation.org/press/2010Dec-newsletter.shtml.
We are fund-raising for 2011 now! Find out more at
http://www.FreeBSDFoundation.org/donate/.
__________________________________________________________________
The FreeBSD German Documentation Project Status Report
URL: http://doc.bsdgroup.de
Contact: Johann Kois <jkois@FreeBSD.org>
Contact: Benedict Reuschling <bcr@FreeBSD.org>
The committers to the German Documentation Project managed to update
the German documentation just in time to get the changes included into
the next FreeBSD releases. The website translations were also kept in
sync with the ones on FreeBSD.org.
We tried to re-activate committers who did not contribute for some time
but most of them are currently unable to free up enough time. We hope
to gain fresh contributor blood as we are getting occasional reports
about bugs and grammar in the german translation.
Open tasks:
1. Submit grammar, spelling or other errors you find in the german
documents and the website.
2. Translate more articles and other open handbook sections.
__________________________________________________________________
The FreeBSD Japanese Documentation Project
URL: http://www.FreeBSD.org/ja/
URL: http://www.jp.FreeBSD.org/doc-jp/
Contact: Hiroki Sato <hrs@FreeBSD.org>
Contact: Ryusuke Suzuki <ryusuke@FreeBSD.org>
Although there is no radical change in this effort since the last
report, the www/ja and doc/ja_JP.eucJP/books/handbook have constantly
been updated. During this period, generating translated RSS feed for
newsflash was started and links to the manual pages were fixed in the
Books and Articles documentation. Some more progress has been made in
the Porter's Handbook and Contributing to FreeBSD as well.
Open tasks:
1. Further translation of the FreeBSD Handbook and contents of the
www.FreeBSD.org website to the Japanese language.
2. Pre-/post-commit review of the translation.
__________________________________________________________________
TRIM support for UFS
Contact: Kirk McKusick <mckusick@FreeBSD.org>
Contact: Konstantin Belousov <kib@FreeBSD.org>
TRIM support for UFS is implemented in HEAD. Potentially, this may
increase the steady speed and longevity of SSDs.
Due to concerns with the speed of TRIM operations on many SSDs, and not
a lot of experience with the real-world behaviour, the support is off
by default, and should be enabled on the per-filesystem basis.
__________________________________________________________________
Webcamd
URL: http://www.selasky.org/hans_petter/video4bsd
URL: http://www.freshports.org/multimedia/webcamd/
Contact: Hans Petter Selasky <hselasky@FreeBSD.org>
Webcamd is a small daemon that enables about 1500 different USB based
webcam, DVB and remote control USB devices under the FreeBSD-8.0 and
later operating system. The webcam daemon is basically an application
which is a port of Video4Linux USB drivers into userspace on FreeBSD.
The daemon currently depends on libc, pthreads, libusb and libcuse4bsd.
During Q3 2010 webcamd got manpages thanks to Dru Lavigne.
Open tasks:
1. I hope to get a Google summer of code project this year building
the default Linux Kernel 2.6.37+ and allowing use of relevant Linux
USB device drivers under FreeBSD. Webcamd is not a replacement for
native FreeBSD kernel drivers and will only be used when no
existing FreeBSD drivers exist for a given device staying clear of
any GPLv2 issues. If you are a student and/or is interested in
participating in such a project feel free to send an e-mail to
hselasky@FreeBSD.org.
__________________________________________________________________
xz Compression for Packages and Log Files
Contact: Martin Matuska <mm@FreeBSD.org>
Creating and processing xz-compressed packages is now supported by
pkg_create(1), pkg_add(1) and bsdtar(1) in both 9-CURRENT and 8-STABLE.
Users can test working with .txz packages by adding "PKG_SUFX=.txz"
into /etc/make.conf.
The ports-mgmt/portupgrade utility supports .txz packages from version
2.4.8 and a patch for ports-mgmt/portmaster has been submitted but not
yet accepted by the author.
A patch for newsyslog(8) with a rewrite of the use of compression tools
supporting xz compression is under maintainer review.
Open tasks:
1. Import xz(1) compression support into newsyslog(8).
2. Add .txz package support to ports-mgmt/portmaster.
3. Add .txz package support to the FreeBSD port building cluster
(pointyhat).
4. Test building all packages in .txz format and compare results with
.tbz.
__________________________________________________________________
ZFS pool version 28
URL:
http://lists.freebsd.org/pipermail/freebsd-fs/2010-December/010292.html
URL:
http://lists.freebsd.org/pipermail/freebsd-fs/2010-December/010321.html
Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>
Contact: Martin Matuska <mm@FreeBSD.org>
A new version of the ZFS pool v28 patch was released for testing, this
time for 9-CURRENT and 8-STABLE. Compared to the previous patch it does
include updated boot support, improved sendfile(2) handling, a
compatibility layer with older ZFS and several other bugfixes.
If there are no major issues we can expect ZFS v28 imported into the
FreeBSD-CURRENT after 8.2 is released.
Open tasks:
1. Import of ZFS v28 into FreeBSD-CURRENT.
__________________________________________________________________
2011 The FreeBSD Project. All rights reserved.