Announcing CVSup 15.0
---------------------
Release 15.0 of CVSup, the CVS-aware network distribution system, is now
available.
Where to Get CVSup
------------------
CVSup is free software. It is available from the following FTP sites:
ftp://hub.freebsd.org/pub/CVSup/ (California)
ftp://ftp.cs.tu-berlin.de/pub/FreeBSD/CVSup/ (Germany)
ftp://ftp.polstra.com/pub/FreeBSD/CVSup/ (slow; avoid if possible)
Full sources as well as FreeBSD binaries are available:
cvsup-bin-15.0.tar.gz FreeBSD static binaries for the client + GUI
cvsup.nogui-bin-15.0.tar.gz FreeBSD static binaries for the client (no GUI)
cvsupd-bin-15.0.tar.gz FreeBSD static binaries for the server
cvsup-15.0.tar.gz Sources **
MD5 signatures for these files are:
MD5 (cvsup-bin-15.0.tar.gz) = 6456b1ec9d54c588ea02899a3292603c
MD5 (cvsup.nogui-bin-15.0.tar.gz) = 89d2e399d812f1969da8ab3c810fab7b
MD5 (cvsupd-bin-15.0.tar.gz) = 1578a568c762e1975676bf02a38c50cc
MD5 (cvsup-15.0.tar.gz) = 3f9cb6e0267be19d8e0627c85e6edeb4
An updated port will appear in the FreeBSD ports and packages
collections soon:
Port:
ftp://ftp.freebsd.org/pub/FreeBSD/ports-current/net/cvsup/
Package:
ftp://ftp.freebsd.org/pub/FreeBSD/packages-current/net/cvsup-15.0.tgz
ftp://ftp.freebsd.org/pub/FreeBSD/packages-2.2/net/cvsup-15.0.tgz
The FreeBSD package now depends only on the "modula-3-lib" package,
a subset of the Modula-3 installation consisting of only the shared
libraries. Because of this, you can now install and use the "cvsup"
package in a reasonable amount of disk space. The package is much
smaller than the statically linked binary distribution, so updates
to new versions of CVSup should be more convenient now. The package
is the recommended distribution for binary-only users. The static
binary distributions may be phased out soon.
If you want SOCKS support, you must also install the "modula-3-socks"
port or package:
Port:
ftp://ftp.freebsd.org/pub/FreeBSD/ports-current/lang/modula-3-socks/
Package:
ftp://ftp.freebsd.org/pub/FreeBSD/packages-current/lang/modula-3-socks-1.0.tgz
ftp://ftp.freebsd.org/pub/FreeBSD/packages-2.2/lang/modula-3-socks-1.0.tgz
SOCKS is supported only under FreeBSD, and only with dynamically linked
executables. The static binary distributions do not support SOCKS.
** If you wish to build CVSup from the sources, be sure to read the
discussion further on in this announcement.
Compatibility with Previous Releases
------------------------------------
This release is backward-compatible with release 14.0, except for a
couple of minor differences in the client:
* On the client, symbolic links are no longer followed inside a
collection. (If the prefix is a symbolic link, it is followed _to_
the collection.) This might affect some users who mirror the
FreeBSD "gnats" collection. This collection typically uses a prefix
of "home", and all updated files are in the subtree
"gnats". It
is important that "gnats" be a true subdirectory of the prefix,
and not a symbolic link to a directory. In other words, adjust
your prefix if necessary, so that it designates the true parent
directory of your "gnats" tree. (In actuality, the client is more
tolerant than this discussion implies. It recognizes the problematic
situations and does the right thing, emitting only a warning.)
* The default verbosity level when the GUI is not used has been
changed from "-L 0" to "-L 1".
* The old "-d" and "-D" command line options no longer
exist. I
doubt that anybody used them anyway.
* There is a new "-d delLimit" option with which you can specify a
limit on the number of files the client will delete before it
decides something is seriously wrong and quits.
[Note that the reuse of "-d" shouldn't cause undetected
problems,
because the old version accepted no arguments while the new
version requires a numeric argument.]
There are a couple of very minor compatibility issues which could affect
a few users upgrading from a release prior to 14.0.
Clients: The default for the "base" directory has changed from
"/usr"
to "/usr/local/etc/cvsup". Practically everybody specifies the base
explicitly in their supfiles, so this change will have no impact for
most people. If you have been using the default value, you will need to
add a line "*default base=/home" to your supfile, or specify "-b
/home"
on the cvsup command line.
Servers: The "hostbase" is no longer taken from the client's
supfile.
It is now controlled on the server host. On the FreeBSD project,
"hostbase=/home" was always used in the past. People operating
servers
will need to specify "-b /home" on the cvsupd command line to get the
same effect. Alternatively, move your server configuration files from
"/home" to the new default location, "/usr/local/etc/cvsup".
(As
before, most of the configuration files appear under a subdirectory
named "sup".)
What Has Changed Since the Previous Release?
--------------------------------------------
Notable changes in release 15.0:
Added the ability to update only selected files from a collection,
by specifying file name patterns with a new "-i " command line
option on the client. The GUI also has a type-in field where
patterns can be entered, separated by spaces.
Added the ability to update all kinds of files, including symbolic
links, hard links, and device nodes.
Added support for mirroring directories exactly. I.e., empty
directories on the server are now preserved on the client.
Added the ability to preseve all file attributes, including owner,
group, modes, and flags. This is under control of a new "preserve"
keyword in the supfile.
Changed the client's default verbosity level from "-L 0" to
"-L 1".
Added support for the "symlink" and "rsymlink" directives
on the
server, to control which symbolic links are followed and which
are updated as links.
Added support for specifying patterns and/or non-directory files
in the server's "upgrade" and "always" directives.
Added support for multiple patterns on the server's "upgrade",
"always", and "omitany" directives, as well as on the new
"symlink"
and "rsymlink" directives.
Enhanced server's "-c" option to accept a colon-separated list
of directories for searching for collections, so that the
collections no longer all need to be in the same place.
Eliminated the client's "-d" and "-D" command line
options. They
were never very useful, and I had other purposes in mind for those
letters. (See the next item.)
Added a new "-d delLimit" option for specifying the maximum number
of files that may be deleted by the client before it decides that
something is seriously wrong. The "-d" conflicts with older
versions' uses of "-d". But there should be little confusion,
because the new version requires a numeric argument while the
old one accepted no arguments.
Significantly sped up the processing of the client's "checkouts"
file. This eliminates a long delay that had been observed between
updating the last collection and shutting down.
Added support for the non-standard RCS keyword "CVSHeader". It
expands the same as "Header", except that the pathname comes out
as relative to the prefix rather than as absolute. Assuming the
prefix is the root of the CVS repository, this gives relative
pathnames within the repository.
Added support for defining aliases for existing RCS keywords, and
for enabling and disabling the recognition of individual keywords.
This is controlled on a repository-wide basis by directives in a
file "<prefix>/CVSROOT/options" on the server.
Added checking of file sizes in addition to modtimes during the
initial culling phase. This adds some network traffic, but it
also adds a degree of safety. It doesn't seem to slow things
down much for typical updates.
Added the "fnmatch" sources to the distribution, so that CVSup
can be built on platforms that lack a full-featured version of
it.
The static binaries for FreeBSD now include a distribution of the
client built without the GUI. It is much smaller, and may be
preferred by people who never use the GUI.
Notable changes in release 14.1.4 (a beta release):
Added an inactivity timeout to the client, enabled only in non-GUI
mode. If 15 minutes pass without any network I/O, the update is
terminated with a transient error (so that it will be retried).
This is useful when the client is run from cron.
Added a new "-r maxRetries" command line option for the client,
to cause it to give up after the specified number of retries.
This is a generalization of the old "-1" option, which is equivalent
to "-r 0".
Added checking to ensure that the client-side "checkouts*" file
is sorted properly. CVSup always keeps this file in sorted order.
But if it gets corrupted such that it is no longer sorted, mass
deletions can result. This actually happened on a system that
had faulty RAM. A 1-bit error changed a "/" to something else.
Added checksumming and fixups for all kinds of file updates.
Formerly, this was done only for updates that modified (edited)
existing files. Now, even files that are sent verbatim are
checked.
Fixed a bug in searching for the revision in effect on a certain
date. Example: Searching on the trunk. The head is 2.2. The
selected revision is 1.7. Because the first components were
different, the code erroneously considered the file to be dead
on the given date. The fix treats the trunk as a special case.
The bug affected just a few files, and only when checkout mode
was used on the main branch with "date=yy.mm.dd.hh.mm.ss".
Added support for the "execute" keyword, for executing
server-specified
commands on the client when certain files are updated.
Added new client options "-e" and "-E", to enable and
disable
the "execute" feature.
Fixed the server's logging so that it now works to specify "-l
/dev/stdout" on the command line.
Added the necessary "libmd" sources to the distribution, so it
is no longer necessary to get them separately and install them.
Changed the server so that it will let you run it as root again.
What Is CVSup?
--------------
CVSup is a software package for distributing and updating collections
of files across a network. CVSup is specifically tailored to
distributing CVS repositories. By taking advantage of the special
properties of the files contained in CVS repositories, CVSup is
able to perform updates much faster than traditional systems. It is
especially valuable for people with slow Internet connections.
CVSup parses and understands the RCS files making up a CVS repository.
When updates occur, CVSup extracts new deltas directly from the RCS
files on the server and edits them into the client's RCS files.
Likewise, CVSup notes the addition of new symbolic tags to the files
on the server and sends only the new tags to the client.
CVSup is able to merge new deltas and tags from the server with deltas
and tags added locally on the client machine. This makes it possible
for the client to check local modifications into his repository
without their being obliterated by subsequent updates from the server.
Note: Although this feature is fully implemented in CVSup, it will
probably not be practical to use it until some small changes have been
made to CVS.
In addition to distributing the RCS files themselves, CVSup is able to
distribute specific checked-out versions. The client can specify a
symbolic tag, a date, or both and CVSup will extract the appropriate
versions from the server's CVS repository. Checked-out versions do
not need to be stored on the server since CVSup can extract any
version directly from the CVS repository.
If the client has an existing checked-out tree, CVSup will apply the
appropriate edits to update the tree or transform it into the
requested version. Only the differences between the existing version
and the desired version are sent across the network.
To update non-RCS files, CVSup uses the highly efficient rsync
algorithm, developed by Andrew Tridgell and Paul Mackerras.
CVSup uses lightweight processes (threads) to implement a streaming
protocol across the network. This completely eliminates the delays
associated with the lock-step, request-reply form of communication
used by many existing protocols, such as sup and NNTP. Information
is transferred at the full available speed of the network in both
directions at once. Network latency and server response delays
are rendered practically irrelevant.
CVSup uses the "zlib" compression package to optionally compress all
communications. This provides an additional 65-75% compression, on
top of the diff-based compression already built into CVSup.
For efficiency, all processing is built into the CVSup package
itself. Neither the client nor the server executes any other
programs.
For further information about how CVSup works, see the "Blurb"
document
in the CVSup distribution.
Using CVSup to Maintain FreeBSD Sources
---------------------------------------
CVSup servers are currently running at the following FreeBSD mirror
sites:
USA:
cvsup.freebsd.org
cvsup2.freebsd.org
Argentina:
cvsup.ar.freebsd.org
Australia:
cvsup.au.freebsd.org
Germany:
cvsup.de.freebsd.org
Japan:
cvsup.jp.freebsd.org
Netherlands:
cvsup.nl.freebsd.org
Norway:
cvsup.no.freebsd.org
South Africa:
cvsup.za.freebsd.org
Taiwan:
sup.tw.freebsd.org
Using CVSup, you can easily receive or update any of the standard
FreeBSD source releases, namely, "cvs", "current", and
"stable".
The manual page for cvsup(1) describes how to do that. For more detailed
instructions, see the section on CVSup in the FreeBSD Handbook:
http://www.freebsd.org/handbook/cvsup.html
Building CVSup from the Sources
-------------------------------
CVSup is written in Modula-3, a modern, compiled, object-oriented
language. Modula-3 integrates threads, exceptions, and garbage
collection, providing an ideal vehicle for this sort of application.
Without Modula-3, CVSup would almost certainly not exist today.
If you wish to build CVSup from the sources, you will first need
to install the free Modula-3 compiler and runtime libraries from
DEC SRC. A port is available in the FreeBSD ports collection, in
"lang/modula-3". The corresponding package is, of course, available
in the packages collection.
You will also need version 1.0.4 or later of the "zlib" library.
In FreeBSD-2.1.6 and later releases, this library has been incorporated
into the system sources, in "src/lib/libz". Prior to that, a
FreeBSD port was available in "devel/libz" of the FreeBSD ports
collection. For other sources of this library, see the "Install"
file. Do not try to use versions earlier than 1.0.4.
Portability Issues
------------------
I intend for CVSup to be portable to most POSIX systems. The
previous release has been run on a number of different platforms,
including FreeBSD, Linux, and DEC OSF/1 ALPHA.
The current release has only been tested under FreeBSD versions
2.1 and later; however, I attempted not to introduce any new
portability problems.
Anybody who succeeds in porting CVSup to other systems is encouraged
to send his changes to <cvsup-bugs@polstra.com>. As long as the
changes are reasonably palatable, they will be incorporated into
future CVSup releases.
CVSup uses several POSIX-specific functions which may make it more
of an effort to port the package to non-POSIX systems such as Win32.
These functions include mmap, fork, syslog, stat, and chmod, among
others.
Status of this Release
----------------------
CVSup has seen heavy use and has been quite stable for months.
Like all software, though, it is not perfect. Please be prepared
to find bugs -- without a doubt, there are some. Please report
bugs to <cvsup-bugs@polstra.com>.
--
John Polstra jdp@polstra.com
John D. Polstra & Co., Inc. Seattle, Washington USA
"Self-knowledge is always bad news." -- John Barth