1I need to build up some systems this week, but it appears that neither the US "releng4" server nor the Japanese snapshot server has snapshots of -STABLE with all of the most recently reported glitches fixed. (The Japanese server seems to be indicating, via log messages in its FTP directories, that it has run out of disk space for builds.) I can't get through to daemon.jp.freebsd.org, which normally holds builds of the latest release with security patches, at all. FreeBSD 4.9-RELEASE isn't due out until the end of next month. What's the best way to install a relatively stable build of FreeBSD, with all known security issues fixed, in the interim? I'd very much NOT like to do a CVS update and rebuild the world on these machines, as hard drive space and time available for installation will be limited. --Brett Glass
On Thursday 28 August 2003 08.38, Brett Glass wrote:> 1I need to build up some systems this week, but it appears that neither the > US "releng4" server nor the Japanese snapshot server has snapshots of > -STABLE with all of the most recently reported glitches fixed. (The > Japanese server seems to be indicating, via log messages in its FTP > directories, that it has run out of disk space for builds.) I can't get > through to > daemon.jp.freebsd.org, which normally holds builds of the latest release > with security patches, at all. > > FreeBSD 4.9-RELEASE isn't due out until the end of next month. What's the > best way to install a relatively stable build of FreeBSD, with all known > security issues fixed, in the interim? I'd very much NOT like to do a CVS > update and rebuild the world on these machines, as hard drive space and > time available for installation will be limited. >Why don't you do a cvs update and a `make release' on one of the machines and do a network install on the others? Or do a `make -DMAKE_ISOS release' if you prefer CDs as your install media. I would choose some pre-PAE RELENG_4, as it seems there are still some stability issues and IIRC I saw some posts about the MFSROOT floppy overflowing. (Maybe that's what the snapshot server is complaining about?) cheerz, m. -- Marton Kenyeres - mkenyeres@konvergencia.hu KVG Konvergencia Kft.
Brett, You can always build the src on one machine and NFS mount it over to the rest for the install part, that is assuming that the machines are fairly similar. Brad Davis LiquidNeon http://www.liquidneon.com On Thu, Aug 28, 2003 at 12:38:36AM -0600, Brett Glass wrote:> 1I need to build up some systems this week, but it appears that neither the US > "releng4" server nor the Japanese snapshot server has snapshots of -STABLE > with all of the most recently reported glitches fixed. (The Japanese server > seems to be indicating, via log messages in its FTP directories, that it has > run out of disk space for builds.) I can't get through to > daemon.jp.freebsd.org, which normally holds builds of the latest release with > security patches, at all. > > FreeBSD 4.9-RELEASE isn't due out until the end of next month. What's the best > way to install a relatively stable build of FreeBSD, with all known security > issues fixed, in the interim? I'd very much NOT like to do a CVS update and > rebuild the world on these machines, as hard drive space and time available > for installation will be limited. > > --Brett Glass > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
Brett Glass <brett@lariat.org> probably said:> I can't get through to daemon.jp.freebsd.org, which normally holds > builds of the latest release with security patches, at all.I'm getting through to; ftp://daemon.jp.freebsd.org/pub/FreeBSD/releases/i386/ ok at the moment. Try it again ? P. -- pir pir-sig@pir.net pir-sig@net.tufts.edu
On Wednesday 27 August 2003 23:38, Brett Glass wrote:> 1I need to build up some systems this week, but it appears that > neither the US "releng4" server nor the Japanese snapshot server has > snapshots of -STABLE with all of the most recently reported glitches > fixed. (The Japanese server seems to be indicating, via log messages > in its FTP directories, that it has run out of disk space for > builds.) I can't get through to > daemon.jp.freebsd.org, which normally holds builds of the latest > release with security patches, at all. > > FreeBSD 4.9-RELEASE isn't due out until the end of next month. What's > the best way to install a relatively stable build of FreeBSD, with > all known security issues fixed, in the interim? I'd very much NOT > like to do a CVS update and rebuild the world on these machines, as > hard drive space and time available for installation will be limited.Brett, I think your best bet would be to install 4.8 on one of the machines, CVSup it to the 4.8 security branch, then upgrade all of the machines from that one or build a distribution set on that one to do the installs from. That way you'll get the security fixes without the PAE instability. Best of luck, and let us know what worked out for you. -- "Where am I, and what am I doing in this handbasket?" Wes Peters wes@softweyr.com
At 00:38 28/08/2003 -0600, Brett Glass wrote:>FreeBSD 4.9-RELEASE isn't due out until the end of next month. What's the best >way to install a relatively stable build of FreeBSD, with all known security >issues fixed, in the interim? I'd very much NOT like to do a CVS update and >rebuild the world on these machines, as hard drive space and time available >for installation will be limited.1. Install FreeBSD 4.8-RELEASE 2. pkg_add ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-4-stable/security/freebsd-update-1.3.tgz 3. mv /usr/local/etc/freebsd-update.conf.sample /usr/local/etc/freebsd-update.conf 4. /usr/local/sbin/freebsd-update fetch 5. /usr/local/sbin/freebsd-update install (Ok, dumb question: Why is my pkg_message not part of the package on ftp.freebsd.org? It works when I `make package` locally.) Colin Percival
Will this fix everything that needs to be recompiled to avoid the realpath() bug? I'd assumed that I'd have to get a fresh build to deal with the problem. --Brett At 11:56 AM 8/28/2003, Colin Percival wrote:>At 00:38 28/08/2003 -0600, Brett Glass wrote: >>FreeBSD 4.9-RELEASE isn't due out until the end of next month. What's the best >>way to install a relatively stable build of FreeBSD, with all known security >>issues fixed, in the interim? I'd very much NOT like to do a CVS update and >>rebuild the world on these machines, as hard drive space and time available >>for installation will be limited. > >1. Install FreeBSD 4.8-RELEASE >2. pkg_add ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-4-stable/security/freebsd-update-1.3.tgz >3. mv /usr/local/etc/freebsd-update.conf.sample /usr/local/etc/freebsd-update.conf >4. /usr/local/sbin/freebsd-update fetch >5. /usr/local/sbin/freebsd-update install > > (Ok, dumb question: Why is my pkg_message not part of the package on ftp.freebsd.org? It works when I `make package` locally.) > >Colin Percival > >
At 12:01 28/08/2003 -0600, Brett Glass wrote:>Will this fix everything that needs to be recompiled to avoid the realpath() >bug?Yes, that's the whole point of FreeBSD Update. Read my paper, or come to BSDCon, for details; but rest assured that if you start with a binary install from the official FTP or ISO releases, and don't recompile any of the world locally, FreeBSD Update will update any binaries which are affected by modifications in the security branch. (Provided, of course, that I'm building updates for the release in question -- I'm doing so for 4.7-RELEASE and 4.8-RELEASE.) Because FreeBSD Update identifies the particular binaries affected by a source code change and only distributes those, it also saves bandwidth -- updating a 4.8-RELEASE system using FreeBSD Update uses less than half the bandwidth required to update the source tree with cvsup. Colin Percival
On Thu, Aug 28, 2003 at 11:16:11AM -0700, Colin Percival wrote:> Because FreeBSD Update identifies the particular binaries affected by a > source code change and only distributes those, it also saves bandwidth -- > updating a 4.8-RELEASE system using FreeBSD Update uses less than half the > bandwidth required to update the source tree with cvsup.Does this include kernel binary patches? Or is this possible at all? (kernel patch) Jiawei Ye -- "Without the userland, the kernel is useless." --inspired by The Tao of Programming
At 12:16 PM 8/28/2003, Colin Percival wrote:>At 12:01 28/08/2003 -0600, Brett Glass wrote: >>Will this fix everything that needs to be recompiled to avoid the realpath() >>bug? > > Yes, that's the whole point of FreeBSD Update. Read my paper, or come to BSDCon, for details; but rest assured that if you start with a binary install from the official FTP or ISO releases, and don't recompile any of the world locally, FreeBSD Update will update any binaries which are affected by modifications in the security branch.That's great. What does one do about packages and ports? It appears that the binary packages on the FreeBSD servers are never updated between releases... which means that if a bug is in a package or is compiled into a package (as with the realpath problem), the FreeBSD servers keep sending out exploitable copies of that package indefinitely. The situation with ports is a bit better, but how does one know which ones to recompile and reinstall? Does your update system handle this situation and/or warn about it? --Brett
At 13:54 28/08/2003 -0600, Brett Glass wrote:>What does one do about packages and ports? It appears that the binary >packages on the FreeBSD servers are never updated between releases... >which means that if a bug is in a package or is compiled into a package >(as with the realpath problem), the FreeBSD servers keep sending out >exploitable copies of that package indefinitely. The situation with ports >is a bit better, but how does one know which ones to recompile and >reinstall? Does your update system handle this situation and/or warn about it?FreeBSD Update only concerns itself with the base FreeBSD distribution -- I simply don't have the resources to build any more than that. However, one simple approach to the ports problem would be to # find /usr/local/ -perm +111 -type f -exec file {} \; | grep "statically linked" | cut -f 1 -d ':' and rebuild the applicable ports. Now that I think about it, I might add some sort of functionality like that (providing a listing of ports which need to be rebuilt) into a future version of FreeBSD Update. Colin Percival
At 02:22 PM 8/28/2003, Colin Percival wrote:> FreeBSD Update only concerns itself with the base FreeBSD distribution -- I simply don't have the resources to build any more than that. However, one simple approach to the ports problem would be to > # find /usr/local/ -perm +111 -type f -exec file {} \; | grep "statically linked" | cut -f 1 -d ':' >and rebuild the applicable ports. Now that I think about it, I might add some sort of functionality like that (providing a listing of ports which need to be rebuilt) into a future version of FreeBSD Update.This would be a big help. It would be even better if it could also identify binary packages that need updating (since this, after all, has historically been one of the biggest problems with updating FreeBSD. Of course, the problem with packages is more serious than with ports, because the project has always (for no reason that I can see other than habit) treated binary packages as "second class citizens" compared to ports. Ports can be updated at any time and recompiled. But if there's a bug in a program which has been installed as a package, there's no way for a user to get a freshened package until the next release of the OS! While the project builds binary snapshots of the OS itself nightly, it doesn't rebuild and post packages in between releases. Yes, a user can sometimes uninstall the package and reinstall the same application as a port (after fixing the relevant libraries). But if disk space is tight, or the system is embedded or doesn't include a compiler (embedded or semi-embedded implementations of the BSDs are becoming more and more common), this might not be possible. I'd like to see the project rebuild binary packages regularly so that a user (or a utility such as your updater) can fetch repaired versions of them as needed. It should be easy to tell which ones need rebuilding, so that it's unnecessary to rebuild the entire collection every night. --Brett
As mentioned earlier, I needed to create a few FreeBSD systems, with all of the latest security updates, this week. (The effort extended into the weekend, for reasons that I'll explain below.) Here's a somewhat long-winded (but, hopefully, useful) account of how I did it and some of the pitfalls I encountered. The first thing I did was create a 4.8-RELEASE system using the standard install. I then brought in the "freebsd-update" package to update the system, which should (in theory) have nuked all of the known security holes in the base install. Alas, what I didn't realize at first (though I should have) was that the package was going to try to update "immutable" files. It couldn't do this, of course, because I'd installed with the "maximum" security settings, which set "securelevel" to 2. It failed to update those files, but gave no warning that it had failed; it was a good thing I noticed. So, I changed /etc/rc.conf, rebooted, and ran freebsd-update again. Alas, freebsd-update told me that the system was fully updated and did nothing. I had to use the "rollback" option to undo all previous changes, and then reapply them, to update the system. It then occurred to me: What would one do if the freebsd-update package itself had been linked with a buggy library? (The package collection, as I've already mentioned, isn't updated in response to security problems, and the package was dated July.) My goal was, after all, to get all buggy code off of the machine, and updated binary packages aren't available between releases. So, the only way I could see to avoid this potential problem was to remove the freebsd-update package and rebuild it as a port if I wanted to use it again. The pkg_delete command issued a warning, however, complaining that it couldn't delete the directory /usr/local/freebsd-update. So, I nuked the directory by hand. (Will this cause future problems? I guess I'll see.) Next, it was time to bring in ported software. Because updated binary packages weren't available, I had to install ALL the ported software I wanted to bring in as ports, not packages. A much slower and messier procedure. Before I did this, I decided that it would be a good idea to update all of my ports. This, of course, required cvsup. And this created a Catch-22. You see, because no updated binary package was available, and I had to update my ports to build the latest version of cvsup, I needed to start by using the old one. Which raised the question: Should I use the old (potentially buggy) binary package to update my ports? Or build the older port of cvsup, use it to update all of my ports (including itself), uninstall it, and then rebuild it if I wanted to use it again? I decided that since it's never a good idea to try uninstalling a port that's been updated, since the uninstall can fail due to the update. (This is another problem that ports have which packages don't, by the way.) So, I realized that the best course of action wasto install cvsup as a binary package (running a small risk of problems if it had been linked with a buggy library), use it to update the ports, and then delete the old binary package. Which I did. Next, I decided that since I'd be building everything from ports until the next release, one of the ports I'd better build first was the cvsup utility itself. What a mess! The build brought in tons of other stuff (including Modula-3, gmake, and many other dependencies). Worse still, it included megabytes of GPLed source code, of which we don't want ANY on our systems for legal reasons. (We really don't want GPLed binaries, either, but the lack of availability of BSD-licensed alternatives -- and the fact that FreeBSD is utterly dependent upon some of them -- forces us to live with a few of these for now.) I had to clean up, removing the source after the build. I then had to build everything else I wanted to bring in from a port. Also very messy. In the end, I managed to get an updated system going, but it wasn't graceful. And a novice user (or even one who was more advanced but hadn't played with FreeBSD) certainly wasn't going to be able to do it. And it's a good thing I had lots of disk space. (I really DIDN'T have the time to go through the whole procedure, and so had to give up a chunk of my holiday weekend.) Bottom line: Updates to the binary packages are an absolute necessity between releases. Without them, creating a secure system between releases is a nightmare... and a novice or newcomer just plain couldn't do it. --Brett
At 12:08 31/08/2003 -0600, Brett Glass wrote:>I then brought in the "freebsd-update" package to update the system, which >should (in theory) have nuked all of the known security holes in the base >install. > >Alas, what I didn't realize at first (though I should have) was that the >package was going to try to update "immutable" files. It couldn't do this, >of course, because I'd installed with the "maximum" security settings, >which set "securelevel" to 2. It failed to update those files, but gave no >warning that it had failed; it was a good thing I noticed.If FreeBSD Update fails to "chflags noschg" a file, it should stop with the message "Error installing $FILE". If it didn't, that's a serious bug; except...> So, I changed /etc/rc.conf, rebooted, and ran freebsd-update again. > Alas, freebsd-update told me that the system was fully updated and did nothing.FreeBSD Update is stateless. It keeps old files available for rollback purposes, but it does not "remember" that it has updated a file. If it decided that nothing needed to be updated, it did so after checking that the MD5 hashes of the currently installed files were correct. In short, provided that you haven't rebuilt the world locally, if FreeBSD Update reports "No updates available", your system is definitely up to date.>It then occurred to me: What would one do if the freebsd-update package >itself had been linked with a buggy library?There's only one binary in freebsd-update, and the only library calls it makes are to malloc, free, fprintf, fopen, fread, fwrite, and fclose; anyway, it's dynamically linked.>The pkg_delete command issued a warning, however, complaining that it >couldn't delete the directory /usr/local/freebsd-update. So, I nuked the >directory by hand. (Will this cause future problems? I guess I'll see.)/usr/local/freebsd-update/work/ contains the rollback files. If you don't want them, nuking the directory is fine. (I spent a long time wondering if I should nuke the directory as part of the uninstall script, until I realized that would cause problems for people who were simply upgrading from one version to another.) > [snip discussion of ports and packages] If you want to save time, you could always install all the (potentially out of date) packages and run portupgrade. For dealing with updated libraries, the following code might be helpful: beastie# cat /root/port-rebuild-statics #!/bin/sh find /usr/local -type f -perm +111 ! -newer \ /usr/lib/`ls -art /usr/lib | tail -1` -print0 | \ xargs -0 file | grep "statically linked" | cut -f 1 -d ':' | \ xargs pkg_which | grep -v '^\?$' | sort -u | \ while read x; do portupgrade -fi $x < /dev/tty; done Colin Percival
At 12:37 PM 8/31/2003, Colin Percival wrote:> In short, provided that you haven't rebuilt the world locally, if FreeBSD Update reports "No updates available", your system is definitely up to date.That's good to know, though it didn't solve the other problems I mentioned. Or a couple I just encountered. First, when I built cvsupdate as a port, I found that the commands "make clean" and "make distclean" removed the detritus left behind by creating cvsupdate itself, but did not nuke the junk that was left behind as the system built other ports on which that one depended. Going around and deleting everything manually (there was no automatic mechanism) was a chore. Then came another zinger. One of the people who will be using the system wants KDE on it. (Not my choice, since it's GPLed, but he's the client.) So, after rebuilding cvsupdate as a port, I went to /stand/sysinstall to install KDE. Two problems here. First was that KDE was installed as a binary package... an OUT-OF-DATE binary package built with the buggy libraries. Second, the install failed. The reason appears to be a conflict between ports and packages. As mentioned above, /stand/sysinstall tried to install KDE as a binary package. (Not a bad idea at all in and of itself, but bringing with it the aforementioned security risks.) Worse still, when the package system tried to install some other packages as dependencies for KDE, it hit a few libraries which had been built as ports when I installed cvsup. The installation stopped with an error. In short, we really have a tangled mess here. Under the current way of doing things, you can't remain updated and secure without using ports -- which is bad because of the time, effort, and disk demands inherent in rebuilding them. What's more, if you do use ports, it messes up your ability to use packages -- even out of /stand/sysinstall -- and leaves junk behind on your disk. Again, what a mess. The only way to avoid it, again, is to make binary packages "first class citizens." And also to resolve the conflicts between them and the use of ports. It's amazing that after installing exactly one port, I couldn't install a package from /stand/sysinstall. --Brett
At 1:44 PM -0600 2003/08/31, Brett Glass wrote:>At 12:37 PM 8/31/2003, Colin Percival wrote: > >> In short, provided that you haven't rebuilt the world locally, if >>FreeBSD Update reports "No updates available", your system is >>definitely up to date. > >That's good to know, though it didn't solve the other problems I mentioned. > >Or a couple I just encountered. First, when I built cvsupdate as a port, >I found that the commands "make clean" and "make distclean" removed the >detritus left behind by creating cvsupdate itself, but did not nuke the >junk that was left behind as the system built other ports on which that one >depended. Going around and deleting everything manually (there was no >automatic mechanism) was a chore.Brett, Have you checked out portsclean (part of portupgrade)?> -C > --workclean Clean out all the working directories of the ports tree. > (cf. WRKDIRPREFIX) > > -D > --distclean Clean out all the distfiles that are not referenced by any > port in the ports tree. Specified twice (i.e. -DD), > clean out all the distfiles that are not referenced by any > port that is currently installed. (cf. DISTDIR)Regards, Chris Pepper -- Chris Pepper: <http://www.reppep.com/~pepper/> Rockefeller University: <http://www.rockefeller.edu/>