Hi, in ancient times there was cvsup. cvsup was a PITA if you wanted (or needed) to install it via ports, the only reasonable way was to use pkg_add for that if you didn't want to pollute your system with otherwise unneeded software. Then there came csup. Small, in the base. You could install FreeBSD and the first task (for me and my environment) was often to simply csup to -STABLE (or a known good version of that) and to build an up-to-date and customised system. Like tayloring make.conf and src.conf to my needs and leave out most of the stuff I don't need on my system and in the kernel. Software and drivers that aren't there can't fail and won't be a security problem. Times have been changing, we're now up to svn. svn is far more modern than cvs and there are pretty good reasons to use it. However, I either overlook something important or we are now at the point we had with cvsup in the early days: The software I need to (source-)update the system doens't come with the base and installing svn is a PITA. It pulls in a whole lot of dependencies, at the time being in FBSD-9.1-R I cannot even pkg_add -r subversion out of the box. And in the end I have my system polluted with software and libraries I don't really need in many cases for anything else. So, is there some alternative small svn client, that leaves a drastically smaller footprint probably somewhere around, probably even in the ports or is there anything I'm missing? The current situaion for me is a bit annoying. From the user's or admin's point of view at least. I didn't even see an option in svn to not build the server components, which would probably already help to make things smaller? Thanx, Oliver -- | Oliver Brandmueller http://sysadm.in/ ob at sysadm.in | | Ich bin das Internet. Sowahr ich Gott helfe. |
On Wed, 23 Jan 2013 15:40:50 +0100, Oliver Brandmueller <ob at e-gitt.net> wrote:> Hi, > > in ancient times there was cvsup. cvsup was a PITA if you wanted (or > needed) to install it via ports, the only reasonable way was to use > pkg_add for that if you didn't want to pollute your system with > otherwise unneeded software. > > Then there came csup. Small, in the base. You could install FreeBSD and > the first task (for me and my environment) was often to simply csup to > -STABLE (or a known good version of that) and to build an up-to-date and > customised system. Like tayloring make.conf and src.conf to my needs and > leave out most of the stuff I don't need on my system and in the kernel. > Software and drivers that aren't there can't fail and won't be a > security problem. > > Times have been changing, we're now up to svn. svn is far more modern > than cvs and there are pretty good reasons to use it. > > However, I either overlook something important or we are now at the > point we had with cvsup in the early days: The software I need to > (source-)update the system doens't come with the base and installing svn > is a PITA. It pulls in a whole lot of dependencies, at the time being in > FBSD-9.1-R I cannot even pkg_add -r subversion out of the box. And in > the end I have my system polluted with software and libraries I don't > really need in many cases for anything else. > > So, is there some alternative small svn client, that leaves a > drastically smaller footprint probably somewhere around, probably even > in the ports or is there anything I'm missing? The current situaion for > me is a bit annoying. From the user's or admin's point of view at least. > I didn't even see an option in svn to not build the server components, > which would probably already help to make things smaller? > > Thanx, > Oliver >I've read about this initiative. http://svnweb.freebsd.org/base/user/des/svnsup/ Maybe you can help there. Ronald.
Oliver Brandmueller <ob at e-Gitt.NET> writes:> <snip> > > So, is there some alternative small svn client, that leaves a > drastically smaller footprint probably somewhere around, probably even > in the ports or is there anything I'm missing? > <snip>This type of question has been asked quite a few times recently. At this point there is no svn version of csup, however there were people working on it (or at least: there is a svnsup project). For details please search recent ports or questions mailing list archives. As far as I know there is also no alternative svn-client. I'm kind of surprised for the need of this though. Why not simply use portsnap if you are not actively developing ports? -- - Frank
On Wed, Jan 23, 2013 at 10:55:32AM -0500, Mike Tancsa wrote:> On 1/23/2013 10:37 AM, Oliver Brandmueller wrote: > > > > But my main concern is the system sources anyway. freebsd-update is not > > feasible for me, as described in the original post. > > > > > Actually, if you build the port minus the NEON option, its as bad in > terms of dependencies. > > # pkg_info -r subversion-1.7.8.tbz > Information for subversion-1.7.8.tbz: > > Depends on: > Dependency: expat-2.0.1_2 > Dependency: pkg-config-0.25_1 > Dependency: gettext-0.18.1.1 > Dependency: sqlite3-3.7.5 > Dependency: gdbm-1.9.1 > Dependency: db42-4.2.52_5 > Dependency: libiconv-1.13.1_1 > Dependency: apr-ipv6-devrandom-gdbm-db42-1.4.5.1.3.12_1 > > ---Mike===> pkg_info -r /usr/ports/packages6/All/subversion-1.7.8.tbz Information for /usr/ports/packages6/All/subversion-1.7.8.tbz: Depends on: === Static linked, NEON included, run on 6+.
Oliver Brandmueller <ob at e-Gitt.NET> wrote:> Hi, > > in ancient times there was cvsup. cvsup was a PITA if you wanted (or > needed) to install it via ports, the only reasonable way was to use > pkg_add for that if you didn't want to pollute your system with > otherwise unneeded software. > > Then there came csup. Small, in the base. You could install FreeBSD > and the first task (for me and my environment) was often to simply > csup to -STABLE (or a known good version of that) and to build an > up-to-date and customised system. Like tayloring make.conf and > src.conf to my needs and leave out most of the stuff I don't need on > my system and in the kernel. Software and drivers that aren't there > can't fail and won't be a security problem. > > Times have been changing, we're now up to svn. svn is far more modern > than cvs and there are pretty good reasons to use it. > > However, I either overlook something important or we are now at the > point we had with cvsup in the early days: The software I need to > (source-)update the system doens't come with the base and installing > svn is a PITA. It pulls in a whole lot of dependencies, at the time > being in FBSD-9.1-R I cannot even pkg_add -r subversion out of the > box. And in the end I have my system polluted with software and > libraries I don't really need in many cases for anything else. > > So, is there some alternative small svn client, that leaves a > drastically smaller footprint probably somewhere around, probably > even in the ports or is there anything I'm missing? The current > situaion for me is a bit annoying. From the user's or admin's point > of view at least. I didn't even see an option in svn to not build the > server components, which would probably already help to make things > smaller? > > Thanx, > Oliverdevel/subversion already has an option to build a static version. A solution could be to create a stub port (devel/subversion-static) similar to: shells/bash-devel shells/bash-static-devel dns/ldns dns/py-ldns That way the package build cluster would create a package of the static version which wouldn't pull in any runtime dependencies. Emanuel
(Please keep me CC'd as I'm not subscribed to the list)> Don't confuse the excessive ports default settings as dependencies. > You can make a quite mean and lean svn client. I did a 100% > BSD-license-compatible src/contrib/svn style proof-of-concept back > when we were planning what to do. Things like gdbm and bdb are not > required and are license contamination that we don't need. But that's > the fault of the port, not a fundamental property of using svn.While I do understand what you're saying, the Oracle DB and GDBM actually aren't the "the fault of the port", they're a result of what has been eluded to as a build cluster configuration problem or something along those lines. I brought up the dependency list mismatch in subversion.tbz on the package servers (ftp.freebsd.org) in mid November 2012; building from actual source (the port itself) did not pull in any of these dependencies: http://lists.freebsd.org/pipermail/freebsd-ports/2012-November/079589.html lev@ (port maintainer) had this to say: http://lists.freebsd.org/pipermail/freebsd-ports/2012-November/079592.html Quote:>>>JC> However, GDBM and Oracle/Sleepycat DB aren't (by default) enabled >>>JC> in 1.7.7 which is what's in ports currently: >>> >>> They weren't enabled for 1.7.6 too, so it is strange, that >>> pointyhat-builded package require it. I need to investigate this.And that still seems to be the case today, as Mike Tancsa pointed out: http://lists.freebsd.org/pipermail/freebsd-stable/2013-January/071804.html What remains on ftp.freebsd.org is still the same today: $ ftp ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-stable/Latest/ ... ftp> dir subversion* 227 Entering Passive Mode (204,152,184,73,242,194). 150 Here comes the directory listing. lrwxr-xr-x 1 967 100 32 Oct 14 14:53 subversion-java.tbz -> ../All/subversion-java-1.7.6.tbz lrwxr-xr-x 1 967 100 27 Oct 13 13:24 subversion.tbz -> ../All/subversion-1.7.6.tbz lrwxr-xr-x 1 967 100 28 Oct 14 01:54 subversion16.tbz -> ../All/subversion-1.6.18.tbz 226 Directory send OK. I'm left to believe lev@ hasn't had the cycles to investigate this, which is perfectly fine -- however given the importance of SVN at this point in FreeBSD's life, some other committer or whoever is responsible for the build cluster should have stepped up to the plate to figure this out, given how the *entire infrastructure* is now dependent upon this one thing. :-/ I can talk about the remaining dependencies that usually concern people (those are commonly sqlite3, expat, and apr) if required, but for now I'll stay squelched. And just as a footnote point: respectfully do not tell me "this is a great opportunity to try pkgng". Please stay focused on the actual problem. -- | Jeremy Chadwick jdc at koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |
(Please keep me CC'd as I'm not subscribed to the list)> Great idea; > > http://www.bayofrum.net/~crees/patches/svn-static.diff > > Lev, do you mind if I commit this? I haven't touched the subversion > port, but it'll have you as maintainer :) > > If you prefer, I don't mind maintaining this.As I understand it this patch would induce the build cluster to build subversion-static.tbz (eventually) and put it on the package servers. So what happens when one of the underlying dependencies that you've included statically (those would possibly be: Oracle/SleepyCat DB, APR, expat, sqlite3, neon, gettext, and iconv) have security holes or major bugs found/addressed in them? As I understand it -- based on history -- the packages on the FTP servers get updated "whenever". My other post shows some haven't been updated in months (and yes I'm aware of the security incident). So how long would a key piece of software containing insecure statically-linked libraries be on the FTP servers? How would the port maintainer(s) even know the libraries/software which subversion is dependent upon had been updated, thus requiring a new subversion package to be pushed out to the package servers ASAP (i.e. immediately, not days, weeks, or months)? My point: ports have always been "best-effort". They are advertised vehemently throughout "everything FreeBSD" as being third-party software and therefore <infinite list of caveats>. Yet now critical pieces to FreeBSD development (and now end-users too, as a result of using the security incident to push SVN) rely upon something in ports. That's quite a conundrum the Project has created for itself, an ouroboros of sorts. -- | Jeremy Chadwick jdc at koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |
On Wed, 23 Jan 2013 15:40:50 +0100 Oliver Brandmueller <ob at e-Gitt.NET> wrote:> Hi,[skipped]> So, is there some alternative small svn client, that leaves a > drastically smaller footprint probably somewhere around, probably > even in the ports or is there anything I'm missing? The current > situaion for me is a bit annoying. From the user's or admin's point > of view at least. I didn't even see an option in svn to not build the > server components, which would probably already help to make things > smaller? >You may: 1/ install subversion on some host/jail 2/ do svn export ( f.e. svn://svn.freebsd.org/base/releng/9 stable_9) 3/ tar it 4/ on 'client' fetch(1)/scp/rsync tarball in that case you don't need svn on 'client', fetch and scp in base :) -- wbr, tiger
On 2013-Jan-23 15:40:50 +0100, Oliver Brandmueller <ob at e-Gitt.NET> wrote:>in ancient times there was cvsup. cvsup was a PITA if you wanted (or >needed) to install it via ports, the only reasonable way was to use >pkg_add for that if you didn't want to pollute your system with >otherwise unneeded software.There was also ctm(1). ctm is small, BSD-licensed and has been part of FreeBSD forever (almost). Thanks to stephen@, ctm deltas for various src trees, as well as the entire SVN repo are still available. c[v]sup can do things than aren't possible with ctm but I would expect that most people who currently use c[v]sup could readily migrate to using ctm. See http://www.freebsd.org/doc/handbook/ctm.html for details. Note that mirroring the actual SVN repo via ctm requires some patches. There is a README and patches in ftp://ftp.freebsd.org/pub/FreeBSD/CTM/svn-cur/ -- Peter Jeremy -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20130124/85cb960a/attachment.sig>
On 23.01.2013 15:40, Oliver Brandmueller wrote:> However, I either overlook something important or we are now at the > point we had with cvsup in the early days: The software I need to > (source-)update the system doens't come with the base and installing svn > is a PITA. [...]It is not a well publicized fact, but I understand that the base utility freebsd-update(8) through it's freebsd-update.conf(5) is able to pull the base sources (/usr/src/) only instead of also updating your binaries. less /etc/freebsd-update.conf # Components of the base system which should be kept updated. Components src world kernel The above setting is the default, but you may easily leave out everything but "src". (Caveat: I have not tried it myself yet.) It also have some optional settings for preserving local changes to the source instead of blowing them away (default). This will allow you to use the sources for a custom build and install yourself. Also for ports we have the portsnap(8) utility, also in base. So it is perfectly possible to get sources for everything using just the tools in base. No csup or svnup is required. Best regards, Gyrd ^_^
Hi folks, thank you for all the answers and fruitful discussion. Special thanks goes to Chris Rees for coming up with the subversion-static ports quite fast, so now we're hoping for package building to kick in here - but that is a quick and very useful way after setting up a fresh machine (specifically if it's not within the bounds of your own infrastructure)! Also I'd like to mention John Mehr, who's work on a "lightweight, dependency-free, BSD licensed program to pull source using the svn protocol" (couldn't say it better, so I use his words :-)). Hope this will make it into ports soon and in the long run even to base! Thanx again, Oliver -- | Oliver Brandmueller http://sysadm.in/ ob at sysadm.in | | Ich bin das Internet. Sowahr ich Gott helfe. |
On Fri, Jan 25, 2013 at 10:27:23AM +0100, Oliver Brandmueller wrote: ... an Ian Smith and des for working on a svnsup solution, which also might in the future be something that solves the problem! - Oliver -- | Oliver Brandmueller http://sysadm.in/ ob at sysadm.in | | Ich bin das Internet. Sowahr ich Gott helfe. |
On Jan 23, 2013, at 9:40 AM, Oliver Brandmueller wrote:> Hi, > > in ancient times there was cvsup....> Then there came csup....> Times have been changing, we're now up to svn....> alternative small svn clientI'm trying to consolidate notes on this topic, here: https://wiki.freebsd.org/UsersFetchingSource (Aiming to condense some clarity for as many parts of the situation as I can). Please feel to make edits (esp. if you are a committer who knows something I don't!), or email me with corrections and I'll put them in! Best, .ike
On Sun, 27 Jan 2013 16:03:35 -0500 ?"Isaac (.ike) Levy" <ike at blackskyresearch.net> wrote:> On Jan 25, 2013, at 8:27 PM, John Mehr wrote: >> On Fri, 25 Jan 2013 10:27:23 +0100 >>??Oliver Brandmueller <ob at e-Gitt.NET> wrote: >> >>> Also I'd like to mention John Mehr, who's work on a >>>"lightweight, dependency-free, BSD licensed program to >>>pull source using the svn protocol" (couldn't say it >>>better, so I use his words :-)). Hope this will make it >>>into ports soon and in the long run even to base! >> >> Thank you for the kind words.??If all goes well (I'm >>still wearing my "Crown of Naive Optimism") I should have >>something ready for show-and-tell in the next week or so >>and I'll be submitting it as a new port soon after that. > > John, Pullin' for you man! > > If you make progress, please give a shout and I'll note >it in this page: > https://wiki.freebsd.org/UsersFetchingSource > > Best, > .ike > > >Hello, I think I've got all of the protocol issues sorted out as good as they'll probably get.? I can't get the exact file permissions from the get-file requests -- it only lets me know which ones are executable.? Setting executables to 0755 and non-executables to 0644 would work for all files in my copy of /usr/src except for /usr/src/usr.bin/lex/mkskel.sh (0744) and /usr/src/usr.sbin/pkg_install/tkpkg (0754).? Since they're both executable for root, unless there's an issue, we could probably let it slide and assign them 0755 permissions. All I have left at this point is to add support for command line configuration options, add comments, tidy everything up, get it to conform to the "man style" guidelines and test it against a remote repository.
On Sun, Jan 27, 2013 at 09:27:11PM -0600, John Mehr wrote:> All I have left at this point is to add support for > command line configuration options, add comments, tidy > everything up, get it to conform to the "man style" > guidelines and test it against a remote repository. >If you have code available to the public, I am sure many would be happy to test. If you do not have a public repository available, email me off-list, and I will help. Glen -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 488 bytes Desc: not available URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20130127/f8c3edee/attachment.sig>
ups, sorry: https://docs.google.com/file/d/0B9Q-zpUXxqCnRVVMTkk1blVfZzA/edit?usp=sharing Please let me know if you have problems with accessing it. -- Markiyan On 31.03.2013 13:18, Kurt Jaeger wrote:> Hi! > >> I also measured svnup basic process resource usage, attaching a complete >> plot (measurements were taken each 2 seconds based on ps(1) and >> procstat(1)). Hopefully it will help you as well. > > The attachment did not make it to the list. Can you put it up somewhere ? >