I am hoping someone else is already doing what I need to do, and can give
me some pointers.
Situation:
1. I am running Centos 4.3 x86_64 on a machine at home, without broadband
access. I have dialup access, but that doesn't work very well for
something like "yum update", so I am still running 4.3 as issued on
the CDs.
2. I have high speed access at work, and I have a USB drive to move files
from work to home.
Proposed solution (other than get broadband):
A. Make a "pretend repo" on the hard drive of home machine by:
1. Install yum "index" file for each repo in directories at home.
Use
rsync via dialup to keep those repo "indexes" up to date (after
initial install). [A sample rsync command line would be helpful.]
2. Persuade yum to give me a list of packages (that I need to download
in order to execute a package install or update) and capture that
list in a format that I can feed into rsync (at work).
3. Have a way to separate package list into three categories:
a. Packages already in correct version my local repository.
b. Packages needing an update in my local repo.
c. Packages I need to download wholesale.
4. Have a way to copy packages needing updates to USB drive.
5. Take list of packages to work, and use rsync to transfer/update the
packages onto the USB drive.
6. Bring USB drive home and dump contents into local repo.
7. Run yum to do the updates.
I think this is how to do things in general terms, but I could use a good
bit of help in coming up with some scripts to automate/semiautomate the
process. I think #3 may be the hardest one to automate.
Anyone doing this? Anyone good with rsync and yum, and care to give me a
starting point, even if your try is untested? Console approach is fine
(GUI OK too), but I am new to yum (moving from Mandriva), and have used
rsync a few times successfully, but am no master at it.
Also, I have 4 install disks. Which repos are on those disks, and which
repos are on which ones? Do any of them include CentOS Plus?
Ted Miller
CentOS 4.3 x86_64
On Sun, 2006-08-06 at 08:30 -0400, Ted Miller wrote:> I am hoping someone else is already doing what I need to do, and can give > me some pointers. > > Situation: > 1. I am running Centos 4.3 x86_64 on a machine at home, without broadband > access. I have dial-up access, but that doesn't work very well for > something like "yum update", so I am still running 4.3 as issued on the CDs. > 2. I have high speed access at work, and I have a USB drive to move files > from work to home.If it's decent sized, just put your masters of everything right there. Then mount appropriate parts (remember that we can mount subdirs too these days, IIRC) over /var yum and rpm dirs or *better*, update config files in /etc to point to the mountable drives dirs. Makes backups to the HDs. If the USB is not large enough, use a CD (CD-RW or DVD-RW would be great). These can be used in standard form (ISO-9660 or whatever, as well as ext2, udf or as raw archives like tar or cpio) to best meet your needs. In the U.S., CD-RW is cheap these days, after rebates or even in big discount stores. Internal especially.> <snip>I haven't had to do what you are looking at, but Dag (rpmforge) has mentioned "yam" in connection with doing things with local repos if this is what you are facing. I don't *know* that it will help, but I thought you should be made aware, if not already on to that.> Proposed solution (other than get broadband): > A. Make a "pretend repo" on the hard drive of home machine by: > 1. Install yum "index" file for each repo in directories at home. Use > rsync via dialup to keep those repo "indexes" up to date (after > initial install). [A sample rsync command line would be helpful.]Again, I haven't done this, but I would think the "index" master should be on the portable media, with backup(s) on the HDs. That let's you know exact status regardless of connect location.> 2. Persuade yum to give me a list of packages (that I need to download > in order to execute a package install or update) and capture that > list in a format that I can feed into rsync (at work).Yum has commands that help determine dependencies, as does rpm. You'll need to do a little (very little?) massaging (bash, perl, sed, (g)awk or some such) to get the format you want out of it.> 3. Have a way to separate package list into three categories: > a. Packages already in correct version my local repository. > b. Packages needing an update in my local repo. > c. Packages I need to download wholesale.I think if your master "index" is on the portable media, b. and c. are side effects of running yum. BTW, a good (but slightly dangerous) way to get this is to run yum update (or install) and then say NO when it asks to confirm install. This NO is very important! Anyway, the screen can be copied and pasted into a file to give you the info that can be edited into a form you like. There are also "deplist", "provides" and "whatprovides" options (yum) and "provides", "requires", "whatprovides" and "whatrequires" (rpm) that might be helpful. With rpm, the "queryformat" option can reduce your editing. Placed in the .rpmmacro file in $HOME, something like this: $ cat .rpmmacros %_query_all_fmt %%{name}-%%{version}-%%{release}.%%{arch} $> 4. Have a way to copy packages needing updates to USB drive.Should be no problem in Linux?> 5. Take list of packages to work, and use rsync to transfer/update the > packages onto the USB drive.Consider the "yam" and my above thoughts to see if maybe the need is adequately addressed. Again, I've not done it, so I don't know. I just know what I would start looking at to design my solution.> 6. Bring USB drive home and dump contents into local repo.As backup, right? If size of portable media permits, it can be master and mounted as needed.> 7. Run yum to do the updates. > I think this is how to do things in general terms, but I could use a good > bit of help in coming up with some scripts to automate/semiautomate the > process. I think #3 may be the hardest one to automate.Unless someone has time and common interest, you may need to develop the functional spec for the script, do some rudimentary coding (essentially, what did you type at the console to do the job once or twice) and then ask for help in refining and automating it. My guess is that will get you more offers of assistance.> > Anyone doing this? Anyone good with rsync and yum, and care to give me a > starting point, even if your try is untested? Console approach is fine > (GUI OK too), but I am new to yum (moving from Mandriva), and have used > rsync a few times successfully, but am no master at it.Just knowing how this list thinks (not me of course, I am such a nice guy ;-) I think a good starting point (*if* you haven't done it yet) would be to hit the yum and rpm docs that the CentOS site can point you to. Also, check out the Yam" stuff in case Dag has a bunch of the heavy lifting already accomplished. Get more familiar with rsync, just in case you have to stroke a little harder on your own than you really would like.> > Also, I have 4 install disks. Which repos are on those disks, and which > repos are on which ones? Do any of them include CentOS Plus?I recall threads that mention that the installs can be used for a lot, but I have no details. A useful command might be find <cd mount point> -type d Should find some directories with interesting names there.> > Ted Miller > CentOS 4.3 x86_64 > <snip sig stuff>Sorry I could not be more specific help. But I figured on a Sunday, you might not get a lot of replies to get you started, so thought I would offer my thoughts. HTH -- Bill -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://lists.centos.org/pipermail/centos/attachments/20060806/c99d4060/attachment-0002.sig>
> > 1. I am running Centos 4.3 x86_64 on a machine at home, without broadband > > access. I have dial-up access, but that doesn't work very well for > > something like "yum update", so I am still running 4.3 as issued on the CDs. > > 2. I have high speed access at work, and I have a USB drive to move files > > from work to home.I'd suggest running yum update answering no, then getting the packages that it lists at work, and copying them into the appriopriate directories on your home machine (/var/cache/yum/[repo]/package if I'm not mistaken). You could probably write a quick sed script to parse the yum output and change it into urls to get at work. Once you've copied the packages you just rerun yum update, this time with a 'y' (and hope there were no extra updates in the mean-time). This if by no means perfect, but it'll get 'most' of the data from your work machines (you'll still be pulling the index and headers files at home). If you're more desperate: than get a big removable USB/whatever (the current update set for centos 4.3 i386 or x86_64 is approximately 640MB). Stick it in to your work computer, rsync the centos update directory onto it [I'm including the script I use for syncing centos, but it syncs _a lot_ more than just the updates - you'd probably want to remove the entire dag-specific part, and add more filters to limit yourself to updates only (or even just change the base path)]. Once that's done (the first time will take looong), unmount it, take it home, configure yum at home to use the mountpoint from your USB as the update repo (/etc/yum.repos.d/CentOS-Base.repo or something like that) and run yum update - voila, the repo's local. Cheers, MaZe. -------------- next part -------------- #!/bin/bash #KBytes/s KBPS="10240" syncit() { #a=rlptgoD #rsync -vv --stats --bwlimit "$KBPS" -aHSzP --numeric-ids \ #--delete --delete-after --delete-excluded ... "$SRC" "$DST" rsync -vv --stats --bwlimit "$KBPS" -rlptDHSzP \ --delete --delete-after --delete-excluded "$@" "$SRC" "$DST" chown maze.maze -Rh "$DST" chcon user_u:object_r:httpd_sys_content_t -Rh "$DST" # chcon user_u:object_r:httpd_user_content_t -Rh "$DST" } #rsync://sunsite.icm.edu.pl/pub/Linux/distributions/centos/ #rsync://ftp.pbone.net/pub/centos/ #rsync://rsync.sunsite.org.uk/sites/msync.centos.org/CentOS/ # rsync.sunsite.org.uk::sites/msync.centos.org/CentOS/ #rsync://mirrors.kernel.org/centos/ #SRC="rsync://rsync.sunsite.org.uk/sites/msync.centos.org/CentOS/" #SRC="rsync://mirrors.kernel.org/centos/" #SRC="rsync://mirrors1.kernel.org/centos/" SRC="rsync://mirrors2.kernel.org/centos/" DST="/opt/mirrors/centos" syncit \ --exclude /[23] \ --exclude /[23].[0-9] \ --exclude /RPM-GPG-KEY-CentOS-[23] \ --exclude /4.[012] \ --exclude /[234].[0-9]beta \ --exclude /*/isos/ \ --exclude /4.3/isos-dvd/SRPMS/CentOS-4.3-i386-srcDVD.iso \ --exclude /4.3/isos-dvd/i386/CentOS-4.3-i386-binDVD.iso \ --exclude /4.3/isos-dvd/x86_64/CentOS-4.3-x86_64-binDVD.iso \ --exclude /*/*/alpha/ \ --exclude /*/*/ia64/ \ --exclude /*/*/ppc/ \ --exclude /*/*/s390/ \ --exclude /*/*/s390x/ \ #--exclude /*/*/x86_64/ \ SRC="rsync://dag.freshrpms.net/pub/freshrpms/pub/dag/" DST="/opt/mirrors/dag" syncit \ --exclude /bert/ \ --exclude /dries/aurora/ \ --exclude /dries/fedora/ \ --exclude /dries/redhat/9/ \ --exclude /dries/redhat/el3/ \ --exclude /dries/packages/*/au1.9[12]-sparc/ \ --exclude /dries/packages/*/fc[12345]-i386/ \ --exclude /dries/packages/*/fc[12345]-x86_64/ \ --exclude /dries/packages/*/el[3]-i386/ \ --exclude /dries/packages/*/el[3]-x86_64/ \ --exclude /fedora/ \ --exclude /packages/*/*-*-*.rh[56789].dag.noarch.rpm \ --exclude /packages/*/*-*-*.rh[56789].dag.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rh[56789].dag.x86_64.rpm \ --exclude /packages/*/*-*-*.rh[56789][0123].dag.noarch.rpm \ --exclude /packages/*/*-*-*.rh[56789][0123].dag.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rh[56789][0123].dag.x86_64.rpm \ --exclude /packages/*/*-*-*.dag.rh[56789][0123].noarch.rpm \ --exclude /packages/*/*-*-*.dag.rh[56789][0123].i[3456]86.rpm \ --exclude /packages/*/*-*-*.dag.rh[56789][0123].x86_64.rpm \ --exclude /packages/*/*-*-*.fc[12345].dag.noarch.rpm \ --exclude /packages/*/*-*-*.fc[12345].dag.i[3456]86.rpm \ --exclude /packages/*/*-*-*.fc[12345].dag.x86_64.rpm \ --exclude /packages/*/*-*-*.rhfc[12345].dag.noarch.rpm \ --exclude /packages/*/*-*-*.rhfc[12345].dag.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rhfc[12345].dag.x86_64.rpm \ --exclude /packages/*/*-*-*.el[23].dag.noarch.rpm \ --exclude /packages/*/*-*-*.el[23].dag.i[3456]86.rpm \ --exclude /packages/*/*-*-*.el[23].dag.x86_64.rpm \ --exclude /packages/*/*-*-*.rhel[23].dag.noarch.rpm \ --exclude /packages/*/*-*-*.rhel[23].dag.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rhel[23].dag.x86_64.rpm \ --exclude /packages/*/*-*-*.rhel[23].[0123456789].dag.noarch.rpm \ --exclude /packages/*/*-*-*.rhel[23].[0123456789].dag.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rhel[23].[0123456789].dag.x86_64.rpm \ --exclude /packages/*/*-*-*.rh[56789].rf.noarch.rpm \ --exclude /packages/*/*-*-*.rh[56789].rf.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rh[56789].rf.x86_64.rpm \ --exclude /packages/*/*-*-*.rh[56789][0123].rf.noarch.rpm \ --exclude /packages/*/*-*-*.rh[56789][0123].rf.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rh[56789][0123].rf.x86_64.rpm \ --exclude /packages/*/*-*-*.rf.rh[56789][0123].noarch.rpm \ --exclude /packages/*/*-*-*.rf.rh[56789][0123].i[3456]86.rpm \ --exclude /packages/*/*-*-*.rf.rh[56789][0123].x86_64.rpm \ --exclude /packages/*/*-*-*.fc[12345].rf.noarch.rpm \ --exclude /packages/*/*-*-*.fc[12345].rf.i[3456]86.rpm \ --exclude /packages/*/*-*-*.fc[12345].rf.x86_64.rpm \ --exclude /packages/*/*-*-*.rhfc[12345].rf.noarch.rpm \ --exclude /packages/*/*-*-*.rhfc[12345].rf.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rhfc[12345].rf.x86_64.rpm \ --exclude /packages/*/*-*-*.el[23].rf.noarch.rpm \ --exclude /packages/*/*-*-*.el[23].rf.i[3456]86.rpm \ --exclude /packages/*/*-*-*.el[23].rf.x86_64.rpm \ --exclude /packages/*/*-*-*.rhel[23].rf.noarch.rpm \ --exclude /packages/*/*-*-*.rhel[23].rf.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rhel[23].rf.x86_64.rpm \ --exclude /packages/*/*-*-*.rhel[23].[0123456789].rf.noarch.rpm \ --exclude /packages/*/*-*-*.rhel[23].[0123456789].rf.i[3456]86.rpm \ --exclude /packages/*/*-*-*.rhel[23].[0123456789].rf.x86_64.rpm \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789].dag.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789].dag.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789].dag.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789][0123].dag.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789][0123].dag.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789][0123].dag.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.dag.rh[56789][0123].noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.dag.rh[56789][0123].i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.dag.rh[56789][0123].x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.fc[12345].dag.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.fc[12345].dag.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.fc[12345].dag.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhfc[12345].dag.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhfc[12345].dag.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhfc[12345].dag.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.el[23].dag.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.el[23].dag.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.el[23].dag.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].dag.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].dag.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].dag.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].[0123456789].dag.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].[0123456789].dag.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].[0123456789].dag.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789].rf.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789].rf.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789].rf.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789][0123].rf.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789][0123].rf.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rh[56789][0123].rf.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.dag.rh[56789][0123].noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.dag.rh[56789][0123].i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.dag.rh[56789][0123].x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.fc[12345].rf.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.fc[12345].rf.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.fc[12345].rf.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhfc[12345].rf.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhfc[12345].rf.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhfc[12345].rf.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.el[23].rf.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.el[23].rf.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.el[23].rf.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].rf.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].rf.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].rf.x86_64.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].[0123456789].rf.noarch.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].[0123456789].rf.i[3456]86.log.gz \ --exclude /packages/*/_buildlogs/*-*-*.rhel[23].[0123456789].rf.x86_64.log.gz \ --exclude /redhat/[45678].[0123]/ \ --exclude /redhat/9/ \ --exclude /redhat/el2.1/ \ --exclude /redhat/el3/ \ #--exclude /redhat/el4/en/x86_64/ \ ln -s i386 dries/redhat/el4/en/x86_64 #end #--exclude /packages/ \ #--exclude /rpmforge/ \ #--exclude /source/ \ exit #/packages/package-name/package-name-version-subversion.dag/rf.src.rpm #/packages/package-name/package-name-version-subversion.build.os.dag/rf.arch.rpm #/packages/package-name/package-name.spec #/packages/package-name/_buildlogs/package-name-version-subversion.build.os.dag/rf.arch.log.gz
I made some progress, so am answering myself, and maybe someone else can help me along.> I am hoping someone else is already doing what I need to do, and can give > me some pointers. > > Situation: > 1. I am running Centos 4.3 x86_64 on a machine at home, without broadband > access. I have dialup access, but that doesn't work very well for > something like "yum update", so I am still running 4.3 as issued on the CDs. > 2. I have high speed access at work, and I have a USB drive to move files > from work to home. > > Proposed solution (other than get broadband): > A. Make a "pretend repo" on the hard drive of home machine by: > 1. Install yum "index" file for each repo in directories at home. Use > rsync via dialup to keep those repo "indexes" up to date (after > initial install). [A sample rsync command line would be helpful.]I got rsync working. So far I have imported all the files in the "repodata" directory, but yum only seems to use primary.xml.gz and repomd.xml. Do I need to have the other files (filelists.xml.gz, other.xml.gz)? They are much bigger than the two that are getting used, so if I don't need them, it will speed up the updating process a lot.> 2. Persuade yum to give me a list of packages (that I need to download > in order to execute a package install or update) and capture that > list in a format that I can feed into rsync (at work).I don't know if I succeeded or failed at this one. I issue a "yum resolvedep xorg-x11" command, all I get back is: Searching Packages for Dependency: Setting up repositories Reading repository metadata in from local files 0:xorg-x11-6.8.2-1.EL.13.25.1.x86_64 I am fairly certain that I have to install the libraries when upgrading a big package like X11, so the response is a bit baffling to me. when I running "yum deplist xorg-x11" I get pages and pages of dependencies without any indication of which packages I already have on my computer, and which I don't. I ran "yum deplist firefox", which I just installed, and yum gave me the pages of dependencies, but the packages are already there. What I need is a list of all the packages I need to download to install that package, so I can go to work, download them all, bring them home, transfer them to the computer, and turn yum loose to do the install.> 3. Have a way to separate package list into three categories: > a. Packages already in correct version my local repository. > b. Packages needing an update in my local repo. > c. Packages I need to download wholesale. > 4. Have a way to copy list of packages needing updates to USB drive. > 5. Take list of packages to work, and use rsync to transfer/update the > packages onto the USB drive.I only wanted two packages today (Firefox 1.5 and Thunderbird 1.5), so I just did the download "by hand".> 6. Bring USB drive home and dump contents into local repo.Did it "by hand"> 7. Run yum to do the updates.Ran just fine, but these packages had no unresolved dependencies.> I think this is how to do things in general terms, but I could use a good > bit of help in coming up with some scripts to automate/semiautomate the > process. I think #3 may be the hardest one to automate. > > Anyone doing this? Anyone good with rsync and yum, and care to give me a > starting point, even if your try is untested? Console approach is fine > (GUI OK too), but I am new to yum (moving from Mandriva), and have used > rsync a few times successfully, but am no master at it. > > Also, I have 4 install disks. Which repos are on those disks, and which > repos are on which ones?I came to the conclusion that my 4 disks constitute the "base" install. Am I right? > Ted Miller> CentOS 4.3 x86_64 > > _______________________________________________ > CentOS mailing list > CentOS at centos.org > http://lists.centos.org/mailman/listinfo/centos >