Hi, On Fri, Mar 25, 2016 at 11:16:47AM -0400, Kevin Korb wrote:> If you were using --link-dest to make multiple backups you wouldn't > need --delete because the target is always a new empty directory (with > - --link-dest pointing to the previous backup run).The source is around 200G and the target box only has 500G total and some of it is used for other data. What I want to do is mirror the source on the target and be able to prune the files that get deleted from the source from the target also. I don't have enough space to back up the whole thing and that is very time consuming anyway over 100M/b link which is why I was using --delete. For a long time it was ok, but now I don't have enough RAM. There is one giant directory that is probably problematic because it has a huge number of files. I suspect this is the one that's causing me problems but it is relatively static. I suppose it could be backed up and cleaned up separately. Is there any way to reduce RAM consumption on the target box while still getting the benefit of the --delete function? I am thinking of trying to back up everything but the gigantic directory with a large number of files, and then backing up only that directory. Is this a reasonable strategy? I just couldn't understand if --delete with --exclude would delete files from the target outside the --exclude path. I guess the answer is no but it would be a very time consuming mistake. I'm trying to make sure before I try it. Thanks for your help and I'm sorry for my poorly worded post(s). /jl -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary / \ http://www.mutt.org attachments / \ Code Blue or Go Home! Encrypted email preferred PGP Key 2048R/DA65BC04
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 You miss-understand the purpose of --link-dest. Yes, it gives you multiple complete backups, but each only consumes the disk space needed to store files that are unique to that backup. Files that are the same in 2 backup runs are actually the same file in multiple directories requiring only 1 to actually be stored. On 03/27/2016 02:39 AM, John Long wrote:> Hi, > > On Fri, Mar 25, 2016 at 11:16:47AM -0400, Kevin Korb wrote: > >> If you were using --link-dest to make multiple backups you >> wouldn't need --delete because the target is always a new empty >> directory (with - --link-dest pointing to the previous backup >> run). > > The source is around 200G and the target box only has 500G total > and some of it is used for other data. What I want to do is mirror > the source on the target and be able to prune the files that get > deleted from the source from the target also. I don't have enough > space to back up the whole thing and that is very time consuming > anyway over 100M/b link which is why I was using --delete. For a > long time it was ok, but now I don't have enough RAM. > > There is one giant directory that is probably problematic because > it has a huge number of files. I suspect this is the one that's > causing me problems but it is relatively static. I suppose it could > be backed up and cleaned up separately. > > Is there any way to reduce RAM consumption on the target box while > still getting the benefit of the --delete function? I am thinking > of trying to back up everything but the gigantic directory with a > large number of files, and then backing up only that directory. Is > this a reasonable strategy? I just couldn't understand if --delete > with --exclude would delete files from the target outside the > --exclude path. I guess the answer is no but it would be a very > time consuming mistake. I'm trying to make sure before I try it. > > Thanks for your help and I'm sorry for my poorly worded post(s). > > /jl >- -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlb3gl4ACgkQVKC1jlbQAQdZ7gCfXzhoaTSU2eW8fFUZQgA/8pjN JQIAn3fQ6ESTB1gm1LT2AsO9kK4MV4ow =Y4bU -----END PGP SIGNATURE-----
Thanks I'll look this up. There is still the issue of how to get the target box cleaned up since I can no longer run --delete. /jl On Sun, Mar 27, 2016 at 02:49:02AM -0400, Kevin Korb wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > You miss-understand the purpose of --link-dest. Yes, it gives you > multiple complete backups, but each only consumes the disk space > needed to store files that are unique to that backup. Files that are > the same in 2 backup runs are actually the same file in multiple > directories requiring only 1 to actually be stored. > > On 03/27/2016 02:39 AM, John Long wrote: > > Hi, > > > > On Fri, Mar 25, 2016 at 11:16:47AM -0400, Kevin Korb wrote: > > > >> If you were using --link-dest to make multiple backups you > >> wouldn't need --delete because the target is always a new empty > >> directory (with - --link-dest pointing to the previous backup > >> run). > > > > The source is around 200G and the target box only has 500G total > > and some of it is used for other data. What I want to do is mirror > > the source on the target and be able to prune the files that get > > deleted from the source from the target also. I don't have enough > > space to back up the whole thing and that is very time consuming > > anyway over 100M/b link which is why I was using --delete. For a > > long time it was ok, but now I don't have enough RAM. > > > > There is one giant directory that is probably problematic because > > it has a huge number of files. I suspect this is the one that's > > causing me problems but it is relatively static. I suppose it could > > be backed up and cleaned up separately. > > > > Is there any way to reduce RAM consumption on the target box while > > still getting the benefit of the --delete function? I am thinking > > of trying to back up everything but the gigantic directory with a > > large number of files, and then backing up only that directory. Is > > this a reasonable strategy? I just couldn't understand if --delete > > with --exclude would delete files from the target outside the > > --exclude path. I guess the answer is no but it would be a very > > time consuming mistake. I'm trying to make sure before I try it. > > > > Thanks for your help and I'm sorry for my poorly worded post(s). > > > > /jl > > > > - -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iEYEARECAAYFAlb3gl4ACgkQVKC1jlbQAQdZ7gCfXzhoaTSU2eW8fFUZQgA/8pjN > JQIAn3fQ6ESTB1gm1LT2AsO9kK4MV4ow > =Y4bU > -----END PGP SIGNATURE----- > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html