Ok, thank you for this extra info. I have experienced exactly what you described. The rsync dry run is _still_ running after being started at 1:30am PST :) But it is finding the right files to update. Most of the entries are:>fc........Which is what I want. So, just because I see:>fat the beginning... That doesn't necessarily mean that the file is going to get updated at the destination? In other words, you're saying that just doing the delta transfer is more time efficient and rsync won't touch the file unless _some_ relevant change is observed? It just concerned me because the file list was extensive, and I definitely don't want anything copied unless for example the checksums don't match. Thanks, -Clint On Wed, Oct 28, 2015 at 5:41 AM, Kevin Korb <kmk at sanitarium.net> wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > - --checksum generally takes a lot longer than --size-only. A delta > transfer generally goes quicker than a checksum. However, if you want > to make a list of what is corrupt a checksumming utility that is less > stupid than rsync can be useful. I say that because rsync's > - --checksum is entirely unintelligent. It will checksum every single > file on both ends including files that only exist on one end and files > that shouldn't match (the size is wrong). At the end of --checksum > you will still have to do the delta xfers that you would do without it. > > OTOH, you are using --dry-run. If you are trying to generate a report > about what files are corrupted then only --checksum an do that. It > will just do it in the dumbest/slowest way possible. > > On 10/28/2015 02:08 AM, Clint Olsen wrote: > > What about -c? It seems I'm getting a lot of spurious file > > transfer candidates when using: > > > > -avvznIi --no-o --no-g --no-p > > > > It's showing transfers (receive) for many files I know haven't > > been tampered with. > > > > Thanks, > > > > -Clint > > > > On Tue, Oct 27, 2015 at 7:53 PM, Kevin Korb <kmk at sanitarium.net > > <mailto:kmk at sanitarium.net>> wrote: > > > > That is correct. Rsync will re-copy (delta xfer unless > > --wholefile) all the files. You can always --dry-run if you aren't > > sure. > > > > On 10/27/2015 10:07 PM, Clint Olsen wrote: > >> Hi: > > > >> I've been using rsync to create backups for a few years. A few > >> months ago I started experiencing sector errors. I ended up > >> replacing the drive and copying the drive data. It turns out > >> that due to the default behavior of rsync "quick check", some of > >> the files were modified without altering the modification time or > >> size, so these files are still clean in the backup. I would like > >> to recover these files, but I need to defeat the quick check in > >> order to do this. It looks like using: > > > >> -I, --ignore-times don't skip files that match size and > >> time > > > >> will work. I just want to confirm that this covers it. The > >> long-form of the switch doesn't mention size, so I was concerned > >> this was the right way to accomplish this. If there are any > >> other switches that would be useful in the context of potentially > >> corrupt files, please feel free to chime in... > > > >> Thanks, > > > >> -Clint > > > > > > > > > > > > -- Please use reply-all for most replies to avoid omitting the > > mailing list. To unsubscribe or change options: > > https://lists.samba.org/mailman/listinfo/rsync Before posting, > > read: http://www.catb.org/~esr/faqs/smart-questions.html > > > > > > > > > > - -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > - -*~ > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > - -*~ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iEYEARECAAYFAlYwwnoACgkQVKC1jlbQAQfHMACfXgY160iVto4Sm5tN7gv+QL39 > 8DsAn3hvPbx3kunv1DIlC9BRXdSVD9fn > =xDso > -----END PGP SIGNATURE----- > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20151028/cb6e6fa5/attachment.html>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 if you see >f it is doing something to the file. At least a delta-xfer. If it was just a metadata change it would show cf. If you see an >fc without a t then that is an example where rsync found a file that didn't match even though the timestamps did. That isn't supposed to happen very often. On 10/28/2015 01:19 PM, Clint Olsen wrote:> Ok, thank you for this extra info. I have experienced exactly what > you described. The rsync dry run is _still_ running after being > started at 1:30am PST :) > > But it is finding the right files to update. Most of the entries > are: > >> fc........ > > Which is what I want. > > So, just because I see: > >> f > > at the beginning... > > That doesn't necessarily mean that the file is going to get updated > at the destination? In other words, you're saying that just doing > the delta transfer is more time efficient and rsync won't touch the > file unless _some_ relevant change is observed? It just concerned > me because the file list was extensive, and I definitely don't want > anything copied unless for example the checksums don't match. > > Thanks, > > -Clint > > > On Wed, Oct 28, 2015 at 5:41 AM, Kevin Korb <kmk at sanitarium.net > <mailto:kmk at sanitarium.net>> wrote: > > --checksum generally takes a lot longer than --size-only. A delta > transfer generally goes quicker than a checksum. However, if you > want to make a list of what is corrupt a checksumming utility that > is less stupid than rsync can be useful. I say that because > rsync's --checksum is entirely unintelligent. It will checksum > every single file on both ends including files that only exist on > one end and files that shouldn't match (the size is wrong). At the > end of --checksum you will still have to do the delta xfers that > you would do without it. > > OTOH, you are using --dry-run. If you are trying to generate a > report about what files are corrupted then only --checksum an do > that. It will just do it in the dumbest/slowest way possible. > > On 10/28/2015 02:08 AM, Clint Olsen wrote: >> What about -c? It seems I'm getting a lot of spurious file >> transfer candidates when using: > >> -avvznIi --no-o --no-g --no-p > >> It's showing transfers (receive) for many files I know haven't >> been tampered with. > >> Thanks, > >> -Clint > >> On Tue, Oct 27, 2015 at 7:53 PM, Kevin Korb <kmk at sanitarium.net >> <mailto:kmk at sanitarium.net> <mailto:kmk at sanitarium.net >> <mailto:kmk at sanitarium.net>>> wrote: > >> That is correct. Rsync will re-copy (delta xfer unless >> --wholefile) all the files. You can always --dry-run if you >> aren't sure. > >> On 10/27/2015 10:07 PM, Clint Olsen wrote: >>> Hi: > >>> I've been using rsync to create backups for a few years. A few >>> months ago I started experiencing sector errors. I ended up >>> replacing the drive and copying the drive data. It turns out >>> that due to the default behavior of rsync "quick check", some >>> of the files were modified without altering the modification >>> time or size, so these files are still clean in the backup. I >>> would like to recover these files, but I need to defeat the >>> quick check in order to do this. It looks like using: > >>> -I, --ignore-times don't skip files that match size >>> and time > >>> will work. I just want to confirm that this covers it. The >>> long-form of the switch doesn't mention size, so I was >>> concerned this was the right way to accomplish this. If there >>> are any other switches that would be useful in the context of >>> potentially corrupt files, please feel free to chime in... > >>> Thanks, > >>> -Clint > > > > > >> -- Please use reply-all for most replies to avoid omitting the >> mailing list. To unsubscribe or change options: >> https://lists.samba.org/mailman/listinfo/rsync Before posting, >> read: http://www.catb.org/~esr/faqs/smart-questions.html > > > > > > > -- Please use reply-all for most replies to avoid omitting the > mailing list. To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync Before posting, > read: http://www.catb.org/~esr/faqs/smart-questions.html > > > >- -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., - -*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., - -*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlYxBG0ACgkQVKC1jlbQAQeUuwCeLWbozz3DbuAXn94/ZnBQaiW5 l94AoNkjkk6QK4Pfjf6qHtOd1ml4xxi8 =lIwU -----END PGP SIGNATURE-----
Hi: I just wanted to thank you for your help. I was able to get a comprehensive list using the checksum switch. To summarize I recovered my data with: -acvzIi --no-o --no-g --no-p -Clint On Wed, Oct 28, 2015 at 10:22 AM, Kevin Korb <kmk at sanitarium.net> wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > if you see >f it is doing something to the file. At least a > delta-xfer. If it was just a metadata change it would show cf. If > you see an >fc without a t then that is an example where rsync found a > file that didn't match even though the timestamps did. That isn't > supposed to happen very often. > > On 10/28/2015 01:19 PM, Clint Olsen wrote: > > Ok, thank you for this extra info. I have experienced exactly what > > you described. The rsync dry run is _still_ running after being > > started at 1:30am PST :) > > > > But it is finding the right files to update. Most of the entries > > are: > > > >> fc........ > > > > Which is what I want. > > > > So, just because I see: > > > >> f > > > > at the beginning... > > > > That doesn't necessarily mean that the file is going to get updated > > at the destination? In other words, you're saying that just doing > > the delta transfer is more time efficient and rsync won't touch the > > file unless _some_ relevant change is observed? It just concerned > > me because the file list was extensive, and I definitely don't want > > anything copied unless for example the checksums don't match. > > > > Thanks, > > > > -Clint > > > > > > On Wed, Oct 28, 2015 at 5:41 AM, Kevin Korb <kmk at sanitarium.net > > <mailto:kmk at sanitarium.net>> wrote: > > > > --checksum generally takes a lot longer than --size-only. A delta > > transfer generally goes quicker than a checksum. However, if you > > want to make a list of what is corrupt a checksumming utility that > > is less stupid than rsync can be useful. I say that because > > rsync's --checksum is entirely unintelligent. It will checksum > > every single file on both ends including files that only exist on > > one end and files that shouldn't match (the size is wrong). At the > > end of --checksum you will still have to do the delta xfers that > > you would do without it. > > > > OTOH, you are using --dry-run. If you are trying to generate a > > report about what files are corrupted then only --checksum an do > > that. It will just do it in the dumbest/slowest way possible. > > > > On 10/28/2015 02:08 AM, Clint Olsen wrote: > >> What about -c? It seems I'm getting a lot of spurious file > >> transfer candidates when using: > > > >> -avvznIi --no-o --no-g --no-p > > > >> It's showing transfers (receive) for many files I know haven't > >> been tampered with. > > > >> Thanks, > > > >> -Clint > > > >> On Tue, Oct 27, 2015 at 7:53 PM, Kevin Korb <kmk at sanitarium.net > >> <mailto:kmk at sanitarium.net> <mailto:kmk at sanitarium.net > >> <mailto:kmk at sanitarium.net>>> wrote: > > > >> That is correct. Rsync will re-copy (delta xfer unless > >> --wholefile) all the files. You can always --dry-run if you > >> aren't sure. > > > >> On 10/27/2015 10:07 PM, Clint Olsen wrote: > >>> Hi: > > > >>> I've been using rsync to create backups for a few years. A few > >>> months ago I started experiencing sector errors. I ended up > >>> replacing the drive and copying the drive data. It turns out > >>> that due to the default behavior of rsync "quick check", some > >>> of the files were modified without altering the modification > >>> time or size, so these files are still clean in the backup. I > >>> would like to recover these files, but I need to defeat the > >>> quick check in order to do this. It looks like using: > > > >>> -I, --ignore-times don't skip files that match size > >>> and time > > > >>> will work. I just want to confirm that this covers it. The > >>> long-form of the switch doesn't mention size, so I was > >>> concerned this was the right way to accomplish this. If there > >>> are any other switches that would be useful in the context of > >>> potentially corrupt files, please feel free to chime in... > > > >>> Thanks, > > > >>> -Clint > > > > > > > > > > > >> -- Please use reply-all for most replies to avoid omitting the > >> mailing list. To unsubscribe or change options: > >> https://lists.samba.org/mailman/listinfo/rsync Before posting, > >> read: http://www.catb.org/~esr/faqs/smart-questions.html > > > > > > > > > > > > > > -- Please use reply-all for most replies to avoid omitting the > > mailing list. To unsubscribe or change options: > > https://lists.samba.org/mailman/listinfo/rsync Before posting, > > read: http://www.catb.org/~esr/faqs/smart-questions.html > > > > > > > > > > - -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > - -*~ > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > - -*~ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iEYEARECAAYFAlYxBG0ACgkQVKC1jlbQAQeUuwCeLWbozz3DbuAXn94/ZnBQaiW5 > l94AoNkjkk6QK4Pfjf6qHtOd1ml4xxi8 > =lIwU > -----END PGP SIGNATURE----- > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20151031/f86844d5/attachment.html>