Richard Michael
2013-Dec-17 23:19 UTC
--itemize-changes not recursive (not printing created files inside created directories)
Hello list,
Using rsync version 3.1.0pre1 protocol version 31.PR14, I ran this
command to sync two local filesystems:
rsync -rvvctpgonh --itemize-changes --progress --stats
--log-file=/root/rsync.log /mnt/ntfs/vmdk/ /mnt/ntfs/restore/
-r = recursive
-vv = verbose
-c = checksum compare
-t = transfer modification times
-p = set destination perms to match source
-g = set destination group to match source
-o = set destination owner to match source
-n = dry run (should cause --progress to be meaningless, but harmless)
-h = human readable output
Output includes this summary:
2013/12/11 16:35:14 [3349] Number of files: 443,967 (reg: 419,427, dir: 24,540)
2013/12/11 16:35:14 [3349] Number of created files: 2,581 (reg: 2,511, dir: 70)
2013/12/11 16:35:14 [3349] Number of deleted files: 0
2013/12/11 16:35:14 [3349] Number of regular files transferred: 2,744
2013/12/11 16:35:14 [3349] Total file size: 560.66G bytes
2013/12/11 16:35:14 [3349] Total transferred file size: 196.26G bytes
2013/12/11 16:35:14 [3349] Literal data: 0 bytes
2013/12/11 16:35:14 [3349] Matched data: 0 bytes
2013/12/11 16:35:14 [3349] File list size: 589.75K
2013/12/11 16:35:14 [3349] File list generation time: 0.047 seconds
2013/12/11 16:35:14 [3349] File list transfer time: 0.000 seconds
2013/12/11 16:35:14 [3349] Total bytes sent: 20.97M
2013/12/11 16:35:14 [3349] Total bytes received: 1.36M
2013/12/11 16:35:14 [3349] sent 20.97M bytes received 1.36M bytes
2.28K bytes/sec
2013/12/11 16:35:14 [3349] total size is 560.66G speedup is 25,111.56 (DRY RUN)
--itemize-changes has output 70 lines pertaining to created
directories (matches the summary output above), similar to this:
2013/12/11 13:54:23 [3349] cd+++++++++
$RECYCLE.BIN/S-1-5-21-947408474-2404634453-201028329-3641/
$ awk '$4 ~ /^cd/' rsync.log | wc -l # ==> 70
However, the log file contains 0 occurrences of '>', '<'
or 'cf++++++++' .
A bit of arithmetic indicates exactly the "Number of regular files
transferred" ( = 2,744 from stats output) are missing from the log:
$ wc -l rsync.log # ==> 441279 ( - 56 lines of stats/info + 2,744
unreported files = 443967 files/dirs, as summarized)
Questions:
1/ Why doesn't --itemize-changes report "cf+++++++++" lines for
files
contained inside created directories ("cd+++++++")? For example, I
expected output similar to: (faked output)
2013/12/11 13:54:23 [3349] cd+++++++++
$RECYCLE.BIN/S-1-5-21-947408474-2404634453-201028329-3641/
2013/12/11 13:54:23 [3349] cf+++++++++
$RECYCLE.BIN/S-1-5-21-947408474-2404634453-201028329-3641/desktop.ini
2/ Of the 2,744 transferred files, 2,581 are created (I expect each,
dir or file, as a "c[fd]+++++++") ; this leaves 163 transferred files,
presumably because they were changed (but not created), also not
reported. Why were they not reported?
Ultimately, I would like a list of all differences, both file and
directory, sent to the client. I suspect --itemize-changes is a bit
archaic, and I should be investigating --info= customized output.
Thank you!
Regards,
Richard
Kevin Korb
2013-Dec-17 23:49 UTC
--itemize-changes not recursive (not printing created files inside created directories)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 You are running with -n (--dry-run) It isn't logging any transfers because it isn't actually doing any. Your command should show the dry run details on stdout but only what it actually transferred (nothing) in the log file. If you drop the -n it will actually make the changes and log them. Note also that you shouldn't use -c unless you have a really good (and unusual) reason. On 12/17/2013 06:19 PM, Richard Michael wrote:> Hello list, > > Using rsync version 3.1.0pre1 protocol version 31.PR14, I ran > this command to sync two local filesystems: > > rsync -rvvctpgonh --itemize-changes --progress --stats > --log-file=/root/rsync.log /mnt/ntfs/vmdk/ /mnt/ntfs/restore/ > > -r = recursive -vv = verbose -c = checksum compare -t = transfer > modification times -p = set destination perms to match source -g > set destination group to match source -o = set destination owner to > match source -n = dry run (should cause --progress to be > meaningless, but harmless) -h = human readable output > > > Output includes this summary: > > 2013/12/11 16:35:14 [3349] Number of files: 443,967 (reg: 419,427, > dir: 24,540) 2013/12/11 16:35:14 [3349] Number of created files: > 2,581 (reg: 2,511, dir: 70) 2013/12/11 16:35:14 [3349] Number of > deleted files: 0 2013/12/11 16:35:14 [3349] Number of regular files > transferred: 2,744 2013/12/11 16:35:14 [3349] Total file size: > 560.66G bytes 2013/12/11 16:35:14 [3349] Total transferred file > size: 196.26G bytes 2013/12/11 16:35:14 [3349] Literal data: 0 > bytes 2013/12/11 16:35:14 [3349] Matched data: 0 bytes 2013/12/11 > 16:35:14 [3349] File list size: 589.75K 2013/12/11 16:35:14 [3349] > File list generation time: 0.047 seconds 2013/12/11 16:35:14 [3349] > File list transfer time: 0.000 seconds 2013/12/11 16:35:14 [3349] > Total bytes sent: 20.97M 2013/12/11 16:35:14 [3349] Total bytes > received: 1.36M 2013/12/11 16:35:14 [3349] sent 20.97M bytes > received 1.36M bytes 2.28K bytes/sec 2013/12/11 16:35:14 [3349] > total size is 560.66G speedup is 25,111.56 (DRY RUN) > > > --itemize-changes has output 70 lines pertaining to created > directories (matches the summary output above), similar to this: > > 2013/12/11 13:54:23 [3349] cd+++++++++ > $RECYCLE.BIN/S-1-5-21-947408474-2404634453-201028329-3641/ > > $ awk '$4 ~ /^cd/' rsync.log | wc -l # ==> 70 > > However, the log file contains 0 occurrences of '>', '<' or > 'cf++++++++' . > > > A bit of arithmetic indicates exactly the "Number of regular files > transferred" ( = 2,744 from stats output) are missing from the > log: > > $ wc -l rsync.log # ==> 441279 ( - 56 lines of stats/info + 2,744 > unreported files = 443967 files/dirs, as summarized) > > > Questions: > > 1/ Why doesn't --itemize-changes report "cf+++++++++" lines for > files contained inside created directories ("cd+++++++")? For > example, I expected output similar to: (faked output) > > 2013/12/11 13:54:23 [3349] cd+++++++++ > $RECYCLE.BIN/S-1-5-21-947408474-2404634453-201028329-3641/ > 2013/12/11 13:54:23 [3349] cf+++++++++ > $RECYCLE.BIN/S-1-5-21-947408474-2404634453-201028329-3641/desktop.ini > > > > 2/ Of the 2,744 transferred files, 2,581 are created (I expect > each, dir or file, as a "c[fd]+++++++") ; this leaves 163 > transferred files, presumably because they were changed (but not > created), also not reported. Why were they not reported? > > > Ultimately, I would like a list of all differences, both file and > directory, sent to the client. I suspect --itemize-changes is a > bit archaic, and I should be investigating --info= customized > output. > > Thank you! > > Regards, Richard >- -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlKw4w0ACgkQVKC1jlbQAQejxgCg+UNQKWWDRHaEwkJK8inIqZYI h8EAn1lejHiC2F3okYDYoHllb+rpzF+v =k7QX -----END PGP SIGNATURE-----