OK. Thanks. Where can I find information regarding how to interpret —itemize-changes? The timestamps aren’t changing, so the target must not be storing them, which I have no idea why. The directory I’m writing to is 777. What is the flag to tell rsync to ignore the timestamps? Thanks, Blake On 6/2/16, 6:18 PM, "rsync on behalf of Kevin Korb" <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote:>It is saying the timestamp is wrong and that it is copying the file and >changing the timestamp. If it does that every time then either the >timestamps are changing on the source or the target isn't storing them. > >On 06/02/2016 06:13 PM, McDowell, Blake wrote: >> Thanks Kevin! I¹m unclear how to read the ‹itemize-changes output. Can >>you >> provide some insight? >> >> This is a local transfer from an external drive to an internal drive all >> attached to one computer. >> >> >> rsync -aPh --itemize-changes -n >> /Volumes/shuttle_05/2012_79_1_14_1__1199_Workprint >>/Volumes/3TB_LTO/LT003A/ >> sending incremental file list >>> f..t....... >>> >>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Derivativ >>>es >>> /2012_79_1_14_1_DER_01.mov >>> f..t....... >>> >>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Derivativ >>>es >>> /2012_79_1_14_1_DER_02.mov >>> f..t....... >>> >>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>19 >>> 9WP_00086400.dpx >>> f..t....... >>> >>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>19 >>> 9WP_00086401.dpx >>> f..t....... >>> >>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>19 >>> 9WP_00086402.dpx >>> f..t....... >>> >>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>19 >>> 9WP_00086403.dpx >>> f..t....... >>> >>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>19 >>> 9WP_00086404.dpx >>> f..t....... >>> >>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>19 >>> 9WP_00086405.dpx >>> f..t....... >>> >>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>19 >>> 9WP_00086406.dpx >> >> >> ~Blake >> >> >> >> On 6/2/16, 5:44 PM, "rsync on behalf of Kevin Korb" >> <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote: >> >>> Instead of the second -v (or even the first) add --itemize-changes. It >>> will tell you why each file is being copied. If the file timestamps >>>are >>> not correct then perhaps the underlying storage doesn't allow arbitrary >>> mtime changes. >>> >>> On 06/02/2016 05:23 PM, McDowell, Blake wrote: >>>> Hi, >>>> >>>> >>>> >>>> At my work we use rsync to move files between drives and to LTO among >>>> other things. >>>> >>>> >>>> >>>> I¹m having an issue using rsync to move material between and external >>>> drive and an internal drive. >>>> >>>> >>>> >>>> We run ³rsync avvPh <src> <dest>² and most of the files keep writing >>>> every time I run this. It appears that the modification times are not >>>> being carried through to the destination resulting in the files >>>> continually wanting to re-write. >>>> >>>> >>>> >>>> I¹m hoping someone can help me figure this out. Any information you >>>>need >>>> from me (logs, etc) I¹m happy to try and provide. >>>> >>>> >>>> >>>> Many thanks, >>>> >>>> Blake >>>> >>>> >>>> >>> >>> -- >>> >>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >>> Kevin Korb Phone: (407) 252-6853 >>> Systems Administrator Internet: >>> FutureQuest, Inc. Kevin at FutureQuest.net (work) >>> Orlando, Florida kmk at sanitarium.net (personal) >>> Web page: http://www.sanitarium.net/ >>> PGP public key available on web site. >>> >>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >>> > >-- >~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. >~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >
The man page has a section on what all the itemize-changes flags do. There is a --ignore-times but the result is what you have now, re-copy everything even if the timestamp matches. The best you can really do with storage that can't handle timestamps is to use --update. But if you do that you need to get rid of --partial (part of -P) or else rsync will never complete a file that was partially copied since the partial copy is newer that the source file. OTOH, are you using rsync to copy files to a tape drive? If so then I would think rsync is not the right tool for dealing with linear storage. On 06/02/2016 06:25 PM, McDowell, Blake wrote:> OK. Thanks. Where can I find information regarding how to interpret > —itemize-changes? > > The timestamps aren’t changing, so the target must not be storing them, > which I have no idea why. The directory I’m writing to is 777. > > What is the flag to tell rsync to ignore the timestamps? > > Thanks, > Blake > > On 6/2/16, 6:18 PM, "rsync on behalf of Kevin Korb" > <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote: > >> It is saying the timestamp is wrong and that it is copying the file and >> changing the timestamp. If it does that every time then either the >> timestamps are changing on the source or the target isn't storing them. >> >> On 06/02/2016 06:13 PM, McDowell, Blake wrote: >>> Thanks Kevin! I¹m unclear how to read the ‹itemize-changes output. Can >>> you >>> provide some insight? >>> >>> This is a local transfer from an external drive to an internal drive all >>> attached to one computer. >>> >>> >>> rsync -aPh --itemize-changes -n >>> /Volumes/shuttle_05/2012_79_1_14_1__1199_Workprint >>> /Volumes/3TB_LTO/LT003A/ >>> sending incremental file list >>>> f..t....... >>>> >>>> 2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Derivativ >>>> es >>>> /2012_79_1_14_1_DER_01.mov >>>> f..t....... >>>> >>>> 2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Derivativ >>>> es >>>> /2012_79_1_14_1_DER_02.mov >>>> f..t....... >>>> >>>> 2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>> 19 >>>> 9WP_00086400.dpx >>>> f..t....... >>>> >>>> 2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>> 19 >>>> 9WP_00086401.dpx >>>> f..t....... >>>> >>>> 2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>> 19 >>>> 9WP_00086402.dpx >>>> f..t....... >>>> >>>> 2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>> 19 >>>> 9WP_00086403.dpx >>>> f..t....... >>>> >>>> 2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>> 19 >>>> 9WP_00086404.dpx >>>> f..t....... >>>> >>>> 2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>> 19 >>>> 9WP_00086405.dpx >>>> f..t....... >>>> >>>> 2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX/1 >>>> 19 >>>> 9WP_00086406.dpx >>> >>> >>> ~Blake >>> >>> >>> >>> On 6/2/16, 5:44 PM, "rsync on behalf of Kevin Korb" >>> <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote: >>> >>>> Instead of the second -v (or even the first) add --itemize-changes. It >>>> will tell you why each file is being copied. If the file timestamps >>>> are >>>> not correct then perhaps the underlying storage doesn't allow arbitrary >>>> mtime changes. >>>> >>>> On 06/02/2016 05:23 PM, McDowell, Blake wrote: >>>>> Hi, >>>>> >>>>> >>>>> >>>>> At my work we use rsync to move files between drives and to LTO among >>>>> other things. >>>>> >>>>> >>>>> >>>>> I¹m having an issue using rsync to move material between and external >>>>> drive and an internal drive. >>>>> >>>>> >>>>> >>>>> We run ³rsync avvPh <src> <dest>² and most of the files keep writing >>>>> every time I run this. It appears that the modification times are not >>>>> being carried through to the destination resulting in the files >>>>> continually wanting to re-write. >>>>> >>>>> >>>>> >>>>> I¹m hoping someone can help me figure this out. Any information you >>>>> need >>>>> from me (logs, etc) I¹m happy to try and provide. >>>>> >>>>> >>>>> >>>>> Many thanks, >>>>> >>>>> Blake >>>>> >>>>> >>>>> >>>> >>>> -- >>>> >>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >>>> Kevin Korb Phone: (407) 252-6853 >>>> Systems Administrator Internet: >>>> FutureQuest, Inc. Kevin at FutureQuest.net (work) >>>> Orlando, Florida kmk at sanitarium.net (personal) >>>> Web page: http://www.sanitarium.net/ >>>> PGP public key available on web site. >>>> >>>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >>>> >> >> -- >> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >> Kevin Korb Phone: (407) 252-6853 >> Systems Administrator Internet: >> FutureQuest, Inc. Kevin at FutureQuest.net (work) >> Orlando, Florida kmk at sanitarium.net (personal) >> Web page: http://www.sanitarium.net/ >> PGP public key available on web site. >> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >> >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/rsync/attachments/20160602/02df0fb1/signature.sig>
Cool Thanks! Specifically, the timestamps on both <src> and <dest> match for "ls -l" but do not match for "ls -lu" or "ls -lc” The storage is just an regular HDD in a mac pro tower. I can’t imagine why it wouldn’t handle timestamps. Also of note - this problem doesn’t exist for every file, just the vast majority. So, that just makes it more confusing. Yes, I imagine rsync is not the best for linear tape but give the choice between cp (which is faster and causes less problems but offers almost zero verbosity) and rsync, I’ll choose rsync. If people know of other options, I’d be very happy to know of them. On 6/2/16, 6:33 PM, "Kevin Korb" <kmk at sanitarium.net> wrote:>The man page has a section on what all the itemize-changes flags do. > >There is a --ignore-times but the result is what you have now, re-copy >everything even if the timestamp matches. > >The best you can really do with storage that can't handle timestamps is >to use --update. But if you do that you need to get rid of --partial >(part of -P) or else rsync will never complete a file that was partially >copied since the partial copy is newer that the source file. > >OTOH, are you using rsync to copy files to a tape drive? If so then I >would think rsync is not the right tool for dealing with linear storage. > >On 06/02/2016 06:25 PM, McDowell, Blake wrote: >> OK. Thanks. Where can I find information regarding how to interpret >> —itemize-changes? >> >> The timestamps aren’t changing, so the target must not be storing them, >> which I have no idea why. The directory I’m writing to is 777. >> >> What is the flag to tell rsync to ignore the timestamps? >> >> Thanks, >> Blake >> >> On 6/2/16, 6:18 PM, "rsync on behalf of Kevin Korb" >> <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote: >> >>> It is saying the timestamp is wrong and that it is copying the file and >>> changing the timestamp. If it does that every time then either the >>> timestamps are changing on the source or the target isn't storing them. >>> >>> On 06/02/2016 06:13 PM, McDowell, Blake wrote: >>>> Thanks Kevin! I¹m unclear how to read the ‹itemize-changes output. Can >>>> you >>>> provide some insight? >>>> >>>> This is a local transfer from an external drive to an internal drive >>>>all >>>> attached to one computer. >>>> >>>> >>>> rsync -aPh --itemize-changes -n >>>> /Volumes/shuttle_05/2012_79_1_14_1__1199_Workprint >>>> /Volumes/3TB_LTO/LT003A/ >>>> sending incremental file list >>>>> f..t....... >>>>> >>>>> >>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Derivat >>>>>iv >>>>> es >>>>> /2012_79_1_14_1_DER_01.mov >>>>> f..t....... >>>>> >>>>> >>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Derivat >>>>>iv >>>>> es >>>>> /2012_79_1_14_1_DER_02.mov >>>>> f..t....... >>>>> >>>>> >>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX >>>>>/1 >>>>> 19 >>>>> 9WP_00086400.dpx >>>>> f..t....... >>>>> >>>>> >>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX >>>>>/1 >>>>> 19 >>>>> 9WP_00086401.dpx >>>>> f..t....... >>>>> >>>>> >>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX >>>>>/1 >>>>> 19 >>>>> 9WP_00086402.dpx >>>>> f..t....... >>>>> >>>>> >>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX >>>>>/1 >>>>> 19 >>>>> 9WP_00086403.dpx >>>>> f..t....... >>>>> >>>>> >>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX >>>>>/1 >>>>> 19 >>>>> 9WP_00086404.dpx >>>>> f..t....... >>>>> >>>>> >>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX >>>>>/1 >>>>> 19 >>>>> 9WP_00086405.dpx >>>>> f..t....... >>>>> >>>>> >>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX >>>>>/1 >>>>> 19 >>>>> 9WP_00086406.dpx >>>> >>>> >>>> ~Blake >>>> >>>> >>>> >>>> On 6/2/16, 5:44 PM, "rsync on behalf of Kevin Korb" >>>> <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote: >>>> >>>>> Instead of the second -v (or even the first) add --itemize-changes. >>>>>It >>>>> will tell you why each file is being copied. If the file timestamps >>>>> are >>>>> not correct then perhaps the underlying storage doesn't allow >>>>>arbitrary >>>>> mtime changes. >>>>> >>>>> On 06/02/2016 05:23 PM, McDowell, Blake wrote: >>>>>> Hi, >>>>>> >>>>>> >>>>>> >>>>>> At my work we use rsync to move files between drives and to LTO >>>>>>among >>>>>> other things. >>>>>> >>>>>> >>>>>> >>>>>> I¹m having an issue using rsync to move material between and >>>>>>external >>>>>> drive and an internal drive. >>>>>> >>>>>> >>>>>> >>>>>> We run ³rsync avvPh <src> <dest>² and most of the files keep >>>>>>writing >>>>>> every time I run this. It appears that the modification times are >>>>>>not >>>>>> being carried through to the destination resulting in the files >>>>>> continually wanting to re-write. >>>>>> >>>>>> >>>>>> >>>>>> I¹m hoping someone can help me figure this out. Any information you >>>>>> need >>>>>> from me (logs, etc) I¹m happy to try and provide. >>>>>> >>>>>> >>>>>> >>>>>> Many thanks, >>>>>> >>>>>> Blake >>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> >>>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._ >>>>>., >>>>> Kevin Korb Phone: (407) 252-6853 >>>>> Systems Administrator Internet: >>>>> FutureQuest, Inc. Kevin at FutureQuest.net (work) >>>>> Orlando, Florida kmk at sanitarium.net (personal) >>>>> Web page: http://www.sanitarium.net/ >>>>> PGP public key available on web site. >>>>> >>>>> >>>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._ >>>>>., >>>>> >>> >>> -- >>> >>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >>> Kevin Korb Phone: (407) 252-6853 >>> Systems Administrator Internet: >>> FutureQuest, Inc. Kevin at FutureQuest.net (work) >>> Orlando, Florida kmk at sanitarium.net (personal) >>> Web page: http://www.sanitarium.net/ >>> PGP public key available on web site. >>> >>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >>> >> > >-- >~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. >~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., >