rsync - Jun 2015 - Question on folder sync with "directory name translation"

OK , but then the solution with symlinks is equivalent, just with the right
options for rsync.

Make the link.
Sync + exclude.
Remove the link.

Don't have to live with the folder on the source.



*_______________Gionata Boccalini*

2015-06-08 22:49 GMT+02:00 Michael Johnson - MJ <mj at revmj.com>:
> Oh, actually, I just thought of a couple other another options that
don't
> require any multiplexing or ssh keys, but it would require that your source
> machine is linux.
>
> The first option would be:
>
> mkdir /A/FolderB
> mount --bind /A/FolderA /A/FolderB
>
> Then just exclude /A/FolderA from the rsync and you are done.  This does
> mean that you have to be ok with /A/FolderB existing on the source.
>
>
> The second option would be to use somthing like aufs or overlayfs to
> create a new mountpoint that contains everything you want and perhaps with
> a little bit of mount --bind thrown in.
>
>
> I just saw your response, and and what you describe makes sense.  Sounds
> like the mount bind option + exclude might be the most elegant option for
> this case.  Just make sure to add the bind mount into your fstab so it
> comes back after a reboot.  :)
>
> On Mon, Jun 8, 2015 at 1:18 PM, Michael Johnson - MJ <mj at
revmj.com> wrote:
>
>> Thought I would chime in here.  To the best of my knowledge what you
are
>> trying to do cannot be done in a single run.  I supposed --fuzzy might
work
>> for you but I've never used that option and it sounds scary to me. 
Perhaps
>> if I spent some time and learned the methodology it uses I would be
less
>> concerned.  If a feature like this were to be added, it seems like it
would
>> make sense to add it as a new "filter" type.
>>
>> But mainly you've piqued my curiosity.  The requirement that it be
able
>> to happen in a single rsync run seems very odd.  Is this just a desire,
or
>> is there really something that bad that would happen if you did one
pass
>> syncing A to B excluding FolderA and FolderB and then a second pass
syncing
>> FolderA to FolderB?
>>
>> The most likely scenario I imagine is that you are running this by hand
>> and manually enter the SSH password.  Given the process take a long
time,
>> you don't want to enter the password again mid stream.  If this is
the
>> case, you could set up SSH keys to allow this to happen without a
manually
>> typed password.  You can find how to set that up here:
>>
>>
http://www.chainsawonatireswing.com/2012/01/15/ssh-into-your-synology-diskstation-with-ssh-keys/
>>
>> If you don't want to do ssh keys w/o a password, you could use ssh
agent
>> with keys.
>>
>> Finally you could also utilize ssh multiplexing (it looks like that
>> should work with the synology nass).  You can find information about
this
>> option here:
>> http://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing
>>
>> The other (very unlikely) scenario I can imagine is that there is
>> something that prevents you from logging in more than once every X
>> hours/days like a time lock safe.  multiplexing would help here as
well.
>> But this scenario seems unlikely, it was probably silly to even mention
it.
>>
>> If there is another case I have not considered I would be very
interested
>> to know where this requirement comes from.
>>
>> Thanks!  Hope there was something useful for you in all this.  :)
>>
>> On Sun, Jun 7, 2015 at 9:38 AM, Gionata Boccalini <
>> gionata.boccalini at gmail.com> wrote:
>>
>>> Hello everyone,
>>>
>>> I'm new to this mailing list but I have been using rsync for
some years
>>> up to now.
>>> I'm trying to synchronize two directory trees, but I want a
special
>>> behavior that I didn't find on the net nor in the manual (or
maybe there is
>>> a combination of options to get what I want but I couldn't find
it).
>>>
>>> Tree A is like:
>>>
>>> A
>>>   *
>>>   *
>>>    FolderA
>>>   *
>>>   *
>>>
>>> And tree B (on a remote filesystem) is like:
>>>
>>> B
>>>   *
>>>   *
>>>   *
>>>    FolderB
>>>   *
>>>   *
>>>
>>>
>>> I have to synchronize everything in one rsync run, like
>>> rysnc  -arv  A/  B/
>>>
>>> but I want FolderA to be synchronized with FolderB.
>>> They must contain the same files but have a different name! I want
>>> something like a "directory name translation" in the
rsync run...
>>> Is it possible? Do you see any another way of doing this? (A part
of
>>> using two rsync runs....)
>>> Please let me know if I didn't explain the problem correctly or
you need
>>> further information.
>>> Thank you for your attention and time.
>>> Best regards.
>>>
>>>
>>> *_______________Gionata Boccalini*
>>>
>>> --
>>> Please use reply-all for most replies to avoid omitting the mailing
list.
>>> To unsubscribe or change options:
>>> https://lists.samba.org/mailman/listinfo/rsync
>>> Before posting, read:
http://www.catb.org/~esr/faqs/smart-questions.html
>>>
>>
>>
>>
>> --
>> Michael Johnson - MJ
>>
>
>
>
> --
> Michael Johnson - MJ
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20150609/49bbc483/attachment.html>

Should be as long as you don't have other symlinks in the tree.

On Mon, Jun 8, 2015, 15:14 Gionata Boccalini <gionata.boccalini at
gmail.com>
wrote:
> OK , but then the solution with symlinks is equivalent, just with the
> right options for rsync.
>
> Make the link.
> Sync + exclude.
> Remove the link.
>
> Don't have to live with the folder on the source.
>
> *_______________*
>
>
> *Gionata Boccalini*
>
> 2015-06-08 22:49 GMT+02:00 Michael Johnson - MJ <mj at revmj.com>:
>
>> Oh, actually, I just thought of a couple other another options that
don't
>> require any multiplexing or ssh keys, but it would require that your
source
>> machine is linux.
>>
>> The first option would be:
>>
>> mkdir /A/FolderB
>> mount --bind /A/FolderA /A/FolderB
>>
>> Then just exclude /A/FolderA from the rsync and you are done.  This
does
>> mean that you have to be ok with /A/FolderB existing on the source.
>>
>>
>> The second option would be to use somthing like aufs or overlayfs to
>> create a new mountpoint that contains everything you want and perhaps
with
>> a little bit of mount --bind thrown in.
>>
>>
>> I just saw your response, and and what you describe makes sense. 
Sounds
>> like the mount bind option + exclude might be the most elegant option
for
>> this case.  Just make sure to add the bind mount into your fstab so it
>> comes back after a reboot.  :)
>>
>> On Mon, Jun 8, 2015 at 1:18 PM, Michael Johnson - MJ <mj at
revmj.com>
>> wrote:
>>
>>> Thought I would chime in here.  To the best of my knowledge what
you are
>>> trying to do cannot be done in a single run.  I supposed --fuzzy
might work
>>> for you but I've never used that option and it sounds scary to
me.  Perhaps
>>> if I spent some time and learned the methodology it uses I would be
less
>>> concerned.  If a feature like this were to be added, it seems like
it would
>>> make sense to add it as a new "filter" type.
>>>
>>> But mainly you've piqued my curiosity.  The requirement that it
be able
>>> to happen in a single rsync run seems very odd.  Is this just a
desire, or
>>> is there really something that bad that would happen if you did one
pass
>>> syncing A to B excluding FolderA and FolderB and then a second pass
syncing
>>> FolderA to FolderB?
>>>
>>> The most likely scenario I imagine is that you are running this by
hand
>>> and manually enter the SSH password.  Given the process take a long
time,
>>> you don't want to enter the password again mid stream.  If this
is the
>>> case, you could set up SSH keys to allow this to happen without a
manually
>>> typed password.  You can find how to set that up here:
>>>
>>>
http://www.chainsawonatireswing.com/2012/01/15/ssh-into-your-synology-diskstation-with-ssh-keys/
>>>
>>> If you don't want to do ssh keys w/o a password, you could use
ssh agent
>>> with keys.
>>>
>>> Finally you could also utilize ssh multiplexing (it looks like that
>>> should work with the synology nass).  You can find information
about this
>>> option here:
>>> http://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing
>>>
>>> The other (very unlikely) scenario I can imagine is that there is
>>> something that prevents you from logging in more than once every X
>>> hours/days like a time lock safe.  multiplexing would help here as
well.
>>> But this scenario seems unlikely, it was probably silly to even
mention it.
>>>
>>> If there is another case I have not considered I would be very
>>> interested to know where this requirement comes from.
>>>
>>> Thanks!  Hope there was something useful for you in all this.  :)
>>>
>>> On Sun, Jun 7, 2015 at 9:38 AM, Gionata Boccalini <
>>> gionata.boccalini at gmail.com> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I'm new to this mailing list but I have been using rsync
for some years
>>>> up to now.
>>>> I'm trying to synchronize two directory trees, but I want a
special
>>>> behavior that I didn't find on the net nor in the manual
(or maybe there is
>>>> a combination of options to get what I want but I couldn't
find it).
>>>>
>>>> Tree A is like:
>>>>
>>>> A
>>>>   *
>>>>   *
>>>>    FolderA
>>>>   *
>>>>   *
>>>>
>>>> And tree B (on a remote filesystem) is like:
>>>>
>>>> B
>>>>   *
>>>>   *
>>>>   *
>>>>    FolderB
>>>>   *
>>>>   *
>>>>
>>>>
>>>> I have to synchronize everything in one rsync run, like
>>>> rysnc  -arv  A/  B/
>>>>
>>>> but I want FolderA to be synchronized with FolderB.
>>>> They must contain the same files but have a different name! I
want
>>>> something like a "directory name translation" in the
rsync run...
>>>> Is it possible? Do you see any another way of doing this? (A
part of
>>>> using two rsync runs....)
>>>> Please let me know if I didn't explain the problem
correctly or you
>>>> need further information.
>>>> Thank you for your attention and time.
>>>> Best regards.
>>>>
>>>>
>>>> *_______________Gionata Boccalini*
>>>>
>>>> --
>>>> Please use reply-all for most replies to avoid omitting the
mailing
>>>> list.
>>>> To unsubscribe or change options:
>>>> https://lists.samba.org/mailman/listinfo/rsync
>>>> Before posting, read:
>>>> http://www.catb.org/~esr/faqs/smart-questions.html
>>>>
>>>
>>>
>>>
>>> --
>>> Michael Johnson - MJ
>>>
>>
>>
>>
>> --
>> Michael Johnson - MJ
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20150609/eba9ac70/attachment.html>

Yeah, I don't have other symlink.
But I'm thinking of changing my folder structure to reflect the data I
really need on the NAS. So, as a side effect, the special rsync is not
needed any more :)
Anyway, thanks for the answers!
Bye



*_______________Gionata Boccalini*

2015-06-09 13:25 GMT+02:00 Michael Johnson - MJ <mj at revmj.com>:
> Should be as long as you don't have other symlinks in the tree.
>
> On Mon, Jun 8, 2015, 15:14 Gionata Boccalini <gionata.boccalini at
gmail.com>
> wrote:
>
>> OK , but then the solution with symlinks is equivalent, just with the
>> right options for rsync.
>>
>> Make the link.
>> Sync + exclude.
>> Remove the link.
>>
>> Don't have to live with the folder on the source.
>>
>> *_______________*
>>
>>
>> *Gionata Boccalini*
>>
>> 2015-06-08 22:49 GMT+02:00 Michael Johnson - MJ <mj at
revmj.com>:
>>
>>> Oh, actually, I just thought of a couple other another options that
>>> don't require any multiplexing or ssh keys, but it would
require that your
>>> source machine is linux.
>>>
>>> The first option would be:
>>>
>>> mkdir /A/FolderB
>>> mount --bind /A/FolderA /A/FolderB
>>>
>>> Then just exclude /A/FolderA from the rsync and you are done.  This
does
>>> mean that you have to be ok with /A/FolderB existing on the source.
>>>
>>>
>>> The second option would be to use somthing like aufs or overlayfs
to
>>> create a new mountpoint that contains everything you want and
perhaps with
>>> a little bit of mount --bind thrown in.
>>>
>>>
>>> I just saw your response, and and what you describe makes sense. 
Sounds
>>> like the mount bind option + exclude might be the most elegant
option for
>>> this case.  Just make sure to add the bind mount into your fstab so
it
>>> comes back after a reboot.  :)
>>>
>>> On Mon, Jun 8, 2015 at 1:18 PM, Michael Johnson - MJ <mj at
revmj.com>
>>> wrote:
>>>
>>>> Thought I would chime in here.  To the best of my knowledge
what you
>>>> are trying to do cannot be done in a single run.  I supposed
--fuzzy might
>>>> work for you but I've never used that option and it sounds
scary to me.
>>>> Perhaps if I spent some time and learned the methodology it
uses I would be
>>>> less concerned.  If a feature like this were to be added, it
seems like it
>>>> would make sense to add it as a new "filter" type.
>>>>
>>>> But mainly you've piqued my curiosity.  The requirement
that it be able
>>>> to happen in a single rsync run seems very odd.  Is this just a
desire, or
>>>> is there really something that bad that would happen if you did
one pass
>>>> syncing A to B excluding FolderA and FolderB and then a second
pass syncing
>>>> FolderA to FolderB?
>>>>
>>>> The most likely scenario I imagine is that you are running this
by hand
>>>> and manually enter the SSH password.  Given the process take a
long time,
>>>> you don't want to enter the password again mid stream.  If
this is the
>>>> case, you could set up SSH keys to allow this to happen without
a manually
>>>> typed password.  You can find how to set that up here:
>>>>
>>>>
http://www.chainsawonatireswing.com/2012/01/15/ssh-into-your-synology-diskstation-with-ssh-keys/
>>>>
>>>> If you don't want to do ssh keys w/o a password, you could
use ssh
>>>> agent with keys.
>>>>
>>>> Finally you could also utilize ssh multiplexing (it looks like
that
>>>> should work with the synology nass).  You can find information
about this
>>>> option here:
>>>> http://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing
>>>>
>>>> The other (very unlikely) scenario I can imagine is that there
is
>>>> something that prevents you from logging in more than once
every X
>>>> hours/days like a time lock safe.  multiplexing would help here
as well.
>>>> But this scenario seems unlikely, it was probably silly to even
mention it.
>>>>
>>>> If there is another case I have not considered I would be very
>>>> interested to know where this requirement comes from.
>>>>
>>>> Thanks!  Hope there was something useful for you in all this. 
:)
>>>>
>>>> On Sun, Jun 7, 2015 at 9:38 AM, Gionata Boccalini <
>>>> gionata.boccalini at gmail.com> wrote:
>>>>
>>>>> Hello everyone,
>>>>>
>>>>> I'm new to this mailing list but I have been using
rsync for some
>>>>> years up to now.
>>>>> I'm trying to synchronize two directory trees, but I
want a special
>>>>> behavior that I didn't find on the net nor in the
manual (or maybe there is
>>>>> a combination of options to get what I want but I
couldn't find it).
>>>>>
>>>>> Tree A is like:
>>>>>
>>>>> A
>>>>>   *
>>>>>   *
>>>>>    FolderA
>>>>>   *
>>>>>   *
>>>>>
>>>>> And tree B (on a remote filesystem) is like:
>>>>>
>>>>> B
>>>>>   *
>>>>>   *
>>>>>   *
>>>>>    FolderB
>>>>>   *
>>>>>   *
>>>>>
>>>>>
>>>>> I have to synchronize everything in one rsync run, like
>>>>> rysnc  -arv  A/  B/
>>>>>
>>>>> but I want FolderA to be synchronized with FolderB.
>>>>> They must contain the same files but have a different name!
I want
>>>>> something like a "directory name translation" in
the rsync run...
>>>>> Is it possible? Do you see any another way of doing this?
(A part of
>>>>> using two rsync runs....)
>>>>> Please let me know if I didn't explain the problem
correctly or you
>>>>> need further information.
>>>>> Thank you for your attention and time.
>>>>> Best regards.
>>>>>
>>>>>
>>>>> *_______________Gionata Boccalini*
>>>>>
>>>>> --
>>>>> Please use reply-all for most replies to avoid omitting the
mailing
>>>>> list.
>>>>> To unsubscribe or change options:
>>>>> https://lists.samba.org/mailman/listinfo/rsync
>>>>> Before posting, read:
>>>>> http://www.catb.org/~esr/faqs/smart-questions.html
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Michael Johnson - MJ
>>>>
>>>
>>>
>>>
>>> --
>>> Michael Johnson - MJ
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20150610/408d2b59/attachment.html>