rsync - Mar 2012 - link(2) EMLINK error behavior with --link-dest and --hard-links

Hi,

I'm having a problem using --link-dest and --hard-links when
the fs hits the hard link limit (link(2) returns EMLINK).

Using rsync 3.0.7 an error is thrown and the target file is
not created.  Glancing at git head it _looks_ like things
could now be a little nicer.  Perhaps the target file is copied
instead of hard linked when hardlinking fails -- I've not
tested it.

Anyway, the behavior I desire when using both --link-dest and
--hard-links and when running out of links is to get an error,
but to "have the --link-dest fail".  In other words, the
end result, for that particular target file, would be
as if --link-dest was not specified.  This leaves the hard links
in the source preserved in the target.  (At least an
"exact copy" happens, which is more desirable than
creating a copy at whatever point the hard link limit
is reached and then hardlinking from that point forward --
if that's what the git head code really does.)

Glancing over the code, it seems to me the way to do this 
is to have the hard_link_one() return value indicate
when failure is EMLINK.  This can then be tested for
during the execution of finish_hard_link().  When
link exhaustion is detected in finish_hard_link(),
&& link_dest, then report an error but also
copy the source file and re-run
finish_hard_link() to undo/fix-up the hard links
already created.  This assumes that the
hard links created so far can be overwritten and
won't break things.  (And there needs to be some
sort of flag to avoid an infinite loop
should link exhaustion happen again.)

If this works there shouldn't be any performance
impact unless hard links are exhausted.

I really don't understand how the code works,
my suggestion could be completely wrong.
I still wanted to supply a brain-dump here
in case my suggestion is useful and to learn
something from the feedback.

Regards,

Karl <kop at meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Should I submit a bug?  Will you take a patch?

On 03/24/2012 11:11:47 PM, Karl O. Pinc wrote:
> Hi,
> 
> I'm having a problem using --link-dest and --hard-links when
> the fs hits the hard link limit (link(2) returns EMLINK).
> 
> Using rsync 3.0.7 an error is thrown and the target file is
> not created.  Glancing at git head it _looks_ like things
> could now be a little nicer.  Perhaps the target file is copied
> instead of hard linked when hardlinking fails -- I've not
> tested it.
> 
> Anyway, the behavior I desire when using both --link-dest and
> --hard-links and when running out of links is to get an error,
> but to "have the --link-dest fail".  In other words, the
> end result, for that particular target file, would be
> as if --link-dest was not specified.  This leaves the hard links
> in the source preserved in the target.  (At least an
> "exact copy" happens, which is more desirable than
> creating a copy at whatever point the hard link limit
> is reached and then hardlinking from that point forward --
> if that's what the git head code really does.)
> 
> Glancing over the code, it seems to me the way to do this 
> is to have the hard_link_one() return value indicate
> when failure is EMLINK.  This can then be tested for
> during the execution of finish_hard_link().  When
> link exhaustion is detected in finish_hard_link(),
> && link_dest, then report an error but also
> copy the source file and re-run
> finish_hard_link() to undo/fix-up the hard links
> already created.  This assumes that the
> hard links created so far can be overwritten and
> won't break things.  (And there needs to be some
> sort of flag to avoid an infinite loop
> should link exhaustion happen again.)
> 
> If this works there shouldn't be any performance
> impact unless hard links are exhausted.
> 
> I really don't understand how the code works,
> my suggestion could be completely wrong.
> I still wanted to supply a brain-dump here
> in case my suggestion is useful and to learn
> something from the feedback.
> 
> Regards,
> 
> Karl <kop at meme.com>
> Free Software:  "You don't pay back, you pay forward."
>                  -- Robert A. Heinlein
> 
> -- 
> Please use reply-all for most replies to avoid omitting the mailing
> list.
> To unsubscribe or change options: https://lists.samba.org/mailman/
> listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-
> questions.html
> 
> 
> 
> 



Karl <kop at meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein