Btrfs devel - Jul 2013 - Problems with incremental btrfs send (out of order instructions?)

Hi all,

For some time, I''ve successfully deployed btrfs send/receive as a
viable backup solution.
It''s fast and flexible and nicely scriptable =)
However, every once in a while, trouble strikes on the receiving end
with a message like:

ERROR: rename
nixpkgs/pkgs/applications/version-management/subversion-1.2.x/default.nix
-> o2979-81788-0 failed. No such file or directory

Judging from looking at the dump-stream, I think sometimes a directory
gets moved (subversion-1.2.x in the example) to some temporary name
(or it gets deleted), but a little later a file within that directory
is being changed.
This seems to happen most often with filesystem-heavy applications
like a git tree or a webbrowser''s cache dir.

I''ve tried many things to recover from such a situation, like removing
the offending directory and retrying from a new snapshot, but nothing
seems to work except for a full (non-incremental) send which is
expensive.

After some narrowing down, I managed to capture a problematic
sendstream. To reproduce:
- I created a separate subvolume (so there are no privacy-related
issues with sharing the dump)
- put a moderately large git tree on it (I used
https://github.com/NixOS/nixpkgs)
- perform an initial btrfs send of the subvolume
- jump around git history (check out very early revision, check out HEAD again)
- perform an incremental (with parent) btrfs send to a file
- try to receive this file on another machine
- about 80% chance this triggers it, if not, jump/branch some more with git

The problematic dump is here:
https://bluescreen303.nl/btrfs-send-error.dump.xz
The error provided above goes with this dump.

The dump was on kernel 3.9.x but it still happens on 3.10.4
btrfs-progs 650e656a8b9c1fbe4ec5cd8c48ae285b8abd3b69
I noticed there have been some commits since that revision but nothing
mentions send/receive.

Can someone have a look at my dumpfile and confirm this is indeed a
problem with btrfs send as opposed to something I am doing wrong?

Thanks!
Mathijs
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Wed, 31 Jul 2013 23:33:59 +0200, Mathijs Kwik wrote:
> Hi all,
> 
> For some time, I''ve successfully deployed btrfs send/receive as a
> viable backup solution.
> It''s fast and flexible and nicely scriptable =)
> However, every once in a while, trouble strikes on the receiving end
> with a message like:
> 
> ERROR: rename
nixpkgs/pkgs/applications/version-management/subversion-1.2.x/default.nix
> -> o2979-81788-0 failed. No such file or directory
I''ve also seen such an errors. It''s an error in the send code.

[...]
> Can someone have a look at my dumpfile and confirm this is indeed a
> problem with btrfs send as opposed to something I am doing wrong?
According to your dumpfile, the file
nixpkgs/pkgs/applications/version-management/subversion-1.2.x/default.nix needs
to be there in the parent subvolume. Apparently it isn''t. Since btrfs
send enforces that sent snapshots are read-only, and btrfs receive sets received
subvolumes to read-only immediately after the reception, it can''t be
your fault. It has to be the fault of the btrfs send logic.

If you enter a bug on bugzilla.kernel.org (refer to
<https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#How_do_I_report_bugs_and_issues.3F>),
you increase the probability that somebody fixes the issue some day :)

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 2013-08-01 10:49, Stefan Behrens wrote:
> On Wed, 31 Jul 2013 23:33:59 +0200, Mathijs Kwik wrote:
>> Hi all,
>>
>> For some time, I''ve successfully deployed btrfs send/receive
as a
>> viable backup solution.
>> It''s fast and flexible and nicely scriptable =)
>> However, every once in a while, trouble strikes on the receiving end
>> with a message like:
>>
>> ERROR: rename 
>>
nixpkgs/pkgs/applications/version-management/subversion-1.2.x/default.nix
>> -> o2979-81788-0 failed. No such file or directory
>
> I''ve also seen such an errors. It''s an error in the send
code.
>
[...]
> If you enter a bug on bugzilla.kernel.org (refer to
> 
>
<https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#How_do_I_report_bugs_and_issues.3F>),
> you increase the probability that somebody fixes the issue some day 
> :)

This bug has been reported in BZ #60673 [1] and was fixed in Josef''s 
btrfs-next commit
titled "Btrfs: check our parent dir when doing a compare send" [2].
This fix hasn''t been merged in the stable kernel yet.

Thanks to all.


[1] https://bugzilla.kernel.org/show_bug.cgi?id=60673
[2] 
http://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/fs/btrfs/send.c?id=441f6d6f7115a927f3e13f7acc53184a6081dbe0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Xavier Bassery <xavier@bartica.org> writes:
> On 2013-08-01 10:49, Stefan Behrens wrote:
>> On Wed, 31 Jul 2013 23:33:59 +0200, Mathijs Kwik wrote:
>>> Hi all,
>>>
>>> For some time, I''ve successfully deployed btrfs
send/receive as a
>>> viable backup solution.
>>> It''s fast and flexible and nicely scriptable =)
>>> However, every once in a while, trouble strikes on the receiving
end
>>> with a message like:
>>>
>>> ERROR: rename
>>>
nixpkgs/pkgs/applications/version-management/subversion-1.2.x/default.nix
>>> -> o2979-81788-0 failed. No such file or directory
>>
>> I''ve also seen such an errors. It''s an error in the
send code.
>>
> [...]
>> If you enter a bug on bugzilla.kernel.org (refer to
>>
>>
<https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#How_do_I_report_bugs_and_issues.3F>),
>> you increase the probability that somebody fixes the issue some day
>> :)
>
>
> This bug has been reported in BZ #60673 [1] and was fixed in
Josef''s
> btrfs-next commit
> titled "Btrfs: check our parent dir when doing a compare send"
[2].
> This fix hasn''t been merged in the stable kernel yet.
>
> Thanks to all.
I have been using the patch for 10 days now, and it''s a huge
improvement, but we''re not out of the woods yet it seems :(

https://bugzilla.kernel.org/show_bug.cgi?id=60673
>
>
> [1] https://bugzilla.kernel.org/show_bug.cgi?id=60673
> [2]
>
http://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/fs/btrfs/send.c?id=441f6d6f7115a927f3e13f7acc53184a6081dbe0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html