I can send snapshots to <volume>, but not <volume>/<dir>.
Please advise
if what I am doing is incorrect.
Rory
Format usb3 disk and mount
root@orchard:/bkp# mkfs.btrfs /dev/sdb1
> WARNING! - Btrfs v0.20-rc1-37-g91d9eec IS EXPERIMENTAL
> WARNING! - see http://btrfs.wiki.kernel.org before using
> fs created label (null) on /dev/sdb1
> nodesize 4096 leafsize 4096 sectorsize 4096 size 698.64GB
> Btrfs v0.20-rc1-37-g91d9eec
mount /dev/sdb1 /mnt
Create snapshots on /bkp share
root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/snap_081012_1715
> Create a readonly snapshot of ''subvol'' in
''snaps/snap_081012_1715''
root@orchard:/bkp# mutt -f subvol/INBOX/
> 1561 kept, 18 deleted.
root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/snap_081012_1716
> Create a readonly snapshot of ''subvol'' in
''snaps/snap_081012_1716''
Send base backup to /mnt
root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt
> At subvol snaps/snap_081012_1715
> At subvol snap_081012_1715
Send incremental backup to /mnt
root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
snaps/snap_081012_1716 | btrfs receive /mnt
> At subvol snaps/snap_081012_1716
> At snapshot snap_081012_1716
root@orchard:/bkp# ls /mnt
snap_081012_1715 snap_081012_1716
Results:
root@orchard:/bkp# btrfs subvolume list /bkp
> ID 259 gen 62 top level 5 path subvol
> ID 278 gen 60 top level 5 path snaps/snap_081012_1715
> ID 279 gen 62 top level 5 path snaps/snap_081012_1716
root@orchard:/bkp# btrfs subvolume list /mnt
> ID 256 gen 8 top level 5 path snap_081012_1715
> ID 259 gen 9 top level 5 path snap_081012_1716
Restart:
root@orchard:/bkp# btrfs subvolume del /mnt/snap_081012_171*
> Delete subvolume ''/mnt/snap_081012_1715''
> Delete subvolume ''/mnt/snap_081012_1716''
Try and snap to /mnt/<subdir>
root@orchard:/bkp# mkdir /mnt/snaps
root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/snaps
> At subvol snaps/snap_081012_1715
> At subvol snap_081012_1715
root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
snaps/snap_081012_1716 | btrfs receive /mnt/snaps
> At subvol snaps/snap_081012_1716
> At snapshot snap_081012_1716
> ERROR: open snaps/snap_081012_1715 failed. No such file or directory
root@orchard:/bkp# ls /mnt/snaps
> snap_081012_1715
--
Rory Campbell-Lange
rory@campbell-lange.net
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On 10/08/12 18:30, Rory Campbell-Lange wrote:> I can send snapshots to <volume>, but not <volume>/<dir>. Please advise > if what I am doing is incorrect. > > Rory > > Format usb3 disk and mount > root@orchard:/bkp# mkfs.btrfs /dev/sdb1 > > WARNING! - Btrfs v0.20-rc1-37-g91d9eec IS EXPERIMENTAL > > WARNING! - see http://btrfs.wiki.kernel.org before using > > fs created label (null) on /dev/sdb1 > > nodesize 4096 leafsize 4096 sectorsize 4096 size 698.64GB > > Btrfs v0.20-rc1-37-g91d9eec > mount /dev/sdb1 /mnt > > > Create snapshots on /bkp share > root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/snap_081012_1715 > > Create a readonly snapshot of ''subvol'' in ''snaps/snap_081012_1715'' > root@orchard:/bkp# mutt -f subvol/INBOX/ > > 1561 kept, 18 deleted. > root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/snap_081012_1716 > > Create a readonly snapshot of ''subvol'' in ''snaps/snap_081012_1716'' > > Send base backup to /mnt > root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt > > At subvol snaps/snap_081012_1715 > > At subvol snap_081012_1715 > > Send incremental backup to /mnt > root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \ > snaps/snap_081012_1716 | btrfs receive /mnt > > At subvol snaps/snap_081012_1716 > > At snapshot snap_081012_1716 > > root@orchard:/bkp# ls /mnt > snap_081012_1715 snap_081012_1716 > > Results: > root@orchard:/bkp# btrfs subvolume list /bkp > > ID 259 gen 62 top level 5 path subvol > > ID 278 gen 60 top level 5 path snaps/snap_081012_1715 > > ID 279 gen 62 top level 5 path snaps/snap_081012_1716 > root@orchard:/bkp# btrfs subvolume list /mnt > > ID 256 gen 8 top level 5 path snap_081012_1715 > > ID 259 gen 9 top level 5 path snap_081012_1716 > > Restart: > root@orchard:/bkp# btrfs subvolume del /mnt/snap_081012_171* > > Delete subvolume ''/mnt/snap_081012_1715'' > > Delete subvolume ''/mnt/snap_081012_1716'' > > Try and snap to /mnt/<subdir> > root@orchard:/bkp# mkdir /mnt/snaps > root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/snaps > > At subvol snaps/snap_081012_1715 > > At subvol snap_081012_1715 > root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \ > snaps/snap_081012_1716 | btrfs receive /mnt/snaps > > At subvol snaps/snap_081012_1716 > > At snapshot snap_081012_1716 > > ERROR: open snaps/snap_081012_1715 failed. No such file or directory > root@orchard:/bkp# ls /mnt/snaps > > snap_081012_1715The target has to be a subvol also. But interestingly enough, it also fails for a subvol. The base send works, the incremental fails, because btrfs receive can''t find snaps/snap_081012_1715. If you give /mnt/snaps as the target for the base and just /mnt for the incremental, it works. There''s clearly something broken there... -arne> > >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 08/10/12, Arne Jansen (sensille@gmx.net) wrote:> On 10/08/12 18:30, Rory Campbell-Lange wrote: > > I can send snapshots to <volume>, but not <volume>/<dir>. Please advise > > if what I am doing is incorrect.> > Try and snap to /mnt/<subdir> > > root@orchard:/bkp# mkdir /mnt/snaps > > root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/snaps > > > At subvol snaps/snap_081012_1715 > > > At subvol snap_081012_1715 > > root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \ > > snaps/snap_081012_1716 | btrfs receive /mnt/snaps > > > At subvol snaps/snap_081012_1716 > > > At snapshot snap_081012_1716 > > > ERROR: open snaps/snap_081012_1715 failed. No such file or directory > > root@orchard:/bkp# ls /mnt/snaps > > > snap_081012_1715 > > The target has to be a subvol also. But interestingly enough, it also > fails for a subvol. The base send works, the incremental fails, because > btrfs receive can''t find snaps/snap_081012_1715. If you give /mnt/snaps > as the target for the base and just /mnt for the incremental, it works. > There''s clearly something broken there...Actually, this does work: root@orchard:~# mkfs.btrfs /dev/sdb1 ...snip... > Btrfs v0.20-rc1-37-g91d9eec root@orchard:/bkp# ls > snaps subvol root@orchard:/bkp# mount /dev/sdb1 /mnt root@orchard:/bkp# btrfs subvolume create /mnt/subvol > Create subvolume ''/mnt/subvol'' root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/subvol > At subvol snaps/snap_081012_1715 > At subvol snap_081012_1715 root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \ snaps/snap_081012_1716 | btrfs receive /mnt/subvol > At subvol snaps/snap_081012_1716 > At snapshot snap_081012_1716 It certainly seems to work if the receive target is a subvolume other than the implicit subvolume at the root of the mount point. So one can only btrfs receive to a subvolume, which makes sense, and is I believe the reason one cannot ''btrfs receive to a subdirectory'' as set out in the subject of my original email. Are there any other problem cases you can see, Arne? Regards Rory -- Rory Campbell-Lange rory@campbell-lange.net -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Rory, Arne,
I think the problem is that currently mnt_fd in struct btrfs_receive
is used both as "mount root" and "directory in which the
subvolume/snapshot needs to be created".
Arne, does the following patch make sense? It uses Jan''s
find_mount_root function. With this patch both Rory''s tests seem to
work for me.
Thanks,
Alex.
diff --git a/cmds-receive.c b/cmds-receive.c
index a8be6fa..ed53326 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -52,11 +52,13 @@ static int g_verbose = 0;
struct btrfs_receive
{
int mnt_fd;
+ int dest_dir_fd;
int write_fd;
char *write_path;
char *root_path;
+ char *dest_dir_path; /* relative to root_path */
char *full_subvol_path;
struct subvol_info *cur_subvol;
@@ -150,8 +152,11 @@ static int process_subvol(const char *path, const
u8 *uuid, u64 ctransid,
r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
r->parent_subvol = NULL;
- r->cur_subvol->path = strdup(path);
- r->full_subvol_path = path_cat(r->root_path, path);
+ if (strlen(r->dest_dir_path) == 0)
+ r->cur_subvol->path = strdup(path);
+ else
+ r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+ r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path,
path);
fprintf(stderr, "At subvol %s\n", path);
@@ -167,7 +172,7 @@ static int process_subvol(const char *path, const
u8 *uuid, u64 ctransid,
memset(&args_v1, 0, sizeof(args_v1));
strcpy(args_v1.name, path);
- ret = ioctl(r->mnt_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
+ ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
if (ret < 0) {
ret = -errno;
fprintf(stderr, "ERROR: creating subvolume %s failed. "
@@ -195,8 +200,11 @@ static int process_snapshot(const char *path,
const u8 *uuid, u64 ctransid,
r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
r->parent_subvol = NULL;
- r->cur_subvol->path = strdup(path);
- r->full_subvol_path = path_cat(r->root_path, path);
+ if (strlen(r->dest_dir_path) == 0)
+ r->cur_subvol->path = strdup(path);
+ else
+ r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+ r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path,
path);
fprintf(stderr, "At snapshot %s\n", path);
@@ -243,7 +251,7 @@ static int process_snapshot(const char *path,
const u8 *uuid, u64 ctransid,
goto out;
}
- ret = ioctl(r->mnt_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
+ ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
close(args_v2.fd);
if (ret < 0) {
ret = -errno;
@@ -790,17 +798,44 @@ struct btrfs_send_ops send_ops = {
int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd)
{
int ret;
+ char *dest_dir_full_path;
int end = 0;
- r->root_path = strdup(tomnt);
- r->mnt_fd = open(tomnt, O_RDONLY | O_NOATIME);
+ dest_dir_full_path = realpath(tomnt, NULL);
+ if (!dest_dir_full_path) {
+ ret = -errno;
+ fprintf(stderr, "ERROR: realpath(%s) failed. %s\n", tomnt,
+ strerror(-ret));
+ goto out;
+ }
+ r->dest_dir_fd = open(tomnt, O_RDONLY | O_NOATIME);
+ if (r->dest_dir_fd < 0) {
+ ret = -errno;
+ fprintf(stderr, "ERROR: failed to open destination directory %s.
%s\n", tomnt,
+ strerror(-ret));
+ goto out;
+ }
+
+ ret = find_mount_root(tomnt, &r->root_path);
+ if (ret < 0) {
+ ret = -EINVAL;
+ fprintf(stderr, "ERROR: failed to determine mount point "
+ "for %s\n", tomnt);
+ goto out;
+ }
+ r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME);
if (r->mnt_fd < 0) {
ret = -errno;
- fprintf(stderr, "ERROR: failed to open %s. %s\n", tomnt,
+ fprintf(stderr, "ERROR: failed to open %s. %s\n", r->root_path,
strerror(-ret));
goto out;
}
+ /* Find the destination dir relative to mount root */
+ r->dest_dir_path = dest_dir_full_path + strlen(r->root_path);
+ if (r->dest_dir_path[0] == ''/'')
+ r->dest_dir_path++;
+
ret = subvol_uuid_search_init(r->mnt_fd, &r->sus);
if (ret < 0)
return ret;
diff --git a/send-utils.h b/send-utils.h
index da407eb..a3e038b 100644
--- a/send-utils.h
+++ b/send-utils.h
@@ -65,5 +65,6 @@ void subvol_uuid_search_add(struct subvol_uuid_search *s,
char *path_cat(const char *p1, const char *p2);
char *path_cat3(const char *p1, const char *p2, const char *p3);
+int find_mount_root(const char *path, char **mount_root);
#endif /* SEND_UTILS_H_ */
On Tue, Oct 9, 2012 at 1:18 PM, Rory Campbell-Lange
<rory@campbell-lange.net> wrote:> On 08/10/12, Arne Jansen (sensille@gmx.net) wrote:
>> On 10/08/12 18:30, Rory Campbell-Lange wrote:
>> > I can send snapshots to <volume>, but not
<volume>/<dir>. Please advise
>> > if what I am doing is incorrect.
>
>> > Try and snap to /mnt/<subdir>
>> > root@orchard:/bkp# mkdir /mnt/snaps
>> > root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs
receive /mnt/snaps
>> > > At subvol snaps/snap_081012_1715
>> > > At subvol snap_081012_1715
>> > root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
>> > snaps/snap_081012_1716 | btrfs receive
/mnt/snaps
>> > > At subvol snaps/snap_081012_1716
>> > > At snapshot snap_081012_1716
>> > > ERROR: open snaps/snap_081012_1715 failed. No such file
or directory
>> > root@orchard:/bkp# ls /mnt/snaps
>> > > snap_081012_1715
>>
>> The target has to be a subvol also. But interestingly enough, it also
>> fails for a subvol. The base send works, the incremental fails, because
>> btrfs receive can''t find snaps/snap_081012_1715. If you give
/mnt/snaps
>> as the target for the base and just /mnt for the incremental, it works.
>> There''s clearly something broken there...
>
> Actually, this does work:
>
> root@orchard:~# mkfs.btrfs /dev/sdb1
> ...snip...
> > Btrfs v0.20-rc1-37-g91d9eec
> root@orchard:/bkp# ls
> > snaps subvol
> root@orchard:/bkp# mount /dev/sdb1 /mnt
> root@orchard:/bkp# btrfs subvolume create /mnt/subvol
> > Create subvolume ''/mnt/subvol''
> root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive
/mnt/subvol
> > At subvol snaps/snap_081012_1715
> > At subvol snap_081012_1715
> root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
> snaps/snap_081012_1716 | btrfs receive
/mnt/subvol
> > At subvol snaps/snap_081012_1716
> > At snapshot snap_081012_1716
>
> It certainly seems to work if the receive target is a subvolume other
> than the implicit subvolume at the root of the mount point.
>
> So one can only btrfs receive to a subvolume, which makes sense, and is
> I believe the reason one cannot ''btrfs receive to a
subdirectory'' as set
> out in the subject of my original email.
>
> Are there any other problem cases you can see, Arne?
>
> Regards
> Rory
>
> --
> Rory Campbell-Lange
> rory@campbell-lange.net
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On 09/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:> Hi Rory, Arne, > I think the problem is that currently mnt_fd in struct btrfs_receive > is used both as "mount root" and "directory in which the > subvolume/snapshot needs to be created". > Arne, does the following patch make sense? It uses Jan''s > find_mount_root function. With this patch both Rory''s tests seem to > work for me.Hi Alex I haven''t had a chance to test this. I''m also on 89fe5b5f of progs recommended to my by Chris for send/receive testing and your patch doesn''t apply for me. Are you using another revision? Regards Rory -- Rory Campbell-Lange rory@campbell-lange.net -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Rory,
thanks for letting me know.
I am using the latest code from Chris''s repo (
git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git),
which has been updated recently (latest commit
91d9eec1ff044394f2b98ee7fcb76713dd33b994).
Do you think you can try the following patch on top of this commit? (I
made another small fix).
Basically, the idea is that the path that you give to "btrfs receive"
should be the destination directory, in which the received subvolume
will be created. And this directory should be somewhere under a mount
point, where destination btrfs is mounted (it can also be the same as
the mount point, but not above it). Does this make sense?
Also it should not matter where the sent subvolume resided in the
source fs, and, if this is a diff-send, where you have received the
parent snapshot to (as long as it is on/under the same mount point).
Thanks,
Alex.
diff --git a/cmds-receive.c b/cmds-receive.c
index a8be6fa..6b7cf12 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -52,11 +52,13 @@ static int g_verbose = 0;
struct btrfs_receive
{
int mnt_fd;
+ int dest_dir_fd;
int write_fd;
char *write_path;
char *root_path;
+ char *dest_dir_path; /* relative to root_path */
char *full_subvol_path;
struct subvol_info *cur_subvol;
@@ -150,8 +152,11 @@ static int process_subvol(const char *path, const
u8 *uuid, u64 ctransid,
r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
r->parent_subvol = NULL;
- r->cur_subvol->path = strdup(path);
- r->full_subvol_path = path_cat(r->root_path, path);
+ if (strlen(r->dest_dir_path) == 0)
+ r->cur_subvol->path = strdup(path);
+ else
+ r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+ r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path,
path);
fprintf(stderr, "At subvol %s\n", path);
@@ -167,7 +172,7 @@ static int process_subvol(const char *path, const
u8 *uuid, u64 ctransid,
memset(&args_v1, 0, sizeof(args_v1));
strcpy(args_v1.name, path);
- ret = ioctl(r->mnt_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
+ ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
if (ret < 0) {
ret = -errno;
fprintf(stderr, "ERROR: creating subvolume %s failed. "
@@ -195,8 +200,11 @@ static int process_snapshot(const char *path,
const u8 *uuid, u64 ctransid,
r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
r->parent_subvol = NULL;
- r->cur_subvol->path = strdup(path);
- r->full_subvol_path = path_cat(r->root_path, path);
+ if (strlen(r->dest_dir_path) == 0)
+ r->cur_subvol->path = strdup(path);
+ else
+ r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+ r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path,
path);
fprintf(stderr, "At snapshot %s\n", path);
@@ -243,7 +251,7 @@ static int process_snapshot(const char *path,
const u8 *uuid, u64 ctransid,
goto out;
}
- ret = ioctl(r->mnt_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
+ ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
close(args_v2.fd);
if (ret < 0) {
ret = -errno;
@@ -790,17 +798,48 @@ struct btrfs_send_ops send_ops = {
int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd)
{
int ret;
+ char *dest_dir_full_path;
int end = 0;
- r->root_path = strdup(tomnt);
- r->mnt_fd = open(tomnt, O_RDONLY | O_NOATIME);
+ dest_dir_full_path = realpath(tomnt, NULL);
+ if (!dest_dir_full_path) {
+ ret = -errno;
+ fprintf(stderr, "ERROR: realpath(%s) failed. %s\n", tomnt,
+ strerror(-ret));
+ goto out;
+ }
+ r->dest_dir_fd = open(dest_dir_full_path, O_RDONLY | O_NOATIME);
+ if (r->dest_dir_fd < 0) {
+ ret = -errno;
+ fprintf(stderr, "ERROR: failed to open destination directory %s.
%s\n",
+ dest_dir_full_path, strerror(-ret));
+ goto out;
+ }
+
+ ret = find_mount_root(dest_dir_full_path, &r->root_path);
+ if (ret < 0) {
+ ret = -EINVAL;
+ fprintf(stderr, "ERROR: failed to determine mount point "
+ "for %s\n", dest_dir_full_path);
+ goto out;
+ }
+ r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME);
if (r->mnt_fd < 0) {
ret = -errno;
- fprintf(stderr, "ERROR: failed to open %s. %s\n", tomnt,
+ fprintf(stderr, "ERROR: failed to open %s. %s\n", r->root_path,
strerror(-ret));
goto out;
}
+ /*
+ * find_mount_root returns a root_path that is a subpath of
+ * dest_dir_full_path. Now get the other part of root_path,
+ * which is the destination dir relative to root_path.
+ */
+ r->dest_dir_path = dest_dir_full_path + strlen(r->root_path);
+ if (r->dest_dir_path[0] == ''/'')
+ r->dest_dir_path++;
+
ret = subvol_uuid_search_init(r->mnt_fd, &r->sus);
if (ret < 0)
return ret;
diff --git a/send-utils.h b/send-utils.h
index da407eb..a3e038b 100644
--- a/send-utils.h
+++ b/send-utils.h
@@ -65,5 +65,6 @@ void subvol_uuid_search_add(struct subvol_uuid_search *s,
char *path_cat(const char *p1, const char *p2);
char *path_cat3(const char *p1, const char *p2, const char *p3);
+int find_mount_root(const char *path, char **mount_root);
#endif /* SEND_UTILS_H_ */
On Thu, Oct 11, 2012 at 12:11 AM, Rory Campbell-Lange
<rory@campbell-lange.net> wrote:> On 09/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
>> Hi Rory, Arne,
>> I think the problem is that currently mnt_fd in struct btrfs_receive
>> is used both as "mount root" and "directory in which the
>> subvolume/snapshot needs to be created".
>> Arne, does the following patch make sense? It uses Jan''s
>> find_mount_root function. With this patch both Rory''s tests
seem to
>> work for me.
>
> Hi Alex
>
> I haven''t had a chance to test this. I''m also on 89fe5b5f
of progs
> recommended to my by Chris for send/receive testing and your patch
> doesn''t apply for me. Are you using another revision?
>
> Regards
> Rory
>
> --
> Rory Campbell-Lange
> rory@campbell-lange.net
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:> I am using the latest code from Chris''s repo ( > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git), > which has been updated recently (latest commit > 91d9eec1ff044394f2b98ee7fcb76713dd33b994). > Do you think you can try the following patch on top of this commit? (I > made another small fix).I had to fix the line wrapping, but your patch applies fine now.> Basically, the idea is that the path that you give to "btrfs receive" > should be the destination directory, in which the received subvolume > will be created. And this directory should be somewhere under a mount > point, where destination btrfs is mounted (it can also be the same as > the mount point, but not above it). Does this make sense? > Also it should not matter where the sent subvolume resided in the > source fs, and, if this is a diff-send, where you have received the > parent snapshot to (as long as it is on/under the same mount point).I don''t think the receive pathing works. Please note below the following issues: [aa] ERROR: open snapdir/subvol_20121011_1214 failed. No such file or directory [bb] ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory [cc] ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory I tested this with the following: send mount point /bkp receive mount point /mnt The ''send'' mount point has the following structure: root@orchard:/bkp# ls > snaps subvol (subvol a btrfs subvolume, snaps a directory for holding snapshots) I then made a directory and two subvolumes in /mnt to receive ''sends'': root@orchard:/bkp# ls /mnt > arbitrary_subvol snapdir subvol I applied your patch to progs 91d9eec1, then did the following rooted in /bkp: 1. Create snapshots root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/subvol_20121011_1214 > Create a readonly snapshot of ''subvol'' in ''snaps/subvol_20121011_1214'' root@orchard:/bkp# mutt -f subvol/INBOX > 1573 kept, 39 deleted. root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/subvol_20121011_1216 > Create a readonly snapshot of ''subvol'' in ''snaps/subvol_20121011_1216'' 2. Send base snapshots to /mnt/{snapdir,arbitrary_subvol,subvol} root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/snapdir > At subvol snaps/subvol_20121011_1214 > At subvol subvol_20121011_1214 root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/arbitrary_subvol > At subvol snaps/subvol_20121011_1214 > At subvol subvol_20121011_1214 root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/subvol > At subvol snaps/subvol_20121011_1214 > At subvol subvol_20121011_1214 3. Send incremental changes root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214 snaps/subvol_20121011_1216/ | btrfs receive /mnt/snapdir At subvol snaps/subvol_20121011_1216/ At snapshot subvol_20121011_1216 ERROR: open snapdir/subvol_20121011_1214 failed. No such file or directory [aa] root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214 snaps/subvol_20121011_1216/ | btrfs receive /mnt/arbitrary_subvol At subvol snaps/subvol_20121011_1216/ At snapshot subvol_20121011_1216 ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory [bb] root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214 snaps/subvol_20121011_1216/ | btrfs receive /mnt/subvol At subvol snaps/subvol_20121011_1216/ At snapshot subvol_20121011_1216 ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory [cc] root@orchard:/bkp# btrfs subvolume list /bkp ID 256 gen 9 top level 5 path subvol ID 260 gen 7 top level 5 path snaps/subvol_20121011_1214 ID 261 gen 9 top level 5 path snaps/subvol_20121011_1216 root@orchard:/bkp# btrfs subvolume list /mnt ID 301 gen 66 top level 5 path arbitrary_subvol ID 302 gen 69 top level 5 path subvol ID 303 gen 66 top level 5 path snapdir/subvol_20121011_1214 ID 304 gen 69 top level 5 path arbitrary_subvol/subvol_20121011_1214 ID 305 gen 72 top level 5 path subvol/subvol_20121011_1214 -- Rory Campbell-Lange rory@campbell-lange.net -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Rory, all your tests work perfectly for me (really, I swear:)), with exact same mount points and subvol/snap/dir names. Just to make that the patch some how did not get messed up in the mail, I attach it. Can you pls post the output of "cat /proc/mounts" and "mount" (if you''re willing to debug this further). Other than that, I will probably need to see the contents of your source subvol (at least file names, not necessarily the content). So if you can expose it, can you perhaps do something like: tree -A --inodes --noreport -s /bkp/subvol and also tree -A --inodes --noreport -s /mnt before you start the test (I assume that two subvolumes at /mnt are empty, right). However, I doubt the contents will really help, because it seems to fail very early, even before it starts processing the commands. Thanks, Alex. On Thu, Oct 11, 2012 at 1:33 PM, Rory Campbell-Lange <rory@campbell-lange.net> wrote:> On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote: >> I am using the latest code from Chris''s repo ( >> git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git), >> which has been updated recently (latest commit >> 91d9eec1ff044394f2b98ee7fcb76713dd33b994). >> Do you think you can try the following patch on top of this commit? (I >> made another small fix). > > I had to fix the line wrapping, but your patch applies fine now. > >> Basically, the idea is that the path that you give to "btrfs receive" >> should be the destination directory, in which the received subvolume >> will be created. And this directory should be somewhere under a mount >> point, where destination btrfs is mounted (it can also be the same as >> the mount point, but not above it). Does this make sense? >> Also it should not matter where the sent subvolume resided in the >> source fs, and, if this is a diff-send, where you have received the >> parent snapshot to (as long as it is on/under the same mount point). > > I don''t think the receive pathing works. Please note below the following issues: > [aa] ERROR: open snapdir/subvol_20121011_1214 failed. No such file or directory > [bb] ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory > [cc] ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory > > I tested this with the following: > > send mount point /bkp > receive mount point /mnt > > The ''send'' mount point has the following structure: > > root@orchard:/bkp# ls > > snaps subvol > (subvol a btrfs subvolume, snaps a directory for holding snapshots) > > I then made a directory and two subvolumes in /mnt to receive ''sends'': > > root@orchard:/bkp# ls /mnt > > arbitrary_subvol snapdir subvol > > I applied your patch to progs 91d9eec1, then did the following rooted in /bkp: > > 1. Create snapshots > > root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/subvol_20121011_1214 > > Create a readonly snapshot of ''subvol'' in ''snaps/subvol_20121011_1214'' > > root@orchard:/bkp# mutt -f subvol/INBOX > > 1573 kept, 39 deleted. > > root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/subvol_20121011_1216 > > Create a readonly snapshot of ''subvol'' in ''snaps/subvol_20121011_1216'' > > 2. Send base snapshots to /mnt/{snapdir,arbitrary_subvol,subvol} > > root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/snapdir > > At subvol snaps/subvol_20121011_1214 > > At subvol subvol_20121011_1214 > > root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/arbitrary_subvol > > At subvol snaps/subvol_20121011_1214 > > At subvol subvol_20121011_1214 > > root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/subvol > > At subvol snaps/subvol_20121011_1214 > > At subvol subvol_20121011_1214 > > 3. Send incremental changes > > root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214 > snaps/subvol_20121011_1216/ | btrfs receive /mnt/snapdir > At subvol snaps/subvol_20121011_1216/ > At snapshot subvol_20121011_1216 > ERROR: open snapdir/subvol_20121011_1214 failed. No such file or directory [aa] > > root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214 > snaps/subvol_20121011_1216/ | btrfs receive /mnt/arbitrary_subvol > At subvol snaps/subvol_20121011_1216/ > At snapshot subvol_20121011_1216 > ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory [bb] > > root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214 > snaps/subvol_20121011_1216/ | btrfs receive /mnt/subvol > At subvol snaps/subvol_20121011_1216/ > At snapshot subvol_20121011_1216 > ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory [cc] > > > root@orchard:/bkp# btrfs subvolume list /bkp > ID 256 gen 9 top level 5 path subvol > ID 260 gen 7 top level 5 path snaps/subvol_20121011_1214 > ID 261 gen 9 top level 5 path snaps/subvol_20121011_1216 > > root@orchard:/bkp# btrfs subvolume list /mnt > ID 301 gen 66 top level 5 path arbitrary_subvol > ID 302 gen 69 top level 5 path subvol > ID 303 gen 66 top level 5 path snapdir/subvol_20121011_1214 > ID 304 gen 69 top level 5 path arbitrary_subvol/subvol_20121011_1214 > ID 305 gen 72 top level 5 path subvol/subvol_20121011_1214 > > > > -- > Rory Campbell-Lange > rory@campbell-lange.net
On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:> Hi Rory, > all your tests work perfectly for me (really, I swear:)), with exact > same mount points and subvol/snap/dir names. > Just to make that the patch some how did not get messed up in the > mail, I attach it.... happy to try things further. Please confirm your btrfs/kernel version. I''m on 5c4e935f44e1aace28988fdc06f233ef5c184103 refs/heads/send-recv Cheers Rory -- Rory Campbell-Lange rory@campbell-lange.net Campbell-Lange Workshop www.campbell-lange.net 0207 6311 555 3 Tottenham Street London W1T 2AF Registered in England No. 04551928 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Rory, my kernel is slightly different, but the send-receive kernel code that I use is on the same commit. However, the issue you are hitting is most probably not a kernel issue. The receive part works totally in user-space. To confirm that, maybe you can post somewhere the full- and diff-streams that you generate: instead of piping to "btrfs receive", pls redirect it into some file (like /tmp/full_stream and /tmp/diff_stream) and post the files. Not sure how sensitive is the content of your files or how big the stream is - it might be not possible for you to do this. I suspect there is some issue with finding subvolume paths relative to your mount root. If possible, pls post "cat /proc/mounts" and also the output of "mount" command. Finally, it might be useful to post strace outputs of the failing receives. To do this, please redirect the stream into some file first, and then receive from this file (and not in one step like you are doing) under strace. Next step, I will probably send you a patch with some prints. Hope in a few iterations we can find the issue (unless you give up:). Thanks, Alex. On Thu, Oct 11, 2012 at 4:56 PM, Rory Campbell-Lange <rory@campbell-lange.net> wrote:> On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote: >> Hi Rory, >> all your tests work perfectly for me (really, I swear:)), with exact >> same mount points and subvol/snap/dir names. >> Just to make that the patch some how did not get messed up in the >> mail, I attach it. > > ... happy to try things further. Please confirm your btrfs/kernel > version. I''m on > > 5c4e935f44e1aace28988fdc06f233ef5c184103 refs/heads/send-recv > > Cheers > Rory > > -- > Rory Campbell-Lange > rory@campbell-lange.net > > Campbell-Lange Workshop > www.campbell-lange.net > 0207 6311 555 > 3 Tottenham Street London W1T 2AF > Registered in England No. 04551928-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Rory, can you pls check if you have a file "/etc/mtab" on your machine? If not or if it''s empty or something like, then this probably explains the issue. In this case, pls apply patch2 that is attached (instead of previous patches). Otherwise, pls try with patch1 (instead of previous) and then with patch2 (instead of any previous). Both will do some amount of prints, pls post them. With both patches your tests work fine for me. Thanks, Alex. On Thu, Oct 11, 2012 at 4:56 PM, Rory Campbell-Lange <rory@campbell-lange.net> wrote:> On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote: >> Hi Rory, >> all your tests work perfectly for me (really, I swear:)), with exact >> same mount points and subvol/snap/dir names. >> Just to make that the patch some how did not get messed up in the >> mail, I attach it. > > ... happy to try things further. Please confirm your btrfs/kernel > version. I''m on > > 5c4e935f44e1aace28988fdc06f233ef5c184103 refs/heads/send-recv > > Cheers > Rory > > -- > Rory Campbell-Lange > rory@campbell-lange.net > > Campbell-Lange Workshop > www.campbell-lange.net > 0207 6311 555 > 3 Tottenham Street London W1T 2AF > Registered in England No. 04551928
On 12/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:> Hi Rory, > can you pls check if you have a file "/etc/mtab" on your machine? If > not or if it''s empty or something like, then this probably explains > the issue. In this case, pls apply patch2 that is attached (instead of > previous patches). Otherwise, pls try with patch1 (instead of > previous) and then with patch2 (instead of any previous). Both will do > some amount of prints, pls post them. With both patches your tests > work fine for me.Hi Alex I reset everything and remade the btrfs partitions. I re-pulled git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git and applied your first patch. (in /bkp) mkdir snaps btrfs snapshot -r mail snaps/snap1 btrfs subvolume snapshot -r mail snaps/snap1 mutt -f mail/INBOX/ btrfs subvolume snapshot -r mail snaps/snap2 # targets (dir, subvol with same name, subvol with different name) mkdir /mnt/snaps btrfs subvolume create /mnt/mail btrfs subvolume create /mnt/snapmail # base backup btrfs send snaps/snap1 | btrfs receive /mnt/snaps btrfs send snaps/snap1 | btrfs receive /mnt/mail btrfs send snaps/snap1 | btrfs receive /mnt/snapmail # update snapshots btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/snaps btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/mail btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/snapmail All worked fine. Sorry for the previous errors, which look to be invalid. Cheers Rory /etc/mtab: rootfs / rootfs rw 0 0 sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,relatime,size=10240k,nr_inodes=749414,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=600748k,mode=755 0 0 /dev/disk/by-uuid/6e899e2d-c401-4a92-8adf-a097bcaf2148 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /run/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=1396860k 0 0 /dev/sda1 /boot ext4 rw,relatime,data=ordered 0 0 rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 /dev/sdb1 /mnt btrfs rw,relatime,space_cache 0 0 /dev/sda6 /bkp btrfs rw,relatime,space_cache 0 0 -- Rory Campbell-Lange rory@campbell-lange.net -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Rory, thanks for confirming. I will put together a more formal patch to the list. Thanks, Alex. On Fri, Oct 12, 2012 at 6:10 PM, Rory Campbell-Lange <rory@campbell-lange.net> wrote:> On 12/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote: >> Hi Rory, >> can you pls check if you have a file "/etc/mtab" on your machine? If >> not or if it''s empty or something like, then this probably explains >> the issue. In this case, pls apply patch2 that is attached (instead of >> previous patches). Otherwise, pls try with patch1 (instead of >> previous) and then with patch2 (instead of any previous). Both will do >> some amount of prints, pls post them. With both patches your tests >> work fine for me. > > Hi Alex > > I reset everything and remade the btrfs partitions. I re-pulled > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git > and applied your first patch. > > (in /bkp) > mkdir snaps > btrfs snapshot -r mail snaps/snap1 > btrfs subvolume snapshot -r mail snaps/snap1 > mutt -f mail/INBOX/ > btrfs subvolume snapshot -r mail snaps/snap2 > > # targets (dir, subvol with same name, subvol with different name) > mkdir /mnt/snaps > btrfs subvolume create /mnt/mail > btrfs subvolume create /mnt/snapmail > # base backup > btrfs send snaps/snap1 | btrfs receive /mnt/snaps > btrfs send snaps/snap1 | btrfs receive /mnt/mail > btrfs send snaps/snap1 | btrfs receive /mnt/snapmail > # update snapshots > btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/snaps > btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/mail > btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/snapmail > > All worked fine. Sorry for the previous errors, which look to be > invalid. > > Cheers > Rory > > > /etc/mtab: > > rootfs / rootfs rw 0 0 > sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 > proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 > udev /dev devtmpfs rw,relatime,size=10240k,nr_inodes=749414,mode=755 0 0 > devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 > tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=600748k,mode=755 0 0 > /dev/disk/by-uuid/6e899e2d-c401-4a92-8adf-a097bcaf2148 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0 > tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 > tmpfs /run/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=1396860k 0 0 > /dev/sda1 /boot ext4 rw,relatime,data=ordered 0 0 > rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 > /dev/sdb1 /mnt btrfs rw,relatime,space_cache 0 0 > /dev/sda6 /bkp btrfs rw,relatime,space_cache 0 0 > > -- > Rory Campbell-Lange > rory@campbell-lange.net-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html