Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 00/17] Btrfs-progs: some receive related patches
Most fixes are trivial. The one from Alex is fixing a real bug that several users have reported. Alex sent the patch half a year ago and it was not yet integrated. The patch "Use /proc/mounts instead of /etc/mtab" is a repost. The patch "btrfs-receive optionally honors the end-cmd" is a preparation step to allow backup tools to multiplex a single communication stream (e.g. a single TCP stream) to carry multiple Btrfs send/receive streams and a request/response handshake. It basically corrects the handling of the "end-cmd" in the send/receive stream. Without, only the closing of the stream (EOF) was terminating the receiver. The change is fully compatible as it is explained in more detail in the commit itself. Unfortunately the change to the sender is only compatible together with a kernel change, it is therefore not part of this patch set. Everything is also on git://btrfs.giantdisaster.de/git/btrfs-progs recv1 Alex Lyakas (1): btrfs-progs: Fix the receive code pathing Stefan Behrens (16): Btrfs-progs: Use /proc/mounts instead of /etc/mtab Btrfs-progs: ignore subvols above BTRFS_LAST_FREE_OBJECTID Btrfs-progs: close file descriptor in cmds-send.c Btrfs-progs: fix a small memory leak in btrfs-list.c Btrfs-progs: add a function to free subvol_uuid_search memory Btrfs-progs: cleanup subvol_uuid_search memory in btrfs send/receive Btrfs-progs: free memory and close file descriptor in btrfs receive Btrfs-progs: Set the root-id for received subvols in btrfs receive Btrfs-progs: btrfs-receive: different levels (amount) of debug output Btrfs-progs: small parent_subvol cleanup for cmds-receive.c Btrfs-progs: fix bug in find_root_gen Btrfs-progs: btrfs-receive optionally honors the end-cmd Btrfs-progs: don''t allocate one byte too much each time Btrfs-progs: Fix that BTRFS_FSID_SIZE is used instead of BTRFS_UUID_SIZE Btrfs-progs: remove some unused code Btrfs-progs: allow to receive to relative directories btrfs-list.c | 14 ++--- cmds-receive.c | 164 ++++++++++++++++++++++++++++++++++++++++++--------------- cmds-send.c | 12 ++++- send-stream.c | 9 ++-- send-stream.h | 3 +- send-utils.c | 49 +++++++++++++++-- send-utils.h | 2 + 7 files changed, 194 insertions(+), 59 deletions(-) -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 01/17] Btrfs-progs: Use /proc/mounts instead of /etc/mtab
/etc/mtab is not working correctly in situations where multiple mount namespaces are used. Use /proc/mounts instead like the rest of the code is doing it. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-send.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-send.c b/cmds-send.c index b2a340e..5a7183d 100644 --- a/cmds-send.c +++ b/cmds-send.c @@ -70,7 +70,7 @@ int find_mount_root(const char *path, char **mount_root) return -errno; close(fd); - mnttab = fopen("/etc/mtab", "r"); + mnttab = fopen("/proc/mounts", "r"); while ((ent = getmntent(mnttab))) { len = strlen(ent->mnt_dir); if (strncmp(ent->mnt_dir, path, len) == 0) { -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 02/17] Btrfs-progs: ignore subvols above BTRFS_LAST_FREE_OBJECTID
Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- send-utils.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/send-utils.c b/send-utils.c index d8d3972..b1d3873 100644 --- a/send-utils.c +++ b/send-utils.c @@ -226,7 +226,7 @@ int subvol_uuid_search_init(int mnt_fd, struct subvol_uuid_search *s) if ((sh->objectid != 5 && sh->objectid < BTRFS_FIRST_FREE_OBJECTID) || - sh->objectid == BTRFS_FREE_INO_OBJECTID) + sh->objectid > BTRFS_LAST_FREE_OBJECTID) goto skip; if (sh->type == BTRFS_ROOT_ITEM_KEY) { -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 03/17] Btrfs-progs: close file descriptor in cmds-send.c
valgrind found this very obvious issue. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-send.c | 1 + 1 file changed, 1 insertion(+) diff --git a/cmds-send.c b/cmds-send.c index 5a7183d..3ba5af6 100644 --- a/cmds-send.c +++ b/cmds-send.c @@ -82,6 +82,7 @@ int find_mount_root(const char *path, char **mount_root) } } } + fclose(mnttab); *mount_root = realpath(longest_match, NULL); free(longest_match); -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 04/17] Btrfs-progs: fix a small memory leak in btrfs-list.c
valgrind found this very obvious issue. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- btrfs-list.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/btrfs-list.c b/btrfs-list.c index a748d5e..38e7e53 100644 --- a/btrfs-list.c +++ b/btrfs-list.c @@ -577,13 +577,13 @@ static int resolve_root(struct root_lookup *rl, struct root_info *ri, add_len = strlen(found->path); - /* room for / and for null */ - tmp = malloc(add_len + 2 + len); - if (!tmp) { - perror("malloc failed"); - exit(1); - } if (full_path) { + /* room for / and for null */ + tmp = malloc(add_len + 2 + len); + if (!tmp) { + perror("malloc failed"); + exit(1); + } memcpy(tmp + add_len + 1, full_path, len); tmp[add_len] = ''/''; memcpy(tmp, found->path, add_len); -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 05/17] Btrfs-progs: add a function to free subvol_uuid_search memory
There was no way to free the memory that was used for the subvol_uuid_search functions. Since this is part of the libbtrfs, add such a cleanup function. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- send-utils.c | 23 +++++++++++++++++++++++ send-utils.h | 1 + 2 files changed, 24 insertions(+) diff --git a/send-utils.c b/send-utils.c index b1d3873..182778a 100644 --- a/send-utils.c +++ b/send-utils.c @@ -304,6 +304,29 @@ out: return ret; } +/* + * It''s safe to call this function even without the subvol_uuid_search_init() + * call before as long as the subvol_uuid_search structure is all-zero. + */ +void subvol_uuid_search_finit(struct subvol_uuid_search *s) +{ + struct rb_root *root = &s->root_id_subvols; + struct rb_node *node; + + while ((node = rb_first(root))) { + struct subvol_info *entry + rb_entry(node, struct subvol_info, rb_root_id_node); + + free(entry->path); + rb_erase(node, root); + free(entry); + } + + s->root_id_subvols = RB_ROOT; + s->local_subvols = RB_ROOT; + s->received_subvols = RB_ROOT; + s->path_subvols = RB_ROOT; +} char *path_cat(const char *p1, const char *p2) { diff --git a/send-utils.h b/send-utils.h index 199dd03..78abf94 100644 --- a/send-utils.h +++ b/send-utils.h @@ -62,6 +62,7 @@ struct subvol_uuid_search { }; int subvol_uuid_search_init(int mnt_fd, struct subvol_uuid_search *s); +void subvol_uuid_search_finit(struct subvol_uuid_search *s); struct subvol_info *subvol_uuid_search(struct subvol_uuid_search *s, u64 root_id, const u8 *uuid, u64 transid, const char *path, -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 06/17] Btrfs-progs: cleanup subvol_uuid_search memory in btrfs send/receive
Call the cleanup function that was introduced with the other commit. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-receive.c | 1 + cmds-send.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/cmds-receive.c b/cmds-receive.c index 6688d0c..fc5094e 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -824,6 +824,7 @@ int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd) ret = 0; out: + subvol_uuid_search_finit(&r->sus); return ret; } diff --git a/cmds-send.c b/cmds-send.c index 3ba5af6..fcde74c 100644 --- a/cmds-send.c +++ b/cmds-send.c @@ -458,6 +458,7 @@ int cmd_send_start(int argc, char **argv) goto out; } add_clone_source(&send, root_id); + subvol_uuid_search_finit(&send.sus); free(subvol); full_send = 0; break; @@ -630,6 +631,7 @@ int cmd_send_start(int argc, char **argv) out: if (send.mnt_fd >= 0) close(send.mnt_fd); + subvol_uuid_search_finit(&send.sus); return ret; } -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 07/17] Btrfs-progs: free memory and close file descriptor in btrfs receive
Nothing really important since this is not part of the library and at the end exit() is called. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-receive.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/cmds-receive.c b/cmds-receive.c index fc5094e..dc72e9e 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -151,6 +151,7 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid, r->parent_subvol = NULL; r->cur_subvol->path = strdup(path); + free(r->full_subvol_path); r->full_subvol_path = path_cat(r->root_path, path); fprintf(stderr, "At subvol %s\n", path); @@ -196,6 +197,7 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, r->parent_subvol = NULL; r->cur_subvol->path = strdup(path); + free(r->full_subvol_path); r->full_subvol_path = path_cat(r->root_path, path); fprintf(stderr, "At snapshot %s\n", path); @@ -803,9 +805,7 @@ int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd) ret = subvol_uuid_search_init(r->mnt_fd, &r->sus); if (ret < 0) - return ret; - - r->write_fd = -1; + goto out; while (!end) { ret = btrfs_read_and_process_send_stream(r_fd, &send_ops, r); @@ -824,7 +824,26 @@ int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd) ret = 0; out: + if (r->write_fd != -1) { + close(r->write_fd); + r->write_fd = -1; + } + free(r->root_path); + r->root_path = NULL; + free(r->write_path); + r->write_path = NULL; + free(r->full_subvol_path); + r->full_subvol_path = NULL; + if (r->cur_subvol) { + free(r->cur_subvol->path); + free(r->cur_subvol); + r->cur_subvol = NULL; + } subvol_uuid_search_finit(&r->sus); + if (r->mnt_fd != -1) { + close(r->mnt_fd); + r->mnt_fd = -1; + } return ret; } @@ -839,6 +858,8 @@ static int do_cmd_receive(int argc, char **argv) int ret; memset(&r, 0, sizeof(r)); + r.mnt_fd = -1; + r.write_fd = -1; while ((c = getopt(argc, argv, "vf:")) != -1) { switch (c) { -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 08/17] Btrfs-progs: Set the root-id for received subvols in btrfs receive
When an entry was added to the subvol search tree, the root_id was always 0 (not set at all) and therefore only the first one was added, all the others had been ignored. This commit adds a function to retrieve the root_id for a given fd, and this function is used to set the root_id before the entry is added. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-receive.c | 3 +++ send-utils.c | 20 ++++++++++++++++++++ send-utils.h | 1 + 3 files changed, 24 insertions(+) diff --git a/cmds-receive.c b/cmds-receive.c index dc72e9e..eedff13 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -125,6 +125,9 @@ static int finish_subvol(struct btrfs_receive *r) goto out; } + ret = btrfs_get_root_id(subvol_fd, &r->cur_subvol->root_id); + if (ret < 0) + goto out; subvol_uuid_search_add(&r->sus, r->cur_subvol); r->cur_subvol = NULL; ret = 0; diff --git a/send-utils.c b/send-utils.c index 182778a..b2d544c 100644 --- a/send-utils.c +++ b/send-utils.c @@ -23,6 +23,26 @@ #include "ioctl.h" #include "btrfs-list.h" +int btrfs_get_root_id(int fd, u64 *root_id) +{ + struct btrfs_ioctl_ino_lookup_args ino_args; + int ret; + + memset(&ino_args, 0, sizeof(ino_args)); + ino_args.objectid = BTRFS_FIRST_FREE_OBJECTID; + + /* this ioctl fills in ino_args->treeid */ + ret = ioctl(fd, BTRFS_IOC_INO_LOOKUP, &ino_args); + if (ret) { + fprintf(stderr, "ERROR: Failed to lookup root_id - %s\n", + strerror(errno)); + return ret; + } + + *root_id = ino_args.treeid; + return 0; +} + static struct rb_node *tree_insert(struct rb_root *root, struct subvol_info *si, enum subvol_search_type type) diff --git a/send-utils.h b/send-utils.h index 78abf94..3c8b7b7 100644 --- a/send-utils.h +++ b/send-utils.h @@ -61,6 +61,7 @@ struct subvol_uuid_search { struct rb_root path_subvols; }; +int btrfs_get_root_id(int fd, u64 *root_id); int subvol_uuid_search_init(int mnt_fd, struct subvol_uuid_search *s); void subvol_uuid_search_finit(struct subvol_uuid_search *s); struct subvol_info *subvol_uuid_search(struct subvol_uuid_search *s, -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 09/17] Btrfs-progs: btrfs-receive: different levels (amount) of debug output
There used to be 2 levels of verbose output, now there are 3: - None at all (no -v option given). - Some information about received snapshots / subvolumes (-v option). - Each received command is printed (-vv option). Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-receive.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/cmds-receive.c b/cmds-receive.c index eedff13..8afffe6 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -268,7 +268,7 @@ static int process_mkfile(const char *path, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "mkfile %s\n", path); ret = creat(full_path, 0600); @@ -292,7 +292,7 @@ static int process_mkdir(const char *path, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "mkdir %s\n", path); ret = mkdir(full_path, 0700); @@ -312,7 +312,7 @@ static int process_mknod(const char *path, u64 mode, u64 dev, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "mknod %s mode=%llu, dev=%llu\n", path, mode, dev); @@ -333,7 +333,7 @@ static int process_mkfifo(const char *path, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "mkfifo %s\n", path); ret = mkfifo(full_path, 0600); @@ -353,7 +353,7 @@ static int process_mksock(const char *path, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "mksock %s\n", path); ret = mknod(full_path, 0600 | S_IFSOCK, 0); @@ -373,7 +373,7 @@ static int process_symlink(const char *path, const char *lnk, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "symlink %s -> %s\n", path, lnk); ret = symlink(lnk, full_path); @@ -394,7 +394,7 @@ static int process_rename(const char *from, const char *to, void *user) char *full_from = path_cat(r->full_subvol_path, from); char *full_to = path_cat(r->full_subvol_path, to); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "rename %s -> %s\n", from, to); ret = rename(full_from, full_to); @@ -416,7 +416,7 @@ static int process_link(const char *path, const char *lnk, void *user) char *full_path = path_cat(r->full_subvol_path, path); char *full_link_path = path_cat(r->full_subvol_path, lnk); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "link %s -> %s\n", path, lnk); ret = link(full_link_path, full_path); @@ -438,7 +438,7 @@ static int process_unlink(const char *path, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "unlink %s\n", path); ret = unlink(full_path); @@ -458,7 +458,7 @@ static int process_rmdir(const char *path, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "rmdir %s\n", path); ret = rmdir(full_path); @@ -631,7 +631,7 @@ static int process_set_xattr(const char *path, const char *name, struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) { + if (g_verbose >= 2) { fprintf(stderr, "set_xattr %s - name=%s data_len=%d " "data=%.*s\n", path, name, len, len, (char*)data); @@ -656,7 +656,7 @@ static int process_remove_xattr(const char *path, const char *name, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) { + if (g_verbose >= 2) { fprintf(stderr, "remove_xattr %s - name=%s\n", path, name); } @@ -680,7 +680,7 @@ static int process_truncate(const char *path, u64 size, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "truncate %s size=%llu\n", path, size); ret = truncate(full_path, size); @@ -702,7 +702,7 @@ static int process_chmod(const char *path, u64 mode, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "chmod %s - mode=0%o\n", path, (int)mode); ret = chmod(full_path, mode); @@ -724,7 +724,7 @@ static int process_chown(const char *path, u64 uid, u64 gid, void *user) struct btrfs_receive *r = user; char *full_path = path_cat(r->full_subvol_path, path); - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "chown %s - uid=%llu, gid=%llu\n", path, uid, gid); @@ -750,7 +750,7 @@ static int process_utimes(const char *path, struct timespec *at, char *full_path = path_cat(r->full_subvol_path, path); struct timespec tv[2]; - if (g_verbose >= 1) + if (g_verbose >= 2) fprintf(stderr, "utimes %s\n", path); tv[0] = *at; -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 10/17] Btrfs-progs: small parent_subvol cleanup for cmds-receive.c
parent_subvol is local to process_snapshot() and not needed outside. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-receive.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/cmds-receive.c b/cmds-receive.c index 8afffe6..081665f 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -63,7 +63,6 @@ struct btrfs_receive char *full_subvol_path; struct subvol_info *cur_subvol; - struct subvol_info *parent_subvol; struct subvol_uuid_search sus; }; @@ -151,7 +150,6 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid, goto out; r->cur_subvol = calloc(1, sizeof(*r->cur_subvol)); - r->parent_subvol = NULL; r->cur_subvol->path = strdup(path); free(r->full_subvol_path); @@ -191,13 +189,13 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, struct btrfs_receive *r = user; char uuid_str[128]; struct btrfs_ioctl_vol_args_v2 args_v2; + struct subvol_info *parent_subvol; ret = finish_subvol(r); if (ret < 0) goto out; r->cur_subvol = calloc(1, sizeof(*r->cur_subvol)); - r->parent_subvol = NULL; r->cur_subvol->path = strdup(path); free(r->full_subvol_path); @@ -221,9 +219,9 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, memset(&args_v2, 0, sizeof(args_v2)); strncpy_null(args_v2.name, path); - r->parent_subvol = subvol_uuid_search(&r->sus, 0, parent_uuid, + parent_subvol = subvol_uuid_search(&r->sus, 0, parent_uuid, parent_ctransid, NULL, subvol_search_by_received_uuid); - if (!r->parent_subvol) { + if (!parent_subvol) { ret = -ENOENT; fprintf(stderr, "ERROR: could not find parent subvolume\n"); goto out; @@ -239,12 +237,12 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, } }*/ - args_v2.fd = openat(r->mnt_fd, r->parent_subvol->path, + args_v2.fd = openat(r->mnt_fd, parent_subvol->path, O_RDONLY | O_NOATIME); if (args_v2.fd < 0) { ret = -errno; fprintf(stderr, "ERROR: open %s failed. %s\n", - r->parent_subvol->path, strerror(-ret)); + parent_subvol->path, strerror(-ret)); goto out; } @@ -253,7 +251,7 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, if (ret < 0) { ret = -errno; fprintf(stderr, "ERROR: creating snapshot %s -> %s " - "failed. %s\n", r->parent_subvol->path, + "failed. %s\n", parent_subvol->path, path, strerror(-ret)); goto out; } -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
A copy & paste error. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- btrfs-list.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/btrfs-list.c b/btrfs-list.c index 38e7e53..1246a25 100644 --- a/btrfs-list.c +++ b/btrfs-list.c @@ -775,7 +775,7 @@ static u64 find_root_gen(int fd) if (sk->min_type != BTRFS_ROOT_ITEM_KEY) break; - if (sk->min_objectid != BTRFS_ROOT_ITEM_KEY) + if (sk->min_objectid != ino_args.treeid) break; } return max_found; -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 12/17] Btrfs-progs: btrfs-receive optionally honors the end-cmd
A new option is added to btrfs-receive to change the behavior when an <end cmd> is received in the Btrfs send stream. The traditional behavior (which still is the default) is to continue to read the stream until an EOF condition is encountered. If an <end cmd> is received, afterwards either an EOF or a new <stream header> is expected. The new behavior (if the -e option is set on the command line) is to terminate after an <end cmd> is read without the need for an EOF. This allows the stream (e.g. a single TCP stream) to carry additional data or even multiple Btrfs send streams. Old btrfs-send tools used to encode multiple snapshots like this (with 2 snapshots in this example): <stream header> + <sequence of commands> + <end cmd> + <stream header> + <sequence of commands> + <end cmd> + EOF If the new -e option is set, the expected format is like this: <stream header> + <sequence of commands> + <sequence of commands> + <end cmd> The btrfs-send tool is changed in a seperate commit to always use the new format, i.e. to send an <end cmd> only at the end. Note that the currently existing receivers treat <end cmd> only as an indication that a new <stream header> is following. This means, you can just skip the sequence <end cmd> <stream header> without loosing compatibility. As long as an EOF is following, the currently existing receivers handle the new format (if the two new flags are used) exactly as the old one. The goal of changing the semantic of <end cmd> is to be able to use a single stream (one TCP connection) to multiplex a request/response handshake plus Btrfs send streams, all in the same stream. In this case you cannot evaluate an EOF condition as an end of the Btrfs send stream. You need something else, and the <end cmd> is just perfect for this purpose. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-receive.c | 16 +++++++++++++--- send-stream.c | 6 ++++-- send-stream.h | 3 ++- 3 files changed, 19 insertions(+), 6 deletions(-) diff --git a/cmds-receive.c b/cmds-receive.c index 081665f..50c1b97 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -65,6 +65,8 @@ struct btrfs_receive struct subvol_info *cur_subvol; struct subvol_uuid_search sus; + + int honor_end_cmd; }; static int finish_subvol(struct btrfs_receive *r) @@ -809,7 +811,8 @@ int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd) goto out; while (!end) { - ret = btrfs_read_and_process_send_stream(r_fd, &send_ops, r); + ret = btrfs_read_and_process_send_stream(r_fd, &send_ops, r, + r->honor_end_cmd); if (ret < 0) goto out; if (ret) @@ -862,7 +865,7 @@ static int do_cmd_receive(int argc, char **argv) r.mnt_fd = -1; r.write_fd = -1; - while ((c = getopt(argc, argv, "vf:")) != -1) { + while ((c = getopt(argc, argv, "evf:")) != -1) { switch (c) { case ''v'': g_verbose++; @@ -870,6 +873,9 @@ static int do_cmd_receive(int argc, char **argv) case ''f'': fromfile = optarg; break; + case ''e'': + r.honor_end_cmd = 1; + break; case ''?'': default: fprintf(stderr, "ERROR: receive args invalid.\n"); @@ -903,7 +909,7 @@ static const char * const receive_cmd_group_usage[] = { }; const char * const cmd_receive_usage[] = { - "btrfs receive [-v] [-f <infile>] <mount>", + "btrfs receive [-ve] [-f <infile>] <mount>", "Receive subvolumes from stdin.", "Receives one or more subvolumes that were previously ", "sent with btrfs send. The received subvolumes are stored", @@ -919,6 +925,10 @@ const char * const cmd_receive_usage[] = { "-f <infile> By default, btrfs receive uses stdin", " to receive the subvolumes. Use this", " option to specify a file to use instead.", + "-e Terminate after receiving an <end cmd>", + " in the data stream. Without this option,", + " the receiver terminates only if an error", + " is recognized or on EOF.", NULL }; diff --git a/send-stream.c b/send-stream.c index a3628e4..1a498f3 100644 --- a/send-stream.c +++ b/send-stream.c @@ -439,7 +439,8 @@ out: } int btrfs_read_and_process_send_stream(int fd, - struct btrfs_send_ops *ops, void *user) + struct btrfs_send_ops *ops, void *user, + int honor_end_cmd) { int ret; struct btrfs_send_stream s; @@ -476,7 +477,8 @@ int btrfs_read_and_process_send_stream(int fd, if (ret < 0) goto out; if (ret) { - ret = 0; + if (!honor_end_cmd) + ret = 0; goto out; } } diff --git a/send-stream.h b/send-stream.h index 9223018..17bc669 100644 --- a/send-stream.h +++ b/send-stream.h @@ -57,7 +57,8 @@ struct btrfs_send_ops { }; int btrfs_read_and_process_send_stream(int fd, - struct btrfs_send_ops *ops, void *user); + struct btrfs_send_ops *ops, void *user, + int honor_end_cmd); #ifdef __cplusplus } -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 13/17] Btrfs-progs: don''t allocate one byte too much each time
str1 + ''/'' + str2 + ''\0'' requires a buffer with the size strlen(str1) + strlen(str2) + 2 bytes. str1 + ''/'' + str2 + ''/'' + str3 + ''\0'' requires a buffer with the size strlen(str1) + strlen(str2) + strlen(str3) + 3 bytes. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- send-utils.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/send-utils.c b/send-utils.c index b2d544c..aff62ad 100644 --- a/send-utils.c +++ b/send-utils.c @@ -352,7 +352,7 @@ char *path_cat(const char *p1, const char *p2) { int p1_len = strlen(p1); int p2_len = strlen(p2); - char *new = malloc(p1_len + p2_len + 3); + char *new = malloc(p1_len + p2_len + 2); if (p1_len && p1[p1_len - 1] == ''/'') p1_len--; @@ -368,7 +368,7 @@ char *path_cat3(const char *p1, const char *p2, const char *p3) int p1_len = strlen(p1); int p2_len = strlen(p2); int p3_len = strlen(p3); - char *new = malloc(p1_len + p2_len + p3_len + 4); + char *new = malloc(p1_len + p2_len + p3_len + 3); if (p1_len && p1[p1_len - 1] == ''/'') p1_len--; -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 14/17] Btrfs-progs: Fix that BTRFS_FSID_SIZE is used instead of BTRFS_UUID_SIZE
Both are 16 but it''s wrong anyway to use FSID_SIZE for UUIDs. Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-receive.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-receive.c b/cmds-receive.c index 50c1b97..f937366 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -565,7 +565,7 @@ static int process_clone(const char *path, u64 offset, u64 len, subvol_search_by_received_uuid); if (!si) { if (memcmp(clone_uuid, r->cur_subvol->received_uuid, - BTRFS_FSID_SIZE) == 0) { + BTRFS_UUID_SIZE) == 0) { /* TODO check generation of extent */ subvol_path = strdup(r->cur_subvol->path); } else { -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- send-stream.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/send-stream.c b/send-stream.c index 1a498f3..88e18e2 100644 --- a/send-stream.c +++ b/send-stream.c @@ -410,9 +410,6 @@ static int read_and_process_cmd(struct btrfs_send_stream *s) break; case BTRFS_SEND_C_UTIMES: TLV_GET_STRING(s, BTRFS_SEND_A_PATH, &path); - if (strstr(path, ".bak_1.log")) { - ret = 0; - } TLV_GET_TIMESPEC(s, BTRFS_SEND_A_ATIME, &at); TLV_GET_TIMESPEC(s, BTRFS_SEND_A_MTIME, &mt); TLV_GET_TIMESPEC(s, BTRFS_SEND_A_CTIME, &ct); -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 16/17] Btrfs-progs: allow to receive to relative directories
Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-receive.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-receive.c b/cmds-receive.c index f937366..e5467d2 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -755,7 +755,7 @@ static int process_utimes(const char *path, struct timespec *at, tv[0] = *at; tv[1] = *mt; - ret = utimensat(-1, full_path, tv, AT_SYMLINK_NOFOLLOW); + ret = utimensat(AT_FDCWD, full_path, tv, AT_SYMLINK_NOFOLLOW); if (ret < 0) { ret = -errno; fprintf(stderr, "ERROR: utimes %s failed. %s\n", -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Stefan Behrens
2013-Apr-09 17:08 UTC
[PATCH 17/17] btrfs-progs: Fix the receive code pathing
From: Alex Lyakas <alex.btrfs@zadarastorage.com> The receive code was not distinguishing properly between the mount root and the directory to create the received subvolume in. Also make sure the find_mount_root reports an error if it cannot find a match at all. Reported-by: Robert Buhren <robert@robertbuhren.de> Reported-by: Rory Campbell-Lange <rory@campbell-lange.net> Reported-by: Stefan Priebe - Profihost AG <s.priebe@profihost.ag> Signed-off-by: Alex Lyakas <alex.btrfs@zadarastorage.com> Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> --- cmds-receive.c | 67 +++++++++++++++++++++++++++++++++++++++++++++++++--------- cmds-send.c | 7 ++++++ 2 files changed, 64 insertions(+), 10 deletions(-) diff --git a/cmds-receive.c b/cmds-receive.c index e5467d2..04704df 100644 --- a/cmds-receive.c +++ b/cmds-receive.c @@ -55,11 +55,13 @@ static int g_verbose = 0; struct btrfs_receive { int mnt_fd; + int dest_dir_fd; int write_fd; char *write_path; char *root_path; + char *dest_dir_path; /* relative to root_path */ char *full_subvol_path; struct subvol_info *cur_subvol; @@ -153,9 +155,12 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid, r->cur_subvol = calloc(1, sizeof(*r->cur_subvol)); - r->cur_subvol->path = strdup(path); + if (strlen(r->dest_dir_path) == 0) + r->cur_subvol->path = strdup(path); + else + r->cur_subvol->path = path_cat(r->dest_dir_path, path); free(r->full_subvol_path); - r->full_subvol_path = path_cat(r->root_path, path); + r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path); fprintf(stderr, "At subvol %s\n", path); @@ -171,7 +176,7 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid, memset(&args_v1, 0, sizeof(args_v1)); strncpy_null(args_v1.name, path); - ret = ioctl(r->mnt_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1); + ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1); if (ret < 0) { ret = -errno; fprintf(stderr, "ERROR: creating subvolume %s failed. " @@ -199,9 +204,12 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, r->cur_subvol = calloc(1, sizeof(*r->cur_subvol)); - r->cur_subvol->path = strdup(path); + if (strlen(r->dest_dir_path) == 0) + r->cur_subvol->path = strdup(path); + else + r->cur_subvol->path = path_cat(r->dest_dir_path, path); free(r->full_subvol_path); - r->full_subvol_path = path_cat(r->root_path, path); + r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path); fprintf(stderr, "At snapshot %s\n", path); @@ -248,7 +256,7 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, goto out; } - ret = ioctl(r->mnt_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2); + ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2); close(args_v2.fd); if (ret < 0) { ret = -errno; @@ -795,17 +803,49 @@ struct btrfs_send_ops send_ops = { int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd) { int ret; + char *dest_dir_full_path; int end = 0; - r->root_path = strdup(tomnt); - r->mnt_fd = open(tomnt, O_RDONLY | O_NOATIME); + dest_dir_full_path = realpath(tomnt, NULL); + if (!dest_dir_full_path) { + ret = -errno; + fprintf(stderr, "ERROR: realpath(%s) failed. %s\n", tomnt, + strerror(-ret)); + goto out; + } + r->dest_dir_fd = open(dest_dir_full_path, O_RDONLY | O_NOATIME); + if (r->dest_dir_fd < 0) { + ret = -errno; + fprintf(stderr, + "ERROR: failed to open destination directory %s. %s\n", + dest_dir_full_path, strerror(-ret)); + goto out; + } + + ret = find_mount_root(dest_dir_full_path, &r->root_path); + if (ret < 0) { + ret = -EINVAL; + fprintf(stderr, "ERROR: failed to determine mount point " + "for %s\n", dest_dir_full_path); + goto out; + } + r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME); if (r->mnt_fd < 0) { ret = -errno; - fprintf(stderr, "ERROR: failed to open %s. %s\n", tomnt, - strerror(-ret)); + fprintf(stderr, "ERROR: failed to open %s. %s\n", r->root_path, + strerror(-ret)); goto out; } + /* + * find_mount_root returns a root_path that is a subpath of + * dest_dir_full_path. Now get the other part of root_path, + * which is the destination dir relative to root_path. + */ + r->dest_dir_path = dest_dir_full_path + strlen(r->root_path); + while (r->dest_dir_path[0] == ''/'') + r->dest_dir_path++; + ret = subvol_uuid_search_init(r->mnt_fd, &r->sus); if (ret < 0) goto out; @@ -838,6 +878,8 @@ out: r->write_path = NULL; free(r->full_subvol_path); r->full_subvol_path = NULL; + r->dest_dir_path = NULL; + free(dest_dir_full_path); if (r->cur_subvol) { free(r->cur_subvol->path); free(r->cur_subvol); @@ -848,6 +890,10 @@ out: close(r->mnt_fd); r->mnt_fd = -1; } + if (r->dest_dir_fd != -1) { + close(r->dest_dir_fd); + r->dest_dir_fd = -1; + } return ret; } @@ -864,6 +910,7 @@ static int do_cmd_receive(int argc, char **argv) memset(&r, 0, sizeof(r)); r.mnt_fd = -1; r.write_fd = -1; + r.dest_dir_fd = -1; while ((c = getopt(argc, argv, "evf:")) != -1) { switch (c) { diff --git a/cmds-send.c b/cmds-send.c index fcde74c..9bb4206 100644 --- a/cmds-send.c +++ b/cmds-send.c @@ -84,6 +84,13 @@ int find_mount_root(const char *path, char **mount_root) } fclose(mnttab); + if (!longest_match) { + fprintf(stderr, + "ERROR: Failed to find mount root for path %s.\n", + path); + return -ENOENT; + } + *mount_root = realpath(longest_match, NULL); free(longest_match); -- 1.8.2.1 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Wang Shilong
2013-Apr-10 01:04 UTC
Re: [PATCH 08/17] Btrfs-progs: Set the root-id for received subvols in btrfs receive
Hello,> When an entry was added to the subvol search tree, the root_id was > always 0 (not set at all) and therefore only the first one was > added, all the others had been ignored. This commit adds a function > to retrieve the root_id for a given fd, and this function is used > to set the root_id before the entry is added. > > Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de> > --- > cmds-receive.c | 3 +++ > send-utils.c | 20 ++++++++++++++++++++ > send-utils.h | 1 + > 3 files changed, 24 insertions(+) > > diff --git a/cmds-receive.c b/cmds-receive.c > index dc72e9e..eedff13 100644 > --- a/cmds-receive.c > +++ b/cmds-receive.c > @@ -125,6 +125,9 @@ static int finish_subvol(struct btrfs_receive *r) > goto out; > } > > + ret = btrfs_get_root_id(subvol_fd, &r->cur_subvol->root_id);There has been a function in btrfs-list.c named ''btrfs_list_get_path_rootid'' which does the same things as ''btrfs_get_root_id''.I think you can reuse it. Thanks, Wang> + if (ret < 0) > + goto out; > subvol_uuid_search_add(&r->sus, r->cur_subvol); > r->cur_subvol = NULL; > ret = 0; > diff --git a/send-utils.c b/send-utils.c > index 182778a..b2d544c 100644 > --- a/send-utils.c > +++ b/send-utils.c > @@ -23,6 +23,26 @@ > #include "ioctl.h" > #include "btrfs-list.h" > > +int btrfs_get_root_id(int fd, u64 *root_id) > +{ > + struct btrfs_ioctl_ino_lookup_args ino_args; > + int ret; > + > + memset(&ino_args, 0, sizeof(ino_args)); > + ino_args.objectid = BTRFS_FIRST_FREE_OBJECTID; > + > + /* this ioctl fills in ino_args->treeid */ > + ret = ioctl(fd, BTRFS_IOC_INO_LOOKUP, &ino_args); > + if (ret) { > + fprintf(stderr, "ERROR: Failed to lookup root_id - %s\n", > + strerror(errno)); > + return ret; > + } > + > + *root_id = ino_args.treeid; > + return 0; > +} > + > static struct rb_node *tree_insert(struct rb_root *root, > struct subvol_info *si, > enum subvol_search_type type) > diff --git a/send-utils.h b/send-utils.h > index 78abf94..3c8b7b7 100644 > --- a/send-utils.h > +++ b/send-utils.h > @@ -61,6 +61,7 @@ struct subvol_uuid_search { > struct rb_root path_subvols; > }; > > +int btrfs_get_root_id(int fd, u64 *root_id); > int subvol_uuid_search_init(int mnt_fd, struct subvol_uuid_search *s); > void subvol_uuid_search_finit(struct subvol_uuid_search *s); > struct subvol_info *subvol_uuid_search(struct subvol_uuid_search *s,-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
David Sterba
2013-Apr-16 12:33 UTC
Re: [PATCH 00/17] Btrfs-progs: some receive related patches
Hi, On Tue, Apr 09, 2013 at 07:08:28PM +0200, Stefan Behrens wrote:> Alex Lyakas (1): > btrfs-progs: Fix the receive code pathing > > Stefan Behrens (16): > Btrfs-progs: Use /proc/mounts instead of /etc/mtab > Btrfs-progs: ignore subvols above BTRFS_LAST_FREE_OBJECTID > Btrfs-progs: close file descriptor in cmds-send.c > Btrfs-progs: fix a small memory leak in btrfs-list.c > Btrfs-progs: add a function to free subvol_uuid_search memory > Btrfs-progs: cleanup subvol_uuid_search memory in btrfs send/receive > Btrfs-progs: free memory and close file descriptor in btrfs receive > Btrfs-progs: Set the root-id for received subvols in btrfs receive > Btrfs-progs: btrfs-receive: different levels (amount) of debug output > Btrfs-progs: small parent_subvol cleanup for cmds-receive.c > Btrfs-progs: fix bug in find_root_gen > Btrfs-progs: btrfs-receive optionally honors the end-cmd > Btrfs-progs: don''t allocate one byte too much each time > Btrfs-progs: Fix that BTRFS_FSID_SIZE is used instead of > BTRFS_UUID_SIZE > Btrfs-progs: remove some unused code > Btrfs-progs: allow to receive to relative directoriesAll merged, thanks. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Possibly Parallel Threads
- btrfs-progs: re-add send-test
- [PATCH] Btrfs-progs: receive: fix the case that we can not find subvolume
- [PATCH v2] Btrfs-progs: receive: fix the case that we can not find the subvolume
- [RFC] Improve btrfs subvolume find-new command
- [PATCH] Btrfs-progs: add send option for using new end-cmd semantic