Eric Blake
2021-Jun-10 15:03 UTC
[Libguestfs] [libnbd PATCH] info: Avoid ambiguous 'allocated' terminology in mapping
Qemu has unfortunately used two distinct meanings for 'allocated' over
time, and that ambiguity has overloaded the word to make it difficult
to know which sense is meant in a given context. In one usage,
'allocated' implies that data is present and occupies space on a block
device (the opposite of sparse). In another usage, 'allocated'
implies that guest contents come from the current layer of a qcow2
backing chain (even if via a sparse zero cluster), rather than
deferring to the backing file. It doesn't help that in qemu source
code, BDRV_BLOCK_ALLOCATED refers solely to this latter usage, while
a preallocated qcow2 image refers to the former usage.
But we can help by using better terminology in nbdinfo output. With
base:allocation, either a cluster is known to be sparse (a hole), or
it (likely) occupies space (remember, NBD block status bits are
advisory - if they are set to 1, we know a given property is true; but
if they are left 0, we cannot assume the opposite of the property).
Instead of calling the latter allocated, we can call it merely "data".
And this maps a bit better to 'qemu-img map --output=json' output; a
sparse section really is "data":false, and everything that is not
known to be sparse is "data":true.
Similarly, in qemu:allocation-depth, either a cluster is known to come
from the backing chain (either locally, or from a given backing
depth), or it is completely absent from the chain (not allocated in
any of the layers). Instead of calling it "unallocated", we can call
it "absent".
Adjust the testsuite to match our improved terminology.
Thanks: Nir Soffer
---
info/info-map-base-allocation-json.sh | 4 ++--
info/info-map-base-allocation-large.sh | 8 ++++----
info/info-map-base-allocation-weird.sh | 4 ++--
info/info-map-base-allocation.sh | 6 +++---
info/info-map-qemu-allocation-depth.sh | 4 ++--
info/map.c | 4 ++--
6 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/info/info-map-base-allocation-json.sh
b/info/info-map-base-allocation-json.sh
index 961eac6..b960295 100755
--- a/info/info-map-base-allocation-json.sh
+++ b/info/info-map-base-allocation-json.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# nbd client library in userspace
-# Copyright (C) 2020 Red Hat Inc.
+# Copyright (C) 2020-2021 Red Hat Inc.
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
@@ -40,7 +40,7 @@ jq . < $out
test $( jq -r '.[0].offset' < $out ) -eq 0
test $( jq -r '.[0].length' < $out ) -eq 32768
test $( jq -r '.[0].type' < $out ) -eq 0
-test $( jq -r '.[0].description' < $out ) = "allocated"
+test $( jq -r '.[0].description' < $out ) = "data"
test $( jq -r '.[3].offset' < $out ) -eq 163840
test $( jq -r '.[3].length' < $out ) -eq 884736
diff --git a/info/info-map-base-allocation-large.sh
b/info/info-map-base-allocation-large.sh
index e7ff904..ab10dc7 100755
--- a/info/info-map-base-allocation-large.sh
+++ b/info/info-map-base-allocation-large.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# nbd client library in userspace
-# Copyright (C) 2020 Red Hat Inc.
+# Copyright (C) 2020-2021 Red Hat Inc.
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
@@ -36,11 +36,11 @@ nbdkit -U - data data='1 @131072 2 @6442450944 3'
size=8G \
cat $out
-if [ "$(tr -s ' ' < $out)" != " 0 32768 0 allocated
+if [ "$(tr -s ' ' < $out)" != " 0 32768 0 data
32768 98304 3 hole,zero
- 131072 32768 0 allocated
+ 131072 32768 0 data
163840 6442287104 3 hole,zero
-6442450944 32768 0 allocated
+6442450944 32768 0 data
6442483712 2147450880 3 hole,zero" ]; then
echo "$0: unexpected output from nbdinfo --map"
exit 1
diff --git a/info/info-map-base-allocation-weird.sh
b/info/info-map-base-allocation-weird.sh
index 20ddec6..0b0343a 100755
--- a/info/info-map-base-allocation-weird.sh
+++ b/info/info-map-base-allocation-weird.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# nbd client library in userspace
-# Copyright (C) 2020 Red Hat Inc.
+# Copyright (C) 2020-2021 Red Hat Inc.
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
@@ -45,7 +45,7 @@ EOF
cat $out
if [ "$(tr -s ' ' < $out)" != " 0 16 1 hole
- 16 4 0 allocated
+ 16 4 0 data
20 12 1 hole" ]; then
echo "$0: unexpected output from nbdinfo --map"
exit 1
diff --git a/info/info-map-base-allocation.sh b/info/info-map-base-allocation.sh
index 9b85c97..76c2024 100755
--- a/info/info-map-base-allocation.sh
+++ b/info/info-map-base-allocation.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# nbd client library in userspace
-# Copyright (C) 2020 Red Hat Inc.
+# Copyright (C) 2020-2021 Red Hat Inc.
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
@@ -36,9 +36,9 @@ nbdkit -U - data data='1 @131072 2' size=1M \
cat $out
-if [ "$(tr -s ' ' < $out)" != " 0 32768 0 allocated
+if [ "$(tr -s ' ' < $out)" != " 0 32768 0 data
32768 98304 3 hole,zero
- 131072 32768 0 allocated
+ 131072 32768 0 data
163840 884736 3 hole,zero" ]; then
echo "$0: unexpected output from nbdinfo --map"
exit 1
diff --git a/info/info-map-qemu-allocation-depth.sh
b/info/info-map-qemu-allocation-depth.sh
index a5dab0b..9aeffd1 100755
--- a/info/info-map-qemu-allocation-depth.sh
+++ b/info/info-map-qemu-allocation-depth.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# nbd client library in userspace
-# Copyright (C) 2020 Red Hat Inc.
+# Copyright (C) 2020-2021 Red Hat Inc.
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
@@ -84,7 +84,7 @@ if [ "$(tr -s ' ' < $out)" != "\
3145728 2097152 3 backing depth 3
5242880 1048576 1 local
6291456 1048576 2 backing depth 2
- 7340032 1048576 0 unallocated" ]; then
+ 7340032 1048576 0 absent" ]; then
echo "$0: unexpected output from nbdinfo --map"
exit 1
fi
diff --git a/info/map.c b/info/map.c
index ae6d4fe..82c9507 100644
--- a/info/map.c
+++ b/info/map.c
@@ -202,7 +202,7 @@ extent_description (const char *metacontext, uint32_t type)
if (strcmp (metacontext, "base:allocation") == 0) {
switch (type) {
- case 0: return strdup ("allocated");
+ case 0: return strdup ("data");
case 1: return strdup ("hole");
case 2: return strdup ("zero");
case 3: return strdup ("hole,zero");
@@ -216,7 +216,7 @@ extent_description (const char *metacontext, uint32_t type)
}
else if (strcmp (metacontext, "qemu:allocation-depth") == 0) {
switch (type) {
- case 0: return strdup ("unallocated");
+ case 0: return strdup ("absent");
case 1: return strdup ("local");
default:
if (asprintf (&ret, "backing depth %u", type) == -1) {
--
2.31.1
Nir Soffer
2021-Jun-10 15:15 UTC
[Libguestfs] [libnbd PATCH] info: Avoid ambiguous 'allocated' terminology in mapping
On Thu, Jun 10, 2021 at 6:03 PM Eric Blake <eblake at redhat.com> wrote:> > Qemu has unfortunately used two distinct meanings for 'allocated' over > time, and that ambiguity has overloaded the word to make it difficult > to know which sense is meant in a given context. In one usage, > 'allocated' implies that data is present and occupies space on a block > device (the opposite of sparse). In another usage, 'allocated' > implies that guest contents come from the current layer of a qcow2 > backing chain (even if via a sparse zero cluster), rather than > deferring to the backing file. It doesn't help that in qemu source > code, BDRV_BLOCK_ALLOCATED refers solely to this latter usage, while > a preallocated qcow2 image refers to the former usage. > > But we can help by using better terminology in nbdinfo output. With > base:allocation, either a cluster is known to be sparse (a hole), or > it (likely) occupies space (remember, NBD block status bits are > advisory - if they are set to 1, we know a given property is true; but > if they are left 0, we cannot assume the opposite of the property). > Instead of calling the latter allocated, we can call it merely "data". > And this maps a bit better to 'qemu-img map --output=json' output; a > sparse section really is "data":false, and everything that is not > known to be sparse is "data":true. > > Similarly, in qemu:allocation-depth, either a cluster is known to come > from the backing chain (either locally, or from a given backing > depth), or it is completely absent from the chain (not allocated in > any of the layers). Instead of calling it "unallocated", we can call > it "absent". > > Adjust the testsuite to match our improved terminology. > > Thanks: Nir Soffer > --- > info/info-map-base-allocation-json.sh | 4 ++-- > info/info-map-base-allocation-large.sh | 8 ++++---- > info/info-map-base-allocation-weird.sh | 4 ++-- > info/info-map-base-allocation.sh | 6 +++--- > info/info-map-qemu-allocation-depth.sh | 4 ++-- > info/map.c | 4 ++-- > 6 files changed, 15 insertions(+), 15 deletions(-) > > diff --git a/info/info-map-base-allocation-json.sh b/info/info-map-base-allocation-json.sh > index 961eac6..b960295 100755 > --- a/info/info-map-base-allocation-json.sh > +++ b/info/info-map-base-allocation-json.sh > @@ -1,6 +1,6 @@ > #!/usr/bin/env bash > # nbd client library in userspace > -# Copyright (C) 2020 Red Hat Inc. > +# Copyright (C) 2020-2021 Red Hat Inc. > # > # This library is free software; you can redistribute it and/or > # modify it under the terms of the GNU Lesser General Public > @@ -40,7 +40,7 @@ jq . < $out > test $( jq -r '.[0].offset' < $out ) -eq 0 > test $( jq -r '.[0].length' < $out ) -eq 32768 > test $( jq -r '.[0].type' < $out ) -eq 0 > -test $( jq -r '.[0].description' < $out ) = "allocated" > +test $( jq -r '.[0].description' < $out ) = "data" > > test $( jq -r '.[3].offset' < $out ) -eq 163840 > test $( jq -r '.[3].length' < $out ) -eq 884736 > diff --git a/info/info-map-base-allocation-large.sh b/info/info-map-base-allocation-large.sh > index e7ff904..ab10dc7 100755 > --- a/info/info-map-base-allocation-large.sh > +++ b/info/info-map-base-allocation-large.sh > @@ -1,6 +1,6 @@ > #!/usr/bin/env bash > # nbd client library in userspace > -# Copyright (C) 2020 Red Hat Inc. > +# Copyright (C) 2020-2021 Red Hat Inc. > # > # This library is free software; you can redistribute it and/or > # modify it under the terms of the GNU Lesser General Public > @@ -36,11 +36,11 @@ nbdkit -U - data data='1 @131072 2 @6442450944 3' size=8G \ > > cat $out > > -if [ "$(tr -s ' ' < $out)" != " 0 32768 0 allocated > +if [ "$(tr -s ' ' < $out)" != " 0 32768 0 data > 32768 98304 3 hole,zero > - 131072 32768 0 allocated > + 131072 32768 0 data > 163840 6442287104 3 hole,zero > -6442450944 32768 0 allocated > +6442450944 32768 0 data > 6442483712 2147450880 3 hole,zero" ]; then > echo "$0: unexpected output from nbdinfo --map" > exit 1 > diff --git a/info/info-map-base-allocation-weird.sh b/info/info-map-base-allocation-weird.sh > index 20ddec6..0b0343a 100755 > --- a/info/info-map-base-allocation-weird.sh > +++ b/info/info-map-base-allocation-weird.sh > @@ -1,6 +1,6 @@ > #!/usr/bin/env bash > # nbd client library in userspace > -# Copyright (C) 2020 Red Hat Inc. > +# Copyright (C) 2020-2021 Red Hat Inc. > # > # This library is free software; you can redistribute it and/or > # modify it under the terms of the GNU Lesser General Public > @@ -45,7 +45,7 @@ EOF > cat $out > > if [ "$(tr -s ' ' < $out)" != " 0 16 1 hole > - 16 4 0 allocated > + 16 4 0 data > 20 12 1 hole" ]; then > echo "$0: unexpected output from nbdinfo --map" > exit 1 > diff --git a/info/info-map-base-allocation.sh b/info/info-map-base-allocation.sh > index 9b85c97..76c2024 100755 > --- a/info/info-map-base-allocation.sh > +++ b/info/info-map-base-allocation.sh > @@ -1,6 +1,6 @@ > #!/usr/bin/env bash > # nbd client library in userspace > -# Copyright (C) 2020 Red Hat Inc. > +# Copyright (C) 2020-2021 Red Hat Inc. > # > # This library is free software; you can redistribute it and/or > # modify it under the terms of the GNU Lesser General Public > @@ -36,9 +36,9 @@ nbdkit -U - data data='1 @131072 2' size=1M \ > > cat $out > > -if [ "$(tr -s ' ' < $out)" != " 0 32768 0 allocated > +if [ "$(tr -s ' ' < $out)" != " 0 32768 0 data > 32768 98304 3 hole,zero > - 131072 32768 0 allocated > + 131072 32768 0 data > 163840 884736 3 hole,zero" ]; then > echo "$0: unexpected output from nbdinfo --map" > exit 1 > diff --git a/info/info-map-qemu-allocation-depth.sh b/info/info-map-qemu-allocation-depth.sh > index a5dab0b..9aeffd1 100755 > --- a/info/info-map-qemu-allocation-depth.sh > +++ b/info/info-map-qemu-allocation-depth.sh > @@ -1,6 +1,6 @@ > #!/usr/bin/env bash > # nbd client library in userspace > -# Copyright (C) 2020 Red Hat Inc. > +# Copyright (C) 2020-2021 Red Hat Inc. > # > # This library is free software; you can redistribute it and/or > # modify it under the terms of the GNU Lesser General Public > @@ -84,7 +84,7 @@ if [ "$(tr -s ' ' < $out)" != "\ > 3145728 2097152 3 backing depth 3 > 5242880 1048576 1 local > 6291456 1048576 2 backing depth 2 > - 7340032 1048576 0 unallocated" ]; then > + 7340032 1048576 0 absent" ]; then > echo "$0: unexpected output from nbdinfo --map" > exit 1 > fi > diff --git a/info/map.c b/info/map.c > index ae6d4fe..82c9507 100644 > --- a/info/map.c > +++ b/info/map.c > @@ -202,7 +202,7 @@ extent_description (const char *metacontext, uint32_t type) > > if (strcmp (metacontext, "base:allocation") == 0) { > switch (type) { > - case 0: return strdup ("allocated"); > + case 0: return strdup ("data"); > case 1: return strdup ("hole"); > case 2: return strdup ("zero"); > case 3: return strdup ("hole,zero"); > @@ -216,7 +216,7 @@ extent_description (const char *metacontext, uint32_t type) > } > else if (strcmp (metacontext, "qemu:allocation-depth") == 0) { > switch (type) { > - case 0: return strdup ("unallocated"); > + case 0: return strdup ("absent"); > case 1: return strdup ("local"); > default: > if (asprintf (&ret, "backing depth %u", type) == -1) { > -- > 2.31.1Looks better, but can break scripts depending on nbdinfo output text. I hope that nbdino is new enough that this is not a problem. Nir
Richard W.M. Jones
2021-Jun-10 17:20 UTC
[Libguestfs] [libnbd PATCH] info: Avoid ambiguous 'allocated' terminology in mapping
I think you need to update the nbdinfo.pod file as well, since there are some references there. As Nir says, it's unfortunate, but I guess we've got to do it. If you push it (with docs fixed) soon then I can put it into the libnbd 1.8.1 stable release. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW