Richard W.M. Jones
2022-Sep-23 14:19 UTC
[Libguestfs] can not get the virt-sparsify code in libguestfs ?
On Fri, Sep 23, 2022 at 02:03:45PM +0000, Li, Feng F wrote:> The log output from command " ./virt-sparsify --inplace -v -x /home/intel/imgStore/POS_DATA.qcow2"Thanks. This is actually a bug in libguestfs. In the following function which is called from virt-sparsify early on: https://github.com/libguestfs/libguestfs-common/blob/9d40590852e0755d4719adf97122758fa98e90f9/options/decrypt.c#L203 if the "lvs" feature is not enabled then calling guestfs_lvs will fail: libguestfs: trace: lvs guestfsd: <= lvs (0xb) request length 40 bytes guestfsd: error: feature 'lvm2' is not available in this build of libguestfs. Read 'AVAILABILITY' in the guestfs(3) man page for how to check for the availability of features. guestfsd: => lvs (0xb) took 0.00 secs libguestfs: trace: lvs = NULL (error) and cause virt-sparsify to exit. Can you try the attached patch to libguestfs/common ? Rich.> > -----Original Message----- > From: Richard W.M. Jones <rjones at redhat.com> > Sent: 2022?9?23? 20:07 > To: Li, Feng F <feng.f.li at intel.com> > Cc: libguestfs at redhat.com > Subject: Re: can not get the virt-sparsify code in libguestfs ? > > On Fri, Sep 23, 2022 at 11:37:01AM +0000, Li, Feng F wrote: > > We download the guestfs-tools from below git and compile a local version . > > > > But when we did the test , please see below: > > > > 1. In the Windows the Drive D:\ , the real disk usage is about 19.9-17.7 > > 2.2G. > > > > [cid] > > > > > > > > 2. Shutdown VM , then we use virt-sparsify on the qcow2 image, but the size is > > still same as before 6.54G. > > > > It is expected the qcow2 is resized to 2.2G , right ? > > > > Any reason for why the disk size is not returned to host ? > > Difficult to say. What is the complete output from virt-sparsify -v -x .... ? > > Rich. > > > > > > > [cid] > > > > > > > > > > > > Thanks a lot ! > > > > lifeng > > > > > > > > -----Original Message----- > > From: Richard W.M. Jones <rjones at redhat.com> > > Sent: 2022?9?22? 21:00 > > To: Li, Feng F <feng.f.li at intel.com> > > Cc: libguestfs at redhat.com > > Subject: Re: can not get the virt-sparsify code in libguestfs ? > > > > > > > > On Thu, Sep 22, 2022 at 11:48:17AM +0000, Li, Feng F wrote: > > > > > 2) https://libguestfs.org/virt-sparsify.1.html > > > > > > > > > > we downloaded the libguestfs (1.48-stable) library ,but after the > > > > > configure and make, we did not found the virt-sparsify source code > > > and > > > > > binary file, would you please help see where to get the > > > virt-sparsify > > > > > source code for debug ? > > > > > > > > virt-sparsify and some other tools were recently moved into a new > > > > repository: > > > > > > > > https://github.com/rwmjones/guestfs-tools > > > > > > > > Most Linux distros already ship these tools. You shouldn't really > > need to compile it all from source yourself (although of course it is > > possible and you are welcome to try). > > > > > > > > Rich. > > > > > > > > -- > > > > Richard Jones, Virtualization Group, Red Hat > > http://people.redhat.com/~rjones Read my programming and > > virtualization blog: http://rwmj.wordpress.com virt-p2v converts > > physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. > > > > http://libguestfs.org/virt-v2v > > > > > > > > > > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit >-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html -------------- next part -------------->From 98f0b3565457c08d14e1f9ab2acecea003ebf6e1 Mon Sep 17 00:00:00 2001From: "Richard W.M. Jones" <rjones at redhat.com> Date: Fri, 23 Sep 2022 15:18:43 +0100 Subject: [PATCH] options: Don't attempt to scan or open LVs if "lvm2" feature is not available Reported-by: Feng Li --- options/decrypt.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/options/decrypt.c b/options/decrypt.c index 6fc7760e3..e9d7d99e0 100644 --- a/options/decrypt.c +++ b/options/decrypt.c @@ -205,19 +205,22 @@ inspect_do_decrypt (guestfs_h *g, struct key_store *ks) CLEANUP_FREE_STRING_LIST char **partitions = guestfs_list_partitions (g); CLEANUP_FREE_STRING_LIST char **lvs = NULL; bool need_rescan; + const char *lvm2_feature[] = { "lvm2", NULL }; if (partitions == NULL) exit (EXIT_FAILURE); need_rescan = decrypt_mountables (g, (const char * const *)partitions, ks); - if (need_rescan) { - if (guestfs_lvm_scan (g, 1) == -1) + if (guestfs_feature_available (g, (char **) lvm2_feature) > 0) { + if (need_rescan) { + if (guestfs_lvm_scan (g, 1) == -1) + exit (EXIT_FAILURE); + } + + lvs = guestfs_lvs (g); + if (lvs == NULL) exit (EXIT_FAILURE); + decrypt_mountables (g, (const char * const *)lvs, ks); } - - lvs = guestfs_lvs (g); - if (lvs == NULL) - exit (EXIT_FAILURE); - decrypt_mountables (g, (const char * const *)lvs, ks); } -- 2.37.0.rc2
Richard W.M. Jones
2022-Sep-23 14:22 UTC
[Libguestfs] can not get the virt-sparsify code in libguestfs ?
On Fri, Sep 23, 2022 at 03:19:47PM +0100, Richard W.M. Jones wrote:> On Fri, Sep 23, 2022 at 02:03:45PM +0000, Li, Feng F wrote: > > The log output from command " ./virt-sparsify --inplace -v -x /home/intel/imgStore/POS_DATA.qcow2" > > Thanks. This is actually a bug in libguestfs. In the following > function which is called from virt-sparsify early on: > > https://github.com/libguestfs/libguestfs-common/blob/9d40590852e0755d4719adf97122758fa98e90f9/options/decrypt.c#L203 > > if the "lvs" feature is not enabled then calling guestfs_lvs will > fail: > > libguestfs: trace: lvs > guestfsd: <= lvs (0xb) request length 40 bytes > guestfsd: error: feature 'lvm2' is not available in this > build of libguestfs. Read 'AVAILABILITY' in the guestfs(3) man page for > how to check for the availability of features. > guestfsd: => lvs (0xb) took 0.00 secs > libguestfs: trace: lvs = NULL (error) > > and cause virt-sparsify to exit. > > Can you try the attached patch to libguestfs/common ?Actually I think you have to apply to patch to guestfs-tools/common ! It should be fairly obvious from the trace if the patch worked or not. Rich.> Rich. > > > > > > > -----Original Message----- > > From: Richard W.M. Jones <rjones at redhat.com> > > Sent: 2022?9?23? 20:07 > > To: Li, Feng F <feng.f.li at intel.com> > > Cc: libguestfs at redhat.com > > Subject: Re: can not get the virt-sparsify code in libguestfs ? > > > > On Fri, Sep 23, 2022 at 11:37:01AM +0000, Li, Feng F wrote: > > > We download the guestfs-tools from below git and compile a local version . > > > > > > But when we did the test , please see below: > > > > > > 1. In the Windows the Drive D:\ , the real disk usage is about 19.9-17.7 > > > 2.2G. > > > > > > [cid] > > > > > > > > > > > > 2. Shutdown VM , then we use virt-sparsify on the qcow2 image, but the size is > > > still same as before 6.54G. > > > > > > It is expected the qcow2 is resized to 2.2G , right ? > > > > > > Any reason for why the disk size is not returned to host ? > > > > Difficult to say. What is the complete output from virt-sparsify -v -x .... ? > > > > Rich. > > > > > > > > > > > [cid] > > > > > > > > > > > > > > > > > > Thanks a lot ! > > > > > > lifeng > > > > > > > > > > > > -----Original Message----- > > > From: Richard W.M. Jones <rjones at redhat.com> > > > Sent: 2022?9?22? 21:00 > > > To: Li, Feng F <feng.f.li at intel.com> > > > Cc: libguestfs at redhat.com > > > Subject: Re: can not get the virt-sparsify code in libguestfs ? > > > > > > > > > > > > On Thu, Sep 22, 2022 at 11:48:17AM +0000, Li, Feng F wrote: > > > > > > > 2) https://libguestfs.org/virt-sparsify.1.html > > > > > > > > > > > > > > we downloaded the libguestfs (1.48-stable) library ,but after the > > > > > > > configure and make, we did not found the virt-sparsify source code > > > > and > > > > > > > binary file, would you please help see where to get the > > > > virt-sparsify > > > > > > > source code for debug ? > > > > > > > > > > > > virt-sparsify and some other tools were recently moved into a new > > > > > > repository: > > > > > > > > > > > > https://github.com/rwmjones/guestfs-tools > > > > > > > > > > > > Most Linux distros already ship these tools. You shouldn't really > > > need to compile it all from source yourself (although of course it is > > > possible and you are welcome to try). > > > > > > > > > > > > Rich. > > > > > > > > > > > > -- > > > > > > Richard Jones, Virtualization Group, Red Hat > > > http://people.redhat.com/~rjones Read my programming and > > > virtualization blog: http://rwmj.wordpress.com virt-p2v converts > > > physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. > > > > > > http://libguestfs.org/virt-v2v > > > > > > > > > > > > > > > > > > > -- > > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit > > > > > > -- > Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones > Read my programming and virtualization blog: http://rwmj.wordpress.com > virt-builder quickly builds VMs from scratch > http://libguestfs.org/virt-builder.1.html> From 98f0b3565457c08d14e1f9ab2acecea003ebf6e1 Mon Sep 17 00:00:00 2001 > From: "Richard W.M. Jones" <rjones at redhat.com> > Date: Fri, 23 Sep 2022 15:18:43 +0100 > Subject: [PATCH] options: Don't attempt to scan or open LVs if "lvm2" feature > is not available > > Reported-by: Feng Li > --- > options/decrypt.c | 17 ++++++++++------- > 1 file changed, 10 insertions(+), 7 deletions(-) > > diff --git a/options/decrypt.c b/options/decrypt.c > index 6fc7760e3..e9d7d99e0 100644 > --- a/options/decrypt.c > +++ b/options/decrypt.c > @@ -205,19 +205,22 @@ inspect_do_decrypt (guestfs_h *g, struct key_store *ks) > CLEANUP_FREE_STRING_LIST char **partitions = guestfs_list_partitions (g); > CLEANUP_FREE_STRING_LIST char **lvs = NULL; > bool need_rescan; > + const char *lvm2_feature[] = { "lvm2", NULL }; > > if (partitions == NULL) > exit (EXIT_FAILURE); > > need_rescan = decrypt_mountables (g, (const char * const *)partitions, ks); > > - if (need_rescan) { > - if (guestfs_lvm_scan (g, 1) == -1) > + if (guestfs_feature_available (g, (char **) lvm2_feature) > 0) { > + if (need_rescan) { > + if (guestfs_lvm_scan (g, 1) == -1) > + exit (EXIT_FAILURE); > + } > + > + lvs = guestfs_lvs (g); > + if (lvs == NULL) > exit (EXIT_FAILURE); > + decrypt_mountables (g, (const char * const *)lvs, ks); > } > - > - lvs = guestfs_lvs (g); > - if (lvs == NULL) > - exit (EXIT_FAILURE); > - decrypt_mountables (g, (const char * const *)lvs, ks); > } > -- > 2.37.0.rc2 >-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
Laszlo Ersek
2022-Sep-25 06:21 UTC
[Libguestfs] can not get the virt-sparsify code in libguestfs ?
On 09/23/22 16:19, Richard W.M. Jones wrote:> From 98f0b3565457c08d14e1f9ab2acecea003ebf6e1 Mon Sep 17 00:00:00 2001 > From: "Richard W.M. Jones" <rjones at redhat.com> > Date: Fri, 23 Sep 2022 15:18:43 +0100 > Subject: [PATCH] options: Don't attempt to scan or open LVs if "lvm2" feature > is not available > > Reported-by: Feng Li > --- > options/decrypt.c | 17 ++++++++++------- > 1 file changed, 10 insertions(+), 7 deletions(-) > > diff --git a/options/decrypt.c b/options/decrypt.c > index 6fc7760e3..e9d7d99e0 100644 > --- a/options/decrypt.c > +++ b/options/decrypt.c > @@ -205,19 +205,22 @@ inspect_do_decrypt (guestfs_h *g, struct key_store *ks) > CLEANUP_FREE_STRING_LIST char **partitions = guestfs_list_partitions (g); > CLEANUP_FREE_STRING_LIST char **lvs = NULL; > bool need_rescan; > + const char *lvm2_feature[] = { "lvm2", NULL }; > > if (partitions == NULL) > exit (EXIT_FAILURE); > > need_rescan = decrypt_mountables (g, (const char * const *)partitions, ks); > > - if (need_rescan) { > - if (guestfs_lvm_scan (g, 1) == -1) > + if (guestfs_feature_available (g, (char **) lvm2_feature) > 0) { > + if (need_rescan) { > + if (guestfs_lvm_scan (g, 1) == -1) > + exit (EXIT_FAILURE); > + } > + > + lvs = guestfs_lvs (g); > + if (lvs == NULL) > exit (EXIT_FAILURE); > + decrypt_mountables (g, (const char * const *)lvs, ks); > } > - > - lvs = guestfs_lvs (g); > - if (lvs == NULL) > - exit (EXIT_FAILURE); > - decrypt_mountables (g, (const char * const *)lvs, ks); > }So this is interesting. (1) If the problem is that guestfs_lvs() is called without checking for the lvm2 feature group first, then the bug was introduced in libguestfs-common commit 2d8c0f8d40b5 ("options: decrypt LUKS-on-LV devices", 2022-02-28). The manual indeed states that guestfs_lvs() depends on the lvm2 feature group. (2) If, however, the problem is *also* that guestfs_lvm_scan() is called without checking for said feature group, then the bug was introduced in either (2.1) super-antique libguestfs commit a232e62dcf50 ("fish: '-i' option automatically handles whole-disk encryption.", 2010-11-05), or (2.2) less antique but still historical libguestfs commit 55dfcb2211a6 ("New API: lvm_scan, deprecate vgscan (RHBZ#1602353).", 2018-07-26). In either of those (2.*) cases, libguestfs-common commit 2d8c0f8d40b5 would only perpetuate the pre-existent bug. Put differently, the guestfs_lvm_scan() call had had its use even before libguestfs-common commit 2d8c0f8d40b5 (i.e., before LUKS-on-LV support), and it was already afflicted by the bug. Now, I'm not obsessing about this just so we can append a proper "Fixes:" line to the commit message. Instead, I think the scope of the fix (= the commit message & the code body of the patch both) depend on the above determination. The subject line and the body in this patch suggest that guestfs_lvm_scan() should be avoided when "lvm2" is missing, but the documentation does not imply that guestfs_lvm_scan() were invalid to call in that case. And those feature group dependency statements in the manual come from the generator. Now, in "generator/actions_core.ml", a bunch of lvm2-related operations (such as "lvs" itself) specify optional = Some "lvm2"; but that is not the case with "lvm_scan". If case (1) applies, then the current patch (code & subject line) are too heavy-handed. That is, the guestfs_lvm_scan() call should not be further restricted than it currently is (depending only on "need_rescan") -- such a restriction would be inconsistent with the fact that the guestfs_lvm_scan() API does not actually depend on the "lvm2" feature group. If case (2) applies however, then, in addition to the current contents of the patch, we should make the "lvm_scan" operation depend on the lvm2 feature group, in the generator. Laszlo