I'm pleased to announce libguestfs 1.26, a library and set of tools for accessing and modifying virtual machine disk images. This release took more than 6 months of work by a considerable number of people, and has many new features (see release notes below). You can get libguestfs 1.26 here: Main website: http://libguestfs.org/ Source: http://libguestfs.org/download/1.26-stable/ You will also need latest supermin from here: http://libguestfs.org/download/supermin/ Fedora 20/21: http://koji.fedoraproject.org/koji/packageinfo?packageID=8391 It will appear as an update for F20 in about a week. Debian/experimental coming soon, see: https://packages.debian.org/experimental/libguestfs0 The Fedora and Debian packages have split dependencies so you can download just the features you need.>From http://libguestfs.org/guestfs-release-notes.1.html :RELEASE NOTES FOR LIBGUESTFS 1.26 New features Tools virt-customize(1) is a new tool for customizing virtual machine disk images. It lets you install packages, edit configuration files, run scripts, set passwords and so on. virt-builder(1) and virt-sysprep(1) use virt-customize, and command line options across all these tools are now identical. virt-diff(1) is a new tool for showing the differences between the filesystems of two virtual machines. It is mainly useful when showing what files have been changed between snapshots. virt-builder(1) has been greatly enhanced. There are many more ways to customize the virtual machine. It can pull templates from multiple repositories. A parallelized internal xzcat implementation speeds up template decompression. Virt-builder uses an optimizing planner to choose the fastest way to build the VM. It is now easier to use virt-builder from other programs. Internationalization support has been added to metadata. More efficient SELinux relabelling of files. Can build guests for multiple architectures. Error messages have been improved. (Pino Toscano) virt-sparsify(1) has a new --in-place option. This sparsifies an image in place (without copying it) and is also much faster. (Lots of help provided by Paolo Bonzini) virt-sysprep(1) can delete and scrub files under user control. You can lock user accounts or set random passwords on accounts. Can remove more log files. Can unsubscribe a guest from Red Hat Subscription Manager. New flexible way to enable and disable operations. (Wanlong Gao, Pino Toscano) virt-win-reg(1) allows you to use URIs to specify remote disk images. virt-format(1) can now pass the extra space that it recovers back to the host. guestfish(1) has additional environment variables to give fine control over the ><fs> prompt. Guestfish reads its (rarely used) configuration file in a different order now so that local settings override global settings. (Pino Toscano) virt-make-fs(1) was rewritten in C, but is unchanged in terms of functionality and command line usage. Language bindings The OCaml bindings have a new Guestfs.Errno module, used to check the error number returned by Guestfs.last_errno. PHP tests now work. (Pino Toscano) Inspection Inspection can recognize Debian live images. Architectures ARMv7 (32 bit) now supports KVM acceleration. Aarch64 (ARM 64 bit) is supported, but the appliance part does not work yet. PPC64 support has been fixed and enhanced. Security Denial of service when inspecting disk images with corrupt btrfs volumes It was possible to crash libguestfs (and programs that use libguestfs as a library) by presenting a disk image containing a corrupt btrfs volume. This was caused by a NULL pointer dereference causing a denial of service, and is not thought to be exploitable any further. See commit d70ceb4cbea165c960710576efac5a5716055486 for the fix. This fix is included in libguestfs stable branches ≥ 1.26.0, ≥ 1.24.6 and ≥ 1.22.8, and also in RHEL ≥ 7.0. Earlier versions of libguestfs are not vulnerable. Better generation of random root passwords and random seeds When generating random root passwords and random seeds, two bugs were fixed which are possibly security related. Firstly we no longer read excessive bytes from /dev/urandom (most of which were just thrown away). Secondly we changed the code to avoid modulo bias. These issues were not thought to be exploitable. (Both changes suggested by Edwin Török) API GUID parameters are now validated when they are passed to API calls, whereas previously you could have passed any string. (Pino Toscano) New APIs guestfs_add_drive_opts: new discard parameter The new discard parameter allows fine-grained control over discard/trim support for a particular disk. This allows the host file to become more sparse (or thin-provisioned) when you delete files or issue the guestfs_fstrim API call. guestfs_add_domain: new parameters: cachemode, discard These parameters are passed through when adding the domain's disks. guestfs_blkdiscard Discard all blocks on a guestfs device. Combined with the discard parameter above, this makes the host file sparse. guestfs_blkdiscardzeroes Test if discarded blocks read back as zeroes. guestfs_compare_* guestfs_copy_* For each struct returned through the API, libguestfs now generates guestfs_compare_* and guestfs_copy_* functions to allow you to compare and copy structs. guestfs_copy_attributes Copy attributes (like permissions, xattrs, ownership) from one file to another. (Pino Toscano) guestfs_disk_create A flexible API for creating empty disk images from scratch. This avoids the need to call out to external programs like qemu-img(1). guestfs_get_backend_settings guestfs_set_backend_settings Per-backend settings (can also be set via the environment variable LIBGUESTFS_BACKEND_SETTINGS). The main use for this is forcing TCG mode in the qemu-based backends, for example: export LIBGUESTFS_BACKEND=direct export LIBGUESTFS_BACKEND_SETTINGS=force_tcg guestfs_part_get_name Get the label or name of a partition (for GPT disk images). Build changes The following extra packages are required to build libguestfs 1.26: supermin ≥ 5 Supermin version 5 is required to build this version of libguestfs. flex, bison Virt-builder now uses a real parser to parse its metadata file, so these tools are required. xz This is now a required build dependency, where previously it was (in theory) optional. Internals PO message extraction rewritten to be more robust. (Pino Toscano) podwrapper gives an error if the --insert or --verbatim argument pattern is not found. Libguestfs now passes the qemu -enable-fips option to enable FIPS, if qemu supports it. ./configure --without-qemu can be used if you don't want to specify a default hypervisor. Copy-on-write [COW] overlays, used for example for read-only drives, are now created through an internal backend API (.create_cow_overlay). Libvirt backend uses some funky C macros to generate XML. These are simpler and safer. The ChangeLog file format has changed. It is now just the same as git log, instead of using a custom format. Appliance start-up has changed: * The libguestfs appliance now initializes LVM the same way as it is done on physical machines. * The libguestfs appliance does not write an empty string to /proc/sys/kernel/hotplug when starting up. Note that you must configure your kernel to have CONFIG_UEVENT_HELPER_PATH="" otherwise you will get strange LVM errors (this applies as much to any Linux machine, not just libguestfs). (Peter Rajnoha) Libguestfs can now be built on arches that have ocamlc(1) but not ocamlopt(1). (Hilko Bengen, Olaf Hering) You cannot use ./configure --disable-daemon --enable-appliance. It made no sense anyway. Now it is expressly forbidden by the configure script. The packagelist file uses m4 for macro expansion instead of cpp. Bugs fixed https://bugzilla.redhat.com/1073906 java bindings inspect_list_applications2 throws java.lang.ArrayIndexOutOfBoundsException: https://bugzilla.redhat.com/1063374 [RFE] enable subscription manager clean or unregister operation to sysprep https://bugzilla.redhat.com/1060404 virt-resize does not preserve GPT partition names https://bugzilla.redhat.com/1057504 mount-local should give a clearer error if root is not mounted https://bugzilla.redhat.com/1056290 virt-sparsify overwrites block devices if used as output files https://bugzilla.redhat.com/1055452 libguestfs: error: invalid backend: appliance https://bugzilla.redhat.com/1054761 guestfs_pvs prints "unknown device" if a physical volume is missing https://bugzilla.redhat.com/1053847 Recommended default clock/timer settings https://bugzilla.redhat.com/1046509 ruby-libguestfs throws "expecting 0 or 1 arguments" on Guestfs::Guestfs.new https://bugzilla.redhat.com/1045450 Cannot inspect cirros 0.3.1 disk image fully https://bugzilla.redhat.com/1045033 LIBVIRT_DEFAULT_URI=qemu:///system breaks libguestfs https://bugzilla.redhat.com/1044585 virt-builder network (eg. --install) doesn't work if resolv.conf sets nameserver 127.0.0.1 https://bugzilla.redhat.com/1044014 When SSSD is installed, libvirt configuration requires authentication, but not clear to user https://bugzilla.redhat.com/1039995 virt-make-fs fails making fat/vfat whole disk: Device partition expected, not making filesystem on entire device '/dev/sda' (use -I to override) https://bugzilla.redhat.com/1039540 virt-sysprep to delete more logfiles https://bugzilla.redhat.com/1033207 RFE: libguestfs inspection does not recognize Free4NAS live CD https://bugzilla.redhat.com/1028660 RFE: virt-sysprep/virt-builder should have an option to lock a user account https://bugzilla.redhat.com/1026688 libguestfs fails examining libvirt guest with ceph drives: rbd: image name must begin with a '/' https://bugzilla.redhat.com/1022431 virt-builder fails if $HOME/.cache doesn't exist https://bugzilla.redhat.com/1022184 libguestfs: do not use versioned jar file https://bugzilla.redhat.com/1020806 All libguestfs LVM operations fail on Debian/Ubuntu https://bugzilla.redhat.com/1008417 Need update helpout of part-set-gpt-type https://bugzilla.redhat.com/953907 virt-sysprep does not correctly set the hostname on Debian/Ubuntu https://bugzilla.redhat.com/923355 guestfish prints literal "\n" in error messages https://bugzilla.redhat.com/660687 guestmount: "touch" command fails: touch: setting times of `timestamp': Invalid argument https://bugzilla.redhat.com/593511 [RFE] function to get partition name https://bugzilla.redhat.com/563450 list-devices returns devices of different types out of order --- Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On Thu, Mar 27, Richard W.M. Jones wrote:> I'm pleased to announce libguestfs 1.26, a library and set of toolsThis fails to link at least in sles11sp3, 1.25.37 was still ok: ... [ 288s] virt_index_validate-index-validate.o: In function `main': [ 288s] index-validate.c:(.text+0x94): undefined reference to `rpl_getopt_long' [ 288s] index-validate.c:(.text+0x165): undefined reference to `rpl_optind' [ 288s] index-validate.c:(.text+0x177): undefined reference to `rpl_optind' [ 288s] index-validate.c:(.text+0x18c): undefined reference to `rpl_optind' [ 288s] index-validate.c:(.text+0x498): undefined reference to `rpl_perror' [ 288s] collect2: ld returned 1 exit status [ 288s] make[2]: *** [virt-index-validate] Error 1 ... Sorry for not testing more recent devel snapshots. Olaf
Richard W.M. Jones
2014-Mar-28 08:50 UTC
Re: [Libguestfs] ANNOUNCE: libguestfs 1.26 released
On Fri, Mar 28, 2014 at 09:42:52AM +0100, Olaf Hering wrote:> On Thu, Mar 27, Richard W.M. Jones wrote: > > > I'm pleased to announce libguestfs 1.26, a library and set of tools > > This fails to link at least in sles11sp3, 1.25.37 was still ok: > > ... > [ 288s] virt_index_validate-index-validate.o: In function `main': > [ 288s] index-validate.c:(.text+0x94): undefined reference to `rpl_getopt_long' > [ 288s] index-validate.c:(.text+0x165): undefined reference to `rpl_optind' > [ 288s] index-validate.c:(.text+0x177): undefined reference to `rpl_optind' > [ 288s] index-validate.c:(.text+0x18c): undefined reference to `rpl_optind' > [ 288s] index-validate.c:(.text+0x498): undefined reference to `rpl_perror' > [ 288s] collect2: ld returned 1 exit status > [ 288s] make[2]: *** [virt-index-validate] Error 1 > ...I'd say this is a gnulib problem, although why gnulib has decided to use replacement functions is a mystery. My understanding is that getopt_long is part of glibc, so gnulib shouldn't need to replace it. Did ./configure output mention getopt*? If there is a config.cache file, can you delete it and run ./configure again. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
Richard W.M. Jones
2014-Mar-28 08:54 UTC
Re: [Libguestfs] ANNOUNCE: libguestfs 1.26 released
On Fri, Mar 28, 2014 at 09:42:52AM +0100, Olaf Hering wrote:> On Thu, Mar 27, Richard W.M. Jones wrote: > > > I'm pleased to announce libguestfs 1.26, a library and set of tools > > This fails to link at least in sles11sp3, 1.25.37 was still ok: > > ... > [ 288s] virt_index_validate-index-validate.o: In function `main': > [ 288s] index-validate.c:(.text+0x94): undefined reference to `rpl_getopt_long' > [ 288s] index-validate.c:(.text+0x165): undefined reference to `rpl_optind' > [ 288s] index-validate.c:(.text+0x177): undefined reference to `rpl_optind' > [ 288s] index-validate.c:(.text+0x18c): undefined reference to `rpl_optind' > [ 288s] index-validate.c:(.text+0x498): undefined reference to `rpl_perror' > [ 288s] collect2: ld returned 1 exit status > [ 288s] make[2]: *** [virt-index-validate] Error 1 > ...Second opinion: I think builder/Makefile.am needs to link these programs with gnulib, ie something like this (not tested): virt_index_validate_LDADD = \ -L../gnulib/lib/.libs -lgnu Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
Richard W.M. Jones
2014-Mar-28 14:41 UTC
Re: [Libguestfs] ANNOUNCE: libguestfs 1.26 released
On Thu, Mar 27, 2014 at 10:26:42PM +0000, Richard W.M. Jones wrote:> I'm pleased to announce libguestfs 1.26, a library and set of tools > for accessing and modifying virtual machine disk images. This release > took more than 6 months of work by a considerable number of people, > and has many new features (see release notes below). > > You can get libguestfs 1.26 here: > > Main website: http://libguestfs.org/ > > Source: http://libguestfs.org/download/1.26-stable/ > You will also need latest supermin from here: > http://libguestfs.org/download/supermin/ > > Fedora 20/21: http://koji.fedoraproject.org/koji/packageinfo?packageID=8391 > It will appear as an update for F20 in about a week.Fedora 20 users can test and give feedback here: https://admin.fedoraproject.org/updates/libguestfs-1.26.0-1.fc20,supermin-5.1.6-3.fc20> Debian/experimental coming soon, see: > https://packages.debian.org/experimental/libguestfs0 > > The Fedora and Debian packages have split dependencies so you can > download just the features you need. > > From http://libguestfs.org/guestfs-release-notes.1.html : > > RELEASE NOTES FOR LIBGUESTFS 1.26 > > New features > > Tools > > virt-customize(1) is a new tool for customizing virtual machine disk > images. It lets you install packages, edit configuration files, run > scripts, set passwords and so on. virt-builder(1) and virt-sysprep(1) > use virt-customize, and command line options across all these tools are > now identical. > > virt-diff(1) is a new tool for showing the differences between the > filesystems of two virtual machines. It is mainly useful when showing > what files have been changed between snapshots. > > virt-builder(1) has been greatly enhanced. There are many more ways to > customize the virtual machine. It can pull templates from multiple > repositories. A parallelized internal xzcat implementation speeds up > template decompression. Virt-builder uses an optimizing planner to > choose the fastest way to build the VM. It is now easier to use > virt-builder from other programs. Internationalization support has been > added to metadata. More efficient SELinux relabelling of files. Can > build guests for multiple architectures. Error messages have been > improved. (Pino Toscano) > > virt-sparsify(1) has a new --in-place option. This sparsifies an image > in place (without copying it) and is also much faster. (Lots of help > provided by Paolo Bonzini) > > virt-sysprep(1) can delete and scrub files under user control. You can > lock user accounts or set random passwords on accounts. Can remove more > log files. Can unsubscribe a guest from Red Hat Subscription Manager. > New flexible way to enable and disable operations. (Wanlong Gao, Pino > Toscano) > > virt-win-reg(1) allows you to use URIs to specify remote disk images. > > virt-format(1) can now pass the extra space that it recovers back to > the host. > > guestfish(1) has additional environment variables to give fine control > over the ><fs> prompt. Guestfish reads its (rarely used) configuration > file in a different order now so that local settings override global > settings. (Pino Toscano) > > virt-make-fs(1) was rewritten in C, but is unchanged in terms of > functionality and command line usage. > > Language bindings > > The OCaml bindings have a new Guestfs.Errno module, used to check the > error number returned by Guestfs.last_errno. > > PHP tests now work. (Pino Toscano) > > Inspection > > Inspection can recognize Debian live images. > > Architectures > > ARMv7 (32 bit) now supports KVM acceleration. > > Aarch64 (ARM 64 bit) is supported, but the appliance part does not work > yet. > > PPC64 support has been fixed and enhanced. > > Security > > Denial of service when inspecting disk images with corrupt btrfs > volumes > > It was possible to crash libguestfs (and programs that use libguestfs > as a library) by presenting a disk image containing a corrupt btrfs > volume. > > This was caused by a NULL pointer dereference causing a denial of > service, and is not thought to be exploitable any further. > > See commit d70ceb4cbea165c960710576efac5a5716055486 for the fix. This > fix is included in libguestfs stable branches ≥ 1.26.0, ≥ 1.24.6 and > ≥ 1.22.8, and also in RHEL ≥ 7.0. Earlier versions of libguestfs are > not vulnerable. > > Better generation of random root passwords and random seeds > > When generating random root passwords and random seeds, two bugs were > fixed which are possibly security related. Firstly we no longer read > excessive bytes from /dev/urandom (most of which were just thrown > away). Secondly we changed the code to avoid modulo bias. These > issues were not thought to be exploitable. (Both changes suggested by > Edwin Török) > > API > > GUID parameters are now validated when they are passed to API calls, > whereas previously you could have passed any string. (Pino Toscano) > > New APIs > > guestfs_add_drive_opts: new discard parameter > > The new discard parameter allows fine-grained control over > discard/trim support for a particular disk. This allows the host file > to become more sparse (or thin-provisioned) when you delete files or > issue the guestfs_fstrim API call. > > guestfs_add_domain: new parameters: cachemode, discard > > These parameters are passed through when adding the domain's disks. > > guestfs_blkdiscard > > Discard all blocks on a guestfs device. Combined with the discard > parameter above, this makes the host file sparse. > > guestfs_blkdiscardzeroes > > Test if discarded blocks read back as zeroes. > > guestfs_compare_* > > guestfs_copy_* > > For each struct returned through the API, libguestfs now generates > guestfs_compare_* and guestfs_copy_* functions to allow you to > compare and copy structs. > > guestfs_copy_attributes > > Copy attributes (like permissions, xattrs, ownership) from one file > to another. (Pino Toscano) > > guestfs_disk_create > > A flexible API for creating empty disk images from scratch. This > avoids the need to call out to external programs like qemu-img(1). > > guestfs_get_backend_settings > > guestfs_set_backend_settings > > Per-backend settings (can also be set via the environment variable > LIBGUESTFS_BACKEND_SETTINGS). The main use for this is forcing TCG > mode in the qemu-based backends, for example: > > export LIBGUESTFS_BACKEND=direct > export LIBGUESTFS_BACKEND_SETTINGS=force_tcg > > guestfs_part_get_name > > Get the label or name of a partition (for GPT disk images). > > Build changes > > The following extra packages are required to build libguestfs 1.26: > > supermin ≥ 5 > > Supermin version 5 is required to build this version of libguestfs. > > flex, bison > > Virt-builder now uses a real parser to parse its metadata file, so > these tools are required. > > xz > > This is now a required build dependency, where previously it was (in > theory) optional. > > Internals > > PO message extraction rewritten to be more robust. (Pino Toscano) > > podwrapper gives an error if the --insert or --verbatim argument > pattern is not found. > > Libguestfs now passes the qemu -enable-fips option to enable FIPS, if > qemu supports it. > > ./configure --without-qemu can be used if you don't want to specify a > default hypervisor. > > Copy-on-write [COW] overlays, used for example for read-only drives, > are now created through an internal backend API (.create_cow_overlay). > > Libvirt backend uses some funky C macros to generate XML. These are > simpler and safer. > > The ChangeLog file format has changed. It is now just the same as git > log, instead of using a custom format. > > Appliance start-up has changed: > > * The libguestfs appliance now initializes LVM the same way as it is > done on physical machines. > > * The libguestfs appliance does not write an empty string to > /proc/sys/kernel/hotplug when starting up. > > Note that you must configure your kernel to have > CONFIG_UEVENT_HELPER_PATH="" otherwise you will get strange LVM > errors (this applies as much to any Linux machine, not just > libguestfs). (Peter Rajnoha) > > Libguestfs can now be built on arches that have ocamlc(1) but not > ocamlopt(1). (Hilko Bengen, Olaf Hering) > > You cannot use ./configure --disable-daemon --enable-appliance. It made > no sense anyway. Now it is expressly forbidden by the configure script. > > The packagelist file uses m4 for macro expansion instead of cpp. > > Bugs fixed > > https://bugzilla.redhat.com/1073906 > > java bindings inspect_list_applications2 throws > java.lang.ArrayIndexOutOfBoundsException: > > https://bugzilla.redhat.com/1063374 > > [RFE] enable subscription manager clean or unregister operation to > sysprep > > https://bugzilla.redhat.com/1060404 > > virt-resize does not preserve GPT partition names > > https://bugzilla.redhat.com/1057504 > > mount-local should give a clearer error if root is not mounted > > https://bugzilla.redhat.com/1056290 > > virt-sparsify overwrites block devices if used as output files > > https://bugzilla.redhat.com/1055452 > > libguestfs: error: invalid backend: appliance > > https://bugzilla.redhat.com/1054761 > > guestfs_pvs prints "unknown device" if a physical volume is missing > > https://bugzilla.redhat.com/1053847 > > Recommended default clock/timer settings > > https://bugzilla.redhat.com/1046509 > > ruby-libguestfs throws "expecting 0 or 1 arguments" on > Guestfs::Guestfs.new > > https://bugzilla.redhat.com/1045450 > > Cannot inspect cirros 0.3.1 disk image fully > > https://bugzilla.redhat.com/1045033 > > LIBVIRT_DEFAULT_URI=qemu:///system breaks libguestfs > > https://bugzilla.redhat.com/1044585 > > virt-builder network (eg. --install) doesn't work if resolv.conf sets > nameserver 127.0.0.1 > > https://bugzilla.redhat.com/1044014 > > When SSSD is installed, libvirt configuration requires > authentication, but not clear to user > > https://bugzilla.redhat.com/1039995 > > virt-make-fs fails making fat/vfat whole disk: Device partition > expected, not making filesystem on entire device '/dev/sda' (use -I > to override) > > https://bugzilla.redhat.com/1039540 > > virt-sysprep to delete more logfiles > > https://bugzilla.redhat.com/1033207 > > RFE: libguestfs inspection does not recognize Free4NAS live CD > > https://bugzilla.redhat.com/1028660 > > RFE: virt-sysprep/virt-builder should have an option to lock a user > account > > https://bugzilla.redhat.com/1026688 > > libguestfs fails examining libvirt guest with ceph drives: rbd: image > name must begin with a '/' > > https://bugzilla.redhat.com/1022431 > > virt-builder fails if $HOME/.cache doesn't exist > > https://bugzilla.redhat.com/1022184 > > libguestfs: do not use versioned jar file > > https://bugzilla.redhat.com/1020806 > > All libguestfs LVM operations fail on Debian/Ubuntu > > https://bugzilla.redhat.com/1008417 > > Need update helpout of part-set-gpt-type > > https://bugzilla.redhat.com/953907 > > virt-sysprep does not correctly set the hostname on Debian/Ubuntu > > https://bugzilla.redhat.com/923355 > > guestfish prints literal "\n" in error messages > > https://bugzilla.redhat.com/660687 > > guestmount: "touch" command fails: touch: setting times of > `timestamp': Invalid argument > > https://bugzilla.redhat.com/593511 > > [RFE] function to get partition name > > https://bugzilla.redhat.com/563450 > > list-devices returns devices of different types out of orderRich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top