Libguestfs - Nov 2016 - Re: Libguestfs debugging issue

Hi. I am currently debugging an issue with libguestfs in Centos 7.2 (Kernel
3.10.0-327.36.3.el7.x86_64).

libguestfs is being called by SaltStack to access a qcow2 image, but it is
failing. I have narrowed the issue down to the command below:

guestmount -i -a /opt/vm-spinning/salt-mnt/salt-images/pb123/system.qcow2
--rw /tmp/guest/opt.vm-spinning.salt-mnt.salt-images.pb123.system.qcow2

I have run libguestfs-test-tool in debug as recommended in the faq but am
at a loss to explain the behavior, Libguestfs-test-tool dumps out when I
run the guestmount.

Please see log below:

libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
LIBGUESTFS_DEBUG=1
LIBGUESTFS_TRACE=1
PATH=/root/perl5/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
SELinux: Disabled
libguestfs: trace: add_drive_scratch 104857600
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: disk_create "/tmp/libguestfs7oWVq5/scratch.1"
"raw"
104857600
libguestfs: trace: disk_create = 0
libguestfs: trace: add_drive "/tmp/libguestfs7oWVq5/scratch.1"
"format:raw"
"cachemode:unsafe"
libguestfs: trace: add_drive = 0
libguestfs: trace: add_drive_scratch = 0
libguestfs: trace: get_append
libguestfs: trace: get_append = "NULL"
guestfs_get_append: (null)
libguestfs: trace: get_autosync
libguestfs: trace: get_autosync = 1
guestfs_get_autosync: 1
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "libvirt"
guestfs_get_backend: libvirt
libguestfs: trace: get_backend_settings
libguestfs: trace: get_backend_settings = []
guestfs_get_backend_settings: []
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
guestfs_get_cachedir: /var/tmp
libguestfs: trace: get_direct
libguestfs: trace: get_direct = 0
guestfs_get_direct: 0
libguestfs: trace: get_hv
libguestfs: trace: get_hv = "/usr/libexec/qemu-kvm"
guestfs_get_hv: /usr/libexec/qemu-kvm
libguestfs: trace: get_memsize
libguestfs: trace: get_memsize = 500
guestfs_get_memsize: 500
libguestfs: trace: get_network
libguestfs: trace: get_network = 0
guestfs_get_network: 0
libguestfs: trace: get_path
libguestfs: trace: get_path = "/usr/lib64/guestfs"
guestfs_get_path: /usr/lib64/guestfs
libguestfs: trace: get_pgroup
libguestfs: trace: get_pgroup = 0
guestfs_get_pgroup: 0
libguestfs: trace: get_program
libguestfs: trace: get_program = "libguestfs-test-tool"
guestfs_get_program: libguestfs-test-tool
libguestfs: trace: get_recovery_proc
libguestfs: trace: get_recovery_proc = 1
guestfs_get_recovery_proc: 1
libguestfs: trace: get_selinux
libguestfs: trace: get_selinux = 0
guestfs_get_selinux: 0
libguestfs: trace: get_smp
libguestfs: trace: get_smp = 1
guestfs_get_smp: 1
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
guestfs_get_tmpdir: /tmp
libguestfs: trace: get_trace
libguestfs: trace: get_trace = 1
guestfs_get_trace: 1
libguestfs: trace: get_verbose
libguestfs: trace: get_verbose = 1
guestfs_get_verbose: 1
host_cpu: x86_64
Launching appliance, timeout set to 600 seconds.
libguestfs: trace: launch
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version *>
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "libvirt"
libguestfs: launch: program=libguestfs-test-tool
libguestfs: launch: version=1.28.1rhel=7,release=1.55.el7.centos.4,libvirt
libguestfs: launch: backend registered: unix
libguestfs: launch: backend registered: uml
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=libvirt
libguestfs: launch: tmpdir=/tmp/libguestfs7oWVq5
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: libvirt version = 1002017 (1.2.17)
libguestfs: guest random name = guestfs-aooe4wmnlsfs1b60
libguestfs: [00000ms] connect to libvirt
libguestfs: opening libvirt handle: URI = qemu:///system, auth default+wrapper,
flags = 0
libguestfs: successfully opened libvirt handle: conn = 0x7feb544f6c30
libguestfs: qemu version (reported by libvirt) = 1005003 (1.5.3)
libguestfs: [00002ms] get libvirt capabilities
libguestfs: [00008ms] parsing capabilities XML
libguestfs: trace: get_backend_setting "force_tcg"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_backend_setting "internal_libvirt_label"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_backend_setting "internal_libvirt_imagelabel"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_backend_setting
"internal_libvirt_norelabel_disks"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: [00008ms] build appliance
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: [00008ms] begin building supermin appliance
libguestfs: [00008ms] run supermin
libguestfs: command: run: /usr/bin/supermin5
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.1.10
supermin: rpm: detected RPM version 4.11
supermin: package handler: fedora/rpm
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: build: /usr/lib64/guestfs/supermin.d
supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type
gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type
gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type
uncompressed excludefiles
supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type
uncompressed hostfiles
supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type
gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type
uncompressed packages
supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz
type gzip base image (tar)
supermin: build: 188 packages, including dependencies
supermin: build: 35821 files
supermin: build: 22640 files, after removing unreadable files
supermin: build: 13906 files, after matching excludefiles
supermin: build: 13911 files, after adding hostfiles
supermin: build: 13947 files, after munging
supermin: kernel: picked kernel vmlinuz-3.10.0-327.36.3.el7.x86_64
supermin: kernel: picked modules path
/lib/modules/3.10.0-327.36.3.el7.x86_64
supermin: kernel: kernel_version 3.10.0-327.36.3.el7.x86_64
supermin: kernel: modules /lib/modules/3.10.0-327.36.3.el7.x86_64
supermin: ext2: creating empty ext2 filesystem
'/var/tmp/.guestfs-0/appliance.d.o5v90gm5/root'
supermin: ext2: populating from base image
supermin: ext2: copying files from host filesystem
supermin5: ext2fs-c.c:584: ext2_copy_file: Assertion `n == 1 || dest[n-1]
!= '/'' failed.
libguestfs: error: /usr/bin/supermin5 killed by signal 6 (Aborted), see
debug messages above
libguestfs: clear_socket_create_context: setsockcreatecon failed: NULL:
Invalid argument [you can ignore this UNLESS using SELinux + sVirt]
libguestfs: trace: launch = -1 (error)
libguestfs-test-tool: failed to launch appliance
libguestfs: trace: close
libguestfs: closing guestfs handle 0x7feb544f6430 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfs7oWVq5

On Tue, Nov 15, 2016 at 10:11:38AM +0000, Paul Braham
wrote:
> libguestfs: command: run: /usr/bin/supermin5
> libguestfs: command: run: \ --build
> libguestfs: command: run: \ --verbose
> libguestfs: command: run: \ --if-newer
> libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
> libguestfs: command: run: \ --copy-kernel
> libguestfs: command: run: \ -f ext2
> libguestfs: command: run: \ --host-cpu x86_64
> libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d
> libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
> supermin5: ext2fs-c.c:584: ext2_copy_file: Assertion `n == 1 || dest[n-1]
> != '/'' failed.
> libguestfs: error: /usr/bin/supermin5 killed by signal 6 (Aborted), see
> debug messages above
This is an actual bug in supermin.  There are a couple of things you
can do:

(1) Try upgrading to a newer version of supermin.  RHEL 7.3 has
supermin 5.1.16, also available here:

https://people.redhat.com/~rjones/libguestfs-RHEL-7.3-preview/

If the bug is still present in the newer version, then:

(2) Run supermin with lots of debugging enabled:

  supermin5 -v -v -v -f ext2 /usr/lib64/guestfs/supermin.d -o /var/tmp/appliance

which should narrow down the problem to a particular file or installed
RPM.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

On Tue, Nov 15, 2016 at 10:23:26AM +0000, Richard W.M. Jones
wrote:
> (1) Try upgrading to a newer version of supermin.  RHEL 7.3 has
> supermin 5.1.16, also available here:
> 
> https://people.redhat.com/~rjones/libguestfs-RHEL-7.3-preview/
> 
> If the bug is still present in the newer version, then:
> 
> (2) Run supermin with lots of debugging enabled:
> 
>   supermin5 -v -v -v -f ext2 /usr/lib64/guestfs/supermin.d -o
/var/tmp/appliance
That should be:

supermin5 -v -v -v --build -f ext2 /usr/lib64/guestfs/supermin.d -o
/var/tmp/appliance

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v