Libguestfs - Jun 2015 - Issue with virt-builder on Ubuntu 14.04

Hello,

I am running into problems with virt-builder on my Ubuntu-14.04 install.
Please find the checklist below:

1. What are you trying to achieve?

    I'm trying to build a standard Ubuntu 14.04 disk image.

2. What exact commands did you run?

    I ran "virt-builder ubuntu-14.04"

3. What was the precise error/output of these commands?

   The process gets stuck indefinitely at "Opening the new disk." I
even
left it over night and it did not get any farther.

4. Enable debugging, run the commands again, and capture the complete
output.

    The file attached contains the output from the command with debugging
enabled.

5. Versions of installed software:

libguestfs: 1.28.10 compiled from source tarball but not installed
supermin: 5.1.13 compiled from source tarball but not installed, used
instructions from
https://rwmj.wordpress.com/2014/03/08/tip-old-supermin-new-libguestfs-and-v-v/
to compile libguestfs with supermin5
operating system: Ubuntu 14.04.2 x64
kernel: 3.13.0-55-generic #94-Ubuntu SMP Thu Jun 18 00:27:10 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux

Additional information:

This is a new issue, I used to use libguestfs 1.28.6 (compiled from source)
and had no issues in the past until this weekend. So I compiled the newest
version (1.28.10) and still have this issue.

Additionally, I tried compiling and running it on a virtual machine (Ubuntu
14.04 image created months ago with virt-builder when it was working) and
that works as expected. Therefore I believe that it is an issue with
something on my host, but I can't seem to narrow it down.

I can still use other libguestfs tools such as virt-df, and virt-inspector
(those are the only two that I've tried)

Any help is appreciated.

Thanks,

Shawn

On Mon, Jun 29, 2015 at 08:49:26PM -0600, Shawn Greig wrote:
[...]

Thanks for the detailed bug report.

The issue is indeed something strange with your host or KVM or your
host kernel (3.13.0-55-generic).  As you can see from the end of the
messages:
> [    0.371018] EXT4-fs (sdb): mounting ext2 file system using the ext4
subsystem
> [    0.393840] EXT4-fs (sdb): mounted filesystem without journal. Opts:
> supermin: chroot
> Starting /init script ...
> ln: failed to create symbolic link '/etc/mtab': File exists
> /init: 55: /init: systemd-tmpfiles: not found
> [    1.687079] systemd-udevd[91]: starting version 204
> [    1.808148] random: nonblocking pool is initialized
> [    1.886029] ACPI Exception: AE_BAD_PARAMETER, Thread 487102464 could not
acquire Mutex [0x1] (20131115/utmutex-285)
> [    1.886591] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100,
revision 0
> /init: 79: /init: cannot create /sys/block/{h,s,ub,v}d*/queue/scheduler:
Directory nonexistent
> /sbin/dhclient-script: line 31: /etc/fstab: No such file or directory
> [    2.935872] input: ImExPS/2 Generic Explorer Mouse as
/devices/platform/i8042/serio1/input/input3
... the kernel boots quite far, and then hangs in the init script (see
appliance/init).

Some things you can try:

 - Force qemu to use TCG (no KVM) mode to see if that makes a difference:

   export LIBGUESTFS_BACKEND_SETTINGS=force_tcg

 - Use 'libguestfs-test-tool' for testing to see if that makes a
   difference.  In the build directory, run:

   make quickcheck

 - Try a different appliance kernel by setting these variables, but
   choose different kernel versions by looking in your /boot directory
   first:

   export SUPERMIN_KERNEL_VERSION=3.13.0-55-generic
   export SUPERMIN_KERNEL=/boot/vmlinuz-$SUPERMIN_KERNEL_VERSION
   export SUPERMIN_MODULES=/lib/modules/$SUPERMIN_KERNEL_VERSION
   rm -rf tmp/.guestfs-*
   make quickcheck

 - Add debugging 'set -x' to the top of appliance/init in the source
   tree to see if it fails while running a specific command.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

- I tried forcing TCG and it gets to the same spot and stops.

- running make quickcheck passes the tests (see attached)

- I tried a different kernel (3.13.0-49-generic) and again it passes the
quickcheck but virt-builder hangs.

- I added 'set -x' to the top of the init script and I don't see any
additional output (see attached)

Shawn

On Tue, Jun 30, 2015 at 2:06 AM, Richard W.M. Jones <rjones@redhat.com>
wrote:
> On Mon, Jun 29, 2015 at 08:49:26PM -0600, Shawn Greig wrote:
> [...]
>
> Thanks for the detailed bug report.
>
> The issue is indeed something strange with your host or KVM or your
> host kernel (3.13.0-55-generic).  As you can see from the end of the
> messages:
>
> > [    0.371018] EXT4-fs (sdb): mounting ext2 file system using the ext4
> subsystem
> > [    0.393840] EXT4-fs (sdb): mounted filesystem without journal.
Opts:
> > supermin: chroot
> > Starting /init script ...
> > ln: failed to create symbolic link '/etc/mtab': File exists
> > /init: 55: /init: systemd-tmpfiles: not found
> > [    1.687079] systemd-udevd[91]: starting version 204
> > [    1.808148] random: nonblocking pool is initialized
> > [    1.886029] ACPI Exception: AE_BAD_PARAMETER, Thread 487102464
could
> not acquire Mutex [0x1] (20131115/utmutex-285)
> > [    1.886591] piix4_smbus 0000:00:01.3: SMBus Host Controller at
> 0xb100, revision 0
> > /init: 79: /init: cannot create
/sys/block/{h,s,ub,v}d*/queue/scheduler:
> Directory nonexistent
> > /sbin/dhclient-script: line 31: /etc/fstab: No such file or directory
> > [    2.935872] input: ImExPS/2 Generic Explorer Mouse as
> /devices/platform/i8042/serio1/input/input3
>
> ... the kernel boots quite far, and then hangs in the init script (see
> appliance/init).
>
> Some things you can try:
>
>  - Force qemu to use TCG (no KVM) mode to see if that makes a difference:
>
>    export LIBGUESTFS_BACKEND_SETTINGS=force_tcg
>
>  - Use 'libguestfs-test-tool' for testing to see if that makes a
>    difference.  In the build directory, run:
>
>    make quickcheck
>
>  - Try a different appliance kernel by setting these variables, but
>    choose different kernel versions by looking in your /boot directory
>    first:
>
>    export SUPERMIN_KERNEL_VERSION=3.13.0-55-generic
>    export SUPERMIN_KERNEL=/boot/vmlinuz-$SUPERMIN_KERNEL_VERSION
>    export SUPERMIN_MODULES=/lib/modules/$SUPERMIN_KERNEL_VERSION
>    rm -rf tmp/.guestfs-*
>    make quickcheck
>
>  - Add debugging 'set -x' to the top of appliance/init in the
source
>    tree to see if it fails while running a specific command.
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-top is 'top' for virtual machines.  Tiny program with many
> powerful monitoring features, net stats, disk stats, logging, etc.
> http://people.redhat.com/~rjones/virt-top
>