libvirt users - Aug 2019 - Detach disk from VM - virsh (working) vs. PHP (not working)

Hi all, 

i created a script in PHP for create a virtual server with two QCOW2 discs … one
is our system for installation and second is target system.

After successfully instalation (create a blank Debian system, prepare all files
and grub partitions) i need a restart virtual without a installation disk.

If i use Virsh:

detach-disk --domain debian-test2 --persistent --target vda
reset debian-test2

everything works well.

If i use a PHP, there is „complicated“ way and „simple“ way:

1, complicated:

libvirt_domain_destroy($domain);
libvirt_domain_undefine($domain);

$xml = domain_create_xml($name,$uuid,$memory,$cpu,$vnc,$mac);
$domain = libvirt_domain_define_xml($server->conn, $xml);
libvirt_domain_disk_add($domain, "/users/".$name.".img",
"vdb", "virtio", "qcow2", NULL);
libvirt_domain_create($domain);

(or instead libvirt_domain_disk_add i can define disk directly in XML)

But in this case, the server will not boot (GRUB error)

2, simple:

libvirt_domain_disk_remove($domain,“vda“);
libvirt_domain_reboot($domain);

The problem of this solution is thats not working. The remove disk is failing
with error „Unable attach disk“ - i looks to source code, and yes, there is a
mystake with „attach“/„detach“, but main problem i see in log from libvirt:

Aug  1 02:57:05 ry libvirtd[19051]: missing source information for device vda

I try to put source detail to xml in source of PHP module 

libvirt-domain.c:

  822	    if (asprintf(&newXml,
  823	                 "    <disk type='file'
device='disk'>\n"
  824	                 "      <target dev='%s'/>\n"
  825	                 "    </disk>", dev) < 0) {
  826	        set_error("Out of memory" TSRMLS_CC);
  827	        goto error;
  828	    }

but my attempts was unsuccesfull (i’m not C programmer).

Questions:

A, why complicated way is not working and system dont want boot (GRUB error) if
virsh works fine
B, why libvirt_domain_disk_remove is not work ? I use libvirt and libvirt-php
latest from git.

Thank you, 
Jan

On Thu, Aug 01, 2019 at 10:12:15AM +0200, Jan Horak
wrote:
>Hi all,
>
>i created a script in PHP for create a virtual server with two QCOW2 discs …
one is our system for installation and second is target system.
>
>After successfully instalation (create a blank Debian system, prepare all
files and grub partitions) i need a restart virtual without a installation disk.
>
>If i use Virsh:
>
>detach-disk --domain debian-test2 --persistent --target vda
>reset debian-test2
>
Hi,

I cannot really help you with the PHP part and I'm not sure about more
things,
but this particular scenario is something that libvirt is actually prepared to
do (kind of) and for example virt-install uses it.  The approach is as follows:

1) You create the XML that you want the VM to have when installing the machine,
   let's call it the "install xml".

2) You create the XML that the VM should start with when it is already installed
   and it is the one that will persist in the libvirt configuration for long
   time (well, longer than the install xml).  Let's call this the
"persistent xml".

3) You define the VM using the persistent xml.  That will get you a VM that is
   not started.

4) Then, you start the machine with the install xml, using the create API
   (analogous to virsh create vm.xml).  Just make sure the name and UUID are the
   same.

That will get you a VM running with the install xml that, when turned off and
then started again (without reboot/reset/restart, you need to have it shut down
completely before starting it) will start with the persistent xml.  No need to
have support for disk hot-(un)plug, no magic.

Even better way to use this is registering a callback for the VM's lifecycle
event, starting the VM after the event was fired.  That way you can automate the
installation to turn off the machine after it is completed and there is no need
for any polling.  I'm not sure, however, how relevant that is or if it is
possible to be used from the PHP bindings.

Have a nice day,
Martin

On 8/1/19 10:12 AM, Jan Horak wrote:
> Hi all,
> 
> i created a script in PHP for create a virtual server with two QCOW2 discs
… one is our system for installation and second is target system.
> 
> After successfully instalation (create a blank Debian system, prepare all
files and grub partitions) i need a restart virtual without a installation disk.
> 
> If i use Virsh:
> 
> detach-disk --domain debian-test2 --persistent --target vda
> reset debian-test2
> 
> everything works well.
> 
> If i use a PHP, there is „complicated“ way and „simple“ way:
> 
> 1, complicated:
> 
> libvirt_domain_destroy($domain);
> libvirt_domain_undefine($domain);
> 
> $xml = domain_create_xml($name,$uuid,$memory,$cpu,$vnc,$mac);
> $domain = libvirt_domain_define_xml($server->conn, $xml);
> libvirt_domain_disk_add($domain,
"/users/".$name.".img", "vdb", "virtio",
"qcow2", NULL);
> libvirt_domain_create($domain);
> 
> (or instead libvirt_domain_disk_add i can define disk directly in XML)
> 
> But in this case, the server will not boot (GRUB error)
Question is, how GRUB refers to the disk where kernel is to be found. 
Also, I suspect it is not GRUB that is complaining, but SeaBIOS which 
hasn't found any bootable device. It's only an assumption becasue I 
don't know how domain_create_xml() works - it's not a libvirt-php 
function. There are two possibilities here:

1) make sure domain_create_xml() sets boot devices
2) construct disk XML yourself, and put "<boot
order='N'/>" into it
> 
> 2, simple:
> 
> libvirt_domain_disk_remove($domain,“vda“);
> libvirt_domain_reboot($domain);
> 
> The problem of this solution is thats not working. The remove disk is
failing with error „Unable attach disk“ - i looks to source code, and yes, there
is a mystake with „attach“/„detach“, but main problem i see in log from libvirt:
Oh, that's only typo in the error message. In fact the detach API is 
called. And it fails.
> 
> Aug  1 02:57:05 ry libvirtd[19051]: missing source information for device
vda
> 
> I try to put source detail to xml in source of PHP module
> 
> libvirt-domain.c:
> 
>    822	    if (asprintf(&newXml,
>    823	                 "    <disk type='file'
device='disk'>\n"
>    824	                 "      <target
dev='%s'/>\n"
>    825	                 "    </disk>", dev) < 0) {
>    826	        set_error("Out of memory" TSRMLS_CC);
>    827	        goto error;
>    828	    }
> 
> but my attempts was unsuccesfull (i’m not C programmer).
Yes, this minimalistic XML is not good as detach API requires full 
device XML. I'll fix this soon.
> 
> Questions:
> 
> A, why complicated way is not working and system dont want boot (GRUB
error) if virsh works fine
> B, why libvirt_domain_disk_remove is not work ? I use libvirt and
libvirt-php latest from git.
I've pushed fixes here:


Please give it a try.

Michal

Thank you for your help!

1, i used vda as install disk and vdb as target disk, because i boot from the
first. After setup boot order flag i was able change install disk to vdb and
target disk to vda. With this configuration when i boot from vdb i can install
OS with grub and after it redefine XML without vdb without problem. Thanks a
lot.

2, maybe i have something wrong, but still not working:

Now in libvirt log i got error: device not found: no target device vdb

but:

from VPS:

root@debian:~# ls /dev/vd*
/dev/vda  /dev/vda1  /dev/vdb  /dev/vdb1

from XML (virsh dumpxml):

    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/users/debian-test2.img'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x01'
slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/users/debian-test2-install.img'/>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x04'
slot='0x00' function='0x0'/>
    </disk>

and from libvirt_domain_get_disk_devices($domain):

Array ( [0] => vda [1] => vdb [num] => 2 )

As you see, there is vdb device in all cases.

PHP code is very simple:

$server =
libvirt_connect("qemu+tls://".$ip.“:“.$port."/system",
false);
$domain = libvirt_domain_lookup_by_name($server, "debian-test2");
libvirt_domain_disk_remove($domain,"vdb");
print_r(libvirt_get_last_error());

res: Unable to detach disk
> 2. 8. 2019 v 14:15, Michal Privoznik <mprivozn@redhat.com>:
> 
> On 8/1/19 10:12 AM, Jan Horak wrote:
>> Hi all,
>> i created a script in PHP for create a virtual server with two QCOW2
discs … one is our system for installation and second is target system.
>> After successfully instalation (create a blank Debian system, prepare
all files and grub partitions) i need a restart virtual without a installation
disk.
>> If i use Virsh:
>> detach-disk --domain debian-test2 --persistent --target vda
>> reset debian-test2
>> everything works well.
>> If i use a PHP, there is „complicated“ way and „simple“ way:
>> 1, complicated:
>> libvirt_domain_destroy($domain);
>> libvirt_domain_undefine($domain);
>> $xml = domain_create_xml($name,$uuid,$memory,$cpu,$vnc,$mac);
>> $domain = libvirt_domain_define_xml($server->conn, $xml);
>> libvirt_domain_disk_add($domain,
"/users/".$name.".img", "vdb", "virtio",
"qcow2", NULL);
>> libvirt_domain_create($domain);
>> (or instead libvirt_domain_disk_add i can define disk directly in XML)
>> But in this case, the server will not boot (GRUB error)
> 
> Question is, how GRUB refers to the disk where kernel is to be found. Also,
I suspect it is not GRUB that is complaining, but SeaBIOS which hasn't found
any bootable device. It's only an assumption becasue I don't know how
domain_create_xml() works - it's not a libvirt-php function. There are two
possibilities here:
> 
> 1) make sure domain_create_xml() sets boot devices
> 2) construct disk XML yourself, and put "<boot
order='N'/>" into it
> 
>> 2, simple:
>> libvirt_domain_disk_remove($domain,“vda“);
>> libvirt_domain_reboot($domain);
>> The problem of this solution is thats not working. The remove disk is
failing with error „Unable attach disk“ - i looks to source code, and yes, there
is a mystake with „attach“/„detach“, but main problem i see in log from libvirt:
> 
> Oh, that's only typo in the error message. In fact the detach API is
called. And it fails.
> 
>> Aug  1 02:57:05 ry libvirtd[19051]: missing source information for
device vda
>> I try to put source detail to xml in source of PHP module
>> libvirt-domain.c:
>>   822	    if (asprintf(&newXml,
>>   823	                 "    <disk type='file'
device='disk'>\n"
>>   824	                 "      <target
dev='%s'/>\n"
>>   825	                 "    </disk>", dev) < 0) {
>>   826	        set_error("Out of memory" TSRMLS_CC);
>>   827	        goto error;
>>   828	    }
>> but my attempts was unsuccesfull (i’m not C programmer).
> 
> Yes, this minimalistic XML is not good as detach API requires full device
XML. I'll fix this soon.
> 
>> Questions:
>> A, why complicated way is not working and system dont want boot (GRUB
error) if virsh works fine
>> B, why libvirt_domain_disk_remove is not work ? I use libvirt and
libvirt-php latest from git.
> 
> I've pushed fixes here:
> 
> 
> Please give it a try.
> 
> Michal