libvirt users - Apr 2016 - Re: Create multiple domains from single saved domain state (is UUID/name fixed?)

(please don't top-post. Put your responses inline, in context)

On 04/19/2016 01:09 PM, Jonas Finnemann Jensen wrote:
> virt-builder looks like some fancy guest/host interaction related to 
> building VM images.
>
> What I'm looking for is more like:
>   virsh save running_domain saved-domain-A.img
>   cp saved-domain-A.img saved-domain-B.img
>   virsh save-image-edit saved-domain-B.img   // Change the network, 
> possibly MAC, VNC port
You'll also need to change the name and uuid of the domain at the very 
least. And I assume these will all be transient domains, not persistent.
>
> Then in parallel I want to do:
>   virsh restore saved-domain-A.img
>   virsh restore saved-domain-B.img
If you do that (restore a previously running image with a different MAC 
address), at the very least the guest OS will be confused about the MAC 
address of the network card, and you'll very likely end up with both 
guests responding to ARP requests for the original MAC address. There's 
likely other problems that I haven't thought of that will happen as well.
>
> So that I have two instances of the same virtual machine starting from 
> the same state.
> This way I can reset the VMs without having to reboot them (booting is 
> rather slow).
>
> I practice I'll probably have ~16 instances at the same time. 
> Constantly being reset to the same state.
> I tried with QEMU, and it's seems totally doable with savevm, copy 
> file, then doing loadvm twice in parallel.
> (I'll be using a separate network for each VM, so I can be sure which 
> one I'm talking to).
Well, as long as they're completely isolated from each other, you may 
have a better chance of success. However there will still be the issue 
of the IP address of the network interface. You can't have two networks 
using the same IP range (since libvirt doesn't use network namespaces 
for its networks), so the guest will need to change its IP, which means 
it will need to be notified of this need, possibly by having the host 
toggle the interface offline and back on - you can use virsh 
domif-setlink to do this.

>
> Is this doable with libvirt, or am I better off using QEMU directly? 
> and how? I couldn't do internal snapshots with --live, and 
> snapshot-revert says it can't revert to external snapshots yet :)
> (using QEMU directly would certainly leave me with a lot of manual 
> network configuration)
Someone else will have to talk about the particulars of snapshots...

2016-04-20 0:26 GMT+03:00 Laine Stump
<laine@laine.org>:
> Well, as long as they're completely isolated from each other, you may
have a
> better chance of success. However there will still be the issue of the IP
> address of the network interface. You can't have two networks using the
same
> IP range (since libvirt doesn't use network namespaces for its
networks), so
> the guest will need to change its IP, which means it will need to be
> notified of this need, possibly by having the host toggle the interface
> offline and back on - you can use virsh domif-setlink to do this.

If you don't need access from external net - you can use SLIRP for
network, in this case you does not have problems with identical mac
address...

-- 
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru

>
> You'll also need to change the name and uuid of the domain at the very
> least.
Agree, but is that possible with libvirt?

If you do that (restore a previously running image with a different
MAC
> address)
Yeah,  probably I wouldn't change the MAC address. As I want to attach the
VMs to different networks.
I rely on the IP being in a different subnet to identify the VM in my
metadata service.
Using IP filters to enforce the subnet seems like the most robust way of
being sure which VM I'm talking to.

possibly by having the host toggle the interface offline and back on

Yeah, I think unplugging the virtual network cable before I save the VM
memory, and plugging it back in after I load the VM, then DHCP would run
immediately.
As an added benefit any guest program I have talking to my meta-data
service would be able to detect that the VM has been loaded, by looking for
network connection.


--
Regards Jonas Finnemann Jensen.

2016-04-19 14:26 GMT-07:00 Laine Stump <laine@laine.org>:
> (please don't top-post. Put your responses inline, in context)
>
> On 04/19/2016 01:09 PM, Jonas Finnemann Jensen wrote:
>
>> virt-builder looks like some fancy guest/host interaction related to
>> building VM images.
>>
>> What I'm looking for is more like:
>>   virsh save running_domain saved-domain-A.img
>>   cp saved-domain-A.img saved-domain-B.img
>>   virsh save-image-edit saved-domain-B.img   // Change the network,
>> possibly MAC, VNC port
>>
>
> You'll also need to change the name and uuid of the domain at the very
> least. And I assume these will all be transient domains, not persistent.
>
>
>> Then in parallel I want to do:
>>   virsh restore saved-domain-A.img
>>   virsh restore saved-domain-B.img
>>
>
> If you do that (restore a previously running image with a different MAC
> address), at the very least the guest OS will be confused about the MAC
> address of the network card, and you'll very likely end up with both
guests
> responding to ARP requests for the original MAC address. There's likely
> other problems that I haven't thought of that will happen as well.
>
>
>> So that I have two instances of the same virtual machine starting from
>> the same state.
>> This way I can reset the VMs without having to reboot them (booting is
>> rather slow).
>>
>> I practice I'll probably have ~16 instances at the same time.
Constantly
>> being reset to the same state.
>> I tried with QEMU, and it's seems totally doable with savevm, copy
file,
>> then doing loadvm twice in parallel.
>> (I'll be using a separate network for each VM, so I can be sure
which one
>> I'm talking to).
>>
>
> Well, as long as they're completely isolated from each other, you may
have
> a better chance of success. However there will still be the issue of the IP
> address of the network interface. You can't have two networks using the
> same IP range (since libvirt doesn't use network namespaces for its
> networks), so the guest will need to change its IP, which means it will
> need to be notified of this need, possibly by having the host toggle the
> interface offline and back on - you can use virsh domif-setlink to do this.
>
>
>
>> Is this doable with libvirt, or am I better off using QEMU directly?
and
>> how? I couldn't do internal snapshots with --live, and
snapshot-revert says
>> it can't revert to external snapshots yet :)
>> (using QEMU directly would certainly leave me with a lot of manual
>> network configuration)
>>
>
> Someone else will have to talk about the particulars of snapshots...
>
>

On Tue, Apr 19, 2016 at 05:26:50PM -0400, Laine Stump
wrote:
>(please don't top-post. Put your responses inline, in context)
>
>On 04/19/2016 01:09 PM, Jonas Finnemann Jensen wrote:
>> virt-builder looks like some fancy guest/host interaction related to
>> building VM images.
>>
>> What I'm looking for is more like:
>>   virsh save running_domain saved-domain-A.img
>>   cp saved-domain-A.img saved-domain-B.img
>>   virsh save-image-edit saved-domain-B.img   // Change the network,
>> possibly MAC, VNC port
>
>You'll also need to change the name and uuid of the domain at the very
>least. And I assume these will all be transient domains, not persistent.
>
>>
>> Then in parallel I want to do:
>>   virsh restore saved-domain-A.img
>>   virsh restore saved-domain-B.img
>
>If you do that (restore a previously running image with a different MAC
>address), at the very least the guest OS will be confused about the MAC
>address of the network card, and you'll very likely end up with both
>guests responding to ARP requests for the original MAC address. There's
>likely other problems that I haven't thought of that will happen as
well.
>
>>
>> So that I have two instances of the same virtual machine starting from
>> the same state.
>> This way I can reset the VMs without having to reboot them (booting is
>> rather slow).
>>
>> I practice I'll probably have ~16 instances at the same time.
>> Constantly being reset to the same state.
>> I tried with QEMU, and it's seems totally doable with savevm, copy
>> file, then doing loadvm twice in parallel.
>> (I'll be using a separate network for each VM, so I can be sure
which
>> one I'm talking to).
>
>Well, as long as they're completely isolated from each other, you may
>have a better chance of success. However there will still be the issue
>of the IP address of the network interface. You can't have two networks
>using the same IP range (since libvirt doesn't use network namespaces
>for its networks), so the guest will need to change its IP, which means
>it will need to be notified of this need, possibly by having the host
>toggle the interface offline and back on - you can use virsh
>domif-setlink to do this.
>
>
>>
>> Is this doable with libvirt, or am I better off using QEMU directly?
>> and how? I couldn't do internal snapshots with --live, and
>> snapshot-revert says it can't revert to external snapshots yet :)
>> (using QEMU directly would certainly leave me with a lot of manual
>> network configuration)
>
>Someone else will have to talk about the particulars of snapshots...
>
I'm sure you can do live snapshot of the VM and restore to it.  It
must've been some issue with your particular setup or something.  But
even if we sole that, I'm not sure it will help you.  Yes, you would be
able to restore to that state, but running multiple domains at once from
the same saved state...

On Tue, Apr 19, 2016 at 03:22:18PM -0700, Jonas Finnemann Jensen
wrote:
>>
>> You'll also need to change the name and uuid of the domain at the
very
>> least.
>
>Agree, but is that possible with libvirt?
>
Not in a supported way.  But you can, technically, edit the save file
(not using virsh), change the name and uuid and restore it.  But don't
seek help when something doesn't work for you after that =)
>If you do that (restore a previously running image with a different MAC
>> address)
>
>Yeah,  probably I wouldn't change the MAC address. As I want to attach
the
>VMs to different networks.
>I rely on the IP being in a different subnet to identify the VM in my
>metadata service.
>Using IP filters to enforce the subnet seems like the most robust way of
>being sure which VM I'm talking to.
>
>possibly by having the host toggle the interface offline and back on
>
>Yeah, I think unplugging the virtual network cable before I save the VM
>memory, and plugging it back in after I load the VM, then DHCP would run
>immediately.
>As an added benefit any guest program I have talking to my meta-data
>service would be able to detect that the VM has been loaded, by looking for
>network connection.
>
>
>--
>Regards Jonas Finnemann Jensen.
>
>2016-04-19 14:26 GMT-07:00 Laine Stump <laine@laine.org>:
>
>> (please don't top-post. Put your responses inline, in context)
>>
>> On 04/19/2016 01:09 PM, Jonas Finnemann Jensen wrote:
>>
>>> virt-builder looks like some fancy guest/host interaction related
to
>>> building VM images.
>>>
>>> What I'm looking for is more like:
>>>   virsh save running_domain saved-domain-A.img
>>>   cp saved-domain-A.img saved-domain-B.img
>>>   virsh save-image-edit saved-domain-B.img   // Change the network,
>>> possibly MAC, VNC port
>>>
>>
>> You'll also need to change the name and uuid of the domain at the
very
>> least. And I assume these will all be transient domains, not
persistent.
>>
>>
>>> Then in parallel I want to do:
>>>   virsh restore saved-domain-A.img
>>>   virsh restore saved-domain-B.img
>>>
>>
>> If you do that (restore a previously running image with a different MAC
>> address), at the very least the guest OS will be confused about the MAC
>> address of the network card, and you'll very likely end up with
both guests
>> responding to ARP requests for the original MAC address. There's
likely
>> other problems that I haven't thought of that will happen as well.
>>
>>
>>> So that I have two instances of the same virtual machine starting
from
>>> the same state.
>>> This way I can reset the VMs without having to reboot them (booting
is
>>> rather slow).
>>>
>>> I practice I'll probably have ~16 instances at the same time.
Constantly
>>> being reset to the same state.
>>> I tried with QEMU, and it's seems totally doable with savevm,
copy file,
>>> then doing loadvm twice in parallel.
>>> (I'll be using a separate network for each VM, so I can be sure
which one
>>> I'm talking to).
>>>
>>
>> Well, as long as they're completely isolated from each other, you
may have
>> a better chance of success. However there will still be the issue of
the IP
>> address of the network interface. You can't have two networks using
the
>> same IP range (since libvirt doesn't use network namespaces for its
>> networks), so the guest will need to change its IP, which means it will
>> need to be notified of this need, possibly by having the host toggle
the
>> interface offline and back on - you can use virsh domif-setlink to do
this.
>>
>>
>>
>>> Is this doable with libvirt, or am I better off using QEMU
directly? and
>>> how? I couldn't do internal snapshots with --live, and
snapshot-revert says
>>> it can't revert to external snapshots yet :)
>>> (using QEMU directly would certainly leave me with a lot of manual
>>> network configuration)
>>>
>>
>> Someone else will have to talk about the particulars of snapshots...
>>
>>