Libguestfs - Dec 2018 - Re: [PATCH FOR DISCUSSION ONLY 0/2] v2v: Copy static IP address information over for Windows guests (RHBZ#1626503).

On Wed, Dec 05, 2018 at 09:01:16AM +0000, Roman Kagan
wrote:
> On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones wrote:
> > This patch is just for discussion.  There are still a couple of issues
> > that I'm trying to fix.
> 
> Sorry if I missed the original discussion on that, but why is copying
> the static IP configuraition even considered to be a good thing?
>
> For one, it's unlikely that the IP address will remain valid in the
> target environment.
For our v2v conversions (generally from VMware to oVirt or OpenStack)
we're trying to maintain a near-identical network configuration on
both sides.  So for example a guest will have the same MAC addresses
before and after and, if using DHCP, will pick up the exact same IP
address (probably served by the same DHCP server) and be able to
access the same set of services etc.

Also we're doing like 1000s of VMs in one go, so manual
reconfiguration of anything is a non-starter.

However let me describe the problem we're seeing, as it is a bit more
general and might even apply in some cloud scenarios.  If you have a
Windows guest which starts out with a static IP address (no DHCP) it
might be configured like this on VMware:

  00:50:56:01:02:03   192.168.128.10   vmxnet3  "Ethernet0" 
EnableDHCP=0

After conversion we preserve the MAC address and add virtio drivers,
but the new netkvm.sys driver just adds a new network interface so
it'll end up like this:

  (no hardware)       192.168.128.10   vmxnet3  "Ethernet0" 
EnableDHCP=0
  00:50:56:01:02:03   (no address)     netkvm   "Ethernet1" 
EnableDHCP=1

It has no address in the second case because this is a non-DHCP
network.  (The DHCP situation is better because the second network
will be assigned the right IP address because of the preserved MAC
address, although it still has the unnecessary network adapter.)

Note that Windows < 10 doesn't have a concept of persistent network
names like we do in Linux which mostly solves this problem.  (Windows
10 on the other hand does save the MAC address in the Registry, but I
haven't fully investigated yet what it's doing.)

Anyway if you have another suggestion how to solve this, I'm all ears ...
For other areas we'd fix things up using an Ansible post-copy script,
but in this case we can't do that since Ansible cannot reconfigure a
guest that has no working network.
> For two, in a typical cloud setup static IP addresses are configured by
> some sort of a cloud-specific management agent, and bypassing it is
> likely to cause conflicts.
> 
> For three, network configuration is a notoriously complex thing.  I
> think I can recall a dozen of network configuration systems in Linux
> that claimed to provide the ultimate solution to all problems and none
> that delivered to the promise.  I guess in Windows the situation is
> hardly much simpler.  Creating yet another universal solution doesn't
> appear to be in scope for v2v (if realistic at all); and I'm not sure
> that having a very limited solution for a very basic use case only is
> better than explicitly telling the user not to rely on v2v for network
> configuration.
Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

On Wed, Dec 5, 2018 at 12:01 PM Richard W.M. Jones <rjones@redhat.com>
wrote:
> On Wed, Dec 05, 2018 at 09:01:16AM +0000, Roman Kagan wrote:
> > On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones wrote:
> > > This patch is just for discussion.  There are still a couple of
issues
> > > that I'm trying to fix.
> >
> > Sorry if I missed the original discussion on that, but why is copying
> > the static IP configuraition even considered to be a good thing?
> >
> > For one, it's unlikely that the IP address will remain valid in
the
> > target environment.
>
> For our v2v conversions (generally from VMware to oVirt or OpenStack)
> we're trying to maintain a near-identical network configuration on
> both sides.  So for example a guest will have the same MAC addresses
> before and after and, if using DHCP, will pick up the exact same IP
> address (probably served by the same DHCP server) and be able to
> access the same set of services etc.
>
> Also we're doing like 1000s of VMs in one go, so manual
> reconfiguration of anything is a non-starter.
>
> However let me describe the problem we're seeing, as it is a bit more
> general and might even apply in some cloud scenarios.  If you have a
> Windows guest which starts out with a static IP address (no DHCP) it
> might be configured like this on VMware:
>
>   00:50:56:01:02:03   192.168.128.10   vmxnet3  "Ethernet0" 
EnableDHCP=0
>
> After conversion we preserve the MAC address and add virtio drivers,
> but the new netkvm.sys driver just adds a new network interface so
> it'll end up like this:
>
>   (no hardware)       192.168.128.10   vmxnet3  "Ethernet0" 
EnableDHCP=0
>   00:50:56:01:02:03   (no address)     netkvm   "Ethernet1" 
EnableDHCP=1
>
> It has no address in the second case because this is a non-DHCP
> network.  (The DHCP situation is better because the second network
> will be assigned the right IP address because of the preserved MAC
> address, although it still has the unnecessary network adapter.)
>
> Note that Windows < 10 doesn't have a concept of persistent network
> names like we do in Linux which mostly solves this problem.  (Windows
> 10 on the other hand does save the MAC address in the Registry, but I
> haven't fully investigated yet what it's doing.)
>
> Anyway if you have another suggestion how to solve this, I'm all ears
...
> For other areas we'd fix things up using an Ansible post-copy script,
> but in this case we can't do that since Ansible cannot reconfigure a
> guest that has no working network.
>
As you mention, using Ansible means that the IP address of the VM is
correctly set.
In another discussion, you mentioned using transient DHCP, but would mean
that we reconfigure one of the NICs before conversion to use DHCP. That
would alter the original VM.
And in the case of multiple NICs, how do we know which one to reconfigure.
That would be quite specific to each customer, meaning customized playbook
/ role.
And IIRC, there's a good chance that it breaks WinRM configuration with
regards to certificates, meaning reconfiguring WinRM after DHCP and after
conversion.
But maybe I'm too pessimistic ;)

> > For two, in a typical cloud setup static IP addresses are configured
by
> > some sort of a cloud-specific management agent, and bypassing it is
> > likely to cause conflicts.
>
In the case of OpenStack, we keep the MAC address by creating the network
ports with it and the IP address pre-allocated. This allows neutron to have
working security groups.
As Richard says, we're looking at the case of a massive migration where
adapting the VM is out of scope. It's a mainly a 1:1 relationship. Anyway,
we're still looking at more complex scenarios with tenant networks and
floating IPs. It promises to be fun :)

> > For three, network configuration is a notoriously complex thing.  I
> > think I can recall a dozen of network configuration systems in Linux
> > that claimed to provide the ultimate solution to all problems and none
> > that delivered to the promise.  I guess in Windows the situation is
> > hardly much simpler.  Creating yet another universal solution
doesn't
> > appear to be in scope for v2v (if realistic at all); and I'm not
sure
> > that having a very limited solution for a very basic use case only is
> > better than explicitly telling the user not to rely on v2v for network
> > configuration.
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-p2v converts physical machines to virtual machines.  Boot with a
> live CD or over the network (PXE) and turn machines into KVM guests.
> http://libguestfs.org/virt-v2v
>
> _______________________________________________
> Libguestfs mailing list
> Libguestfs@redhat.com
> https://www.redhat.com/mailman/listinfo/libguestfs
>

-- 

*Fabien Dupont*

PRINCIPAL SOFTWARE ENGINEER

Red Hat - Solutions Engineering

fabien@redhat.com     M: +33 (0) 662 784 971 <+33662784971>

<http://redhat.com>  *TRIED. TESTED. TRUSTED.*

Twitter: @redhatway <https://twitter.com/redhatway> | Instagram:
@redhatinc
<https://www.instagram.com/redhatinc/> | Snapchat: @redhatsnaps

On Wed, Dec 05, 2018 at 01:18:49PM +0100, Fabien Dupont
wrote:
> On Wed, Dec 5, 2018 at 12:01 PM Richard W.M. Jones
<rjones@redhat.com>
> wrote:
> 
> > On Wed, Dec 05, 2018 at 09:01:16AM +0000, Roman Kagan wrote:
> > > On Tue, Dec 04, 2018 at 05:29:31PM +0000, Richard W.M. Jones
wrote:
> > > > This patch is just for discussion.  There are still a couple
of issues
> > > > that I'm trying to fix.
> > >
> > > Sorry if I missed the original discussion on that, but why is
copying
> > > the static IP configuraition even considered to be a good thing?
> > >
> > > For one, it's unlikely that the IP address will remain valid
in the
> > > target environment.
> >
> > For our v2v conversions (generally from VMware to oVirt or OpenStack)
> > we're trying to maintain a near-identical network configuration on
> > both sides.  So for example a guest will have the same MAC addresses
> > before and after and, if using DHCP, will pick up the exact same IP
> > address (probably served by the same DHCP server) and be able to
> > access the same set of services etc.
> >
> > Also we're doing like 1000s of VMs in one go, so manual
> > reconfiguration of anything is a non-starter.
> >
> > However let me describe the problem we're seeing, as it is a bit
more
> > general and might even apply in some cloud scenarios.  If you have a
> > Windows guest which starts out with a static IP address (no DHCP) it
> > might be configured like this on VMware:
> >
> >   00:50:56:01:02:03   192.168.128.10   vmxnet3  "Ethernet0" 
EnableDHCP=0
> >
> > After conversion we preserve the MAC address and add virtio drivers,
> > but the new netkvm.sys driver just adds a new network interface so
> > it'll end up like this:
> >
> >   (no hardware)       192.168.128.10   vmxnet3  "Ethernet0" 
EnableDHCP=0
> >   00:50:56:01:02:03   (no address)     netkvm   "Ethernet1" 
EnableDHCP=1
> >
> > It has no address in the second case because this is a non-DHCP
> > network.  (The DHCP situation is better because the second network
> > will be assigned the right IP address because of the preserved MAC
> > address, although it still has the unnecessary network adapter.)
> >
> > Note that Windows < 10 doesn't have a concept of persistent
network
> > names like we do in Linux which mostly solves this problem.  (Windows
> > 10 on the other hand does save the MAC address in the Registry, but I
> > haven't fully investigated yet what it's doing.)
> >
> > Anyway if you have another suggestion how to solve this, I'm all
ears ...
> > For other areas we'd fix things up using an Ansible post-copy
script,
> > but in this case we can't do that since Ansible cannot reconfigure
a
> > guest that has no working network.
> >
> 
> As you mention, using Ansible means that the IP address of the VM is
> correctly set.
> In another discussion, you mentioned using transient DHCP, but would mean
> that we reconfigure one of the NICs before conversion to use DHCP. That
> would alter the original VM.
No I don't think so.  The newly added netkvm NIC seems to come up with
EnableDHCP=1 (although NB I did not test in the end to end case
whether this means the Windows guest will get an address if a DHCP
server is available).
> And in the case of multiple NICs, how do we know which one to reconfigure.
The current patch has the same problem.  In the multiple NIC case for
Windows < 10 we don't know how the NICs were assigned to guest
hardware on the source and it seems hard to know without looking at
PCI address assignments which opens up a whole can of worms.

In Windows >= 10 it looks as if Windows is finally storing the
hardware MAC in the registry associated with each interface, so maybe
this problem will disappear in future.
> That would be quite specific to each customer, meaning customized
> playbook / role.
This is correct.
> And IIRC, there's a good chance that it breaks WinRM configuration with
> regards to certificates, meaning reconfiguring WinRM after DHCP and after
> conversion.
What is “WinRM configuration”?
> But maybe I'm too pessimistic ;)
> 
> 
> > > For two, in a typical cloud setup static IP addresses are
configured by
> > > some sort of a cloud-specific management agent, and bypassing it
is
> > > likely to cause conflicts.
> >
> 
> In the case of OpenStack, we keep the MAC address by creating the network
> ports with it and the IP address pre-allocated. This allows neutron to have
> working security groups.
> As Richard says, we're looking at the case of a massive migration where
> adapting the VM is out of scope. It's a mainly a 1:1 relationship.
Anyway,
> we're still looking at more complex scenarios with tenant networks and
> floating IPs. It promises to be fun :)
> 
> 
> > > For three, network configuration is a notoriously complex thing. 
I
> > > think I can recall a dozen of network configuration systems in
Linux
> > > that claimed to provide the ultimate solution to all problems and
none
> > > that delivered to the promise.  I guess in Windows the situation
is
> > > hardly much simpler.  Creating yet another universal solution
doesn't
> > > appear to be in scope for v2v (if realistic at all); and I'm
not sure
> > > that having a very limited solution for a very basic use case
only is
> > > better than explicitly telling the user not to rely on v2v for
network
> > > configuration.
> >
> > Rich.
Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top