Libguestfs - Feb 2018 - Re: [PATCH] v2v: remove MAC address related information

On Tue, Feb 27, 2018 at 12:53:08PM +0100, Pino Toscano
wrote:
> On Tuesday, 27 February 2018 12:35:36 CET Tomáš Golembiovský wrote:
> > Remove ties to MAC address because it is likely to change.
> 
> v2v tries to preserve the MAC address of network interfaces; few months
> ago we did a fix regarding this:
> https://bugzilla.redhat.com/show_bug.cgi?id=1506572
> 
> The approach of this patch is IMHO not good, since it removes the MAC
> address from the network-scripts, but still the rest of v2v will try
> to preserve the MAC addresses.
We preserve the MAC address in metadata.  On the other hand AIUI this
patch only removes the association in the ifcfg files and the guest
will reassociate it when it boots (albeit it might then mix up the
ethernet interfaces so that's not good).

There's IMHO a bigger problem which is not being addressed:

https://bugzilla.redhat.com/show_bug.cgi?id=1318922
> What's the reason behind this patch?
There's a bit of background which is missing.  Tomáš and I had some
discussions (privately, unfortunately) with the ManageIQ developers
who are integrating virt-v2v into MIQ/CloudForms.  Their existing
software runs a separate virt-sysprep step on guests after they have
been converted by virt-v2v.  They disable all sysprep the operations
except for just a couple, including ‘net-hwaddr’, so the effect is
roughly the same as this patch.

The question was raised why they need to do that as a separate step
and why virt-v2v doesn't do it.  And indeed there was some discussion
about whether or not converted guests need a new MAC address -- it's
at best unclear -- it is thought that VMware might reuse MAC addresses
which have "left" the hypervisor, although no one knows if that's
really true or not.

I don't have much opinion on this.  Maybe it's best for CFME to
continue to run virt-sysprep as a separate step.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Running sysprep is quite easy today because we use the export domain, which
is frequently NFS. So we simply mount it on the conversion host and run
sysprep against the disks of the converted VM before it is imported.
Soon, we want to skip the export domain, so the disks of the VM will be in
a separate domain. We can still find them and run the sysprep, but I have
the feeling that it would be more logical if virt-v2v did the sysprep when
target is oVirt / RHV.

On 27 February 2018 at 13:34, Richard W.M. Jones <rjones@redhat.com>
wrote:
> On Tue, Feb 27, 2018 at 12:53:08PM +0100, Pino Toscano wrote:
> > On Tuesday, 27 February 2018 12:35:36 CET Tomáš Golembiovský wrote:
> > > Remove ties to MAC address because it is likely to change.
> >
> > v2v tries to preserve the MAC address of network interfaces; few
months
> > ago we did a fix regarding this:
> > https://bugzilla.redhat.com/show_bug.cgi?id=1506572
> >
> > The approach of this patch is IMHO not good, since it removes the MAC
> > address from the network-scripts, but still the rest of v2v will try
> > to preserve the MAC addresses.
>
> We preserve the MAC address in metadata.  On the other hand AIUI this
> patch only removes the association in the ifcfg files and the guest
> will reassociate it when it boots (albeit it might then mix up the
> ethernet interfaces so that's not good).
>
> There's IMHO a bigger problem which is not being addressed:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1318922
>
> > What's the reason behind this patch?
>
> There's a bit of background which is missing.  Tomáš and I had some
> discussions (privately, unfortunately) with the ManageIQ developers
> who are integrating virt-v2v into MIQ/CloudForms.  Their existing
> software runs a separate virt-sysprep step on guests after they have
> been converted by virt-v2v.  They disable all sysprep the operations
> except for just a couple, including ‘net-hwaddr’, so the effect is
> roughly the same as this patch.
>
> The question was raised why they need to do that as a separate step
> and why virt-v2v doesn't do it.  And indeed there was some discussion
> about whether or not converted guests need a new MAC address -- it's
> at best unclear -- it is thought that VMware might reuse MAC addresses
> which have "left" the hypervisor, although no one knows if
that's
> really true or not.
>
> I don't have much opinion on this.  Maybe it's best for CFME to
> continue to run virt-sysprep as a separate step.
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~
> rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines.  Supports shell scripting,
> bindings from many languages.  http://libguestfs.org
>


-- 

*Fabien Dupont*

PRINCIPAL SOFTWARE ENGINEER

Red Hat - Solutions Engineering

fabien@redhat.com     M: +33 (0) 662 784 971 <+33662784971>

<http://redhat.com>  *TRIED. TESTED. TRUSTED.*

Twitter: @redhatway <https://twitter.com/redhatway> | Instagram:
@redhatinc
<https://www.instagram.com/redhatinc/> | Snapchat: @redhatsnaps

On Tue, 27 Feb 2018 13:43:59 +0100
Fabien Dupont <fdupont@redhat.com> wrote:
> We can still find them and run the sysprep, but I have
> the feeling that it would be more logical if virt-v2v did the sysprep when
> target is oVirt / RHV.
This is trickier than you think. For LVM volumes somebody (VDSM) has to
lock and prepare the disks for you first and there is no external API to
do that AFAIK.

    Tomas

> 
> On 27 February 2018 at 13:34, Richard W.M. Jones <rjones@redhat.com>
wrote:
> 
> > On Tue, Feb 27, 2018 at 12:53:08PM +0100, Pino Toscano wrote:  
> > > On Tuesday, 27 February 2018 12:35:36 CET Tomáš Golembiovský
wrote:
> > > > Remove ties to MAC address because it is likely to change.  
> > >
> > > v2v tries to preserve the MAC address of network interfaces; few
months
> > > ago we did a fix regarding this:
> > > https://bugzilla.redhat.com/show_bug.cgi?id=1506572
> > >
> > > The approach of this patch is IMHO not good, since it removes the
MAC
> > > address from the network-scripts, but still the rest of v2v will
try
> > > to preserve the MAC addresses.  
> >
> > We preserve the MAC address in metadata.  On the other hand AIUI this
> > patch only removes the association in the ifcfg files and the guest
> > will reassociate it when it boots (albeit it might then mix up the
> > ethernet interfaces so that's not good).
> >
> > There's IMHO a bigger problem which is not being addressed:
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=1318922
> >  
> > > What's the reason behind this patch?  
> >
> > There's a bit of background which is missing.  Tomáš and I had
some
> > discussions (privately, unfortunately) with the ManageIQ developers
> > who are integrating virt-v2v into MIQ/CloudForms.  Their existing
> > software runs a separate virt-sysprep step on guests after they have
> > been converted by virt-v2v.  They disable all sysprep the operations
> > except for just a couple, including ‘net-hwaddr’, so the effect is
> > roughly the same as this patch.
> >
> > The question was raised why they need to do that as a separate step
> > and why virt-v2v doesn't do it.  And indeed there was some
discussion
> > about whether or not converted guests need a new MAC address --
it's
> > at best unclear -- it is thought that VMware might reuse MAC addresses
> > which have "left" the hypervisor, although no one knows if
that's
> > really true or not.
> >
> > I don't have much opinion on this.  Maybe it's best for CFME
to
> > continue to run virt-sysprep as a separate step.
> >
> > Rich.
> >
> > --
> > Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~
> > rjones
> > Read my programming and virtualization blog: http://rwmj.wordpress.com
> > libguestfs lets you edit virtual machines.  Supports shell scripting,
> > bindings from many languages.  http://libguestfs.org
> >  
> 
> 
> 
> -- 
> 
> *Fabien Dupont*
> 
> PRINCIPAL SOFTWARE ENGINEER
> 
> Red Hat - Solutions Engineering
> 
> fabien@redhat.com     M: +33 (0) 662 784 971 <+33662784971>
> 
> <http://redhat.com>  *TRIED. TESTED. TRUSTED.*
> 
> Twitter: @redhatway <https://twitter.com/redhatway> | Instagram:
@redhatinc
> <https://www.instagram.com/redhatinc/> | Snapchat: @redhatsnaps

-- 
Tomáš Golembiovský <tgolembi@redhat.com>

On Tuesday, 27 February 2018 13:34:09 CET Richard W.M. Jones
wrote:
> On Tue, Feb 27, 2018 at 12:53:08PM +0100, Pino Toscano wrote:
> > On Tuesday, 27 February 2018 12:35:36 CET Tomáš Golembiovský wrote:
> > > Remove ties to MAC address because it is likely to change.
> > 
> > v2v tries to preserve the MAC address of network interfaces; few
months
> > ago we did a fix regarding this:
> > https://bugzilla.redhat.com/show_bug.cgi?id=1506572
> > 
> > The approach of this patch is IMHO not good, since it removes the MAC
> > address from the network-scripts, but still the rest of v2v will try
> > to preserve the MAC addresses.
> 
> We preserve the MAC address in metadata.  On the other hand AIUI this
> patch only removes the association in the ifcfg files and the guest
> will reassociate it when it boots (albeit it might then mix up the
> ethernet interfaces so that's not good).
Yes, this is my concern, i.e. that the guest network configuration
might break.
> There's IMHO a bigger problem which is not being addressed:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1318922
Possibly there could be also
https://bugzilla.redhat.com/show_bug.cgi?id=1451597
> > What's the reason behind this patch?
> 
> There's a bit of background which is missing.  Tomáš and I had some
> discussions (privately, unfortunately) with the ManageIQ developers
> who are integrating virt-v2v into MIQ/CloudForms.  Their existing
> software runs a separate virt-sysprep step on guests after they have
> been converted by virt-v2v.  They disable all sysprep the operations
> except for just a couple, including ‘net-hwaddr’, so the effect is
> roughly the same as this patch.
> 
> The question was raised why they need to do that as a separate step
> and why virt-v2v doesn't do it.
This is what my question was about: I do not understand why this
sysprep step (to remove the MAC addresses) is done in the first place,
and without this information it is hard to put extra conversion steps
in v2v.

What issues were fixed by sysprep'ing the converted guest?
> And indeed there was some discussion
> about whether or not converted guests need a new MAC address -- it's
> at best unclear -- it is thought that VMware might reuse MAC addresses
> which have "left" the hypervisor, although no one knows if
that's
> really true or not.
IMHO the first thing to do would be deciding on whether the MAC address
need to be removed from the guest or not.

If it is decided they need to go, then my suggestion would be to add a
new command line option to tune this behaviour:
- if not specified, then the MAC addresses are preserved in both guest
  and metadata
- if specified, then the MAC addresses will not be preserved even in
  the metadata

-- 
Pino Toscano

Apart from the MAC address, the network interface name may change. In my
tests, my network interface is renamed from ens160 in VMware to eth0 in RHV.

On 27 February 2018 at 14:01, Pino Toscano <ptoscano@redhat.com> wrote:
> On Tuesday, 27 February 2018 13:34:09 CET Richard W.M. Jones wrote:
> > On Tue, Feb 27, 2018 at 12:53:08PM +0100, Pino Toscano wrote:
> > > On Tuesday, 27 February 2018 12:35:36 CET Tomáš Golembiovský
wrote:
> > > > Remove ties to MAC address because it is likely to change.
> > >
> > > v2v tries to preserve the MAC address of network interfaces; few
months
> > > ago we did a fix regarding this:
> > > https://bugzilla.redhat.com/show_bug.cgi?id=1506572
> > >
> > > The approach of this patch is IMHO not good, since it removes the
MAC
> > > address from the network-scripts, but still the rest of v2v will
try
> > > to preserve the MAC addresses.
> >
> > We preserve the MAC address in metadata.  On the other hand AIUI this
> > patch only removes the association in the ifcfg files and the guest
> > will reassociate it when it boots (albeit it might then mix up the
> > ethernet interfaces so that's not good).
>
> Yes, this is my concern, i.e. that the guest network configuration
> might break.
>
> > There's IMHO a bigger problem which is not being addressed:
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=1318922
>
> Possibly there could be also
> https://bugzilla.redhat.com/show_bug.cgi?id=1451597
>
> > > What's the reason behind this patch?
> >
> > There's a bit of background which is missing.  Tomáš and I had
some
> > discussions (privately, unfortunately) with the ManageIQ developers
> > who are integrating virt-v2v into MIQ/CloudForms.  Their existing
> > software runs a separate virt-sysprep step on guests after they have
> > been converted by virt-v2v.  They disable all sysprep the operations
> > except for just a couple, including ‘net-hwaddr’, so the effect is
> > roughly the same as this patch.
> >
> > The question was raised why they need to do that as a separate step
> > and why virt-v2v doesn't do it.
>
> This is what my question was about: I do not understand why this
> sysprep step (to remove the MAC addresses) is done in the first place,
> and without this information it is hard to put extra conversion steps
> in v2v.
>
> What issues were fixed by sysprep'ing the converted guest?
>
> > And indeed there was some discussion
> > about whether or not converted guests need a new MAC address --
it's
> > at best unclear -- it is thought that VMware might reuse MAC addresses
> > which have "left" the hypervisor, although no one knows if
that's
> > really true or not.
>
> IMHO the first thing to do would be deciding on whether the MAC address
> need to be removed from the guest or not.
>
> If it is decided they need to go, then my suggestion would be to add a
> new command line option to tune this behaviour:
> - if not specified, then the MAC addresses are preserved in both guest
>   and metadata
> - if specified, then the MAC addresses will not be preserved even in
>   the metadata
>
> --
> Pino Toscano



-- 

*Fabien Dupont*

PRINCIPAL SOFTWARE ENGINEER

Red Hat - Solutions Engineering

fabien@redhat.com     M: +33 (0) 662 784 971 <+33662784971>

<http://redhat.com>  *TRIED. TESTED. TRUSTED.*

Twitter: @redhatway <https://twitter.com/redhatway> | Instagram:
@redhatinc
<https://www.instagram.com/redhatinc/> | Snapchat: @redhatsnaps