CentOS - Dec 2014 - NetworkManager fights with DHCP-only backup NIC

We ship servers to remote sites, which are rarely staffed with techs familiar
with Linux.  We have them tell us the static IP configuration for the box before
we ship it, then we set it up for them here and ship it out to the site, where
they just plug it in, turn it on, and walk away.

That?s the ideal, anyway.

What often happens in reality is either:

1. They give us incorrect static IP info, so the box arrives and won?t connect
to the Internet, which means we often have to arrange to get someone clueful
on-site to fix it.

2. The site is in the middle of some major deployment, a small piece of which is
our server, so the LAN isn?t ready, but they demand the box be shipped early
anyway for some handwavy business reason.  "No, we can?t tell you what
static IP to use," they say.  "Just configure it on-site," they
say.  Sigh.

Since these systems have 2+ Ethernet ports and we really only need one in normal
operations, we?ve taken to configuring the second one for DHCP, so that they can
just move the cable from the primary port to the secondary.

This works fine in CentOS 5: DHCP comes up and takes over, giving us the access
we need to fix/configure the static IP on the primary port.

What happens in CentOS 7 depends on whether you plug in one cable or two:

1. If you plug in only one cable, NetworkManager sees that the static interface
is unplugged, so it *helpfully* moves that IP to the secondary NIC, apparently
on the assumption that static is always better than DHCP.  This is of no use to
us, since all it does is move the problem to the other NIC.

2. If you plug both cables in, both interfaces come up configured as you?d
expect, but since both configurations provided a gateway address, you still
can?t get out to the Internet since the static one came up first, and it?s
pointing at an unreachable box.

I think all we need to do to fix this is convince NetworkManager not to be
clever about moving the static IP to the second NIC.  Alas, there is no checkbox
in the NM GUI labeled ?This is a 4U server, dummy, not a laptop.?

Anyone know how to convince NM to obey the MAC binding in the ifcfg-* file, to
prevent NM from moving the broken static IP info to the second NIC?

Yes, we know we can still disable NetworkManager and edit
network-scripts/ifcfg-* directly.  We?d just prefer not to fight the OS.  Also,
unlike EL6, disabling NM on EL7 breaks the network GUI, which we?ve occasionally
found helpful, as when we have a semi-clueful tech at the remote site.

On Mon, Dec 1, 2014 at 3:35 PM, Warren Young <wyml at etr-usa.com> wrote:
> We ship servers to remote sites, which are rarely staffed with techs
familiar with Linux.  We have them tell us the static IP configuration for the
box before we ship it, then we set it up for them here and ship it out to the
site, where they just plug it in, turn it on, and walk away.
>
> That?s the ideal, anyway.
>
Is there anyone who has more than a few boxes at more than one
location who _doesn't_ have this issue?  I'd like to see a FAQ or
something by whoever designed the network configuration system about
how they planned for it to work (with and without GUI availability).
Likewise for what is supposed to happen when you restore a backup onto
different hardware.

-- 
   Les Mikesell
      lesmikesell at gmail.com

> On Dec 1, 2014, at 14:48, Les Mikesell <lesmikesell at gmail.com>
wrote:
> 
> On Mon, Dec 1, 2014 at 3:35 PM, Warren Young <wyml at etr-usa.com>
wrote:
> 
>> We ship servers to remote sites, which are rarely staffed with techs
familiar with Linux.  We have them tell us the static IP configuration for the
box before we ship it, then we set it up for them here and ship it out to the
site, where they just plug it in, turn it on, and walk away.
>> 
>> That?s the ideal, anyway.
>> 
> 
> Is there anyone who has more than a few boxes at more than one
> location who _doesn't_ have this issue?  I'd like to see a FAQ or
> something by whoever designed the network configuration system about
> how they planned for it to work (with and without GUI availability).
> Likewise for what is supposed to happen when you restore a backup onto
> different hardware.
Most of the time, I end up nuking HWADDR from orbit on most boxes.  It just
causes more trouble than it fixes.

--
Nate Duehr
denverpilot at me.com

On 12/02/2014 10:35 AM, Warren Young wrote:
> We ship servers to remote sites, which are rarely staffed with techs
familiar with Linux.  We have them tell us the static IP configuration for the
box before we ship it, then we set it up for them here and ship it out to the
site, where they just plug it in, turn it on, and walk away.
>
> That?s the ideal, anyway.
>
> What often happens in reality is either:
>
> 1. They give us incorrect static IP info, so the box arrives and won?t
connect to the Internet, which means we often have to arrange to get someone
clueful on-site to fix it.
>
> 2. The site is in the middle of some major deployment, a small piece of
which is our server, so the LAN isn?t ready, but they demand the box be shipped
early anyway for some handwavy business reason.  "No, we can?t tell you
what static IP to use," they say.  "Just configure it on-site,"
they say.  Sigh.
>
> Since these systems have 2+ Ethernet ports and we really only need one in
normal operations, we?ve taken to configuring the second one for DHCP, so that
they can just move the cable from the primary port to the secondary.
>
> This works fine in CentOS 5: DHCP comes up and takes over, giving us the
access we need to fix/configure the static IP on the primary port.
>
> What happens in CentOS 7 depends on whether you plug in one cable or two:
>
> 1. If you plug in only one cable, NetworkManager sees that the static
interface is unplugged, so it *helpfully* moves that IP to the secondary NIC,
apparently on the assumption that static is always better than DHCP.  This is of
no use to us, since all it does is move the problem to the other NIC.
>
> 2. If you plug both cables in, both interfaces come up configured as you?d
expect, but since both configurations provided a gateway address, you still
can?t get out to the Internet since the static one came up first, and it?s
pointing at an unreachable box.
>
> I think all we need to do to fix this is convince NetworkManager not to be
clever about moving the static IP to the second NIC.  Alas, there is no checkbox
in the NM GUI labeled ?This is a 4U server, dummy, not a laptop.?
>
> Anyone know how to convince NM to obey the MAC binding in the ifcfg-* file,
to prevent NM from moving the broken static IP info to the second NIC?
Have you put
NM_CONTROLLED="no"
in the ifcfg-eth0 script?
> Yes, we know we can still disable NetworkManager and edit
network-scripts/ifcfg-* directly.  We?d just prefer not to fight the OS.  Also,
unlike EL6, disabling NM on EL7 breaks the network GUI, which we?ve occasionally
found helpful, as when we have a semi-clueful tech at the remote site.
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos

On Dec 1, 2014, at 10:27 PM, Rob Kampen <rkampen at reaching-clients.com>
wrote:
> Have you put
> NM_CONTROLLED="no"
> in the ifcfg-eth0 script?
How is that better than

    systemctl stop NetworkManager
    systemctl disable NetworkManager

Again, I?m not really after a way to make this work without NetworkManager. 
We?ve already got that.  What I want is a way to tell NM to obey the MAC
binding.  This configuration *here* goes with that MAC chip *there*.

Given that, we don?t need to disable NetworkManager.