libvirt users - Feb 2018 - Re: libvirt and NAT on a system that already has a DHCP server

On 2/1/2018 9:28 AM, Laine Stump wrote:
> On 02/01/2018 09:22 AM, Daniel P. Berrangé wrote:
>> On Thu, Feb 01, 2018 at 09:19:11AM -0500, Laine Stump wrote:
>>> On 01/30/2018 07:37 PM, john@bluemarble.net wrote:
>>>> I'm trying to use virt-manager and qemu/kvm on Arch Linux.
The box I'm
>>>> using is also the router for my house. It runs a kea DHCP
server. When I
>>>> try to start the default NAT network, it can't start
dnsmasq because that
>>>> port is already bound. Is there a way to have it not bind on
this
>>>> interface? I see there is an except-on statement in the
dnsmasq.conf, but
>>>> I can't add lines to that directly, and I didn't see
any way to add
>>>> special options using virsh net-edit default.
>>>
>>> The dnsmasq processes run by libvirt to serve dhcp for the virtual
>>> networks already does this - they listen *only* on the bridge
created
>>> for their particular network, nothing else. Your problem is that
your
>>> host system's dhcp server has been configured to automatically
listen on
>>> all interfaces.
>>>
>>> So it's not the configuration of the libvirt network that needs
to
>>> change, it's the configuration of the host system's dhcp
server. It
>>> needs to be told that it shouldn't automatically listen on all
new
>>> interfaces, but to just listen on certain specific interfaces.
>>
>> Checkout this
>>
>> https://wiki.libvirt.org/page/Libvirtd_and_dnsmasq
> 
> 
> Useful for dnsmasq, but he says his host is using "kea dhcp
server",
> which appears to be some off-shoot of ISC dhcpd, so the config would be
> different.
> 
Thanks. I asked on the kea list and they say they don't have a method to 
do this. Something about raw packets. I may try to switch to dnsmasq for 
my DHCP server on the machine. For now, I'm back to VirtualBox.

Thanks.

On 02/07/2018 03:07 PM, John Ratliff wrote:
> On 2/1/2018 9:28 AM, Laine Stump wrote:
>> On 02/01/2018 09:22 AM, Daniel P. Berrangé wrote:
>>> On Thu, Feb 01, 2018 at 09:19:11AM -0500, Laine Stump wrote:
>>>> On 01/30/2018 07:37 PM, john@bluemarble.net wrote:
>>>>> I'm trying to use virt-manager and qemu/kvm on Arch
Linux. The box I'm
>>>>> using is also the router for my house. It runs a kea DHCP
server.
>>>>> When I
>>>>> try to start the default NAT network, it can't start
dnsmasq
>>>>> because that
>>>>> port is already bound. Is there a way to have it not bind
on this
>>>>> interface? I see there is an except-on statement in the
>>>>> dnsmasq.conf, but
>>>>> I can't add lines to that directly, and I didn't
see any way to add
>>>>> special options using virsh net-edit default.
>>>>
>>>> The dnsmasq processes run by libvirt to serve dhcp for the
virtual
>>>> networks already does this - they listen *only* on the bridge
created
>>>> for their particular network, nothing else. Your problem is
that your
>>>> host system's dhcp server has been configured to
automatically
>>>> listen on
>>>> all interfaces.
>>>>
>>>> So it's not the configuration of the libvirt network that
needs to
>>>> change, it's the configuration of the host system's
dhcp server. It
>>>> needs to be told that it shouldn't automatically listen on
all new
>>>> interfaces, but to just listen on certain specific interfaces.
>>>
>>> Checkout this
>>>
>>> https://wiki.libvirt.org/page/Libvirtd_and_dnsmasq
>>
>>
>> Useful for dnsmasq, but he says his host is using "kea dhcp
server",
>> which appears to be some off-shoot of ISC dhcpd, so the config would be
>> different.
>>
> 
> Thanks. I asked on the kea list and they say they don't have a method
to
> do this. Something about raw packets. I may try to switch to dnsmasq for
> my DHCP server on the machine. For now, I'm back to VirtualBox.
Really? That seems like a serious limitation - imagine a machine that's
acting as a router from a public network to your own private network,
and you want that same machine to serve DHCP only on the private side
(to avoid making the admin of the public side angry :-). I could see how
using raw sockets could muddy the waters, but surely they must have a
way to configure their server to only listen on a particular interface?

On 2/8/2018 9:05 AM, Laine Stump wrote:
> On 02/07/2018 03:07 PM, John Ratliff wrote:
>> On 2/1/2018 9:28 AM, Laine Stump wrote:
>>> On 02/01/2018 09:22 AM, Daniel P. Berrangé wrote:
>>>> On Thu, Feb 01, 2018 at 09:19:11AM -0500, Laine Stump wrote:
>>>>> On 01/30/2018 07:37 PM, john@bluemarble.net wrote:
>>>>>> I'm trying to use virt-manager and qemu/kvm on Arch
Linux. The box I'm
>>>>>> using is also the router for my house. It runs a kea
DHCP server.
>>>>>> When I
>>>>>> try to start the default NAT network, it can't
start dnsmasq
>>>>>> because that
>>>>>> port is already bound. Is there a way to have it not
bind on this
>>>>>> interface? I see there is an except-on statement in the
>>>>>> dnsmasq.conf, but
>>>>>> I can't add lines to that directly, and I
didn't see any way to add
>>>>>> special options using virsh net-edit default.
>>>>>
>>>>> The dnsmasq processes run by libvirt to serve dhcp for the
virtual
>>>>> networks already does this - they listen *only* on the
bridge created
>>>>> for their particular network, nothing else. Your problem is
that your
>>>>> host system's dhcp server has been configured to
automatically
>>>>> listen on
>>>>> all interfaces.
>>>>>
>>>>> So it's not the configuration of the libvirt network
that needs to
>>>>> change, it's the configuration of the host system's
dhcp server. It
>>>>> needs to be told that it shouldn't automatically listen
on all new
>>>>> interfaces, but to just listen on certain specific
interfaces.
>>>>
>>>> Checkout this
>>>>
>>>> https://wiki.libvirt.org/page/Libvirtd_and_dnsmasq
>>>
>>>
>>> Useful for dnsmasq, but he says his host is using "kea dhcp
server",
>>> which appears to be some off-shoot of ISC dhcpd, so the config
would be
>>> different.
>>>
>>
>> Thanks. I asked on the kea list and they say they don't have a
method to
>> do this. Something about raw packets. I may try to switch to dnsmasq
for
>> my DHCP server on the machine. For now, I'm back to VirtualBox.
> 
> Really? That seems like a serious limitation - imagine a machine that's
> acting as a router from a public network to your own private network,
> and you want that same machine to serve DHCP only on the private side
> (to avoid making the admin of the public side angry :-). I could see how
> using raw sockets could muddy the waters, but surely they must have a
> way to configure their server to only listen on a particular interface?
> 
This is the response I got from the kea list. It's from a member of the 
ISC, Francis Dupont.

--------------
There is no good solution: Kea uses LPF raw sockets on Linux by default
with a fallback socket which is used to:
  1- send some packets back
  2- avoid the kernel to return ICMP port unreachables because no socket
   is bound to the service port
The result is that it is complex and sometime impossible to run multiple
DHCP services on the same system. BTW unfortunately it is not a new
problem...

Some extra comments:
  - the openFallbackSocket() method is generic so does not use the Linux
   specific SO_BINDTODEVICE.
  - SO_REUSEADDR won't help on Linux because its implementation is broken:
   when set to 1 it simply disables conflict detection.
  - I saw the word NAT in your message: if dnsmasq is run behind a NAT the
   best solution is to translate the DHCP server port and to use for
   dnsmasq this alternate port.
--------------------------