Hi @all,
I'm having trouble to realize my use case and hope somebody could help me.
# Use case
For a home lab I want to deploy several guest domains. These domains
must not have a direct or NAT connection to the internet or my LAN. They
should only be able to reach my LAN and the internet through a proxy.
# What I've done
I've created the following virtual switch in isolated mode:
$ sudo virsh net-dumpxml private1
<network connections='3'>
<name>private1</name>
<uuid>THE-UUID</uuid>
<bridge name='virbr1' stp='on' delay='0'/>
<mac address='DE:AD:BE:EF:FF:FF'/>
<domain name='private1'/>
<ip address='192.168.100.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.100.128'
end='192.168.100.254'/>
</dhcp>
</ip>
</network>
I've setup a guest domain that serves as a proxy and several other guests.
# My issue
Nameresolution for *.private1 works fine on this network. But I'm not
able to resolve domains from the outside world like github.com.
I understood that libvirt is forwarding dns resolution requests to the
hosts nameserver configured in /etc/resolv.conf in case the dnsmasq
instance for the virtual network is not able to resolve the name.
My guess, in my setup this don't work, because the virtual switch is in
isolated mode, right?
# My questions
* What can I do to achieve my use case described above?
* Is it possible to use the isolated mode here or do I have to use a
different mode?
It's important that the guest domains could only connect to the internet
by using the proxy.
Regards,
Joerg
Laine Stump
2020-Nov-11 13:47 UTC
Re: DNS forwarding for guest domains on isolated network
On 11/11/20 3:40 AM, Jörg Kastning wrote:> Hi @all, > > I'm having trouble to realize my use case and hope somebody could help me. > > # Use case > > For a home lab I want to deploy several guest domains. These domains > must not have a direct or NAT connection to the internet or my LAN. They > should only be able to reach my LAN and the internet through a proxy. > > # What I've done > > I've created the following virtual switch in isolated mode: > > $ sudo virsh net-dumpxml private1 > <network connections='3'> > <name>private1</name> > <uuid>THE-UUID</uuid> > <bridge name='virbr1' stp='on' delay='0'/> > <mac address='DE:AD:BE:EF:FF:FF'/> > <domain name='private1'/> > <ip address='192.168.100.1' netmask='255.255.255.0'> > <dhcp> > <range start='192.168.100.128' end='192.168.100.254'/> > </dhcp> > </ip> > </network> > > I've setup a guest domain that serves as a proxy and several other guests. > > # My issue > > Nameresolution for *.private1 works fine on this network. But I'm not > able to resolve domains from the outside world like github.com.This behavior is intentional: https://gitlab.com/libvirt/libvirt/-/commit/513122ae93> > I understood that libvirt is forwarding dns resolution requests to the > hosts nameserver configured in /etc/resolv.conf in case the dnsmasq > instance for the virtual network is not able to resolve the name.Not for isolated networks, because a DNS request could be used to break out of an isolated network (by using "IP over DNS")> > My guess, in my setup this don't work, because the virtual switch is in > isolated mode, right?When DNS traffic is forwarded by a DNS server, it is at application level, not IP level, so any filtering of forwarded traffic on the switch is not involved.> > # My questions > > * What can I do to achieve my use case described above? > > * Is it possible to use the isolated mode here or do I have to use a > different mode?"no-resolv" will always be in the dnsmasq config file for an isolated network, and there isn't any way to remove it (other than using a different kind of network). And since there is not (as far as I know) a different dnsmasq option to counteract a "no-resolv" that's already there, you can't eliminate the effect of no-resolv by adding something to the conf file with <dnsmasq:options>. A few things to try: 1) try adding <forwarder addr='x.x.x.x'/> in the the <dns> section of the network, pointing to your normal DNS server. Possibly that directive to dnsmasq will make a "side run" around the restriction on forwarding. (this can also have "domain='blah'" added, in which case it only forwards requests for names within the 'blah' domain ). https://libvirt.org/formatnetwork.html#elementsAddress 2) use a <forward mode='route'> network, but also add in nwfilter rules that only allow traffic on the local network. https://libvirt.org/formatnwfilter.html 3) again, use <forward mode='route'>, but also manually add a rule to the host iptables that rejects all traffic from the guest network outbound on the host's egress interface.> It's important that the guest domains could only connect to the internet > by using the proxy.Have you tried putting the guests
Jörg Kastning
2020-Nov-11 14:02 UTC
Re: DNS forwarding for guest domains on isolated network
On 11.11.20 14:47, Laine Stump wrote:> On 11/11/20 3:40 AM, Jörg Kastning wrote:[...]>> # My issue >> >> Nameresolution for *.private1 works fine on this network. But I'm not >> able to resolve domains from the outside world like github.com. > > This behavior is intentional: > > https://gitlab.com/libvirt/libvirt/-/commit/513122ae93 > >> >> I understood that libvirt is forwarding dns resolution requests to the >> hosts nameserver configured in /etc/resolv.conf in case the dnsmasq >> instance for the virtual network is not able to resolve the name. > > Not for isolated networks, because a DNS request could be used to break > out of an isolated network (by using "IP over DNS") > >> >> My guess, in my setup this don't work, because the virtual switch is >> in isolated mode, right? > > When DNS traffic is forwarded by a DNS server, it is at application > level, not IP level, so any filtering of forwarded traffic on the switch > is not involved. > >> >> # My questions >> >> * What can I do to achieve my use case described above? >> >> * Is it possible to use the isolated mode here or do I have to use a >> different mode? > > "no-resolv" will always be in the dnsmasq config file for an isolated > network, and there isn't any way to remove it (other than using a > different kind of network). And since there is not (as far as I know) a > different dnsmasq option to counteract a "no-resolv" that's already > there, you can't eliminate the effect of no-resolv by adding something > to the conf file with <dnsmasq:options>. A few things to try: > > 1) try adding <forwarder addr='x.x.x.x'/> in the the <dns> section of > the network, pointing to your normal DNS server. Possibly that directive > to dnsmasq will make a "side run" around the restriction on forwarding. > (this can also have "domain='blah'" added, in which case it only > forwards requests for names within the 'blah' domain ). > > https://libvirt.org/formatnetwork.html#elementsAddress > > 2) use a <forward mode='route'> network, but also add in nwfilter rules > that only allow traffic on the local network. > > https://libvirt.org/formatnwfilter.html > > 3) again, use <forward mode='route'>, but also manually add a rule to > the host iptables that rejects all traffic from the guest network > outbound on the host's egress interface. > >> It's important that the guest domains could only connect to the >> internet by using the proxy. > > > Have you tried putting the guests >Hi, Thanks for your reply Laine. I solved this by finding a way where I don't need the DNS resolution at all. To be able to reach internet domains through a proxy I configured the proxy via the enironment variables http_proxy and https_proxy. This way tools like `curl` or `git` hand over the request to the proxy and the proxy resolves the domain. Didn't think about this option earlier. Thanks again for your good explanation. Regards, Joerg