I have a Proxmox VE (http://pve.proxmox.com/wiki/Main_Page) cluster setup. In this cluster I have a bunch of KVM machines. I am using Shorewall to firewall the host nodes and the virtual machines. Shorewall is running on each node of the cluster and I wrote a small script to syncronize the /etc/shorewall/ directory on each node of the cluster. I have recently switched from a bridged network to a routed one using proxy ARP for my KVM virtual machines. My question is will having bogus entries in the proxyarp file cause issues? Since the entire directory is synchronized the files are all same across the cluster, which allows for easy migration of the virtual machines, the proxyarp is the same on each box. I have an entry in the proxyarp file for an IP that is not currently on that cluster node it is however on the other cluster node. Will that mess anything up? In testing everything seems to still work but I want to be sure before I put this change into production. Is there a better way to accomplish what I am trying to do? I am using Shorewall version 4.0.15 Thanks, _ /-\ ndrew Niemantsverdriet ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Andrew Niemantsverdriet wrote:> I have a Proxmox VE (http://pve.proxmox.com/wiki/Main_Page) cluster > setup. In this cluster I have a bunch of KVM machines. I am using > Shorewall to firewall the host nodes and the virtual machines. > Shorewall is running on each node of the cluster and I wrote a small > script to syncronize the /etc/shorewall/ directory on each node of the > cluster. I have recently switched from a bridged network to a routed > one using proxy ARP for my KVM virtual machines. > > My question is will having bogus entries in the proxyarp file cause > issues? Since the entire directory is synchronized the files are all > same across the cluster, which allows for easy migration of the > virtual machines, the proxyarp is the same on each box. I have an > entry in the proxyarp file for an IP that is not currently on that > cluster node it is however on the other cluster node. Will that mess > anything up?Potentially -- I don''t have a clear picture of your configuration.> In testing everything seems to still work but I want to > be sure before I put this change into production. Is there a better > way to accomplish what I am trying to do?If your /etc/shorewall/proxyarp entries currently have ''no'' in the HAVE ROUTE column, you could simply not use /etc/shorewall/proxyarp and rather just set the proxyarp option on all interfaces. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Tom Eastep wrote:> Andrew Niemantsverdriet wrote: >> I have a Proxmox VE (http://pve.proxmox.com/wiki/Main_Page) cluster >> setup. In this cluster I have a bunch of KVM machines. I am using >> Shorewall to firewall the host nodes and the virtual machines. >> Shorewall is running on each node of the cluster and I wrote a small >> script to syncronize the /etc/shorewall/ directory on each node of the >> cluster. I have recently switched from a bridged network to a routed >> one using proxy ARP for my KVM virtual machines. >> >> My question is will having bogus entries in the proxyarp file cause >> issues? Since the entire directory is synchronized the files are all >> same across the cluster, which allows for easy migration of the >> virtual machines, the proxyarp is the same on each box. I have an >> entry in the proxyarp file for an IP that is not currently on that >> cluster node it is however on the other cluster node. Will that mess >> anything up? > > Potentially -- I don''t have a clear picture of your configuration. > >> In testing everything seems to still work but I want to >> be sure before I put this change into production. Is there a better >> way to accomplish what I am trying to do? > > If your /etc/shorewall/proxyarp entries currently have ''no'' in the HAVE > ROUTE column, you could simply not use /etc/shorewall/proxyarp and > rather just set the proxyarp option on all interfaces.Hi again, Andrew -- that should have been ''yes'' in the HAVE ROUTE column. The idea is that if you don''t need Shorewall to add any routes for you, then setting the proxyarp interface option will work okay. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Andrew Niemantsverdriet
2009-Nov-03 21:49 UTC
Re: Proxy ARP file the same on different boxes
Hi, On Tue, Nov 3, 2009 at 2:03 PM, Tom Eastep <teastep@shorewall.net> wrote:> Tom Eastep wrote: >> Andrew Niemantsverdriet wrote: >>> I have a Proxmox VE (http://pve.proxmox.com/wiki/Main_Page) cluster >>> setup. In this cluster I have a bunch of KVM machines. I am using >>> Shorewall to firewall the host nodes and the virtual machines. >>> Shorewall is running on each node of the cluster and I wrote a small >>> script to syncronize the /etc/shorewall/ directory on each node of the >>> cluster. I have recently switched from a bridged network to a routed >>> one using proxy ARP for my KVM virtual machines. >>> >>> My question is will having bogus entries in the proxyarp file cause >>> issues? Since the entire directory is synchronized the files are all >>> same across the cluster, which allows for easy migration of the >>> virtual machines, the proxyarp is the same on each box. I have an >>> entry in the proxyarp file for an IP that is not currently on that >>> cluster node it is however on the other cluster node. Will that mess >>> anything up? >> >> Potentially -- I don''t have a clear picture of your configuration. >> >>> In testing everything seems to still work but I want to >>> be sure before I put this change into production. Is there a better >>> way to accomplish what I am trying to do? >> >> If your /etc/shorewall/proxyarp entries currently have ''no'' in the HAVE >> ROUTE column, you could simply not use /etc/shorewall/proxyarp and >> rather just set the proxyarp option on all interfaces. > > Hi again, Andrew -- that should have been ''yes'' in the HAVE ROUTE > column. The idea is that if you don''t need Shorewall to add any routes > for you, then setting the proxyarp interface option will work okay. > > -TomBy setting the proxyarp interface option you mean something like: echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp ? If I set ''yes'' under HAVEROUTE do I need to do anything else on the host node to make it work? Also what about PERSISTENT in the proxyarp file what does that do? As far as my network setup goes it is fairly simple. I have two machines running KVM hosts connected to the same switch each one has one network interface eth0 and then another interface vmbr0. eth0 has a public IP vmbr0 has a dummy IP. For more information about the setup you can also see: http://montanalinux.org/proxmox-ve-with-shorewall.html which documents how I setup Shorewall on a single Proxmox VE host. I am trying to extend this setup to my cluster. Thanks for the response. _ /-\ ndrew Niemantsverdriet ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Andrew Niemantsverdriet wrote:> Hi, > > On Tue, Nov 3, 2009 at 2:03 PM, Tom Eastep <teastep@shorewall.net> wrote: >> Tom Eastep wrote: >>> Andrew Niemantsverdriet wrote: >>>> I have a Proxmox VE (http://pve.proxmox.com/wiki/Main_Page) cluster >>>> setup. In this cluster I have a bunch of KVM machines. I am using >>>> Shorewall to firewall the host nodes and the virtual machines. >>>> Shorewall is running on each node of the cluster and I wrote a small >>>> script to syncronize the /etc/shorewall/ directory on each node of the >>>> cluster. I have recently switched from a bridged network to a routed >>>> one using proxy ARP for my KVM virtual machines. >>>> >>>> My question is will having bogus entries in the proxyarp file cause >>>> issues? Since the entire directory is synchronized the files are all >>>> same across the cluster, which allows for easy migration of the >>>> virtual machines, the proxyarp is the same on each box. I have an >>>> entry in the proxyarp file for an IP that is not currently on that >>>> cluster node it is however on the other cluster node. Will that mess >>>> anything up? >>> Potentially -- I don''t have a clear picture of your configuration. >>> >>>> In testing everything seems to still work but I want to >>>> be sure before I put this change into production. Is there a better >>>> way to accomplish what I am trying to do? >>> If your /etc/shorewall/proxyarp entries currently have ''no'' in the HAVE >>> ROUTE column, you could simply not use /etc/shorewall/proxyarp and >>> rather just set the proxyarp option on all interfaces. >> Hi again, Andrew -- that should have been ''yes'' in the HAVE ROUTE >> column. The idea is that if you don''t need Shorewall to add any routes >> for you, then setting the proxyarp interface option will work okay. >> >> -Tom > > By setting the proxyarp interface option you mean something like: echo > 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp ?Add ''proxyarp=1'' in the OPTIONS column of /etc/shorewall/interfaces> > If I set ''yes'' under HAVEROUTE do I need to do anything else on the > host node to make it work?No.> Also what about PERSISTENT in the proxyarp file what does that do?It prevents ''shorewall stop'' from deleting ARP cache entries that it adds at ''shorewall start''. That won''t be an issue if you set the interface ''proxyarp'' option since shorewall doesn''t touch /proc/sys/net/ipv4/conf/*/proxy_arp during stop processing.> > As far as my network setup goes it is fairly simple. I have two > machines running KVM hosts connected to the same switch each one has > one network interface eth0 and then another interface vmbr0. eth0 has > a public IP vmbr0 has a dummy IP. For more information about the setup > you can also see: > http://montanalinux.org/proxmox-ve-with-shorewall.html which documents > how I setup Shorewall on a single Proxmox VE host. I am trying to > extend this setup to my cluster.Okay -- what I recommend then will work. But you will have to 1) add routes to the proxy arped hosts in your /etc/network/interfaces file. auto vmbr0 iface vmbr0 inet static address 10.1.1.1 netmask 255.255.255.0 bridge_ports none bridge_stp off bridge_fd 0 post-up ip route add 192.0.2.11/32 dev vmbr0 <======== ... 2) Remove entry (entries) from /etc/shorewall/proxyarp 3) Change /etc/shorewall/interfaces to: net eth0 detect tcpflags,routefilter,nosmurfs,logmartians,proxyarp=1 ----------- dmz venet0 detect routeback dmz vmbr0 detect routeback,bridge,proxyarp=1 ----------- -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference
Andrew Niemantsverdriet
2009-Nov-03 22:47 UTC
Re: Proxy ARP file the same on different boxes
Hi, On Tue, Nov 3, 2009 at 3:12 PM, Tom Eastep <teastep@shorewall.net> wrote:> Andrew Niemantsverdriet wrote: >> On Tue, Nov 3, 2009 at 2:03 PM, Tom Eastep <teastep@shorewall.net> wrote: >>> Tom Eastep wrote: >>>> Andrew Niemantsverdriet wrote: >>>>> I have a Proxmox VE (http://pve.proxmox.com/wiki/Main_Page) cluster >>>>> setup. In this cluster I have a bunch of KVM machines. I am using >>>>> Shorewall to firewall the host nodes and the virtual machines. >>>>> Shorewall is running on each node of the cluster and I wrote a small >>>>> script to syncronize the /etc/shorewall/ directory on each node of the >>>>> cluster. I have recently switched from a bridged network to a routed >>>>> one using proxy ARP for my KVM virtual machines. >>>>> >>>>> My question is will having bogus entries in the proxyarp file cause >>>>> issues? Since the entire directory is synchronized the files are all >>>>> same across the cluster, which allows for easy migration of the >>>>> virtual machines, the proxyarp is the same on each box. I have an >>>>> entry in the proxyarp file for an IP that is not currently on that >>>>> cluster node it is however on the other cluster node. Will that mess >>>>> anything up? >>>> Potentially -- I don't have a clear picture of your configuration. >>>> >>>>> In testing everything seems to still work but I want to >>>>> be sure before I put this change into production. Is there a better >>>>> way to accomplish what I am trying to do? >>>> If your /etc/shorewall/proxyarp entries currently have 'no' in the HAVE >>>> ROUTE column, you could simply not use /etc/shorewall/proxyarp and >>>> rather just set the proxyarp option on all interfaces. >>> Hi again, Andrew -- that should have been 'yes' in the HAVE ROUTE >>> column. The idea is that if you don't need Shorewall to add any routes >>> for you, then setting the proxyarp interface option will work okay. >>> >>> -Tom >> >> By setting the proxyarp interface option you mean something like: echo >> 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp ? > > Add 'proxyarp=1' in the OPTIONS column of /etc/shorewall/interfaces > >> >> If I set 'yes' under HAVEROUTE do I need to do anything else on the >> host node to make it work? > > No. > >> Also what about PERSISTENT in the proxyarp file what does that do? > > It prevents 'shorewall stop' from deleting ARP cache entries that it > adds at 'shorewall start'. That won't be an issue if you set the > interface 'proxyarp' option since shorewall doesn't touch > /proc/sys/net/ipv4/conf/*/proxy_arp during stop processing. > >> >> As far as my network setup goes it is fairly simple. I have two >> machines running KVM hosts connected to the same switch each one has >> one network interface eth0 and then another interface vmbr0. eth0 has >> a public IP vmbr0 has a dummy IP. For more information about the setup >> you can also see: >> http://montanalinux.org/proxmox-ve-with-shorewall.html which documents >> how I setup Shorewall on a single Proxmox VE host. I am trying to >> extend this setup to my cluster. > > Okay -- what I recommend then will work. But you will have to > > 1) add routes to the proxy arped hosts in your /etc/network/interfaces > file. > > auto vmbr0 > iface vmbr0 inet static > address 10.1.1.1 > netmask 255.255.255.0 > bridge_ports none > bridge_stp off > bridge_fd 0 > post-up ip route add 192.0.2.11/32 dev vmbr0 <========> ... > > 2) Remove entry (entries) from /etc/shorewall/proxyarp > > 3) Change /etc/shorewall/interfaces to: > > net eth0 detect tcpflags,routefilter,nosmurfs,logmartians,proxyarp=1 > ----------- > dmz venet0 detect routeback > dmz vmbr0 detect routeback,bridge,proxyarp=1 > ----------- > > -TomOkay but the major downside is when I migrate a virtual machine from one cluster node to the other I would have to issue that route statement. I could script it but have no way to know the IP of the box that is migrated. The ultimate goal is high availability for these servers so having to enter and keep track of route statements does not work all that well. Would bridging be a better solution? Proxy ARP is nicer because you can do everything with it bridging has some limitations but I would trade the limitations to be able to migrate stuff with out having to add route statements. Thanks Tom, _ /-\ ndrew Niemantsverdriet ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Andrew Niemantsverdriet wrote:> Okay but the major downside is when I migrate a virtual machine from > one cluster node to the other I would have to issue that route > statement. I could script it but have no way to know the IP of the box > that is migrated. The ultimate goal is high availability for these > servers so having to enter and keep track of route statements does not > work all that well. Would bridging be a better solution? Proxy ARP is > nicer because you can do everything with it bridging has some > limitations but I would trade the limitations to be able to migrate > stuff with out having to add route statements.Sounds like bridging is the way to go if you want to be able to move the VMs around. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what''s new with Crystal Reports now. http://p.sf.net/sfu/bobj-july