CentOS - Mar 2009 - Monitoring IP masquerading on LVS load-balancing

I've got small numbers of connections moving through a load balancer
configured in NAT mode.  So I've got an iptables table called
"nat", which
has in it a line "-A POSTROUTING -o eth0 -j MASQUERADE" (lan connect
is
eth0, private lan inside the cluster is eth1).

The load balancer is working; connections made to the virtual ip on that
host do get routed to one of the real servers behind this load load
balancer.

But I want to observe the connections on the load balancer.

My first attempt was to use netstat with the --masquerade switch.  This
produced the result "netstat: no support for `ip_masquerade' on this
system."  Consistent with this, there is no /proc/net/ip_masquerade.

On the other hand, the load balancer *IS* working; those connections *are*
getting NATted and routed.

Also, lsmod shows varous relevant modules loaded:
iptable_nat            40773  1
ip_nat                 53101  2 ipt_MASQUERADE,iptable_nat
ip_conntrack           91237  5
xt_state,ip_conntrack_netbios_ns,ipt_MASQUERADE,iptable_nat,ip_nat
nfnetlink              40457  2 ip_nat,ip_conntrack
ip_tables              55329  2 iptable_filter,iptable_nat
x_tables               50377  7
xt_state,ipt_REJECT,xt_tcpudp,ipt_MASQUERADE,xt_multiport,iptable_nat,ip_tables

So, netstat just isn't somehow the right monitoring tool, right?  So what
is the right monitoring tool?  I need to know the source IP and
real-server IP of connections being handled by the load balancer.  I don't
need a lot showing exactly how each one was handled, but I'd like to be
able to determine the state of any connection currently active.  How can I
do this?





-- 
David Dyer-Bennet, dd-b at dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

David Dyer-Bennet wrote:
>
> So, netstat just isn't somehow the right monitoring tool, right?  So
what
> is the right monitoring tool?  I need to know the source IP and
Shot in the dark since I've never used LVS but perhaps
/proc/net/ip_conntrack

If that is right then there is a program called netstat-nat that
is out there, not sure if there is a ready-made package for CentOS
or if it's included by default but here is the debian version(source
code on the right)

http://packages.debian.org/lenny/netstat-nat

I statically compiled it for a ipcop firewall recently and it worked
pretty well.

nate

Quoting David Dyer-Bennet <dd-b at dd-b.net>:
> I've got small numbers of connections moving through a load balancer
> configured in NAT mode.  So I've got an iptables table called
"nat", which
> has in it a line "-A POSTROUTING -o eth0 -j MASQUERADE" (lan
connect is
> eth0, private lan inside the cluster is eth1).
>
> The load balancer is working; connections made to the virtual ip on that
> host do get routed to one of the real servers behind this load load
> balancer.
>
> But I want to observe the connections on the load balancer.
>
> My first attempt was to use netstat with the --masquerade switch.  This
> produced the result "netstat: no support for `ip_masquerade' on
this
> system."  Consistent with this, there is no /proc/net/ip_masquerade.
>
> On the other hand, the load balancer *IS* working; those connections *are*
> getting NATted and routed.
>
> Also, lsmod shows varous relevant modules loaded:
> iptable_nat            40773  1
> ip_nat                 53101  2 ipt_MASQUERADE,iptable_nat
> ip_conntrack           91237  5
> xt_state,ip_conntrack_netbios_ns,ipt_MASQUERADE,iptable_nat,ip_nat
> nfnetlink              40457  2 ip_nat,ip_conntrack
> ip_tables              55329  2 iptable_filter,iptable_nat
> x_tables               50377  7
>
xt_state,ipt_REJECT,xt_tcpudp,ipt_MASQUERADE,xt_multiport,iptable_nat,ip_tables
>
> So, netstat just isn't somehow the right monitoring tool, right?  So
what
> is the right monitoring tool?  I need to know the source IP and
> real-server IP of connections being handled by the load balancer.  I
don't
> need a lot showing exactly how each one was handled, but I'd like to be
> able to determine the state of any connection currently active.  How can I
> do this?
ipvsadm -L -c -n should do the trick.  Also, you shouldn't need that MASQ
rule
unless you need to MASQ traffic originating from inside your private network. 
LVS handles all LVS related NATing.

Be careful .. you must use the lower case 'c' in this command as the
uppercase
'C' will CLEAR your ipvs table and break things.

Hope this helps.

Barry