Eric Low
2006-Apr-10 17:10 UTC
[Xen-users] RE: [Ebtables-user] complex bridge and nat problem
Steffen, Do you have the proper rules in place to make the firewall stateful? For example: iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT Also, can you see the reply packets with a logging rule on the Gateway Bridge? Someplace early... like the mangle prerouting or nat prerouting chain. If I recall, the packets should still appear to have the translated address (as the destination on the reply) all the way through the chains. But don''t quote me on it. Anyhow, if iptables can see the packets there, you can narrow down where to look. Put in logging rules on each chain and see where it dies. Here''s what I think is happening... I think that because you''re double bridging, your Gateway Bridge does not know where to route the packets back to. I know that doesn''t make a lot of sense with a bridge, but you''re obviously not doing straight bridging when you SNAT. From experience, it seems that iptables will just drop the packet in certain NAT''ing circumstances if it doesn''t have either an ARP entry or a ROUTE... or possibly both. Try the following things: 1) Double SNAT - SNAT on the first bridge, then SNAT again on the second. 2) Add an alias from the client subnet to xenbr0 (xenbr0:0) 3) Check your routing table and try adding routes to the appropriate subnets (a route to the client subnet through vif0.0). Hope this helps. I''ve tried to do a lot of weird NAT''ing like this in the past, but it''s been a while. Eric -----Original Message----- From: ebtables-user-admin@lists.sourceforge.net on behalf of Steffen Heil Sent: Mon 4/10/2006 8:10 AM To: netfilter@lists.netfilter.org; ebtables-user@lists.sourceforge.net; xen-users@lists.xensource.com Cc: Subject: [Ebtables-user] complex bridge and nat problem Hi I have problems with nat. My rather complex setup is as follows: I use a server (running xen), which has two bridges in on linux kernel: physical eth0 (renamed to peth0) is connected to the first bridge (xenbr0). veth0 (renamed to eth0) is connceted to the first bridge (xenbr0). I call this first bridge the gateway-bridge, it has no ip address. I call this domain (=VM) my gateway. It has a public ip and is connected to xenbr0, which is connected to the physical ethernet card, which is connected to my isps router... Then I have another bridge (xenbr1), which has a ip-address on it''s own in my $clientnet. The client domain (=VM) has a virtual network interface which is connected to that bridge. So my bridge setup is as: bridge name bridge id STP enabled - xenbr0 8000.feffffffffff no interfaces - peth0 == physical interface - vif0.0 == gateways eth0 bridge name bridge id STP enabled - xenbr1 8000.feffffffffff no interfaces - vif22.0 == clients eth0 Finally I have the following SNAT rule (ip4_forward is enabled.) Chain POSTROUTING (policy ACCEPT 113K packets, 18M bytes) pkts bytes target prot opt in out source destination 23 1380 SNAT all -- * eth0 $clientnet/24 0.0.0.0/0 to:$gateway This somehow works: If I try to reach my outside ssh server (by ip), I get: GI = Gateway Interface [tcpdump -nni peth0 host $server] GB = Gateway Bridge [tcpdump -nni xenbr0 host $server] CI = Client Interface [tcpdump -nni vif22.0 host $server] CB = Client Bridge [tcpdump -nni xenbr1 host $server] SI = Server [tcpdump -nni eth0 host $gateway] CI> 11:40:00.770005 IP $client.2958 > $server.22: S 3227338208:3227338208(0) win 5840 <mss 1460,sackOK,timestamp 12927453 0,nop,wscale 2> CB> 11:40:00.770250 IP $client.2958 > $server.22: S 3227338208:3227338208(0) win 5840 <mss 1460,sackOK,timestamp 12927453 0,nop,wscale 2> GB> 11:40:00.770416 IP $gateway.2958 > $server.22: S 3227338208:3227338208(0) win 5840 <mss 1460,sackOK,timestamp 12927453 0,nop,wscale 2> GI> 11:40:00.770571 IP $gateway.2958 > $server.22: S 3227338208:3227338208(0) win 5840 <mss 1460,sackOK,timestamp 12927453 0,nop,wscale 2> SI> 13:40:01.108827 IP $gateway.2958 > $server.22: S 3227338208:3227338208(0) win 5840 <mss 1460,sackOK,timestamp 12927453 0,nop,wscale 2> SI> 13:40:01.108863 IP $server.22 > $gateway.2958: S 1070006580:1070006580(0) ack 3227338209 win 5792 <mss 1460,sackOK,timestamp 4101418364 12927453,nop,wscale 2> GI> 11:40:00.779428 IP $server.22 > $gateway.2958: S 1070006580:1070006580(0) ack 3227338209 win 5792 <mss 1460,sackOK,timestamp 4101418364 12927453,nop,wscale 2> So the client sends the SYN, the client bridge passes this to the gateway, the gateway does SNAT and forwards it to the gateway bridge, the gateway bridge sends this through the physical interface. The ssh-Server responds with SYN ACK, and this arrives at the gateway''s physical interface. However, it doesn''t make it till the first bridge. I would expect it at leat to reach the gateway bridge and then the gateway. I even hoped SNAT would do it''s job, rewrite the address and forward it to the client over the client bridge. But it DOES NEVER reach the gateway bridge. Can someone tell me, why packets that where SNATed earlier don''t make it up to the gateway bridge? The gateway itself (and other domains connected to the gateway bridge work just fine. Regards, Steffen _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users