Hello, I am facing difficulties with my chain : client - ipsec - shorewall - openswan - ipvs - Real servers. It seems that the return packets never arrive to the clients. Architecture : client :10.44.0.254 | | \ +----+----+ | node A | | | +---+-----+ | | tunnel : 10.44.0.254/32 <-> 10.4.0.30/32 | | | +------+--------+ | node B | kernel 2.6.18 | shorewall | v: 4.0.11 | openswa | swan = ipsec zone | ipvs | VIP: 10.4.0.30:80 +------X--------+ -/ \ / \- -/ \ -/ \ / \ RealServer1 RealServer2 10.0.3.99:8080 10.0.3.100:8080 /etc/shorewall/hosts : swan eth0:10.44.0.254 1. the access: client -> 10.4.0.30 is working OK Done with /etc/shorewall/rules ACCEPT swan:10.44.0.0/24 fw all 2. The masq for real servers to exit with 10.4.0.30 is OK Done with /etc/shorewall/masq eth0::10.44.0.254 10.0.3.99 10.4.0.30 - - 3. The forward from ipvs to real server is OK when doing a : telnet 10.4.0.30 80 I have the following tcpdump on Node B 10:40:48.682340 IP 10.44.0.254.36701 > 10.0.3.99.webcache: S1067349055:1067349055(0) win 5840 <mss 1460,sackOK,timestamp2887838843 0,nop,wscale 5> 10:40:48.682479 IP 10.0.3.99.webcache > 10.44.0.254.36701: S3408723439:3408723439(0) ack 1067349056 win 5792 <mss 1460,sackOK,timestamp 696330234 2887838843,nop,wscale 7> 10:40:51.681631 IP 10.44.0.254.36701 > 10.0.3.99.webcache: S1067349055:1067349055(0) win 5840 <mss 1460,sackOK,timestamp 2887841843 0,nop,wscale 5> 10:40:51.681748 IP 10.0.3.99.webcache > 10.44.0.254.36701: S3408723439:3408723439(0) ack 1067349056 win 5792 <mss 1460,sackOK,timestamp 696333233 2887838843,nop,wscale 7> 10:40:52.282769 IP 10.0.3.99.webcache > 10.44.0.254.36701: S3408723439:3408723439(0) ack 1067349056 win 5792 <mss 1460,sackOK,timestamp 696333834 2887838843,nop,wscale 7> 10:40:58.283227 IP 10.0.3.99.webcache > 10.44.0.254.36701: S3408723439:3408723439(0) ack 1067349056 win 5792 <mss 1460,sackOK,timestamp 696339834 2887838843,nop,wscale 7> However the return never arrives to the client. I don''t seen any drop/reject on the firewall. But I don''t know what is missing. When I bypass the ipvs by a DNAT rules like this one : DNAT:info swan:10.44.0.254 loc:10.0.3.99:8080 tcp 80 - 10.4.0.30 it works, but I am loosing the loadbalancer ipvs. I am obviouly missing a rule but I don''t know which one. Can someone help me ? Thanks ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer''s challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/
Sebastien COUPPEY wrote:> I am facing difficulties with my chain : > > client - ipsec - shorewall - openswan - ipvs - Real servers. > > It seems that the return packets never arrive to the clients. > > Architecture :<folded and mutilated ASCII art omitted> Your mailer folded your ASCII art to the point where it was unreadable.> > /etc/shorewall/hosts : > swan eth0:10.44.0.254 >We really need to see the output of ''shorewall dump'' collected as described at http://www.shorewall.net/support.htm#Guidelines. Snippets of your configuration are not really useful. -Tom -- Tom Eastep \ The ultimate result of shielding men from the effects of Shoreline, \ folly is to fill the world with fools. Washington, USA \ -- Herbert Spencer ------------------------------------------------------------------------ http://www.shorewall.net ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer''s challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/
Sorry for the previous email,
Here are the missing info and the dump with the used IPs
Hello,
I am facing difficulties with my chain :
client - ipsec - shorewall - openswan - ipvs - Real servers.
It seems that the return packets never arrive to the clients.
Architecture :
client :10.44.0.254
|
|
\
+----+----+
| node A |
| |
+---+-----+
|
|
|
|
|
+------+--------+
| node B |
| shorewall | 4.0.11
| openswan | 2.4.9
| ipvs | VIP: 10.4.0.30
+------X--------+
-/\____
/ \-
-/ \
-/ \
/ \
RealServer1 RealServer2
10.0.3.99 10.0.3.100
/etc/shorewall/hosts :
swan eth0:10.44.0.254
1. the access: client -> 10.4.0.30 is working OK
Done with /etc/shorewall/rules
ACCEPT swan:10.44.0.0/24 fw all
2. The masq for real servers to exit with 10.4.0.30 is OK
Done with /etc/shorewall/masq
eth0::10.44.0.254 10.0.3.99 10.4.0.30 - -
3. The forward from ipvs to real server is OK
when doing a : telnet 10.4.0.30 80
I have the following tcpdump on Node B
10:40:48.682340 IP 10.44.0.254.36701 > 10.0.3.99.webcache:
S1067349055:1067349055(0) win 5840 <mss 1460,sackOK,timestamp2887838843
0,nop,wscale 5>
10:40:48.682479 IP 10.0.3.99.webcache > 10.44.0.254.36701:
S3408723439:3408723439(0) ack 1067349056 win 5792 <mss 1460,sackOK,timestamp
696330234 2887838843,nop,wscale 7>
10:40:51.681631 IP 10.44.0.254.36701 > 10.0.3.99.webcache:
S1067349055:1067349055(0) win 5840 <mss 1460,sackOK,timestamp 2887841843
0,nop,wscale 5>
10:40:51.681748 IP 10.0.3.99.webcache > 10.44.0.254.36701:
S3408723439:3408723439(0) ack 1067349056 win 5792 <mss 1460,sackOK,timestamp
696333233 2887838843,nop,wscale 7>
10:40:52.282769 IP 10.0.3.99.webcache > 10.44.0.254.36701:
S3408723439:3408723439(0) ack 1067349056 win 5792 <mss 1460,sackOK,timestamp
696333834 2887838843,nop,wscale 7>
10:40:58.283227 IP 10.0.3.99.webcache > 10.44.0.254.36701:
S3408723439:3408723439(0) ack 1067349056 win 5792 <mss 1460,sackOK,timestamp
696339834 2887838843,nop,wscale 7>
However the return never arrives to the client. I don''t seen any
drop/reject on the firewall. But I don''t know what
is missing.
When I bypass the ipvs by a DNAT rules like this one :
DNAT:info swan:10.44.0.254 loc:10.0.3.99:8080 tcp 80 - 10.4.0.30
it works, but I am loosing the loadbalancer ipvs.
I am obviouly missing a rule but I don''t know which one. Can someone
help me ?
Thanks
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer''s
challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Sebastien COUPPEY wrote:> Sorry for the previous email, > Here are the missing info and the dump with the used IPsThe nat table in this dump makes no sense at all -- please forward a tarball of /etc/shorewall/. You can send it to support@shorewall.net if you like. Thanks, -Tom -- Tom Eastep \ The ultimate result of shielding men from the Shoreline, \ effects of folly is to fill the world with fools. Washington, USA \ -Herbert Spencer http://shorewall.net \________________________________________________ ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer''s challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/
Hello,
I re-created a test platform with a lighter configuration.
Here are all the information.
I am facing difficulties with my chain :
client - ipsec - shorewall - openswan - ipvs - Real servers.
It seems that the return packets never arrive to the clients.
Architecture :
client :10.44.0.254
|
|
\
+----+----+
| node A |
| |
+---+-----+
|
|
|
|
|
+------+--------+
| node B |
| shorewall | 4.0.11
| openswan | 2.4.14
| ipvs | VIP: 10.4.0.30
+------X--------+
-/\____
/ \-
-/ \
-/ \
/ \
RealServer1 RealServer2
10.0.1.60
Ldirector configuration :
virtual=10.4.0.30:80
real=10.0.1.60:80 masq
service=http
protocol=tcp
checktype=on
1. the ping: client -> 10.4.0.30 is working OK
Done with /etc/shorewall/rules
ACCEPT swan:10.44.0.0/24 fw all
2. The masq for real servers to exit with 10.4.0.30 is OK
Done with /etc/shorewall/masq
eth2::10.44.0.254 10.0.1.60 10.4.0.30 - -
3. The forward from ipvs to real server is OK
when doing a : telnet 10.4.0.30 80
I have the following tcpdump on Node B
Tcpdump from the shorewall :
15:36:27.558268 IP 10.44.0.254.49598 > 10.4.0.30.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991974947
0,nop,wscale 5>
15:36:27.558310 IP 10.44.0.254.49598 > 10.0.1.60.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991974947
0,nop,wscale 5>
15:36:27.558312 IP 10.44.0.254.49598 > 10.0.1.60.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991974947
0,nop,wscale 5>
15:36:27.558426 IP 10.0.1.60.http > 10.44.0.254.49598: S
3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp
2588542865 2991974947,nop,wscale 7>
15:36:27.558816 arp who-has 10.44.0.254 tell 10.4.0.30
15:36:28.558764 arp who-has 10.44.0.254 tell 10.4.0.30
15:36:29.558589 arp who-has 10.44.0.254 tell 10.4.0.30
15:36:30.557790 IP 10.0.1.60.http > 10.44.0.254.49598: S
3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp
2588545865 2991974947,nop,wscale 7>
15:36:30.557797 IP 10.44.0.254.49598 > 10.4.0.30.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991977947
0,nop,wscale 5>
15:36:30.557826 IP 10.44.0.254.49598 > 10.0.1.60.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991977947
0,nop,wscale 5>
15:36:30.557828 IP 10.44.0.254.49598 > 10.0.1.60.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991977947
0,nop,wscale 5>
15:36:30.557930 IP 10.0.1.60.http > 10.44.0.254.49598: S
3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp
2588545865 2991974947,nop,wscale 7>
15:36:36.557900 IP 10.0.1.60.http > 10.44.0.254.49598: S
3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp
2588551865 2991974947,nop,wscale 7>
15:36:36.558100 IP 10.44.0.254.49598 > 10.4.0.30.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991983947
0,nop,wscale 5>
15:36:36.558148 IP 10.44.0.254.49598 > 10.0.1.60.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991983947
0,nop,wscale 5>
tcpdump from the Realserver :
15:36:27.509438 IP 10.44.0.254.49598 > 10.0.1.60.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991974947
0,nop,wscale 5>
15:36:27.509510 IP 10.0.1.60.http > 10.44.0.254.49598: S
3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp
2588542865 2991974947,nop,wscale 7>
15:36:30.508811 IP 10.0.1.60.http > 10.44.0.254.49598: S
3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp
2588545865 2991974947,nop,wscale 7>
15:36:30.508944 IP 10.44.0.254.49598 > 10.0.1.60.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991977947
0,nop,wscale 5>
15:36:30.508950 IP 10.0.1.60.http > 10.44.0.254.49598: S
3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp
2588545865 2991974947,nop,wscale 7>
15:36:36.508971 IP 10.0.1.60.http > 10.44.0.254.49598: S
3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp
2588551865 2991974947,nop,wscale 7>
15:36:36.509314 IP 10.44.0.254.49598 > 10.0.1.60.http: S
3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991983947
0,nop,wscale 5>
15:36:36.509320 IP 10.0.1.60.http > 10.44.0.254.49598: S
3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp
2588551865 2991974947,nop,wscale 7>
However the return never arrives to the client. I don''t seen any
drop/reject on the firewall. But I don''t know what
is missing.
When I bypass the ipvs by a DNAT rules like this one :
DNAT:info swan:10.44.0.254 loc:10.0.1.60:80 tcp 80 - 10.4.0.30
it works, but I am loosing the loadbalancer ipvs.
I am obviouly missing a rule to link packet from
loc -> ipvs -> shorewall -> openswan
but I don''t know which one. Can someone help me ?
Thanks
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer''s
challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Sebastien COUPPEY wrote:> Hello, > > I re-created a test platform with a lighter configuration. > Here are all the information. > > > I am facing difficulties with my chain : > > client - ipsec - shorewall - openswan - ipvs - Real servers. > > It seems that the return packets never arrive to the clients.Does this work with Shorewall turned off (''shorewall clear'')? -Tom -- Tom Eastep \ The ultimate result of shielding men from the Shoreline, \ effects of folly is to fill the world with fools. Washington, USA \ -Herbert Spencer http://shorewall.net \________________________________________________ ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer''s challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/
Thanks Tom for the reply, On Thu, Oct 09, 2008 at 07:35:24AM -0700, Tom Eastep wrote:> > I am facing difficulties with my chain : > > > > client - ipsec - shorewall - openswan - ipvs - Real servers. > > > > It seems that the return packets never arrive to the clients. > > Does this work with Shorewall turned off (''shorewall clear'')?No, and I didn''t even thought about performing this test. Thanks for pointing me to the right direction, because with shorewall down this is not a shorewall problem. Sorry for disturbing ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer''s challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/